Personal Accounting Software

I have long desired a personal accounting software that can be populated with data from downloaded bank statements and other source documents with minimal manual intervention. GnuCash is a widely used open source software with enough capability to be used by small businesses. However, its ability to import data from other formats is very limited. It can import from Quicken, but since Quicken is not a proper double entry software, this facility is not particularly useful. So even if I convert a downloaded bank statement into accounting entries, I still have to manually input these into GnuCash. That is more work than I am willing to do.

A couple of months back, I discovered Plain Text Accounting and in particular Ledger. The brilliant idea behind this class of software is that the entire accounting data resides in a plain text file as a series of journal entries. Importing data into Ledger is just a question of writing text files. It is hard to imagine anything simpler than this. The only reports that Ledger produces are again plain text and parsing them is not difficult to anybody familiar with regular expressions.

Suddenly, personal accounting became something that could be done relatively painlessly. I do not participate in the underground (black money) economy and so all my income flows through my bank account. Most of my large expenses are paid for through credit cards, cheques or online transfers. All of this is visible in my bank statements that I can download from the web sites of my banks or in credit card statements that also come to me by email. If I buy or sell stocks or mutual funds, I get statements by email from the broker, or the mutual fund or from the depository. The only thing that is left out is the relatively small expenses incurred in cash. Since I am too lazy to manually record and keep track of these, I use the simple accounting policy of treating cash withdrawals as expenses even though they may not yet have been spent. This may appear strange to corporate accountants, but is not unreasonable. If a company gets 5,000 copies of its letterhead printed, it expenses them in the same period even though 4,000 of those may still be in stock. I so no reason to treat pieces of paper printed by the central bank any differently. Accountants who might want to object can be silenced with one magic word – “materiality”. In any case, my personal accounting is not subject to any accounting standards.

So I choose to base all my personal accounting only on the statements received from banks and other various financial firms. These statements come in many different formats – XLS, TXT, HTML and PDF. All these formats other than PDF can be read by a spreadsheet program which can then save them as CSV text files. PDF files are harder, but over a period of time, I have figured out that I can extract the text from these file and then use regular expressions to parse this text into CSV files. (After trying various PDF conversion tools I have found that good old ghostscript does the best job: even if the PDF file is ill-formed or slightly damaged, gs -dBATCH -dNOPAUSE -sDEVICE=txtwrite will produce usable text.)

I decided to use Python to manage the whole process from parsing bank statements to creating the journal entries to running ledger and generating balance sheets and other statements. CSV files can be read into a Python pandas DataFrame which can be sliced and diced in many ways quite easily. It is quite straightforward to use a series of regular expressions to determine the journal entries for each transaction. For example, the regular expression ATM WDL identifies cash withdrawals in one of my bank accounts. To identify online purchases , I use a long regular expression, a part of which reads as follows:

(INB Flipkart)|(INB Avenues)|(INB Indigo)|(INB Amazon)||(INB Makemytrip)|(INB EBAY)

Using such techniques and some manual intervention, each line item in each source document is turned into a journal entry. I choose to represent a journal entry as a Python list whose first element is the date and narration of the entry and the remaining elements are the line items of the entry each of which consists of an account name and amount (a Python tuple). The whole journal is simply a list of such journal entries. This internal representation will be turned into a Ledger input text file at a later stage.
Continue Reading


SWIFT hacking threatens to erode confidence in financial sector

I had a short blog post on the Bangladesh-Bank SWIFT hacking shortly before I went on a two month long vacation. Since then, the story has become more and more frightening. It is no longer about Bangladesh Bank and its cheap routers: the hacking now appears to be global in scope and sophisticated in approach:

  • BAE Systems have identified parts of the malware that was used in the Bangladesh-Bank hacking. This malware “contains sophisticated functionality” and “appears to be just part of a wider attack toolkit”.

The tools are highly configurable and given the correct access could feasibly be used for similar attacks in the future.

The wider lesson learned here may be that criminals are conducting more and more sophisticated attacks against victim organisations, particularly in the area of network intrusions (which has traditionally been the domain of the ‘APT’ actor).

  • More than a year before the Bangladesh-Bank hacking, a total of $12 million was stolen from Banco del Austro (BDA) in Ecuador through SWIFT instructions to Wells Fargo in the US to transfer funds to a number of accounts around the world. The matter came to light only when BDA sued Well Fargo to recover the money.

Neither bank reported the theft to SWIFT, which said it first learned about the cyber attack from a Reuters inquiry.

  • In 2015, there had been an attempt to steal more than 1 million euros from Vietnam’s Tien Phong Bank through fraudulent SWIFT messages using infrastructure of an outside vendor hired to connect it to the SWIFT bank messaging system. TP Bank did not suffer losses because it detected the fraud quickly enough to stop the transfers.

  • SWIFT now admits that there were “a number of fraudulent payment cases where affected customers suffered a breach in their local payment infrastructure”. The whole set of press releases issued by SWIFT on this issue is worth reading.

The picture that emerges out of this is that on the one side there are well organized criminals who are building sophisticated tools to attack the banks. They may or may not be linked to each other, but they are certainly borrowing and building on each others’ tools. Their arsenal is gradually beginning to rival that of the APT (Advanced Persistent Threat) actors (who are traditionally focused on espionage or strategic benefits rather than financial gains). Very soon global finance could be attacked by criminals wielding Stuxnet-like APT tools re-purposed for stealing money.

On the other side is a banking industry that is unable to get its act together. Instead of hiring computer security professionals to shore up their defences, they are busy hiring lawyers to try and deflect the losses on to each other. It is evident that the banks are not sharing information with each other. Worse, my experience is that information is not even being shared within the banks. I have heard horror stories in India of security firms who have detected vulnerabilities in the IT systems of banks being told by the IT departments not to mention these to the top management. These IT people think that everything is fine so long as top management does not know about the problems. The top management in turn thinks that things are fine so long as the regulator does not know that there is a problem. I hear reports of banks quietly reimbursing a customer’s losses without either fixing the problem or reporting it to the regulators or other authorities. Most of the stories that I hear are from India, but the evidence suggests that the situation is not any different elsewhere in the world.

This state of denial and discord in the banking industry provides the hackers the perfect opportunity to learn the vulnerabilities of the banks, improve their hacking tools, and increase the scale and scope of their attacks. At some point, of course, the losses to the banking system would become too big to sweep under the carpet. That is when the confidence in the financial sector would begin to erode.

Another problem for the banks is that in their lawsuits against the paying banker, the victim bank is raising the issue of “red flags” and “suspicious transactions” to argue that the paying banker should have halted the payment. With large amounts of money at stake, this argument would be made by skilled lawyers and may even be successful in court. If that happens, it would set up a dangerous precedent against the banks themselves. So far, banks have taken the stand that their customers are responsible for the transactions so long as the valid authentication was provided. Bank customers typically do not have the resources and inside knowledge to challenge this stand. The inter-bank litigation is very different and has the potential to overturn the established distribution of liability.

I have not so far talked about nation state actors getting into the attack. Any nation state would love to hack the banks of an enemy country. Some rogue states that are excluded from global finance might even want to try and disrupt the global financial system. India is one of the countries at serious risk of an attack from a resourceful nation state, but as I look around, I see only complacency and no sense of concern let alone paranoia.