None of my banks or financial institutions has an option to deliver statements by e-mail attaching a pdf. The statements have to be downloaded from their websites. Of course every institution handles the location of the applicable download and the process differently. Is there a way to create an automation to do this for me? My manual process works fine, but this seems like the kind of task the computer should be able to do without my intervention. I don’t use IFTTT, Zapier, or any other services like that, so I’m not interested in a solution that requires them. Other than that constraint, I’m open to anything.
The best way to do this is to use
curl or the equivalent to download the document directly based on its URL. This means, of course, that you have to be able to predict the URLs of future documents when you create the automation. Sometimes you can do this by looking at the URLs of documents already available at your bank’s website and working out the naming system they use. There are, unfortunately, no rules you can apply across the board.
That’s exactly what I needed. Thanks for pointing me in this direction. I’ll try it out and will report back about my victory… or any obstacles I encounter. Thank you.
Might be worth highlighting that many consumer accounting packages have banking integrations. Not so sure on what other financial company integrations they might have.
Any direct access via
curl and the like I would expect to run afoul of authentication issues as the authentication should be being securely handled within the browser and not be leaked to separate command line sessions. But I guess there is always a way to exfiltrate and reuse it when you have the will and the access, so that may not be an insurmountable issue.
I certainly wouldn’t be happy if my bank was putting financial statements out on unauthenticated public URLs.
Yeah, authentication is a problem, but there are ways (not easy ways, but ways) of adding authentication to an automation. On the other hand, if there’s no means to predict a document’s URL, it’s game over, and you can forget about downloading it automatically.
I hear you on this. All my financial institutions enable direct access to my financial software, but I still keep and reconcile statements. My software can enable me to get the statements, but that’s not really different from the manual approach.
Authentication is a hurdle, although
curl does provide techniques that enable you to comply with the authentication system (whether they’d work in my case or not, I don’t know). Regardless, I just pulled up a few arbitrary statement-download urls and there is no discernible pattern I can detect. There is plenty of information that I can parse, but most of the useful information is what looks like a very long sequence of arbitrary hexadecimal numbers.
Most financial institutions use ofx. Your best bet might be to use python and an ofx library (e.g. ofxtools)
There are two online services that make this easy:
FileThis and HubDoc.
Both allow you to add your financial institutions credentials into their systems. Their systems advertise that they use “bank-level” security and encryption.
The services leverage your credentials to automatically download statements and other documents.
Both services can connect to more than banks and credit card companies. Insurance companies’ documents and statements are another one that comes to mind.
They both do an initial sump of documents and then check periodically for new docs. If they find any, they download them.
Where do they go? The services also allow you do link a Dropbox or google drive account so that the files (PDFs) show up in whatever folder/directory you’ve set up.
I started using Hazel to move them from Google Drive to my NAS.
And now, I havea one-way sync on my Synology NAS so that the NAS maintains a copy even if I remove the files from my cloud storage provider.
This results in a completely automated way to gather statements, etc., without me needing to do anything, except occasionally update a credential with the service.
Not all companies have the same auth schemes, or support OAUTH, making me login every time I want to have the service scan for any updates files and download them.
Also, if you’ve enabled 2FA, it causes headaches.
I have subs to both services, and I like HubDoc better because I can give my BookKeeper and CPA access to all my statements, making my life a heck of a lot easier.
It’s not perfect, but it’s good enough for me to keep using them.
Thank you for this really helpful response. I will check these options out.
I use OFX for direct access to my banks to download transaction data. I still reconcile my monthly statements, so I like to download them and save them in my financial records archive.