Any business, freelance or otherwise, would be wise to retain everything related to that business for at least seven years, if not longer.
You should have files for communications, taxes, accounting, transaction records, contracts, etc.
For physical records (receipts, print outs, letters, etc), I have files where I store and sort such records. Well, not really "files". I simply throw these things in a box I have in my office. Then in January of every year I close the box. Write the preceding year on the exterior, and store the box out of the way. Then start a new box for the new year.
For digital records, I create a directory titled for the year, then zip the directory and back it up in at least 3 locations, one off-site. Although I do this in January for the preceding year, I do tend to keep these digital items available for at least another year. Most of the records are in the form of emails, so I do an archive of my email accounts and back those up. Knowing they've already been backed up, I delete items when it's clear I no longer need to be able to access anything with regularly - i.e. if a client is gone, I can delete their emails knowing I have them backed up. I can always unarchive the emails and gain access again should I need. When I started out would print emails, but that quickly became untenable due to the lack of any "search" function and the sheer amount of paper/toner it took. Keeping things digital makes it much easier to find something later.
I'm not saying this is the absolute proper way to do anything, merely how I've handled it.
