One neat feature that's been in Ubuntu since Intrepid Ibex is the ability to encrypt a private directory. This allows one to put sensitive data, such as one's ssh and gnupg keys as well as data such as email into an encrypted directory. Thanks to Dustin Kirkland for writing up how to do this.
Basically, an encrypted Private directory is created and one can move the directories you want to be encrypted into this directory. Then one create symbolic links to these files in the Private directory. At login time, the encrypted Private directory is automatically mounted and also unmounted at logout time.
Note that hibernating is not a good idea with this solution, since private data in memory is written out to swap, which is not encrypted.