Amazon, SimpleDB, and Base64
The other day I made some changes to the way I am storing data in SimpleDB, part of Amazon Web Services. Instead of storing one data field per attribute, I store several delimited data fields in one attribute, which is more efficient for my needs. I used to use a ~ as a delimiter, but I decided to change the delimiter to a non-printable character code. Suddenly, my data was not returning the way I had saved it and my application was choking on it.
It turns out that Amazon changed the way they handle data storage a few months ago. It didn’t apply to me at the time I read the release notes, so I skipped over it. If there is a non-standard XML character in your data, they will automatically Base64 encode it. Took a while to figure this out. I decided to base64 encode when saving the delimited data and base64 decode upon retrieval. This way, I am not relying on Amazon to provide only encoded data. It is better to have the same data return as what you put in.