Amazon SimpleDB NextToken edge condition
Protip:
When you query SimpleDB using the "select" expression, the default setting is to return a maximum of 100 items. If there are more items to be retrieved, SimpleDB returns a NextToken and you can supply that into subsequent queries to continue getting data where you left off. The subtle part is that if your query has EXACTLY 100 items, or a multiple of 100....then on the last dataset, you will still get a NextToken even though there is no more data left! The next query using that token will result in an empty dataset.
So don't be fooled: Getting a NextToken does not necessarily mean there will be data on the next trip. This caused me a headache tracing why my custom SimpleDB enumerator was behaving badly.
