2011/Stream Storage DB+S3

Session Notes

Notes from Al Partridge

What to store?
1) Breakout the fields you operate on often
2) Capture the original JSON for the msg (Blob)
3) Meta data about capture (Date/Time, Source, etc)
4) Flags (Spam, Auto-filter, Human Filter, Publish?)

Potential Feed Storage Options:
Your DB - MySQL / MS SQL / PostGres / Oracle / Etc
Cloud RDS - Amazon RDS / MS SQL Cloud / Etc
Flat - Amazon Simple Table / Google Big Table / Twitter FlockDB
CDN - publish from one of the above to a flat file on your CDN

A new way of looking at storing the data storage for your indie web site = distinct services

Auth Service - User Data / Solution: Jan Rain, Facebook Connect, etc
Central RDS - Main site functions / Solution: Local User Profile, app tables, etc
High Availability - Stream Data(Twitter, Facebook, etc) / Solution: Amazon Simple Table and/or flat file to CDN