Amazon S3
This article is a stub. You can help the IndieWeb wiki by expanding it.
Amazon S3 is a file hosting service from Amazon.
IndieWeb Examples
lionzan.me
Leonardo Zangrando hosts his website on S3.
These are the notes that I took while creating my S3 Jekyll Indieweb site.
- Create account AWS
- Create account AS3 in AWS
- Create bucket in AS3 and initialise CloudFront for website delivery via AWS - instructions here https://github.com/laurilehmijoki/s3_website#how-to-use-cloudfront-to-deliver-your-blog
- Install Jekyll on local - instructions here http://jekyllrb.com/docs/home/
- Write a post and commit (?) - from terminal, in blog local folder type command $ jekyll build
- Upload blog to AWS - in terminal in blog local folder type command $ s3_website push
Other sources
- Other instructions for deployment on GitHub here http://jekyllbootstrap.com/ - not tried
- for tutorial check also http://www.andrewmunsell.com/tutorials/jekyll-by-example/#why-jekyll - used to install jekyll on AWS S3, but now apparently the tutorial is $15 :(
- check also blogging framework http://octopress.org/ - installed a GitHub version, but still not working, site doesn't build (dunno why)
- http://yeswejekyll.com/ very interesting, with links to different templates by http://foundation.zurb.com/
- http://klepas.org/jekyll-a-static-site-generator/
Date Edge
Date Edge is a straighteedge dating website made by Shane Becker and Bookis. It uses S3 for serving assets (images now, audio/videos later) and Cloudfront as a CDN. They pay $0.12 per GB transferred up to 10 TB. The first 1 GB transferred is free. For the month of 2014-05, they transferred 27.941 GB and paid $3.35 USD.
Use local copy or rsync to S3
Frédéric de Villamil is hosting a static website on S3 using plain html pages.
This setup allows to sync the pages using your operating system file manager using drag and drop or local rsync:
- Create an account on AWS – Create bucket in AS3 and initialise Cloudfront for website delivery via AWS - If you don't have it done yet, setup your domain on Amazon Route53. - Install S3 Fuse on your system. It was tested sucessfully on Mac OS, Linux and FreeBSD. - Create a directory to mount the remote file system (the S3 bucket) on the local machine. - Mount the remote file system locally. - Locally copy the files from your work directory to the remote one. Uing rxync here avoids you to transfer the whole files each time.
This setup is quite slow because Fuse is slow, but it was made for someone not really tech savvy who had to update the HTML and push the files on S3. Since that person already had to learn HTML, it was decided to make the process easier by allowing the local copy
notenoughneon.com
Emma Kuo has been hosting her personal site on S3 since March 2015. She is using Skein to publish static html to S3.
Jonathan LaCour
Jonathan LaCour uses S3 to store his generated website files and his location logs.
Michael Bishop
Michael Bishop uses S3 to store and serve images uploaded to his WP instance
Considerations
- If your site has modest traffic then you will have a low hosting bill. There is no minimum cost per month.
- Amazon is responsible for keeping the hardware and software running, and Amazon probably has more resources for fixing problems than you do :-)
- Content on S3 is stored redundantly.
- It is very difficult to serve custom HTTP status codes (e.g. 301 redirects): you must create a dummy file at the appropriate URL and then use Amazon’s web-based management system to set up the status.
- It is not possible to serve HTTP Link headers for webmention or micropub, you will need to use
<link>
elements in the html. - It is not possible to serve HTTPS directly from S3, you will need to use Cloudfront.
- It is not possible to perform advanced URL rewriting on the server.
- You cannot run your own software (e.g. PHP or Perl scripts) server-side.
Brainstorming
S3 as a Media Endpoint
After trying AWS Lambda as a media endpoint, Eddie Hinkle decided to try Amazon s3 as a media endpoint. The idea was that the micropub endpoint would return a presigned s3 url inside the Micropub config query for the media endpoint. This would allow a user that retrieves the url to be able to upload a file directly to s3. This would be less secure because there would be no way to verify the authorization token so it would be assuming the person has access simply because they had a valid access token when they fetched the Micropub config query.
The issue with this is that Micropub clients tend to hold onto Media Endpoint urls for awhile and presigned s3 urls expire in a maximum of 7 days, which means I would need a new Media Endpoint url every 7 days which doesn't work with most Micropub apps currently.
Outages
Times when there was a noticeable S3 outage:
- ...
- 2017-02-28 Portions of S3 were down for quite a while, causing lots of sites to stop working. [1] [2]. Sites that folks noticed were not working:
- ...
See Also
- Amazon
- hosting
- Jekyll
- Building and hosting a static site with Jekyll and Amazon S3 on Mac OS X
- http://aws.amazon.com/s3/
- AWS walkthrough on setting up a static site on S3
- 2017-07-12 Verizon Data Of 6 million Users Leaked Online
…[ NICE Systems ] made a security setting public, instead of private, on an Amazon S3 storage server … . This means Verizon data stored in the cloud was temporarily visible to anyone who had the public link.
[…]
The is the latest leak to surface from a misconfigured Amazon S3 storage unit. In June, an analytics firm exposed the data of almost 200 million voters, and earlier this month, an insecure server leaked 3 million WWE fans’ data last week.
Why does this keep happening? Amazon secures these servers by default. This means the errors that occur are due to changes someone makes with a security setting — typically by accident … .
- s3 being used for static site serving
- 2017 Outage: 2017-03-02 The Verge: How a typo took down S3, the backbone of the internet
“Unfortunately, one of the inputs to the command was entered incorrectly and a larger set of servers was removed than intended,” Amazon said.
- 2017-02-28 outage summary from Amazon: Summary of the Amazon S3 Service Disruption in the Northern Virginia (US-EAST-1) Region