Hosting a Website on AWS S3

Amazon Web Services (AWS) Simple Storage Service (S3) can host static web sites at an extremely low cost. The cost for most websites is a few cents to a few dollars per month (for a very large, high traffic websites).

In this post I will discuss two software tools, written in Java, that allow large websites to be moved to S3 and maintained once the website is on S3.

My personal website, bearcave.com, went live in 1995, in the early days of the Web.

Bear Products International Home Page_Edit description_bearcave.com

Over the years, bearcave.com has grown to over 2300 web pages. These pages include work that I have done on wavelet signal processing and computational finance.

Until recently I hosted bearcave.com on a commercial Linux Web host (names are omitted to protect the guilty). At $35/month (for “platinum support”) this is very costly compared to hosting on AWS.

After building my consulting website, topstonesoftware.com, on Amazon S3 I meant to move bearcave.com to S3. I kept putting off this project, since it is a big task. Finally the Web host I was using provided sufficient motivation. They moved my Linux home directory and websites to a new sever, losing photographs of my late Mother’s art work. Several weeks ago my photo galleries, containing several years of travel photos, stopped working (although fortunately the photos were not lost as they were with my Mother’s photo galleries).

When I complained to the Web host, they were completely unapologetic. Although I had been a customer for many years, they didn’t care that they had lost photographs and broken my website. This cavalier treatment finally provided the impetus to move bearcave.com to S3.

Most of bearcave.com consists of static web pages. Other than some dead links, the only part of bearcave.com that was broken were the photo galleries.

I originally hosted my photo galleries using the PHP based Gallery.

Gallery | Your photos on your website_So what's next? Sometime soon we'll make the website read only. All the forum posts and topics will still be there, but…_galleryproject.org

To support my photo galleries on AWS S3, I needed to move away from the dynamic PHP based Gallery to a static (or at least JavaScript) based photo gallery.

When I built the photo galleries for the nderground social network, I used the Galleria JavaScript based galleries.

Galleria | Responsive JavaScript Image Gallery_A free JavaScript image gallery framework that simplifies the process of creating beautiful image galleries for the web…_galleria.io

With this experience, I decided to move the bearcave.com photo galleries to Galleria.

The version of the PHP based Gallery I used stored all of the photos and photo metadata (e.g., captions) in files. The photo metadata, including the caption information, was stored in a set of files named photo.dat, photos.dat.0, photos.dat.1, …

I wanted to recover the captions from the metadata and build the HTML that would be used by Galleria. This was a task that would be almost impossible to do by hand, especially since I have hundreds of photos.

I wrote the GalleryBuilder Java application to process the Gallery metadata and build Galleria HTML. This application can be found on GitHub:

IanLKaplan/GalleryBuilder_GalleryBuilder - A Java application that will build static photo gallery HTML from Gallery directories_github.com

GalleryBuilder is a specialized tool for converting Gallery based photo galleries. It will only work on the file based version of Gallery (a later version used mySQL to store photos and photo metadata).

Once I had generated the HTML for the new Galleria based photo galleries, I used the JetBrains IntelliJ integrated development environment to build and test my new Galleria based web pages.

IntelliJ IDEA: The Java IDE for Professional Developers by JetBrains_Capable and Ergonomic Java IDE for Enterprise Java, Scala, Kotlin and much more..._www.jetbrains.com

When the Galleria photo galleries were complete, I had a (mostly) working website. The next step was to move the web pages over to AWS S3.

S3 has a web based user interface that allows the user to move files to S3 and create directories. Unfortunately, directories on the local computer cannot be moved without laboriously creating a directory and then moving all of the files in that directory. Moving all 2300 of the bearcave.com web pages this way would be extremely time consuming and error prone.

I wrote the Java S3Update application to move directory trees from the local system to S3. This code can be downloaded from GitHub (you will need to know how to build, compile and execute Java code).

IanLKaplan/S3Update_S3Update - A Java application that copies/updates files in an Amazon Web Services S3 bucket with files from a local…_github.com

The S3Update application was developed with Eclipse and the GitHub repository includes the Eclipse project files.

The S3Update application will also update files on S3 when copies are changed on the local system. This makes maintaining a website on S3 much easier.

The S3 storage is a web resource and is accessed via HTTP. Operations that access S3 files spend most of their time waiting for the HTTP transactions to complete. To improve performance, the S3Update application is multi-threaded. This allows one thread to read or write while another thread is waiting. With 32-threads this saturates my local network connection (I only have a 10Mb/sec network connection, so your mileage may vary).

To use the S3Update code you will need to create an AWS ID and secret key using the IAM service. These keys should have permissions that are limited to reading and writing S3 (AmazonS3FullAccess).

Before you can move your website to S3, you will need to build the base static website on Amazon. They have published a fairly good guide on how to do this:

Example: Setting up a Static Website Using a Custom Domain - Amazon Simple Storage Service_Walk through a code example of how to set up a static website using a custom domain._docs.aws.amazon.com

This will require an Amazon Web Services account.

When you set up your S3 hosting you will need to change your DNS so that it is handled by Amazon. This will require you to login to your domain provider (e.g., godaddy.com, networksolutions.com, etc…) to change the DNS to the DNS servers that Amazon provides.

The low cost of hosting a website on S3 makes it an attractive alternative for hosting static websites. The S3Update tool makes hosting and maintaining a website easier.

A final caveat: The process of building and maintaining an S3 website is complicated and may be outside the comfort-zone of many non-technical users. Sites like wix.com make building a website easier and include a variety of attractive templates that you can use to build simple websites. Although these commercial web hosting platforms are more expensive than AWS S3, the cost may be worth it for people who are not web developers.