30 Jul 2016 7 min read

Wercker and AWS S3 for a static middleman site

Recently I've been working on a few projects that are working really well as simple static sites. I've been toying with Middleman and I'm really enjoying it!

My current setup for development projects is using Middleman as my SSG (static-site generator), Wercker as my CI and deployment, and Amazon's S3 as my hosting platform. I like this for a few reasons:

AWS has a free tier for a year which I'm making use of. For a static site I can use S3 as the entire hosting platform for free for now.
Middleman is free and open source. It's not another SAAS platform so you can just pick it up and play.
Wercker also has a free tier meaning my CI/CD is free if I make a few compromises.

I thought I'd share my current setup, it takes me on average about 5-10 minutes to set this process up for a new project. It's simple, reliable, and cheap!

1) The repository

Create a new repository on github, this is where we'll host our code. Other providers should work just fine but may be a bit more fiddly with web hooks and such.

2) The base project

You're going to need to install the Middleman gem with gem install middleman, this may take a few minutes to install and setup so be willing to wait.

Once that's completed we can use the generator to create our base project. Use middleman init middleman-wercker-s3-sample, this will create a middleman project inside the directory middleman-wercker-s3-sample so make sure you're in the directory you wish to execute this.

Middleman will ask some questions, answer these to suit your project. None of the options effect how the rest of our process works at time of writing.

Then push this to github, the commands will be on your github repo if you didn't initialise with any files, here are the ones I'm using.

# Enter the project
cd middleman-wercker-s3-sample

# Create git repo and commit the base project
git init
git add -A
git commit -m "Initial commit"

# Add the github remote and push the changes
git remote add origin https://github.com/BlueHatbRit/middleman-wercker-s3-sample.git
git push -u origin master

We now have the default middleman site and it's nearly ready for us to start working on deployment.

3) Adding a JavaScript runtime

Wercker uses linux based "containers" to run builds. This will cause a problem for us as Middleman requires a JavaScript runtime. Most Linux operating systems don't come with a pre-installed with one.

We can fix this by adding the gem therubyracer to our project, just dump this at the bottom of your Gemfile. You could add it just for linux environments but I'm not going to bother here.

# Include a JavaScript runtime
gem 'therubyracer', '~> 0.12.2'

Commit this and push the changes up to github so that we can start working on our build and deployment.

4) The Wercker project

Log into Wercker using your github account. If you already have a different account you wish to use then ensure it's connected to your github account.

Create a new "application" and select the relevant settings for you. Just make sure it's feeding form the right repository.

As a side note for werckers step 3. Public projects can get away without using any SSH keys generally; if you're making a private repository then you'll need to attach SSH keys which wercker will happily do for you if you select the right option.

5) The S3 Bucket

I'm assuming you're somewhat familiar with AWS and S3 buckets. If you're not then I'll speed through creating one.

Log into your AWS console and under "Storage and Content Delivery" open up S3.

Create a new bucket with the big blue button. Choose a name that makes sense to you and a region that fits your needs, I'm in the UK so I'm choosing Ireland just for the sake of it.

Once it's created, select it and open up the "Static Website Hosting" tab. Click enable and add index.html as the index document. You can add an error page later but at the moment the project doesn't have one.

Note the "Endpoint" given to you, this is the URL that you'll be able to use to access the website later.

Now we need some API keys that'll give us access to our S3 bucket. These will be used by wercker to deploy our site later on. To do this head on over to "IAM" in the Services menu, then click on "Users" on the lefthand menu.

6) Creating an IAM user for our S3 bucket

We're not going to create a new user that only has access to our new bucket. This means wercker can never access anything else on our S3 platform.

Hit "Create New User", then choose a name that makes sense to you. Make sure to tick the box asking if you want to generate access keys for the user.

It's important to make note of the access keys as we'll need these in a moment. Then close our of this screen and go back to the IAM console.

Currently this user can't do anything, so we need to assign it some "policies". We're going to create a custom policy to only allow access to our bucket.

Go to the "Policies" tab on the lefthand menu and hit "Create Policy" then choose "Create Your Own Policy".

Create a name that makes sense to you and a useful description, then use the follow policy document. You'll need to change the bucket names to fit the name you used earlier for your bucket, these are in the two "Resource" nodes. For example if my bucket name was my-bucket then my two resources would be arn:aws:s3:::www.middleman-wercker-s3-sample.com and arn:aws:s3:::www.middleman-wercker-s3-sample.com/*

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::www.middleman-wercker-s3-sample.com"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:DeleteObject",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::www.middleman-wercker-s3-sample.com/*"
            ]
        }
    ]
}

Now that we've created that, we can go back to the "Users" tab, find our new user, and under "Permissions" we can hit "Attach Policy". Now all you need to do is attach the policy we just created and we're ready to move on.

7) Automating the deployment pipeline

We now need to define a "Pipeline" for our deployment on our Wercker app. Head back over to the Wercker app and click on "Manage Workflows", then click "Add New Pipeline" in the second section of the page.

We'll name this one "deploy" and this also happens to be what we'll call our YML identifier in a moment.

Now we can connect this to our default workflow. Head back to the Workflow manger and attach our new pipeline to the "build" pipeline for our default workflow but only for the master branch. This means that if a build succeeds from the master branch, wercker will attempt to deploy it for us.

8) Adding IAM access keys to our Wercker application

We now need to add the IAM access keys to our new pipeline. We're going to do this as "protected environment variables". This means they won't appear in any logs and even if someone gains access to that panel they won't be able to see them.

AWS_ACCESS_KEY_ID - The IAM user access ID
AWS_SECRET_ACCESS_KEY - The IAM user access key
AWS_BUCKET_URL - The aws bucket url, this is just the name we created prepended with s3:// such as s3://www.middleman-wercker-s3-sample.com. This doesn't need to be protected but it can be if you want.

9) The wercker.yml file

So we have our S3 bucket, our deployment user, our wercker application, and we've accounted for our build environment. It's time to tell wercker how to build and deploy our application.

For this, we will create file in the root of our repository called "wercker.yml", the content will be the following.

box: ruby:2.3.1
build:
    steps:
        # Install dependencies
        - bundle-install

        # Execute middleman build
        - script:
            name: middleman build
            code: bundle exec middleman build --verbose
deploy:
    steps:
        # Deploy to S3
        - s3sync:
            key_id: $AWS_ACCESS_KEY_ID
            key_secret: $AWS_SECRET_ACCESS_KEY
            bucket_url: $AWS_BUCKET_URL
            source_dir: build/

For those unfamiliar with Wercker files, I'll briefly describe what this does.

Tell Wercker that we want to run our build within the ruby 2.3.1 container, that's right, we're using containers. There's nothing to worry about though, this is as close as we come to caring about them.
build: defines the steps we want to carry out for the build stage of the process. Here we're simply using the pre-build step provided by Wercker to install dependencies using bundler. Then we're running the build command that wercker gives us. This results in a directory called build which contains our generated html, css, and javascript!
deploy: defines our deployment step. Here we're using the pre-built step called s3-sync to say that we want to sync an s3 bucket. We're then providing all the details as environment variables from the Wercker application. These are the ones we set earlier in the S3 step.

Commit that and push it to github and you should see wercker pick it up and begin the process. If you're not on a paid wercker account then your process can take a while as it uses a public queue.

Everything should now work like a dream. When the build and deployment have passed, you'll be able to see your website at the endpoint we got from your bucket earlier.

Closing notes and resource links

There's a lot to do and keep track of, but once the process is done you can update your website simply by pushing to master. The process is now self documented through code and configurations, and the deployment is far better than any manual attempt.

I like the simplicity of this build and deployment process and I find it works nicely with github flow.

As always, everything I've done here is now open source! You can view the code on github, the builds on wercker, and the final product at the bucket endpoint.