I've spent the better part of my last two weeks working on implementing a new form of authentication for the Make API. This post is going to detail what has changed, how it works, and how you can set up your local webmaker-dev environment to use it. It may also be helpful to anyone who's interested in implementing Hawk's authentication scheme in their own app.
Background
At the Mozilla Foundation, I'm a part of the Webmaker.org engineering team. For some time now, I've been focusing my efforts on developing the Make API. The Make API is a service for indexing things that have been created on the web (or even off the web, in some cases), exposing them through a search API backed by Elastic Search. The general idea is that when applications like Thimble and Popcorn Maker publish content, they can then send meta data about the make to the Make API. The MakeAPI takes this metadata and Indexes it into an ElasticSearch cluster. We unified user accounts in these tools using a custom implementation of single sign on using Persona. The result of this work was a single place for Webmaker.org to query for data about makes, enabling the creation of galleries and search tools on Webmaker.org. Not only that, but for the first time, users have their own personal galleries of things they've made.
The Problem
During our initial push to release the new Webmaker, we did some quick and dirty things. Some would call it "technical debt". Early on, I realised that the Make API would need to protect its create, update and delete paths, in order to prevent abusive behaviours. We agreed that the minimum viable product (MVP) was to have only the three applications that needed these route use basic authentication over SSL to authenticate calls. While this works great, It doesn't scale at all to a system that accepts metadata about makes from applications outside of the Webmaker universe.
The Solution
The obvious solution was to create an API key system for authentication. With a system like this in place, we can receive requests from apps that wish to index their user's creations on the MakeAPI and issue them keys that grant access. I've got no previous experience doing something like this, so it was going to be a learning experience. A colleague of mine suggested using Hawk. Hawk is an HTTP authentication scheme that can be used to verify the authenticity of messages using cryptographically generated message authentication codes (MACs). I chose to give hawk a try as it provides us with several other wins as well, such as: replay protection, payload and response payload validation and most notably, the shared secret never gets transferred between the client and server.
WAT?
To give you a better idea of what Hawk is doing I'll use my somewhat limited understanding of the protocol to breakdown the steps. (assuming a client-server set up where both implement the Hawk protocol)
- The server generates and stores an ID and a secret key. The ID and key are sent to the client over some secure channel.
- To make a request to the server, the client generates an Authorization header using hawk, passing Hawk its ID, key, request URL, payload, a nonce, and a time stamp. Behind the scenes, Hawk uses this information to generate a MAC. The header looks something like this:
Authorization: Hawk id="dh37fgj492je", ts="1353832234", nonce="j4h3g2", mac="6R4rV5iE+NPoym+WwjeHzjAGXUtLNIxmo1vpMofpLAE="
- Notice that the Key is not a part of the Authorization header!
- The request is sent to the server. Upon receipt of the request, the server must parse this header, grabbing the secret key from its database for the specified ID.
- Using the information in the header, it does the same cryptographic steps the client did to generate another MAC.
- If the two MACs match, the request is valid!
The How
Implementing Hawk in the Make API was not too difficult. After specifying Hawk as a dependency, I created a middleware function that would authenticate requests using the steps I outline above (4-6). I created a function to DRY out attaching SERVER-AUTHORIZATION headers to responses, and wrote a small script to test out the new authentication model using static keys and ids. Once I had success here, I defined a simple Mongo model to store the key pairs and make them available for look-up by the Hawk authenticate middleware.
It's worth mentioning that I decided to go with a public/private key naming convention. In my description of the Hawk protocol, I described the use an ID and a key. The MakeAPI refers to the ID as the "public key" and the key as the "private key". Both of these pairs are UUID's generated by the node-uuid module.
Once that was in place I put the pieces together. In the interest of reducing inline code, I've linked several gists below with the specific bits of code that make up the resulting implementation.
This Middleware show how I authenticate requests with Hawk. I've added extra comments to make the code clearer
This is the respond function that us used to generate MACs for server responses.
This is how the makeapi-client code to generate MACs.
With this code in place, applications and sites like Thimble, Popcorn Maker and Webmaker.org can be issued keys to use when they do create/update/delete actions with the Make API!
All this would be useless if there was no way to generate these keys, so I built two ways to do this. The first way is using a very simple tool on the Make Editor page. If you scroll below the Make grid view, you'll see an input box, a button and a text area. The input box accepts an email address to associate with a pair of keys. Clicking the button instructs the server to generate a key pair for you, which is presented to you as a JSON object in the text area.
The second (and recommended) method for generating keys is to use the command line script I wrote. It can be located in the scripts folder and is invoked with two parameters. The fist indicates the email address to use with the key pairs, the second indicates the number of pairs you wish to generate (great for issuing keys in bulk).
node scripts/generateKeys chris@chrisdecairos.ca 3
This will generate 3 pairs of keys associated to the email address in the example. Output of the program is sent to stdout and returns exit codes based on the result, so it can be used in scripts.
Setting up an app to use the key pairs is really easy. You just place the credentials in the apps respective environment variable/config file ( or invoke it with the variables on the command line ). Your best bet to figuring out the expected name of the variable in webmaker apps is to look in there sample/default environment variable files.
I CAN HAZ KEYS?
Not Yet.
There's still more to do before we can issue keys to third parties. We need to implement response validation for the client code. Also, we need to implement server logic that prevents a key pair from being used to delete a make that it did not create. That will prevent a third party app from deleting makes from any and everyone using the API.
We also have to determine how tightly coupled makes will be with Webmaker accounts. To be clearer: How do third party makes determine who "owns" them? Does the consuming app have to link their users account to a valid Webmaker account via some API we have yet to write? are they "anonymous"? How do third party apps update the metadata in an already published make?
These are some of the questions I'll be asking over the next few weeks while we work out the details. For now, I'm just glad we're taking a big step towards a real, open API for sharing and showcasing all the awesome stuff people create around the world!
Questions, Comments and Suggestions are welcome! If you would like to get involved, visit webmaker.org/getinvolved and check out all the ways you can help make the web!