A majority of my work on Webmaker has been dedicated to something we call the Make API. It is a service for storing meta data about things people create online and making that meta data publicly searchable. Ever since we first stood up the service back in June, it has only been allowed to accept meta data from trusted web apps like Popcorn Maker and Thimble. This has worked great, but it has become increasingly more obvious to me that there are applications out there that would like to be able to share metadata about their user's creations with the Make API.
Having third parties publishing to the Make API is an exciting idea, but it's also a very complicated one. In our ten week sprint to launch our Webmaker.org beta, the Make API wasn't optimized to play nicely with non-webmaker entities. In order to prepare the Make API for third party publishing, there are now a lot of tough problems to solve, problems that I haven't had to solve before, and ones that will likely require changes across the entire Webmaker application stack. In this post I'm going to present the big issues facing the Make API today and also touch lightly on some possible solutions. Writing about these problems helps me because it will force me to think critically about the tasks at hand, and hopefully will also generate some feedback from my team and community members. So if you have anything to add, please leave a comment below, email me, message me on IRC or hunt me down IRL.
Make API records have a very strong link to Webmaker accounts, in order to determine and enforce ownership. Due to a "lets ship this" mentality in the early weeks of the Webmaker.org sprint, make records returned by search queries have their email fields changed into a Webmaker usernames using a single authenticated Webmaker loginAPI request per record. This is not sustainable or scalable, and needs to be rectified as soon as possible. I personally want to just use the Webmaker username directly, but it would require a lot of changes upstream and downstream. Work would need to be done to ensure that username changes (if they are implemented) trickle down to make records. Additionally, makes published by third parties would need a system for proving that a make should be associated with a Webmaker account, if we allow such a process to exist (ie. "anonymously" owned makes, not tied to a specific user).
The Make API's security model for Create, Update and Delete actions practically limit it to server-to-server authentication (first implemented using HTTP basic authentication, then using Hawk ). Hawk is great for establishing identity and ensuring payload integrity without transmitting credentials, but it lacks the ability to delegate permissions for protected resources to a third party on behalf of a user. This problem has been solved with the OAuth protocol, and the OAuth 2.0 "Framework", but I have heard negative things about both. What I would love to see, is a token based system built into persona, where a logged in persona user can grant an application a token which can then be used in conjunction with server-server authentication protocols (like Hawk) to verify that a request is being made on a logged in user's behalf. Maybe something similar to this crude flowchart I just made in LibreOffice (haha):
Another major roadblock is the need for Capability-based security. There are some actions that we certainly don't want to grant to external entities. Actions such as deleting makes should be limited only to records which an application created. A clear list of ACLs should be defined and used to determine what actions a specific set of API keys is allowed to do.
We will have to build a front end system where potential consumers can submit requests for API keys. It will require a commitment on our end to respond to requests in a timely manner, and we will need to establish TOS so that we can establish what we commit to as a provider of the service, and what is expected of the consumer of the service.
I've covered pretty much all of the bigger issues facing the move to third party access, but there's plenty smaller things that need to be solved as well. Here's an incomplete list:
- The concept of URL vs. Content URLs, and the details of *.makes.org has to get refactored out of the Make API, such that Makes are separate from what Webmaker apps make.
- Remixability shouldn't be a requirement for make records; the idea of /edit and /remix needs to be handled outside of the Make API
- ContentType as a make field is confusing and misleading. Probably could be refactored into some kind of identifier for the tool used to create the make
- We need to expand and better understand the value of having 3rd party apps publish into the Make API, and then build our API, UI, UX to meet those requirements.
- End the dual database nonsense, dropping Mongo and going with just Elasticsearch.
- Improve admin tools, build for external use cases.
- Improve Analytics and metrics for makes