Sergey Shishkin

software craftsmanship in practice

Designing Web API

Earlier this week I gave a workshop on Web API Design at OOP 2013 in Munich, which I want to summarize in a blog post.

With only a flip chart and markers I started designing from scratch a web API with heavy use of hypermedia for an imaginary “groups and events” website a la meetup.com. The following features have been covered iteratively, one by one, showing the benefits of the hypermedia approach:

  • events – open and link together the core data of the site;
  • personalization – user-specific resources like “my events”;
  • search – allow for parameterized querying;
  • groups and topics – simply more linked data mainly to illustrate addition of core features at a later stage;
  • RSVP – doing updates driven by hypermedia.

Start

I asked the audience which format they prefer for the representations and they voted for JSON although I mentioned that HTML would make many further steps easier. Anyways, format is in my opinion subject to content negotiation and a new one can always be added later.

Usually the next step would be to come up with a URI tree of resources, which will form the API. This wasn’t what I did though. Instead I suggested to decide on a hypermedia-driven way the clients will discover available resources. With absence of hypermedia RESTful APIs rely on developer documentation for that purpose. A much better approach to me is a single well-known resource, serving as an entry point into the API and containing only the valid of the potential next interaction steps. The Home Document draft for example specifies how such resource might look like in JSON.

Linked Data

Having defined the entry point into the API, we can move to modeling the core domain resources. Since we solely rely on hypermedia for discovering resources starting from a single well-known entry point, it is logical to visualize the design process with a directed graph, where nodes represent resources (or states of application interaction) and edges represent hypermedia controls leading from one resource to another.

An example of such a graph is my poor flip chart drawing on the right. Here we start with the Home resource and by means of a hypermedia link known as upcoming-events navigate to a resource of type Events representing all the upcoming events on the site. Nodes on the graph really represent whole classes of resources or resource types – showing individual resources on a graph would hinder the overview.

The upcoming-events hypermedia control (a green arrow) is simply a link in a JSON Home Document, but the Events representation has a collection semantics, containing individual events, which are separate resources on their own right. To avoid reinventing the wheel, we’ll use the Collection+JSON media type to represent event collections. Collection+JSON resources can reference other resources via links, which comes in handy e.g. for pagination with link relation types next and prev.

There is a more ubiquitous representation of event collections though, which is iCalendar. iCalendar format automatically opens our data to all sort of calendar applications like Outlook, iCal and Google Calendar. It is a low hanging fruit in terms of interoperability, which is the primary aim for open data and APIs. Solutions like iCalendar or Atom feeds should probably be considered as Lo-Fi API alternatives at the early stage of API development.

In REST terms iCalendar and JSON are just different representations of the same resource and thus should share the same URI. The client then will have to ask for the desired representation with the Accept request header. This is unpractical though, since the iCalendar feed will be requested by clients like browsers and calendar applications, which won’t bother requesting the special iCalendar media type. A much more interoperable option would be to provide a link with the relation type alternate from the JSON representation (or add it directly to the Home).

Personalization

From the hypermedia perspective a resource like “my events” is just another resource, linked from somewhere and linking to somewhere else. The trick is that it can only be accessed with a valid authentication. The drawing above shows two ways of of discovering personalized resources via hypermedia: The first one is the link my-upcoming-events, which — when activated — responds with a standard HTTP authentication challenge (depicted with the red padlock). Another option is a loopback link (red arrow Auth) from Home to itself, meaning that the client receives upon authentication a personalized representation of the Home resource with a (previously not present) my-upcoming-events link.

Querying

Event search results are basically just a collection of events, for which we already have a resource type Events. We will reuse the same linking mechanisms to expose the event search results via the search-events link. Search however requires some user input to generate a representation of search results. The URI Template specifies a way of formatting parameterized URIs. There are already libraries available for expanding URI templates, so the clients shall have no problems consuming those.

More Links

We can now search for and navigate upcoming events, which are already valuable features. Even without topics and groups the API is a viable product worth exposing and consuming. Adding topics and groups to the mix at this stage is neither late nor difficult. We add all-topics link from Home to the Topics resource type. It might as well be popular-topics or any other reasonable default collection semantic, or even several of them to choose from. The Topics collection contains individual topics and may be searched or filtered using a parameterized link. A Topic resource references a collection of events and a collection of groups, both filtered by the topic. Groups and Group resources follow the same pattern of collections and links.

Collection+JSON provides collection semantics and linking capabilities. It is however not the only one option. The Hypertext Application Language (HAL) allows to embed multiple resources in a single JSON (or XML) representation and to link to other resources. The Web Linking describes a format-neutral way of putting links in a special Link HTTP response header.

Updates

Until now we’ve dealt with safe read-only HTTP GET requests, but rsvp’ing an event is an update and should be modeled with an appropriate HTTP verb.

In case of a simple yes/no answer it is sufficient to mark the respective accept or decline link (see the drawing on the left) with the "method": "POST" attribute. The client will POST the request to the specified URI and the server will redirect the client back to the Event resource. But what if we want to receive a short text note from the user along with the answer?

We could also model the RSVP resource as a Collection+JSON type with a built-in item template and defined semantics for adding new items. That way the client navigates to the RSVP collection from the corresponding event, then it finds the item template, fills the template out (adding the attendance status and the optional text note), POSTs the filled-out template to the specified URI and gets redirected to the updated RSVP collection.

Another option would be to add the notion of a POST payload template to other types of links (e.g. HAL links). The benefit of explicit accept and decline links — compared to a generic collection update — is that the server only exposes the valid options in the current context: the user can’t accept an event twice.

Bottom Line

This approach to designing Web APIs is very simple and boils down to defining resources and deciding which formats suit representations better and which hypermedia controls to use to navigate from one resource to another. In this example we used the following hypermedia controls: collections, simple links, authenticated links, query links and update links. There are already standards for implementing some these basic building blocks and more standards will emerge. Focusing on the semantics of hypermedia controls during design will help to evolve and tweak the implementation down the road.

Let’s consider the amount of out-of-band information, which client developers will have to look up in the API documentation in order to make API calls:

  • The single URI of the API entry point (the Home Document);
  • Link relation types used in the API (e.g. my-upcoming-events and accept) and their meaning;
  • Formats of the resource representations (e.g. Events, Event, Group etc.);
  • URI Template parameters and their meaning (e.g. searchTerms as the search query input);
  • Parameters used in the item templates for update requests and their meaning (e.g. note for the text note of an RSVP answer).

All this information is domain-specific and is basically what differentiates a social events API from an accounting API. All the technical details are encapsulated into generic media types (e.g. Collection+JSON and HAL) and is ideally handled by generic hypermedia client libraries.

References