Database Backwards Compatibility

The popularity of the “backend-as-a-service” (BaaS) model, or products that bundle the relational database, security, storage, and numerous other integrations into a single interface, continues to grow. For good reason: setting up a full-featured backend and its continuous deployment pipeline to a cloud provider can take even the most experienced engineer weeks or even months.

BaaS works by having the database host the API as well - no additional service needed. This is done through an extension that maps tables, views, columns, and functions to a well-known API implementation format, such as REST (see PostgREST) or GraphQL (see pg_graphql).

Backwards Compatibility

The thing about successful software is that it grows, changes, and transforms. For the backend, that means you need to have a strategy for deploying breaking changes. In some environments, it may be possible to deploy the backend and its clients atomically, elegantly side-stepping the need for any sort of backwards compatibility in exchange for periodic downtime.

However, not all environments have this luxury, and one in particular - mobile app development - absolutely requires the backend to support backwards compatibility explicitly in order to maintain even a semblance of good user experience.

In my personal experience, the brunt of the work of backwards compatibility has been done above the persistence layer, in the application that sits on top. The database is the stone-cold truth, and everything conforms to it. It’s the application’s responsibility to handle version headers and funnel requests to past implementations.

Let’s say you’ve selected a BaaS for the next project. The database is the application. How can we implement backwards compatibility nonetheless?

Database Tools for Encapsulation

I learned about encapsulation early on, in my second computer science class at university. “Essentially, encapsulation prevents external code from being concerned with the internal workings of an object.”

This is precisely what backwards compatibility requires: a controlled, version-aware layer to read and write the actual, underlying data. In short, we need getters and setters at the database level. Fortunately, SQL databases come with a two particularly well-suited mechanisms:

How can we use these tools to design a database that can have breaking changes, without breaking its existing clients?

Backwards-Compatible Database Rules

With the following three rules, we can implement encapsulation, and therefore backwards compatibility, inside the database:

  1. Create two schemas, public and internal.
  2. internal holds the data for the application. This schema is never accessed by clients directly. There is no notion of version, and it is always up to date.
  3. public holds objects that clients may access directly. Use views as getters and functions as setters. Each view and function in this schema has a version suffix, ie. profiles_v1 and update_profile_v1().

When modifying internal tables and columns in a breaking way, update the affected views and functions to continue functioning. When the latest view or function is no longer sufficient to support all functionality, cut a new version.

In this way, old clients continue to function and updated clients can benefit from the latest features. Execute a parallel change to enable deletion of the older versions, if possible. Backwards compatibility is achieved.

Conclusion

Relational databases have always had the tools necessary to implement backwards compatibility, we just need to use them. With a few simple rules, it’s feasible to use a BaaS for your next project, knowing that it will be possible to expand the backend as you would with any normal backend application.