When I started in web development, the architecture of an application always radiated out from the database. Any application was firmly rooted by its data schema and the first step was sketching out the tables and relationships that would define how data was organized and retrieved.
But that’s where the web was, not where it’s headed. Today, I’m struck by how little developers need to think about the database at all.
Databases are still very much at the heart of the modern web, just as servers still dutifully power the expanding array of serverless offerings. But it’s today possible—and common—to author and deploy rich, interactive web applications without managing database infrastructure or even knowing how the data is ultimately stored.
RELATED CONTENT: Jamstack brings front-end development back into focus
It’s a shift that’s been in the making: developing directly against the database became less common with the rise of web frameworks like Rails and the ORM. Even with these abstractions, it was still advisable to understand the schema of tables beneath so you could drop down to SQL to optimize critical queries as you tweaked performance. To know your app, you had to know your database.
Breaking the monolith: API services impact the data layer
The mighty, monolithic database and its role as an oracle—as the singular source of truth—is being challenged on two fronts.
The first challenge comes from inside each company as more development teams adopt microservice architectures, structuring each service to focus on a single domain complete with its own datastore. While companies sometimes try building a microservice stack on top of a monolithic database, this tends to be the worst of both worlds. Individual teams still need to own the data layer.
The second challenge to the central database actually comes from outside the company: the explosion of API services — the economy of developer-centric offerings powering everything (payment, search, content authoring, artificial intelligence). The productivity lift from instant access to APIs and services makes the benefits of decoupling an application too large to ignore.
This requires a shift in thinking from “my database” to “my data,” trading direct access to each table for indirect access via APIs. It’s the right tradeoff, even if data spreads across third-party services. Monolithic databases were straining under the weight of additional requirements as applications grew.
A service like Stripe bundles compute and data behind a payment API, managing, scaling, securing and optimizing databases for payment processing in ways that would be out of reach to all but the largest web teams. New “content APIs”—hosted services like Contentful and Sanity—allow entire teams to author all types of content without ever managing a CMS or the database behind it.
Prioritizing Performance: Making API-centric apps faster with the Jamstack
Querying out to a remote API can be slower than requesting the same data from a database local to the application. However, developers have tackled this problem head-on with a new performance-oriented architecture for the web called the Jamstack.
For years, the web worked like this: as each incoming web request arrives at the server, a response is built by calling application software that fetches the required data with calls out to the database and/or API services. Once the right data is fetched, the html page is assembled and returned. This process is repeated for each and every request.
Jamstack works differently. Instead of pulling the data at request time, Jamstack reverses the flow. The majority of compute happens ahead of the request cycle. API services use webhooks to notify an automated build process to re-render html pages whenever the data they are storing changes. Prebuilt pages are published out to the edge of the network as close to users as possible. At request time, there’s little left to do but serve the final html.
This “precompute” rendering happens prior to the actual web request made by a visitor. A complex e-commerce page with multiple data sources from multiple APIs can be pre-assembled and still served in milliseconds, with a faster time to first byte than a traditional server. The response can be personalized for each user via Javascript, which may run inside of an edge node on a modern CDN or inside the browser on the user’s device. Developers optimize applications by reaching out for the right data from the right APIs during build time for what can be precomputed or during request time for what is custom to the user.
Querying across APIs
New API services continue to accelerate new applications being developed. New architectures like the Jamstack are accelerating web applications towards the same level of performance once only enjoyed by native applications.
As we make use of an increasing number of services, we’ll need to guard against fragmentation. It’s now common to have user and authentication data housed in one service, content in another service, and subscriptions in another service, all using different providers. While the variety of services is empowering, underlying vendors will need to be better managed and easier to develop against.
How do we create a unified way of thinking about it? Thankfully, a host of new answers are emerging. In the Jamstack ecosystem, TakeShape, OneGraph, Apollo GraphQL, and Prisma are working on unifying the new generation of data layers.
Takeaways for web application architects
With all the new services available for instant API consumption, my advice to modern web developer teams:
- Embrace the move away from the monolith. Modern apps will be easier to deploy, maintain, and scale than the old monoliths.
- Start by decoupling the frontend of your application from the backend and use APIs to talk to your own internal services as well as external ones.
- Performance is everything on the web. Test your application under throttled bandwidth conditions to better approximate real-world usage.
- Have your architecture do the hard work in advance. Remove as much as possible from the request path by pulling data during the build process so that you can prerender pages into high-performance assets. Services like Netlify help make the Jamstack architecture easy to adopt.
- Orient your thinking away from the centralized database towards the distributed data layer.
The new mindset about “my data” can help you change the way you build systems. Our answer to that is the Jamstack.
