The Query: From Life to Death

Have you ever wondered how to build a database? I don't mean CREATE DATABASE employee. I mean from scratch using C++ to implement a fully functional production-ready database? Don't you dare say "WhY nOt rUsT".

Not to worry! This site aims to demonstrate that, specifically relating to Postgres as much as possible. Sources used includes:

  • Andy Pavlo’s Database Systems lectures at CMU
  • Let’s Build a Simple Database – Writing a SQLite Clone from Scratch in C by Connor Stack
  • The Internals of PostgreSQL by Hironobu Suzuki
  • Architecture of a Database System by Joseph M. Hellerstein, Michael Stonebraker and James Hamilton
  • and many more...

So you have a query to grab some data from a Postgres table. You hover over the magical green arrow button and click execute. What happens?

Most likely, you’ll see an error similar to this:

Connection to Server Failed: Connection Refused

What do servers have to do with databases? It turns out databases are servers too — they just speak SQL instead of HTTP. This leads us into our first piece of architecture when following the life of a query: The Postmaster.

The Postmaster

Postmaster diagram

Figure 1.1: Starting the Postmaster

We connect to servers all the time. When we navigate to youtube.com, we connect to YouTube’s servers that hand us back videos. When we listen to music, we connect to Spotify’s servers that deliver audio streams. Postgres is no different — when we want to run a query, we must connect to the Postgres server. Internally, this main server process is called the Postmaster.

One common reason you might see the error above is that the Postmaster isn’t running yet. Here’s one way to start it:

postgres -D /usr/local/pgsql/data >logfile 2>&1 &

This command starts the PostgreSQL server — which includes the Postmaster (the parent process) and several background processes like the checkpointer and WAL writer. Once it’s up and running, we’re ready to connect!


Postmaster connection diagram

Figure 1.2: Connecting to the Postmaster

There are many ways to connect to Postgres once the Postmaster is running. We can connect locally, over the network using TCP, or through libraries like libpq. The simplest option is a local connection, which usually uses a Unix domain socket instead of TCP. All we need to run is:

psql -d my_database

Here, my_database is the database we want to connect to. After running that, you’ll be connected to Postgres and see something like this in the command line:

postgres=#

Behind the scenes, the Postmaster has forked a new backend process just for you — one per connection. This process will handle all your SQL commands until you disconnect.

Meanwhile, background processes (like the checkpointer or WAL writer) continue running alongside to perform system-level tasks such as checkpointing, writing to the transaction log, and maintaining table health.

This design — one process per client connection plus supporting background workers — is known as the process-per-DBMS-worker model. It’s one of PostgreSQL’s core architectural decisions and a key reason for its reliability and stability.

Another reason for Postgres using the Process model is a historical one. The Postgres project began all the way back in 1986. At that time Threads in the OS were not as mature as Processes. Here we can see that threads became more popular in the early 2000's and since then the ecosystem has matured greatly. This tell us something very important when trying to understand the architectural decisions behind systems, history matters!

I’ll go deeper into the process model in Database Architectural Decisions, but for now, remember this: the Postmaster is the parent of everything. Every query, every connection, and every background task begins with it.

The Query Processor

Query processor diagram

The Storage Manager

Storage manager diagram

The Background Systems

Background processes diagram