Understanding WAL Internals in PostgreSQL

Introduction

If you have worked with PostgreSQL for any serious workload, you have heard the term WAL many times. Write-Ahead Logging is often explained in a single line: “PostgreSQL writes changes to WAL before writing data pages.” While that statement is technically correct, it omits several important details that are crucial when tuning performance, debugging crashes, or designing highly available systems.

This blog goes deeper into WAL internals. We will examine why WAL exists, how it operates internally, and what actually happens during commits, checkpoints, and crash recovery.

Why PostgreSQL Needs WAL?

Databases face a fundamental problem. Writing data to disk is slow, and crashes can happen at any moment. If PostgreSQL were to update data files on disk for every change directly, it would be both slow and unsafe. A crash halfway through a write could leave the database in an inconsistent state.

WAL solves this by separating durability from data file updates.

The core idea is simple:
Before PostgreSQL changes any data page on disk, it first writes a description of that change to a sequential log file. This log is append-only, fast to write, and easy to replay.

If PostgreSQL crashes, it can read this log and reapply any changes that were committed but not yet written to the main data files.

The WAL Architecture at a High Level

Internally, PostgreSQL stores WAL records in files located in the pg_wal directory.

Some key concepts:

• WAL records: Logical descriptions of changes.
• WAL segments: Fixed-size files, usually 16 MB each.
• LSN (Log Sequence Number): A pointer that identifies a specific position in the WAL stream.

Every WAL record is assigned an LSN. LSNs are monotonic and increase as new records are written. PostgreSQL uses LSNs everywhere to track progress, consistency, and replication state.

What Exactly Is Written to WAL?

PostgreSQL typically does not write full data rows to the WAL.

Instead, it writes physical change descriptions such as:

• Insert tuple X at offset Y on page Z
• Update the tuple at offset A to offset B
• Mark transaction ID as committed

These records are compact and efficient. However, there are cases where PostgreSQL writes a full page image. This typically happens when:

• A page is modified for the first time after a checkpoint
• The full_page_writes setting is enabled

Full page writes protect against partial page writes caused by crashes or power failures. They increase WAL size but significantly improve safety.

Commit Processing and WAL

When you issue a COMMIT, PostgreSQL performs several important steps:

1. All required WAL records for the transaction are generated.
2. A commit record is written to WAL.
3. PostgreSQL ensures the WAL up to the commit LSN is flushed to disk.
4. The client is informed that the transaction has been committed.

This is why WAL is central to PostgreSQL’s ACID guarantees. The data pages themselves may still be dirty in memory and not written to disk yet, but the commit is safe because the WAL can always be replayed.

Checkpoints and Their Role

If PostgreSQL relied solely on WAL replay, recovery would become increasingly slower as the WAL grows. This is where checkpoints come in.

A checkpoint is a point where PostgreSQL ensures that:

• All dirty data pages up to a certain LSN are written to disk
• A checkpoint record is written to WAL

After a checkpoint, PostgreSQL knows that it does not need to replay WAL records older than that checkpoint during crash recovery.

Checkpoints reduce recovery time but are expensive because they cause many disk writes. This is why checkpoint tuning is important for write-heavy systems.

Key parameters that influence checkpoints include:

• checkpoint_timeout
• checkpoint_completion_target
• max_wal_size

Crash Recovery Using WAL

When PostgreSQL starts after a crash, it enters crash recovery mode.

The process is roughly:

1. PostgreSQL finds the last valid checkpoint.
2. Data files are restored to the state they were in at that checkpoint.
3. WAL records after the checkpoint are replayed sequentially.
4. Only committed transactions are applied.
5. Uncommitted changes are ignored.

Because WAL records describe physical changes, recovery is deterministic and reliable. This is one of the reasons PostgreSQL is trusted for critical systems.

WAL and Replication

WAL is also the foundation of PostgreSQL replication.

In physical replication:

• The primary server streams WAL records to replicas.
• Replicas replay WAL to stay in sync with the primary.

Replication slots use LSNs to ensure that the WAL needed by replicas is not deleted too early. This makes WAL retention management crucial. If a replica stops consuming WAL, the primary’s pg_wal directory can grow rapidly.

Understanding WAL internals helps avoid common production issues, such as disk space exhaustion due to stalled replication.

Performance Implications of WAL

WAL is both a safety net and a performance factor.

Some important trade-offs:

• More WAL means more disk I/O.
• Turning off durability settings can improve performance, but it also increases the risk.
• Frequent checkpoints reduce recovery time but increase write pressure.
• Full-page writes increase the WAL size but protect against corruption.

Tuning WAL is about finding the right balance between performance, durability, and recovery guarantees for your workload.

Conclusion

Write-Ahead Logging is the backbone of PostgreSQL’s reliability. It enables PostgreSQL to be fast, crash-safe, and highly available simultaneously. While the concept is simple, the internal mechanics are carefully engineered and deeply integrated into almost every part of the system.

If you work with PostgreSQL in production, learning the internals of WAL is not optional. It is one of the most valuable investments you can make in understanding how your database really works.

About CloudThat