You're running PostgreSQL 15 or 16 on a Linux server, and suddenly it won't start. The log shows something like FATAL: WAL file corruption detected in segment 000000010000000000000001. This usually happens after a sudden power loss, a disk write error, or a filesystem crash. I've seen it on RHEL 9 and Ubuntu 22.04 servers with ext4 filesystems that didn't flush writes properly.
Why This Happens
PostgreSQL writes all changes to a Write-Ahead Log (WAL) before writing to the main data files. If a crash hits while PostgreSQL is writing to a WAL file, that file can end up half-written or garbled. The database can't trust it for crash recovery, so it stops cold.
Don't panic. You have options.
Fix Step by Step
Step 1: Find the Bad WAL File
Check your PostgreSQL log file. It tells you the exact segment number. Look for a line like:
FATAL: WAL file corruption detected in segment 000000010000000000000001
That 000000010000000000000001 is your bad WAL segment. Write it down.
Step 2: Stop PostgreSQL
If it's still running, stop it cleanly:
sudo systemctl stop postgresql-15
Or for older versions:
sudo service postgresql stop
Step 3: Check Your Backup Strategy
Before doing anything, ask yourself: Do I have a recent base backup plus all WAL archives? If yes, you can skip the next steps and restore from backup. That's the cleanest fix. If you don't have a backup, keep reading.
Step 4: Force the Corrupt WAL to Be Skipped (Danger Zone)
This is the nuclear option. Only do this if you're willing to lose the transactions in that WAL segment. PostgreSQL has a setting called ignore_wal_corruption in the postgresql.conf file. Set it to on.
ignore_wal_corruption = on
Then restart PostgreSQL. It will skip the bad segment and continue recovery. After the database starts, you need to check for data loss. Run VACUUM and ANALYZE on all tables to rebuild indexes and stats.
Warning: This can leave your database in an inconsistent state. Only use it if you have no other choice and you'll do a full dump and restore afterward.
Step 5: Workaround Using pg_resetwal (Better Option)
The tool pg_resetwal resets the WAL and forces PostgreSQL to ignore all existing WAL files. You lose any transactions not yet applied to the data files, but the database starts. Here's the command:
sudo -u postgres pg_resetwal -f /var/lib/postgresql/15/main
Replace the path with your data directory. PostgreSQL will start from scratch with a fresh WAL. After it's up, run VACUUM ANALYZE on every database to clean up.
Step 6: Restore from Backup (The Real Fix)
If you have a base backup and WAL archives, this is the right path. Shut down PostgreSQL, restore your base backup to the data directory, then restore all WAL archives into the pg_wal directory. Start PostgreSQL. It will replay all the archives and get you back to the point of failure minus maybe a few seconds.
sudo systemctl stop postgresql-15
sudo rm -rf /var/lib/postgresql/15/main
sudo pg_basebackup -h backup-server -D /var/lib/postgresql/15/main -U replicator -P
sudo -u postgres cp /backup/wal_archive/* /var/lib/postgresql/15/main/pg_wal/
sudo systemctl start postgresql-15
What to Check If It Still Fails
If PostgreSQL still won't start after these steps, you might have multiple corrupt WAL segments. Run pg_controldata to see the current WAL position:
sudo -u postgres pg_controldata /var/lib/postgresql/15/main
Look at the Latest checkpoint's REDO WAL file line. If that file is also corrupt, you're in deep trouble. You'll need to restore from a full backup or use pg_resetwal again with the -x flag to skip more segments.
Also check your disk health. Run smartctl -a /dev/sda (or your disk device) to see if the drive is failing. A dying disk can corrupt WAL files repeatedly. Replace it if you see reallocated sectors.
One more thing: if you're using replication, check that your standby servers don't have the same corruption. They might need the same fix.
Honestly, the best fix is prevention. Set wal_level = replica, enable archive_command to copy WAL files to a separate server, and run pg_basebackup weekly. This error sucks, but you can beat it.