← All topics

Topic 02

Backups

Every backup system reports success. The question is what it checked before it said so — and whether anyone has ever put the result back.

Backups

Failure patterns

Four ways this goes wrong quietly.

Each one happened. Each one showed green while it was happening.

01

Verify skips what it already blessed

What you saw

The verify job completes. Status OK. Every snapshot shows as verified.

What was happening

--ignore-verified defaults to true. A plain verify quietly skips every snapshot already marked verified, so it re-read almost nothing. The green came from a flag, not from the data.

Force it to re-read the chunks

proxmox-backup-manager verify <datastore> --ignore-verified false

# and schedule it, so it is not a thing you remember to do
proxmox-backup-manager verify-job create nightly \
  --store <datastore> --schedule daily

A default verify re-reads nothing. Only forcing it reads the chunks.

02

The index header is not the data

What you saw

The archive lists. The snapshot browses. File names and sizes all look right.

What was happening

Listing an archive reads its index, not its chunks. The index was intact and some of the data behind it was not. Nothing that only reads metadata can tell you this.

Read the chunks, then actually restore one

# a listing proves the index exists
# only a verify reads the chunk store
# only a restore proves the result boots

A backup you have never restored is a rumour, not a backup.

03

The backup that froze the guest

What you saw

The backup job is running. The guest pings. Monitoring is green.

What was happening

The snapshot froze the guest's filesystem and the thaw never came back. The guest answered ICMP for hours while every write blocked. Ping proves the kernel is up, nothing more.

Ask the filesystem, not the network

# is this guest frozen right now?
qm guest cmd <vmid> fsfreeze-status

# thaw it
qm guest cmd <vmid> fsfreeze-thaw

A guest that answers ping can still be frozen solid.

04

Coverage you assumed, not measured

What you saw

The job runs nightly and the backup set grows every day.

What was happening

A path had been renamed months earlier. The job kept succeeding because it was backing up what it could still find, and nobody compared the archive's contents to what was supposed to be in it.

Prove scope from the archive, not from the config

# list what is actually inside, and diff it against what should be
proxmox-backup-client list
proxmox-backup-client catalog dump <snapshot>

A job reporting success for finding nothing still reports success.

House rules: no real addresses, no real hostnames, no topology — every screen is generic, scale figures only. No sponsors. Open-source tools get named; anything for-profit stays generic. Every failure here happened in this rack.

← Monitoring Virtualization → All topics

Not a wrong answer.A check that cannot be wrong.

That is the whole channel.