01
Failover that pins to the dead link
What you saw
Two internet connections, failover configured, dashboard shows both WANs.
What was happening
A policy route sent that traffic out of a specific WAN by name. Policy routing is evaluated before failover, so when that link died the traffic kept being handed to it. Failover was working; it was never consulted.
Check for routes that pin traffic before failover runs
# list policy routes and confirm none name a WAN directly
# then fail the link on purpose and count the failures
A rule that names a link will keep using it after it dies.
02
A tunnel that is up with nothing behind it
What you saw
The tunnel shows healthy. The connector is connected.
What was happening
Healthy means the connector reached the edge. It says nothing about whether the service it fronts is answering. The tunnel was perfectly up and proxying to a dead port.
Probe through the tunnel to the real service
# tunnel health is the connector, not the origin
curl -sS -o /dev/null -w '%{http_code}\n' https://<public-host>/healthz
A healthy tunnel can carry you to a dead service.
03
A long slot that is electrically short
What you saw
The card is seated in a full-length slot and the link comes up.
What was happening
The slot was physically long and electrically one lane. The ten-gig card negotiated and ran at a fraction of its rate. Nothing was broken, so nothing alerted — it was just slow forever.
Read the negotiated width, not the slot length
lspci -vv | grep -E 'LnkCap|LnkSta'
# compare LnkSta (negotiated) against LnkCap (capable)
A slot that fits is not a slot that is wired.
04
Measuring from inside the thing you are measuring
What you saw
Throughput looked fine from the test host.
What was happening
The test ran on a guest whose traffic never crossed the link in question. The result was true and irrelevant. A reading with no named vantage point is not a measurement.
Name the host before you trust the number
hostname; ip -br addr
# then say which path the test actually crossed
A lane is not a vantage point. Say where you measured from.
House rules: no real addresses, no real hostnames, no topology — every screen is
generic, scale figures only. No sponsors. Open-source tools get named; anything for-profit stays
generic. Every failure here happened in this rack.