advanced · ~12 min
Not dead, just slow
Ticket from Priya Raman, senior backend developer
The 2.0 rollout has been stuck since yesterday. Its pods restart forever — Killing … failed liveness probe in the events — and 1.0 is still carrying all traffic.
2.0 isn't broken: it loads its fraud model at startup and takes about 50 seconds to listen. I don't want to lose the liveness probe; it caught a real deadlock last month. payments.yaml is the manifest.
Startup probes
A probe that runs first and alone. Until it succeeds, liveness and readiness probes don't run, so a slow starter isn't killed for being slow. It gets failureThreshold × periodSeconds to start; once it passes, the liveness probe takes over at its usual, tighter settings. Docs
Objectives
- payments: 2 ready replicas on 2.0
- Still guarded by a liveness probe
- A startup probe gives it time to start
- Bonus: Keep the liveness probe exactly as it was
Best on a computer with a keyboard. It works on a phone, with shortcuts under the terminal.