[OisinKyne]

Summary

Follow-ups from the rc1 runs on the dev box (Hoodi and mainnet Silver).

• Silver startup and liveness probes are now HTTP. Both call GET /eth/v1/node/version, matching fix(silver): HTTP probes, info logs, amd64 node selector; bump to alpha.12 ethpandaops/ethereum-helm-charts#505. Until then they were tcpSocket probes, which pass as long as the port accepts connections.
  • Why: Silver sometimes stalls at boot until its first engine-API exchange with the EL completes. While stalled, its API accepts connections but never answers, so the TCP probes never restarted it. The host saw stalls of 2.5 min, 23 min and indefinite.
  • Startup probe: fails after 180 × 10 s, so a stalled boot is restarted after 30 min.
  • Liveness probe: fails after 3 × 120 s, so an API that stops answering is restarted after about 6 min.
  • Readiness is unchanged (/eth/v1/node/health, from fix(network): Silver logs at info and is ready only when its beacon API answers #901).
  • These overrides can go once ethereum-node ships chore(frontend): bump obol-stack-front-end v0.1.24 → v0.1.25-rc1 (digest-pinned) #505.
• Silver v0.1.0-alpha.13 ("clustering fixes", clustering fixes gattaca-com/silver#470). The version fix, Report the crate version in the agent version gattaca-com/silver#473, merged after alpha.13 was cut, so it lands in the next alpha.
• The arch check asks the cluster.
  • network install and network sync used to compare Silver's amd64-only image against the CLI's runtime.GOARCH.
  • They now read the nodes' kubernetes.io/arch label, with a 5 s request timeout. If the cluster doesn't answer, they fall back to runtime.GOARCH.
  • With mixed nodes, a non-amd64 node fails the check, since nothing pins the beacon pod to an amd64 node.
  • Only amd64-only clients trigger the lookup, so other installs make no extra cluster call.
• network sync warns when it will also restart the EL.
  • Deployments made before obol set per-network memory requests run their EL StatefulSet with none. The next sync adds them, which restarts the EL even when the user only meant to change the CL.
  • The host hit this on mainnet: reth restarted during the lighthouse → Silver switch.
  • Sync now reads the live ethereum-execution StatefulSet. If it has no memory request (and the node isn't stopped), sync prints: This sync also restarts reth (the execution client) once: ….