perf: Prometheus recording rules for lz/bridge histogram queries

[Flotapponnier]

Summary

• Add recording rules to /opt/ocb/prom-rules/histogram_recording_rules.yml (already deployed on VPS, Prometheus reloaded)
• Replace raw rate(lz_message_latency_milliseconds_bucket[24h]) and rate(bridge_quote_latency_ms_bucket[24h]) in bench YAMLs with pre-computed ocb:*:rate24h recording rules
• Fixes materialize-worker "This operation was aborted" timeouts: expensive 24h histogram scans were spiking Prometheus CPU, causing even fast recording-rule lookups to queue past the 10s AbortController deadline

Root cause

layerzero-message-latency and bridge-quote-latency both used raw histogram_quantile(... rate([24h])) queries. These scan the full 24h TSDB block on every worker sweep (every ~5 min), brown-out Prometheus, and cause adjacent recording-rule queries (kusama, kava, juno RPC benches) to abort under the 10s timeout.

Fix

Recording rules pre-compute the expensive aggregations every 5 minutes. Query-time cost drops from O(24h scan) to O(1 TSDB lookup).

Test plan

• Verify Prometheus rule groups include lz_recording (4 rules) and bridge_quote_recording (6 rules)
• After 5 min, confirm ocb:lz_message_latency_ms_bucket:rate24h returns data
• Rebuild materialize-worker on VPS from dev branch, confirm no more "This operation was aborted" logs
• /benchmarks hub shows live data instead of n/a for layerzero and bridge-quote cards

🤖 Generated with Claude Code