🖥️

Real incidents need a real screen.

Open senioreng.dev on your laptop for the full experience.

Live·00:00elapsed

Incident Workspace

Service Overview

Environment

Production

inventory-service

No events · 18 min

847

msg lag

notification-service

No events · 18 min

847

msg lag

fulfillment-service

No events · 18 min

847

msg lag

loyalty-service

No events · 18 min

847

msg lag

analytics-pipeline

No events · 18 min

847

msg lag

reporting-service

No events · 18 min

847

msg lag

audit-log-service

No events · 18 min

847

msg lag

warehouse-service

No events · 18 min

847

msg lag

Service logs — last 18 minutes

[WARN] inventory-service: no order events received in 18m — inventory may be stale

[WARN] notification-service: event queue idle — 14,000+ notifications pending

[WARN] fulfillment-service: order assignments paused — no routing events

[WARN] loyalty-service: point awards delayed — consumer group lag: 847

[INFO] All services healthy otherwise — API endpoints responding normally

P1 · Production Incident

Consumer Running. Events Dead.

PagerDuty · 10:29 UTC

ALERT: order-events-consumer-group lag critical. 8 downstream services reporting no events received. Consumer restart count: 94.

8 services — inventory, fulfillment, notifications, loyalty, analytics, reporting, audit, and warehouse — have all stopped receiving order events. Consumer lag is at 847 messages and growing.

The consumer service shows status: running. Restart count: 94. No recent deployments on any of the affected downstream services.

You are the on-call engineer. Investigate the available telemetry, identify the root cause, and restore event processing.