AWS/MediaLive ActiveAlerts correlated against CloudFront cache-miss latency.
How the +Amazon CloudWatch integration pulls AWS-side metrics and logs as evidence during incident investigation, joins them against the streaming-side CDN signals on the same bus, and emits the remediation back as a CloudWatch metric so the same dashboard panel that flagged the ActiveAlerts drift is the panel that re-renders the loop-closure window.
This is a worked-incident editorial replay. Not a real outage.
All AWS-side metric + log pulls, the Streamwake-direction trace, and the cross-layer recovery checklist below are reconstructed for illustration. No real customer data appears in the fixtures; the AWS-side metric names + log group + IAM role posture are documented public examples from the AWS MediaLive + Amazon CloudFront pair the +AWS MediaLive + +CloudFront pages already name. No claim is made that the remediation this page illustrates would have been the one an operator ran in production.
The CloudWatch evidence pull.
The CloudwatchCorrelationsPanel on /app/incidents/[id] POSTs /api/v1/cloudwatch-evidence/collect with the active incidentId. The route signs one GetMetricData call against monitoring.<region>.amazonaws.com AND one FilterLogEvents call against logs.<region>.amazonaws.com using a handwritten SigV4 signer + sts:AssumeRole if the source row carries a roleArn.
POST /api/v1/cloudwatch-evidence/collect HTTP/2
host: api.streamwake.example
content-type: application/json
x-incident-id: ckq3xrenamexz-p95jadi-cmh
----- POST body -----
{ "incidentId": "ckq3xrenamexz-p95jadi-cmh" }
----- CloudWatch metrics pull + AWS signing (reconstructed) -----
POST monitoring.us-east-2.amazonaws.com/ HTTP/2
authorization: AWS4-HMAC-SHA256 Credential=AKIA***/20260821/us-east-2/monitoring/aws4_request, SignedHeaders=...; Signature=...
host: monitoring.us-east-2.amazonaws.com
x-amz-content-sha256: <sha256 of body>
x-amz-date: 20260821T153400Z
content-type: application/x-amz-json-1.0
{
"StartTime": "<from, 5 minutes before now>",
"EndTime": "<to, now>",
"MetricDataQueries": [
{
"Id": "m1",
"MetricStat": {
"Metric": {
"Namespace": "AWS/MediaLive",
"MetricName": "ActiveAlerts",
"Dimensions": [
{ "Name": "ChannelId", "Value": "primary-na" }
]
},
"Period": 60,
"Stat": "Average"
},
"ReturnData": true
}
]
}
----- CloudWatch Logs pull (reconstructed) -----
GET logs.us-east-2.amazonaws.com/?logGroupName=%2Faws%2Fmedialive%2Fchannel-events&startTime=...&endTime=...&limit=100&filterPattern=%7B+%24.eventType%3D%22ChannelStateChange%22+%7D HTTP/2
authorization: AWS4-HMAC-SHA256 Credential=AKIA***/20260821/us-east-2/logs/aws4_request, SignedHeaders=...; Signature=...
host: logs.us-east-2.amazonaws.com
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date: 20260821T153400Z
----- response (reconstructed) -----
{
"MetricDataResults": [
{
"Id": "m1",
"Label": "AWS/MediaLive ActiveAlerts — primary-na",
"Timestamps": ["…", "…", "…", "2026-08-21T15:32:00Z"],
"Values": [0, 0, 0, 14],
"StatusCode": "Complete"
}
]
}
{
"events": [
{
"logStreamName": "primary-na/2026/08/21/15",
"timestamp": 1755787900000,
"message": "EVENT ChannelStateChange ChannelId=primary-na State=DEGRADED PreviousState=RUNNING",
"ingestionTime": 1755787903000
}
],
"searchedLogStreams": [
{ "logStreamName": "primary-na/2026/08/21/15", "searchedCompletely": true }
]
}The Streamwake-direction trace.
The agent joins the captured CloudWatch evidence against the CDN/encoder/DRM signals on the same bus, ranks root cause with a typed confidence score, and tags the remediation actions on the same incident timeline. Every action is tagged direction: 'reconstructed' so a reader cannot mistake this trace for an actual Streamwake observation.
{
"incident_id": "ckq3xrenamexz-p95jadi-cmh",
"direction": "reconstructed",
"checked_at": "2026-08-21T15:34:00Z",
"context": "Editorial reconstruction of an AWS ActiveAlerts + CloudFront cache-miss latency fan-out to demonstrate the +Amazon CloudWatch evidence pull + bi-directional emit. No actual outage; figures reconstructed from documented AWS MediaLive + Amazon CloudFront behavior for illustration.",
"ranked_hypotheses": [
{
"rank": 1,
"slug": "aws_medialive_active_alerts_vs_origin_classification",
"confidence": 0.91,
"evidence_signals": [
"aws.medialive.ActiveAlerts = 14 vs 0 baseline, on primary-na channel",
"aws.medialive.ChannelStateChange events captured by CloudWatch Logs FilterLogEvents on /aws/medialive/channel-events (DEGRADED after RUNNING)",
"cloudfront.cache_miss_latency_p95_ms = 2,184 ms vs 812 ms baseline on iad-cmh regional edge PoPs (paired, not collapsed — same window)",
"streamwake.cloudwatch.evidence_datapoints += 1 metric + N log rows per the collect response"
],
"classification_cross_layer": {
"origin_side": "aws_medialive_active_alerts_with_encoder_profile_drift_remediation",
"edge_side": "cloudfront_cache_miss_latency_fan_out_vs_origin_shield_miss_remediation"
},
"recommended_fix": [
"retune_aws_medialive_encoder_profile_for_primary_na",
"reroute_cloudfront_egress_off_iad_cmh_affected_PoPs",
"repin_origin_shield_tier_assignment",
"rewrite_cloudwatch_logs_filter_pattern_to_routing_contract_revision_next"
]
},
{
"rank": 2,
"slug": "cloudwatch_logs_filter_skew",
"confidence": 0.07,
"evidence_signals": [
"channel-state events matched by { $.eventType = "ChannelStateChange" } — narrow but unfiltered by ChannelId",
"existing filter does not yet include the next routing-contract revision"
]
},
{
"rank": 3,
"slug": "metric_vs_classification_skew",
"confidence": 0.02,
"evidence_signals": [
"ActiveAlerts metric vs source rows in CloudwatchEvidence show consistent burned-vs-baseline delta across 4 datapoints"
]
}
],
"evidence_pulled": [
{
"kind": "metric",
"namespace": "AWS/MediaLive",
"metric_name": "ActiveAlerts",
"datapoints": 4,
"max": 14,
"captured_at": "2026-08-21T15:34:00Z"
},
{
"kind": "log",
"log_group": "/aws/medialive/channel-events",
"filter_pattern": "{ $.eventType = "ChannelStateChange" }",
"events": 1,
"captured_at": "2026-08-21T15:34:00Z"
}
],
"governed_actions": [
{
"id": "evt-2026-08-21-retune-medialive-profile",
"type": "retune_aws_medialive_encoder_profile",
"scope": "channel=primary-na",
"verifies": ["aws.medialive.ActiveAlerts trend down within window", "no regression on AWS/MediaLive encoder_health metric"],
"approval_required": false
},
{
"id": "evt-2026-08-21-reroute-cloudfront",
"type": "reroute_cloudfront_egress_off_brownout_PoP",
"scope": "deployment=primary-na",
"verifies": ["cloudfront.cache_miss_latency_p95_ms trend down on affected PoPs"],
"approval_required": false
},
{
"id": "evt-2026-08-21-repin-origin-shield",
"type": "repin_cloudfront_origin_shield_tier_assignment",
"scope": "distribution=primary-na",
"verifies": ["aws.cloudfront.origin_shield_miss counter cleared on the affected regional tier"],
"approval_required": false
},
{
"id": "evt-2026-08-21-emit-cloudwatch-metric",
"type": "publish_metric_to_cloudwatch",
"scope": "namespace=Streamwake/Remediation",
"metric_payload": {
"MetricName": "streamwake.<account>.cloudwatch.remediation.applied",
"Value": 1,
"Unit": "Count",
"Dimensions": [
{ "Name": "IncidentId", "Value": "ckq3xrenamexz-p95jadi-cmh" },
{ "Name": "Classification", "Value": "aws_medialive_active_alerts_vs_origin_classification" },
{ "Name": "Confidence", "Value": "0.91" }
]
},
"verifies": ["Dashboard panel that flagged the ActiveAlerts drift re-renders the loop-closure metric the same window"],
"approval_required": false
}
]
}Why this lands cleanly on the CloudWatch side.
CloudWatch is the AWS-side metrics + logs bus the rest of the AWS stack already writes into — AWS/MediaLive, AWS/MediaPackage, AWS/CloudFront, AWS/Lambda, plus the CloudWatch Logs log groups an AWS account surfaces as first-class signals. The +Amazon CloudWatch integration reads that bus via SigV4 + sts:AssumeRole, layers the streaming-domain classification the CloudwatchCorrelationsPanel renders under the ranked hypothesis list, and emits the remediation back as streamwake.<account>.cloudwatch.remediation.applied on the same account so the same CloudWatch Dashboard panel the operator already renders re-renders the loop-closure the same window the incident closes.
The cross-layer recovery checklist.
What to verify on the AWS side AND the streaming side before closeout, in the order the agent loop runs them.
- 1.
AWS/MediaLive ActiveAlerts → 0 across all channel cohorts; encoder_health metric back inside its baseline band.
- 2.
Amazon CloudFront CacheHitRate back inside its per-PoP baseline; p95 OriginLatency cleared on iad-cmh affected PoPs.
- 3.
Origin-shield miss counter cleared on the affected regional tier (no fallback-to-origin direct path).
- 4.
CloudWatch Logs /aws/medialive/channel-events continues to report DEGRADED → RUNNING transitions on the next window without regressing.
- 5.
streamwake.<account>.cloudwatch.remediation.applied = 1 emitted on the same CloudWatch namespace, with Dimensions { IncidentId, Classification, Confidence } the Dashboard panel renders.
- 6.
Players in the affected region back to < 0.5% rebuffer rate across a 5-minute window. POSTMORTEM posted; PRs routed to the encoder-profile + origin-shield teams.
See the CloudWatch evidence pull live. On your AWS account.
Book a 20-minute walkthrough on your AWS MediaLive + Amazon CloudFront distribution — we'll show how the +Amazon CloudWatch integration pulls the AWS/MediaLive ActiveAlerts drift + CloudWatch Logs FilterLogEvents as evidence, layers the streaming-domain classification the CloudwatchCorrelationsPanel renders under the ranked hypothesis list, and emits the loop-closure back as a CloudWatch metric the same Dashboard panel renders the next window.