Commit Graph
6 Commits
Author SHA1 Message Date
Craig Lawson 4fcb13694b Exclude degraded checks from the docker and hardware aggregates
Those aggregations were reverted when docker and hardware could not carry the
flag. Both types are now egress attributable, so their checks can be degraded
and their totals, bucket counts and latest-check reads must exclude them like
every other aggregate.

🔧
2026-09-17 22:06:34 +01:00
Craig Lawson d865103fb5 Exclude degraded-egress checks from every aggregate
A degraded episode now reads as a gap in monitoring rather than as downtime
or as a dip in response time. The exclusion is applied once per query, in the
top-level $match, so no pipeline can count a degraded check by omission.

- check.query.ts: EXCLUDE_DEGRADED_EGRESS_MATCH is the only helper;
  IS_NOT_DEGRADED_EGRESS_EXPR and the $cond accumulators it fed are gone.
- findUptimeDateRangeChecks: excludes at the match, so the response-time
  series, the up/down series and the averages drop degraded checks too.
- findPageSpeedDateRangeChecks: excludes as well; pagespeed is egress
  attributable and was previously missed.
- getDailyStatusBuckets: excludes at the match; the group shapes and the
  totalChecks > 0 guard revert to their original form.
- check.docker.aggregation.ts, check.hardware.aggregations.ts: reverted.
  Neither type is egress attributable, so their checks never carry the flag
  and the filters were dead code. A test now pins that decision.
- Tests: the gap behaviour, a window that was entirely degraded, pagespeed,
  the team listing and the resolve filter, and an incident that stays open
  when egress drops during it.

🔧
2026-09-17 22:06:16 +01:00
Craig Lawson 4335e6b7b6 Address review of degraded-egress check persistence
CheckProducer consults egress only for a transport failure: the check
failed, no HTTP status came back (isHttpStatusCode from PR 1), and the
monitor type is one whose failure can be a loss of egress
(EgressAttributableTypes; hardware and docker are excluded). A 4xx/5xx
or a content mismatch is evaluated as usual while the instance is
degraded. A rejected assessment is logged and treated as unknown so the
check is still buffered.

EXCLUDE_DEGRADED_EGRESS_MATCH moves out of the model into
check.query.ts, with an expression form for $group accumulators.
Daily status buckets and docker stats buckets exclude degraded checks
from their up/total counts while keeping them in the response-time
average, matching the uptime facet. toEntity maps egressStatus like its
siblings.

Adds heartbeatJobDegradedEgress integration tests, extends the
repository exclusion tests, and makes the harness egress stub settable.

🔧
2026-09-17 22:06:16 +01:00
Craig Lawson f52acb3f68 Flag checks taken during degraded egress and exclude them from uptime (#3937)
CheckProducer asks EgressService to assess the instance's own connectivity
after a failing check and stamps the result on the check as egressStatus.
CheckEvaluator short-circuits degraded checks so no incident opens and no
spurious recovery fires later. Uptime and down-count aggregations exclude
degraded checks; the response-time series keeps them.

🔧
2026-09-17 22:06:16 +01:00
Alex Holliday 8ccc7cbe9a new container endpoint,service, repo methods 2026-09-01 10:05:34 -07:00
Alex Holliday 1a1dc7d807 add docker check aggregation 2026-08-28 13:30:32 -07:00