Clear the status window when a degraded spell starts

The window was carried across a spell, so the results either side of it ended up
adjacent: a monitor at [t, t, t, f, f] when egress dropped resumed at
[t, t, f, f, f] on its first real failure afterwards, crossing the threshold on
results taken before the outage.

Cleared in the producer beside the maintenance gate, which already clears the
window on entering a window, and guarded on its length so it is one write per
spell rather than one per degraded check. The monitor still goes down once the
rebuilt window holds enough real failures.

The two integration tests that asserted the old behaviour are rewritten: one
expected the window preserved through a spell, the other expected failures
either side of a spell to combine.

🔧
This commit is contained in:
Craig Lawson committed 2026-09-19 14:43:16 +01:00
1 parent b55560e2f1
commit 1b3b8e5312
3 files changed
+63 -7

No files matched your search

+10 -1
View File
@@ -133,10 +133,19 @@ export class CheckProducer implements ICheckProducer {
if (egressStatus !== null) {
check.egressStatus = egressStatus;
}
// Step 2c: Add to buffer
this.bufferService.addToBuffer(check);
// Step 2d: Handle docker logs
// Step 2d: As with the maintenance gate above, the window is cleared at the point the monitor stops
// being evaluated, so the results either side of a spell never end up adjacent and cross the threshold
// on stale data. The length guard keeps this to one write per spell rather than one per degraded check,
// and it runs after the check is buffered so that a failed write cannot cost us the check.
if (egressStatus === "degraded" && monitor.statusWindow?.length) {
await this.monitorsRepository.updateById(monitor.id, monitor.teamId, { statusWindow: [] });
}
// Step 2e: Handle docker logs
if (status.type === "docker") {
const dockerLogs = await this.dockerLogsService.buildDockerLogs(status as MonitorStatusResponse<DockerStatusPayload>);
for (const dockerLog of dockerLogs) {