Commit Graph
523 Commits
Author SHA1 Message Date
Alexander Holliday d7f8bcc9e1 Merge pull request #3973 from fabiangigler/feat/capture-docker-monitors
Support Capture-backed Docker monitors
2026-09-21 10:05:51 -07:00
Alexander Holliday d51fdbd772 Merge pull request #3978 from bluewave-labs/chore/readability-refactor
chore: refactor worker services into a pipeline for readability
2026-09-21 09:38:56 -07:00
Alexander Holliday 199d68e8a2 Merge pull request #3974 from nikhilsaxena04/fix/grpc-monitor-proto-path
fix(network): correctly resolve health.proto path for gRPC monitors and include in build
2026-09-21 09:17:36 -07:00
Alex Holliday 8dd57d45e2 port egress to pipeline 2026-09-20 17:37:42 -07:00
fabiangigler 2f3de9797d Use network types for Capture responses 2026-09-19 22:51:23 +02:00
Craig Lawson b55560e2f1 Report whether the peer answered from every provider that can tell
An egress spell masked a service on the surviving network going down: with
`peerResponded` set only by the DNS and gRPC providers, every other failure fell
back to the HTTP-status heuristic, which answers "nothing replied" for anything
that is not speaking HTTP. A refused connection on the local network was read as
the instance's own loss of egress, so the check was recorded as degraded and the
monitor stayed up.

`peerAnsweredFromError` holds the errno codes that require the target's own stack
to have replied — a refusal, a reset, a TLS handshake that got as far as a
certificate. Http, Port, WebSocket and Docker use it; Http also treats any status
code as an answer, and Docker any status from the daemon. Hardware and PageSpeed
go through Http and inherit it.

`true` is the claim that costs something, since it blames the target, so it is
only set on evidence. Two cases stay conservative and are commented as such: a
ping that draws no echo reply, and a gamedig query that fails, neither of which
can be told apart from the instance losing its egress. Nor can a proxied HTTP
request that fails with no response, where the refusal may be the proxy's.

🔧
2026-09-19 14:43:11 +01:00
nikhil2004 57ddbc474f fix(network): correctly resolve health.proto path for gRPC monitors and include in build
Fixes #3970 by pointing protoPath to the correct relative directory (../protos) and adding a step in the package.json build script to copy the protos directory to the final dist output.
2026-09-19 01:03:42 +05:30
fabiangigler 862ff15eef Support Capture-backed Docker monitors 2026-09-18 19:54:33 +02:00
Craig Lawson 3dc529733a Address the inline review on the egress check pipeline
Distinguish "the peer answered" from "nothing answered", widen attribution to
every check that leaves the instance, make the maintenance exit explicit, and
fix the recovery poll cadence.

Providers can now say outright whether the peer answered, via peerResponded on
MonitorStatusResponse. Previously the producer inferred it from an HTTP status
code, which two providers do not report:

- GrpcProvider: NOT_SERVING shares NETWORK_ERROR with a connection failure, and
  gRPC codes 0-16 are not HTTP statuses, so PERMISSION_DENIED, UNIMPLEMENTED
  and the rest were all read as transport failures. Only DEADLINE_EXCEEDED and
  UNAVAILABLE, or a failure carrying no gRPC code, now count as unreachable.
- DNSProvider: every resolver error became NETWORK_ERROR, so an authoritative
  NXDOMAIN, SERVFAIL or REFUSED was read as unreachable. The c-ares codes where
  the server replied are now separated from timeout and connection refused.

Left unset, the HTTP-status heuristic still applies, so other providers are
unchanged.

Attribution now follows whether the check leaves the instance, not the monitor
type. A hardware check is an ordinary outbound HTTP request to the Capture
agent, and a Docker daemon can be reached over TCP, so both are attributable;
only a Docker daemon on a unix socket is local and exempt.

Leaving a maintenance window is now explicit in the producer, alongside the
entry it already owned. It was a side effect of evaluating a check, which the
degraded-egress short-circuit skips, so a monitor whose window ended during an
outage kept reporting "maintenance" for the length of it.

The recovery poll interval is fixed at 5 seconds and no longer configurable.
While degraded, checks are flagged from stored state without probing, so the
interval is the window in which a recovered instance still discards its checks;
the only effect of raising it was a longer blind window. egressPollIntervalSeconds
is removed from the settings, the validator and the OpenAPI spec.

🔧
2026-09-17 22:06:34 +01:00
Alex Holliday f299098548 bug fixes 2026-09-17 12:30:56 -07:00
Alex Holliday fa0c0abcac Fix: Use Check IDs instead of timestamp to process checks 2026-09-17 11:57:19 -07:00
Alex Holliday 50d0721822 cache DB settings, used cached db settings in ProxyResolver, PageSpeedProvider, and EgressService 2026-09-16 11:37:29 -07:00
Craig Lawson b68367b04f Harden the egress recovery job after a second review pass (#3937)
- Schedule the recovery job before recording the degraded transition,
  so a crash between the two leaves a stray row (removed on the job's
  first run) rather than a degraded state nothing is polling
- Count any HTTP response, including 4xx/5xx, as reachable for http
  targets; only a transport failure counts against egress
- Cascade a deleted notification out of egressNotifications with an
  atomic $pull in the settings repository, after the team-scoped delete
- Handle two processes racing the first egress state insert
- Address global job rows by one key (deleteGlobalJob on _id) and
  restore the deleteByIdAndType signature
- Inject net into PingProvider like PortProvider
- Integration tests for MongoEgressStateRepository and the global-row
  delete methods on the jobs repository

🔧
2026-09-15 08:59:13 +01:00
Craig Lawson 08f104e283 Address review on the egress recovery job (#3937)
- The egress job runs through WorkerHelper.getEgressRecoveryJob(), like
  the cleanup jobs, instead of a direct service call from the queue
- Toggling egressCheckEnabled resets the egress state and removes any
  pending recovery job
- Probes resolve and pass the proxy, so http targets honour the global
  proxy as monitors do
- PingProvider keeps IPv6 addresses intact instead of truncating at the
  first colon
- Egress targets are parsed by one function shared by the validator and
  the probe; malformed addresses such as unbalanced brackets are rejected
- Notification id pattern drops the /i flag, which produced an invalid
  OpenAPI pattern
- EgressState findSingleton reads before it writes, so a read no longer
  bumps updatedAt
- "disabled" is a distinct assessment result and is cached like any other
- Concurrent failures share an in-flight assessment for as long as it
  runs, then reuse the settled result for 5 s
- The recovery job row is only removed if nobody rescheduled it since
  this run claimed it, so a concurrent new episode keeps its job
- Deleting a notification removes it from egressNotifications

🔧
2026-09-15 08:42:09 +01:00
Alex Holliday ba0d685c6d commetns 2026-09-11 12:07:45 -07:00
Alex Holliday 912f1f960d add missing timeout 2026-09-11 11:50:26 -07:00
Alex Holliday b14b2142e3 add secrets rotation service and tests 2026-09-11 10:09:30 -07:00
Alex Holliday 5eb16fdbdb add validation, helpers, tests 2026-09-10 12:23:20 -07:00
Alex Holliday f9bc02ed1c update msgs 2026-09-10 11:58:12 -07:00
Alex Holliday 9c788561a7 update docker provider to add a TLS path 2026-09-10 11:49:53 -07:00
Alex Holliday f009aa17d9 Add encryption service and tests. Update README 2026-09-10 10:31:41 -07:00
Alex Holliday 71fed33ecd purge ssh 2026-09-04 14:33:13 -07:00
Alex Holliday 431cada08e add opt in docker logs flag 2026-09-04 14:06:46 -07:00
Alex Holliday 6587ddf389 wire up services/repositories 2026-09-03 11:32:48 -07:00
Alex Holliday 3e06cd39b1 add docker logs to provider 2026-09-02 11:09:38 -07:00
Alex Holliday efcc7829d4 move docker types to docker domain 2026-09-02 10:08:48 -07:00
Alex Holliday 786c803847 add port/mount display 2026-09-02 09:45:28 -07:00
Alex Holliday ca896fa22f implement docker host details page 2026-08-31 12:02:40 -07:00
Alex Holliday 8f6c909acd add missing const, add key check 2026-08-28 12:28:26 -07:00
Alex Holliday 1ca38a31c9 add new docker payload types, promote docker type monitor to first class monitor 2026-08-28 12:24:44 -07:00
Alex Holliday 252067142e apply proxy agents to http monitors 2026-08-21 12:19:23 -07:00
Alex Holliday 1152a70afb wire up proxy resolver 2026-08-21 11:41:26 -07:00
Alex Holliday ffd7ea7517 add tests 2026-08-21 11:13:22 -07:00
Alex Holliday a56bff2305 add proxy resolver 2026-08-21 11:06:42 -07:00
egeoztass c35b22e1ac fix: propagate SMTP transport errors from EmailService.sendEmail
sendEmail caught transport failures internally and resolved to false or
undefined, so every caller's .catch() was dead code and send failures were
silently suppressed. Password recovery in particular reported "email sent
successfully" while the user received nothing.

sendEmail now throws an AppError carrying the underlying cause, and returns
Promise<string> instead of Promise<string | false | undefined>.

Call sites updated for the new contract:
- UserService.registerUser: the existing .catch() now actually fires
- UserService.requestRecovery: the failure reaches the client instead of a
  misleading success response
- InviteService and the test-email endpoint: keep their user-facing messages
  and attach the SMTP cause
- EmailProvider: catches and returns false, because NotificationsService fans
  providers out through Promise.all and a rejection there would abort the
  other channels' alerts

Closes #3757
2026-08-14 14:11:16 +03:00
Alex Holliday dfcb3b4b21 slim check snapshot model/types 2026-08-04 11:27:32 -07:00
Alexander Holliday ff28e74aca Merge pull request #3797 from LvckyAPI/fix-incorrect-status
Fix monitoring status stuck
2026-07-30 09:44:15 -07:00
Alex Holliday ed54fa4668 first byte to total repsonse 2026-07-29 12:30:18 -07:00
Iven Schlenther 04f7ef252a Fix indentation in statusService.ts for isUnknownStatus assignment. 2026-07-27 15:07:00 +02:00
Iven Schlenther 3201653f19 Fix monitor status handling: replace isKnownStatus with isUnknownStatus for clarity and simplify initialization logic. 2026-07-27 15:03:43 +02:00
Iven Schlenther 0988a61c1d Resolve legacy monitor status issues: handle unknown statuses, fix stuck states from old boolean values, and add regression tests. Adjust BaseChart text color for theme compatibility. 2026-07-27 15:03:41 +02:00
Alex Holliday ed36db172a dedupe hardware stat definitions, fix tests, remove misleading null return types 2026-07-24 11:32:00 -07:00
Alex Holliday c021d04f53 extract swtich 2026-07-24 11:09:53 -07:00
Alex Holliday 4949acac1c remove unused email templates, remove duplicate constants 2026-07-22 10:26:00 -07:00
Alex Holliday 586e79efb2 remove unused getters 2026-07-22 10:13:37 -07:00
Alex 0df71d6356 use re2 for regex 2026-07-06 09:39:43 -07:00
Alex Holliday 8579edd56e add a shutdown method to buffer service 2026-06-24 11:58:32 -07:00
Alex Holliday 93b9c5033a add http metod option, add monitor.method guard to buildResponse 2026-06-22 10:26:10 -07:00
Alex Holliday 85f674ee08 resovle merge conflict 2026-06-19 09:55:52 -07:00
Alex Holliday a058c0518c change status on exit maintenance window 2026-06-19 09:37:24 -07:00