docs: verify and document every sink use case (Loki, Wazuh, HTTP, syslog) #40
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,22 +1,3 @@ | ||
| # Scratch PR/issue bodies used when filing GitHub issues | ||
| .issue-bodies/ | ||
| .pr-bodies/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -57,18 +57,27 @@ names and troubleshooting. | |
|
|
||
| ## Verified | ||
|
Owner
There was a problem hiding this comment. #31 was asked to keep visual proof ("where are the images?") and this PR deletes the three screenshots outright. The captured log output is solid evidence, but decide consciously: keep screenshots + logs, or logs only. Also - #31 already covers this exact work (same title, same files, CHANGES_REQUESTED from the maintainer): pick one PR to carry forward and close the other, or you'll have two conflicting rewrites of the same sections. |
||
|
|
||
| Every sink type is verified end to end in [examples/local-demo](examples/local-demo/README.md): | ||
| real Loki, real TCP and UDP syslog receivers, a real HTTP webhook receiver, and | ||
| AuditBridge's actual binary — only the NetBird API is stubbed to fixed sample | ||
| data, no live account involved. The evidence below is real output captured from | ||
| that compose stack, reproduced with one `docker compose up -d --build`. | ||
|
|
||
| | Sink | Delivery | Captured evidence | | ||
| |---|---|---| | ||
| | Grafana Loki | `SINKS=loki` | Loki query response, one stream per activity, nanosecond timestamps | | ||
| | Wazuh | `SINKS=wazuh`, `SINK_WAZUH_ADDR` | RFC 3164 frames received over TCP | | ||
| | Generic HTTP | `SINK_<NAME>_TRANSPORT=http` | `application/x-ndjson` `POST /ingest` received at the webhook | | ||
| | Generic syslog | `SINK_<NAME>_TRANSPORT=syslog` | RFC 5424 frames received over UDP | | ||
|
|
||
| Each section in [Sinks](docs/SINKS.md) pairs the complete configuration with | ||
| its verification commands and the captured output, including readyz/metrics | ||
| showing per-sink delivered totals. | ||
|
|
||
| > [!TIP] | ||
| > Don't take this page's word for it: `examples/local-demo/` reproduces the | ||
| exact setup with one command, and every block above is re-capturable from its | ||
| logs and endpoints. See [examples/local-demo/README.md](examples/local-demo/README.md). | ||
|
|
||
| ## License | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,21 +2,161 @@ | |
|
|
||
| Set `SINKS` to a comma-separated list. Each sink maintains its own cursor, so | ||
| a failing destination does not block successful deliveries to another one. | ||
| Every scenario below is verified end to end in | ||
| [examples/local-demo](../examples/local-demo/): the evidence blocks are real | ||
| output captured from that compose stack against a real Loki, real TCP and UDP | ||
| syslog receivers, and a real HTTP webhook receiver. Only the NetBird API is | ||
| stubbed to fixed sample data, no live account is involved. | ||
|
|
||
| ## Grafana Loki | ||
|
|
||
| `loki` is enabled by default. Events are pushed to Loki's | ||
| `/loki/api/v1/push` endpoint as per-label-set streams with nanosecond | ||
| timestamps, which is what Loki requires to accept a batch at all. | ||
|
|
||
| | Variable | Default | Meaning | | ||
| |---|---|---| | ||
| | `LOKI_URL` | `http://loki:3100` | Loki base URL; the push path is appended | | ||
| | `SINK_LOKI_URL` | unset | Exact push URL, used verbatim when set (overrides `LOKI_URL`) | | ||
| | `SINK_LOKI_ENCODING` | `loki` | Keep the `loki` encoding unless you have a reason not to | | ||
|
|
||
| The Loki label set is fixed: `job="netbird-events"`, `account_id` (defaults to | ||
| `unknown` when missing), `activity`, and `activity_code`. The higher-cardinality | ||
| `initiator_*` fields stay in the JSON log line rather than as labels, so streams | ||
| fragment per activity, not per user. | ||
|
|
||
| ### Worked example | ||
|
|
||
| ```bash | ||
| docker run -d --rm --name auditbridge \ | ||
| -v "$PWD/netbird-token:/run/secrets/netbird-token:ro" \ | ||
| -e NETBIRD_API_TOKEN_FILE=/run/secrets/netbird-token \ | ||
| -e SINKS=loki \ | ||
| -e LOKI_URL=https://loki.example.com \ | ||
| -p 9090:9090 \ | ||
| ghcr.io/onelrian/auditbridge:<immutable-tag> | ||
| ``` | ||
|
|
||
| ### Verify | ||
|
|
||
| Query Loki for the delivered stream: | ||
|
|
||
| ```bash | ||
| docker compose exec loki wget -qO- \ | ||
| 'http://localhost:3100/loki/api/v1/query_range?query=%7Bjob%3D%22netbird-events%22%7D' | ||
| ``` | ||
|
|
||
| Real response from the demo (three events, one stream per activity; Loki 3.x | ||
| adds the `service_name` label it derives from `job`): | ||
|
|
||
| ```json | ||
| {"status":"success","data":{"resultType":"streams","result":[ | ||
| {"stream":{"account_id":"acc-demo-01","activity":"User login","activity_code":"user.login","job":"netbird-events","service_name":"netbird-events"}, | ||
| "values":[["1786538589000000000","{\"account_id\":\"acc-demo-01\",\"activity\":\"User login\",\"activity_code\":\"user.login\",\"event_id\":\"evt-1003\",\"initiator_email\":\"alice@example.com\",\"initiator_id\":\"user-alice\",\"initiator_name\":\"Alice Example\",\"meta\":null,\"target_id\":null,\"timestamp\":\"2026-08-12T12:43:09Z\"}"]]}, | ||
| {"stream":{"account_id":"acc-demo-01","activity":"Peer added","activity_code":"peer.add","job":"netbird-events","service_name":"netbird-events"}, | ||
| "values":[["1786538581000000000","{\"account_id\":\"acc-demo-01\",\"activity\":\"Peer added\",\"activity_code\":\"peer.add\",\"event_id\":\"evt-1001\",\"initiator_email\":\"alice@example.com\",\"initiator_id\":\"user-alice\",\"initiator_name\":\"Alice Example\",\"meta\":{\"peer_name\":\"laptop-alice\"},\"target_id\":\"peer-7f3a\",\"timestamp\":\"2026-08-12T12:43:01Z\"}"]]}, | ||
| {"stream":{"account_id":"acc-demo-01","activity":"Group created","activity_code":"group.add","job":"netbird-events","service_name":"netbird-events"}, | ||
| "values":[["1786538585000000000","{\"account_id\":\"acc-demo-01\",\"activity\":\"Group created\",\"activity_code\":\"group.add\",\"event_id\":\"evt-1002\",\"initiator_email\":\"bob@example.com\",\"initiator_id\":\"user-bob\",\"initiator_name\":\"Bob Example\",\"meta\":{\"group_name\":\"engineering\"},\"target_id\":\"group-eng\",\"timestamp\":\"2026-08-12T12:43:05Z\"}"]]} | ||
| ]}} | ||
| ``` | ||
|
|
||
| In Grafana, add a Loki data source and query with LogQL, e.g. all audit events, | ||
| then filter to logins: | ||
|
|
||
| ```logql | ||
| {job="netbird-events"} | ||
| {job="netbird-events"} |= "user.login" | ||
| ``` | ||
|
|
||
| ## Wazuh | ||
|
|
||
| Add `wazuh` to `SINKS` and point it at the Wazuh manager's syslog listener. | ||
| The preset uses RFC 3164 framing over TCP, the combination most Wazuh | ||
| deployments accept from an agentless sender. | ||
|
|
||
| | Variable | Default | Meaning | | ||
| |---|---|---| | ||
| | `SINK_WAZUH_ADDR` / `WAZUH_ADDR` | none | Manager `host:port` (required) | | ||
| | `SINK_WAZUH_PROTOCOL` | `tcp` | `tcp` or `udp` | | ||
| | `SINK_WAZUH_ENCODING` | `syslog3164` | `syslog3164` or `syslog5424` | | ||
|
|
||
| ### Worked example (TCP, the default) | ||
|
|
||
| ```bash | ||
| docker run -d --rm --name auditbridge \ | ||
| -v "$PWD/netbird-token:/run/secrets/netbird-token:ro" \ | ||
| -e NETBIRD_API_TOKEN_FILE=/run/secrets/netbird-token \ | ||
| -e SINKS=wazuh \ | ||
| -e SINK_WAZUH_ADDR=wazuh-manager:1514 \ | ||
|
Owner
There was a problem hiding this comment. This contradicts #31 (same docs work by t-desmond): #31 says a stock Wazuh manager's syslog listener is UDP and that TCP 1514 is the agent "secure" port, not a syslog listener - with a real Wazuh alert as proof. The demo's |
||
| ghcr.io/onelrian/auditbridge:<immutable-tag> | ||
| ``` | ||
|
|
||
| For UDP, add `-e SINK_WAZUH_PROTOCOL=udp` and set the manager's UDP listener | ||
| port in `SINK_WAZUH_ADDR`. | ||
|
|
||
| ### Wazuh-side configuration | ||
|
|
||
| On the Wazuh manager, register a decoder and ruleset so delivered frames | ||
| become alerts. AuditBridge emits the event JSON as the MSG part with keys in | ||
| stable alphabetical order, e.g.: | ||
|
|
||
| ``` | ||
| <134>2026-08-12T12:43:01Z auditbridge netbird-audit: {"account_id":"acc-demo-01","activity":"Peer added","activity_code":"peer.add","event_id":"evt-1001",...} | ||
| ``` | ||
|
|
||
| `/var/ossec/etc/decoders/local_decoder.xml`: | ||
|
|
||
| ```xml | ||
| <decoder name="netbird-audit"> | ||
| <prematch>auditbridge netbird-audit</prematch> | ||
| <!-- The JSON keys are alphabetical, so activity_code is not the first | ||
| key: match it anywhere after the tag instead of anchoring on '{'. --> | ||
| <regex>"activity_code":"(\S+)"</regex> | ||
| <order>status</order> | ||
| </decoder> | ||
| ``` | ||
|
|
||
| `/var/ossec/etc/rules/local_rules.xml`: | ||
|
|
||
| ```xml | ||
| <group name="netbird,"> | ||
| <rule id="100200" level="3"> | ||
| <decoded_as>netbird-audit</decoded_as> | ||
| <description>NetBird audit event delivered</description> | ||
| </rule> | ||
| <rule id="100201" level="8"> | ||
| <decoded_as>netbird-audit</decoded_as> | ||
| <match>peer.add</match> | ||
| <description>NetBird peer added</description> | ||
| </rule> | ||
| <rule id="100202" level="12"> | ||
| <decoded_as>netbird-audit</decoded_as> | ||
| <match>user.login</match> | ||
| <description>NetBird user login</description> | ||
| </rule> | ||
| </group> | ||
| ``` | ||
|
|
||
| Restart the manager, then confirm alerts land in | ||
| `/var/ossec/logs/alerts/alerts.json` (`grep netbird-audit ...`). Adapt the | ||
| decoder's `<regex>` to your Wazuh version's regex flavour; the wire frames | ||
| below are the proof the manager is being fed. | ||
|
|
||
| ### Verify | ||
|
|
||
| The demo's `wazuh-receiver` is a TCP listener standing in for the manager. | ||
| Real frames received (RFC 3164, one per event): | ||
|
|
||
| ``` | ||
| [syslog-tcp] <134>2026-08-12T12:43:01Z auditbridge netbird-audit: {"account_id":"acc-demo-01","activity":"Peer added","activity_code":"peer.add","event_id":"evt-1001","initiator_email":"alice@example.com","initiator_id":"user-alice","initiator_name":"Alice Example","meta":{"peer_name":"laptop-alice"},"target_id":"peer-7f3a","timestamp":"2026-08-12T12:43:01Z"} | ||
| [syslog-tcp] <134>2026-08-12T12:43:05Z auditbridge netbird-audit: {"account_id":"acc-demo-01","activity":"Group created","activity_code":"group.add","event_id":"evt-1002","initiator_email":"bob@example.com","initiator_id":"user-bob","initiator_name":"Bob Example","meta":{"group_name":"engineering"},"target_id":"group-eng","timestamp":"2026-08-12T12:43:05Z"} | ||
| [syslog-tcp] <134>2026-08-12T12:43:09Z auditbridge netbird-audit: {"account_id":"acc-demo-01","activity":"User login","activity_code":"user.login","event_id":"evt-1003","initiator_email":"alice@example.com","initiator_id":"user-alice","initiator_name":"Alice Example","meta":null,"target_id":null,"timestamp":"2026-08-12T12:43:09Z"} | ||
| ``` | ||
|
|
||
| ## Generic HTTP sinks | ||
|
|
||
| Every sink name other than `loki` and `wazuh` is fully configurable: pick the | ||
| transport, the encoding, and the destination. HTTP is the `http` transport. | ||
|
|
||
| | Variable | Required for | Values | | ||
| |---|---|---| | ||
|
|
@@ -26,8 +166,6 @@ case and underscores: | |
| | `SINK_<NAME>_METHOD` | HTTP | HTTP method, default `POST` | | ||
| | `SINK_<NAME>_HEADERS` | HTTP | Comma-separated `Name:Value` headers | | ||
| | `SINK_<NAME>_HEADERS_FILE` | HTTP | File containing headers instead of the direct variable | | ||
|
|
||
| > [!TIP] | ||
| > Use the `_HEADERS_FILE` form for bearer tokens and API keys, mounted from a | ||
|
|
@@ -47,12 +185,71 @@ docker run -d --rm --name auditbridge \ | |
| -e LOKI_URL=https://loki.example.com \ | ||
| -e SINK_WEBHOOK_TRANSPORT=http \ | ||
| -e SINK_WEBHOOK_URL=https://collector.example.com/ingest \ | ||
| -e SINK_WEBHOOK_ENCODING=ndjson \ | ||
| -e SINK_WEBHOOK_HEADERS="Authorization:Bearer your-webhook-token" \ | ||
| ghcr.io/onelrian/auditbridge:<immutable-tag> | ||
| ``` | ||
|
|
||
| A replay of the full account history (fresh install, lost cursor) is split | ||
| into `BATCH_SIZE`-event requests, default 500, so payloads stay under intake | ||
| limits; see [Configuration](CONFIGURATION.md). | ||
|
|
||
| ### Verify | ||
|
|
||
| The demo's `webhook-receiver` logs each request. Real delivery with | ||
| `ndjson` encoding and an `X-Demo` header: | ||
|
|
||
| ``` | ||
| [webhook-receiver] POST /ingest content-type=application/x-ndjson | ||
| [webhook-receiver] {"id":"evt-1001","timestamp":"2026-08-12T12:43:01Z","activity":"Peer added","activity_code":"peer.add","initiator_id":"user-alice","initiator_email":"alice@example.com","initiator_name":"Alice Example","target_id":"peer-7f3a","account_id":"acc-demo-01","meta":{"peer_name":"laptop-alice"}} | ||
| [webhook-receiver] {"id":"evt-1002","timestamp":"2026-08-12T12:43:05Z","activity":"Group created","activity_code":"group.add","initiator_id":"user-bob","initiator_email":"bob@example.com","initiator_name":"Bob Example","target_id":"group-eng","account_id":"acc-demo-01","meta":{"group_name":"engineering"}} | ||
| [webhook-receiver] {"id":"evt-1003","timestamp":"2026-08-12T12:43:09Z","activity":"User login","activity_code":"user.login","initiator_id":"user-alice","initiator_email":"alice@example.com","initiator_name":"Alice Example","target_id":null,"account_id":"acc-demo-01","meta":null} | ||
| ``` | ||
|
|
||
| ## Generic syslog sinks | ||
|
|
||
| Syslog is the `syslog` transport. Any address, either protocol, either | ||
| framing; RFC 5424 for consumers confirmed to support it, RFC 3164 otherwise. | ||
|
|
||
| | Variable | Required for | Values | | ||
| |---|---|---| | ||
| | `SINK_<NAME>_TRANSPORT` | every generic sink | `http` or `syslog` | | ||
| | `SINK_<NAME>_ENCODING` | every generic sink | `syslog3164` or `syslog5424` | | ||
| | `SINK_<NAME>_ADDR` | syslog | Destination `host:port` | | ||
| | `SINK_<NAME>_PROTOCOL` | syslog | `tcp` or `udp`, default `tcp` | | ||
|
|
||
| ### Worked example: RFC 5424 over UDP | ||
|
|
||
| ```bash | ||
| docker run -d --rm --name auditbridge \ | ||
| -v "$PWD/netbird-token:/run/secrets/netbird-token:ro" \ | ||
| -e NETBIRD_API_TOKEN_FILE=/run/secrets/netbird-token \ | ||
| -e SINKS=sysloggen \ | ||
| -e SINK_SYSLOGGEN_TRANSPORT=syslog \ | ||
| -e SINK_SYSLOGGEN_ADDR=collector.example:1515 \ | ||
| -e SINK_SYSLOGGEN_PROTOCOL=udp \ | ||
| -e SINK_SYSLOGGEN_ENCODING=syslog5424 \ | ||
| ghcr.io/onelrian/auditbridge:<immutable-tag> | ||
| ``` | ||
|
|
||
| For TCP, drop `SINK_SYSLOGGEN_PROTOCOL` (TCP is the default) and point | ||
| `SINK_SYSLOGGEN_ADDR` at the TCP listener; the Wazuh section above shows the | ||
| RFC 3164-over-TCP variant in action. | ||
|
|
||
| ### Verify | ||
|
|
||
| The demo's `wazuh-receiver` also listens on UDP 1515. Real frames received | ||
| (RFC 5424: `<PRI>VERSION TIMESTAMP HOSTNAME APP-NAME PROCID MSGID | ||
| STRUCTURED-DATA MSG`): | ||
|
|
||
| ``` | ||
| [syslog-udp] <134>1 2026-08-12T12:43:01Z auditbridge netbird-audit - AUDIT - {"account_id":"acc-demo-01","activity":"Peer added","activity_code":"peer.add","event_id":"evt-1001","initiator_email":"alice@example.com","initiator_id":"user-alice","initiator_name":"Alice Example","meta":{"peer_name":"laptop-alice"},"target_id":"peer-7f3a","timestamp":"2026-08-12T12:43:01Z"} | ||
| ``` | ||
|
|
||
| ## Reproduce everything | ||
|
|
||
| `examples/local-demo/` runs all four scenarios at once: one `auditbridge` | ||
| instance with `SINKS=loki,wazuh` and a second, `auditbridge-generic`, with | ||
| `SINKS=webhook,sysloggen` covering the generic HTTP and generic syslog paths. | ||
| See [examples/local-demo/README.md](../examples/local-demo/README.md) for the | ||
| one-command setup and the verification commands whose output appears above. | ||
Binary file not shown.
Binary file not shown.
Binary file not shown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This replaces the entire .gitignore with just the two scratch dirs -
target/,debug/,.env,.vscode/,*.pdbare no longer ignored, socargo buildartifacts and local secrets can get committed. Looks accidental: restore the original entries and add.issue-bodies//.pr-bodies/alongside them.