diff --git a/.github/workflows/ci_asyncapi.yml b/.github/workflows/ci_asyncapi.yml new file mode 100644 index 0000000..bae6902 --- /dev/null +++ b/.github/workflows/ci_asyncapi.yml @@ -0,0 +1,36 @@ +# Validates the generated AsyncAPI spec via @asyncapi/cli on push and PR to main. +name: AsyncAPI + +on: + push: + branches: + - main + pull_request: + branches: + - main + +permissions: + contents: read + +jobs: + validate: + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Set up Node.js + uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 + with: + node-version: '22' + + - name: Set up Task + uses: go-task/setup-task@a00fbb05ce67b35648be3c78cbc9fd85354c757e # v2.2.0 + with: + version: '3.40.1' + + - name: Validate AsyncAPI spec + run: task asyncapi-lint + diff --git a/.github/workflows/ci_test.yml b/.github/workflows/ci_test.yml index 8fc1441..3a9b9f5 100644 --- a/.github/workflows/ci_test.yml +++ b/.github/workflows/ci_test.yml @@ -29,4 +29,4 @@ jobs: run: go vet ./... - name: Run tests - run: go test ./... -v + run: go test -race -count=1 ./... -v diff --git a/README.md b/README.md index 7990ca7..5d5a3a9 100644 --- a/README.md +++ b/README.md @@ -4,9 +4,10 @@ Go library providing event contract types for the [ComplyTime](https://github.com/complytime) ecosystem. Events use [CloudEvents](https://cloudevents.io/) v1.0 envelopes with -JSON-encoded payloads. The canonical event contract is defined in the -[AsyncAPI 3.0 spec](api/events/asyncapi.yaml); the Go types in this -library must match that spec. +JSON-encoded payloads. Go types in `events/events.go` are the source of +truth for event contracts; the [AsyncAPI 3.0 spec](api/events/asyncapi.yaml) +and [JSON Schema files](api/events/schemas/) are generated from those types +via `go generate`. Do not edit the generated files manually. ## Installation @@ -47,6 +48,73 @@ func main() { |------|----------|-------------| | `dev.complytime.evidence.ingested` | `events.TypeEvidenceIngested` | Evidence accepted for processing | +## Development + +### Prerequisites + +- [Go](https://go.dev/) (version per `go.mod`) +- [Task](https://taskfile.dev/) (task runner) +- [golangci-lint](https://golangci-lint.run/) +- [Node.js](https://nodejs.org/) with `npx` (required by `task asyncapi-lint`; first run downloads the AsyncAPI CLI from npm) + +After modifying Go structs in `events/events.go`, regenerate derived artifacts: + +```bash +task generate +``` + +This runs `go generate ./events/...` which rebuilds `api/events/asyncapi.yaml` +and the JSON Schema files in `api/events/schemas/`. + +To validate the generated AsyncAPI spec: + +```bash +task asyncapi-lint +``` + +To run all checks (lint, vet, test, asyncapi validation): + +```bash +task check +``` + +### Adding a new event type + +1. Define a new `*Data` struct in `events/events.go` with a sentinel blank + field carrying the `asyncapi` tag. The tag is a comma-separated list of + `key:value` pairs. **Values must not contain commas** (the parser splits + on commas, and a comma inside a value silently truncates it). Use + semicolons for natural pauses. Recognised keys: + + | Key | Required | Format | Description | + |-----|----------|--------|-------------| + | `channel` | yes | NATS subject with `{param}` placeholders | Channel address | + | `param` | no | `name=description` (repeatable) | Channel parameter | + | `stream` | yes | Upper-case stream name | NATS JetStream stream | + | `type` | yes | Reverse-DNS CloudEvents type | CloudEvents `type` attribute | + | `send` | yes | Free text (no commas) | Send operation summary | + | `receive` | yes | Free text (no commas) | Receive operation summary | + | `description` | no | Free text (no commas) | Channel description | + + Example sentinel field: + ```go + _ struct{} `asyncapi:"channel:core.widget.created.{ownerId},param:ownerId=The widget owner,stream:WIDGETS,type:dev.complytime.widget.created,send:Published when a widget is created,receive:Consume widget-created events,description:Widget creation pipeline"` + ``` + +2. Add `asyncapi-field:"description:..."` tags on each struct field for + schema descriptions. +3. Run `task generate` to regenerate all derived artifacts. +4. Add a constructor function (e.g., `NewYourEventEvent()`) following the + existing pattern. + +### Evolving an event contract + +Before changing an existing event's payload, read the +[Event Versioning Strategy](docs/versioning.md). It defines which field to bump +for additive versus breaking changes (CloudEvents `type` and AsyncAPI +`info.version`), and why the NATS subject stays stable so subscribers never +re-subscribe. + ## License [Apache-2.0](LICENSE) diff --git a/Taskfile.yml b/Taskfile.yml index 28e4301..5883a02 100644 --- a/Taskfile.yml +++ b/Taskfile.yml @@ -5,7 +5,7 @@ tasks: test: desc: Run all tests cmds: - - go test ./... + - go test -race -count=1 ./... lint: desc: Run golangci-lint @@ -17,9 +17,21 @@ tasks: cmds: - go vet ./... + generate: + desc: Regenerate derived artifacts (asyncapi.yaml and JSON Schemas) + cmds: + - go generate ./events/... + + asyncapi-lint: + desc: Validate asyncapi.yaml with the AsyncAPI CLI + cmds: + # Pinned to 2.16.1 — latest has broken npm dependency (@asyncapi/studio-ui@0.5.0 404) + - npx --yes @asyncapi/cli@2.16.1 validate api/events/asyncapi.yaml + check: - desc: Run lint and tests + desc: Run lint, vet, tests, and asyncapi validation cmds: - task: lint - task: vet - task: test + - task: asyncapi-lint diff --git a/api/events/asyncapi.yaml b/api/events/asyncapi.yaml index b3cba81..fd5a5ac 100644 --- a/api/events/asyncapi.yaml +++ b/api/events/asyncapi.yaml @@ -1,122 +1,116 @@ # SPDX-License-Identifier: Apache-2.0 asyncapi: 3.0.0 info: - title: ComplyTime API Events - version: 0.1.0 - description: | - Event contract for the ComplyTime evidence lifecycle. - - All public events use CloudEvents v1.0 envelope (JSON format). - The AsyncAPI spec is the source of truth for event contracts; - Go types in the events package must match these schemas. - license: - name: Apache-2.0 - contact: - name: ComplyTime - url: https://github.com/complytime/complyapi + title: ComplyTime API Events + version: 0.1.0 + description: |- + Event contract for the ComplyTime evidence lifecycle. + All public events use CloudEvents v1.0 envelope (JSON format). + This spec is generated from Go types in the events package via cmd/asyncapi-gen. + Do not edit manually — run 'go generate ./events/...' to regenerate. + license: + name: Apache-2.0 + contact: + name: ComplyTime + url: https://github.com/complytime/complyapi defaultContentType: application/cloudevents+json - +servers: + nats: + host: localhost:4222 + protocol: nats channels: - evidenceIngested: - address: core.evidence.ingested.{subjectId} - description: | - Published when evidence is ingested, before sealing. - parameters: - subjectId: - description: The compliance subject identifier (e.g. `my-app-v1`) - messages: - evidenceIngested: - $ref: '#/components/messages/evidenceIngested' - + evidenceIngested: + address: core.evidence.ingested.{subjectId} + description: Evidence ingestion pipeline for compliance artifacts + parameters: + subjectId: + description: The compliance subject identifier + messages: + EvidenceIngested: + $ref: '#/components/messages/EvidenceIngested' operations: - publishEvidenceIngested: - action: send - channel: - $ref: '#/channels/evidenceIngested' - summary: Published when evidence is accepted for processing. - - consumeEvidenceIngested: - action: receive - channel: - $ref: '#/channels/evidenceIngested' - summary: Consume evidence-ingested events. - + consumeEvidenceIngested: + action: receive + summary: Consume evidence-ingested events + channel: + $ref: '#/channels/evidenceIngested' + publishEvidenceIngested: + action: send + summary: Published when evidence is accepted for processing; before sealing + channel: + $ref: '#/channels/evidenceIngested' + bindings: + nats: + x-stream: EVIDENCE + bindingVersion: 0.1.0 components: - messages: - evidenceIngested: - name: EvidenceIngested - title: Evidence Ingested - contentType: application/cloudevents+json - payload: - $ref: '#/components/schemas/EvidenceIngestedCloudEvent' - - schemas: - EvidenceIngestedCloudEvent: - type: object - description: CloudEvents v1.0 envelope for evidence.ingested - required: - - specversion - - id - - type - - source - - subject - - time - - datacontenttype - - data - properties: - specversion: - type: string - const: "1.0" - id: - type: string - format: uuid - type: - type: string - const: dev.complytime.evidence.ingested - source: - type: string - description: URI identifying the producing service - examples: - - complytime-gateway - subject: - type: string - description: The compliance subject identifier - time: - type: string - format: date-time - datacontenttype: - type: string - const: application/json - data: - $ref: '#/components/schemas/EvidenceIngestedData' - - EvidenceIngestedData: - type: object - description: Payload for evidence.ingested events. - required: - - contentDigest - - artifactType - - subjectId - properties: - contentDigest: - type: string - description: SHA-256 digest of the evidence artifact - examples: - - sha256:abc123... - artifactType: - type: string - description: Gemara artifact type - examples: - - application/vnd.gemara.evaluation-log+json - storageRef: - type: string - description: Internal storage reference - subjectId: - type: string - description: Compliance subject identifier - examples: - - my-app-v1 - shardId: - type: string - description: Subject shard identifier (null when sharding is not configured) + messages: + EvidenceIngested: + name: EvidenceIngested + title: Evidence Ingested + contentType: application/cloudevents+json + payload: + $ref: '#/components/schemas/EvidenceIngestedCloudEvent' + schemas: + EvidenceIngestedCloudEvent: + type: object + description: CloudEvents v1.0 envelope for dev.complytime.evidence.ingested + required: + - specversion + - id + - type + - source + - subject + - time + - datacontenttype + - data + properties: + data: + $ref: '#/components/schemas/EvidenceIngestedData' + datacontenttype: + type: string + const: application/json + id: + type: string + format: uuid + source: + type: string + description: URI identifying the producing service + specversion: + type: string + const: "1.0" + subject: + type: string + description: The compliance subject identifier + time: + type: string + format: date-time + type: + type: string + const: dev.complytime.evidence.ingested + EvidenceIngestedData: + type: object + description: |- + EvidenceIngestedData is the CloudEvents data payload for + evidence.ingested events. + required: + - contentDigest + - artifactType + - subjectId + properties: + artifactType: + type: string + description: Gemara artifact type + contentDigest: + type: string + description: SHA-256 digest of the evidence artifact + shardId: + type: string + description: Subject shard identifier (null when sharding is not configured) + storageRef: + type: string + description: Internal storage reference + subjectId: + type: string + description: Compliance subject identifier diff --git a/api/events/schemas/EvidenceIngestedCloudEvent.schema.json b/api/events/schemas/EvidenceIngestedCloudEvent.schema.json new file mode 100644 index 0000000..a842717 --- /dev/null +++ b/api/events/schemas/EvidenceIngestedCloudEvent.schema.json @@ -0,0 +1,50 @@ +{ + "$comment": "Generated by cmd/asyncapi-gen from events/events.go — do not edit manually; run 'go generate ./events/...' to regenerate.", + "$id": "EvidenceIngestedCloudEvent.schema.json", + "$schema": "https://json-schema.org/draft/2020-12/schema", + "description": "CloudEvents v1.0 envelope for dev.complytime.evidence.ingested", + "properties": { + "data": { + "$ref": "EvidenceIngestedData.schema.json" + }, + "datacontenttype": { + "const": "application/json", + "type": "string" + }, + "id": { + "format": "uuid", + "type": "string" + }, + "source": { + "description": "URI identifying the producing service", + "type": "string" + }, + "specversion": { + "const": "1.0", + "type": "string" + }, + "subject": { + "description": "The compliance subject identifier", + "type": "string" + }, + "time": { + "format": "date-time", + "type": "string" + }, + "type": { + "const": "dev.complytime.evidence.ingested", + "type": "string" + } + }, + "required": [ + "specversion", + "id", + "type", + "source", + "subject", + "time", + "datacontenttype", + "data" + ], + "type": "object" +} diff --git a/api/events/schemas/EvidenceIngestedData.schema.json b/api/events/schemas/EvidenceIngestedData.schema.json new file mode 100644 index 0000000..309b1bc --- /dev/null +++ b/api/events/schemas/EvidenceIngestedData.schema.json @@ -0,0 +1,34 @@ +{ + "$comment": "Generated by cmd/asyncapi-gen from events/events.go — do not edit manually; run 'go generate ./events/...' to regenerate.", + "$id": "EvidenceIngestedData.schema.json", + "$schema": "https://json-schema.org/draft/2020-12/schema", + "description": "EvidenceIngestedData is the CloudEvents data payload for\nevidence.ingested events.", + "properties": { + "artifactType": { + "description": "Gemara artifact type", + "type": "string" + }, + "contentDigest": { + "description": "SHA-256 digest of the evidence artifact", + "type": "string" + }, + "shardId": { + "description": "Subject shard identifier (null when sharding is not configured)", + "type": "string" + }, + "storageRef": { + "description": "Internal storage reference", + "type": "string" + }, + "subjectId": { + "description": "Compliance subject identifier", + "type": "string" + } + }, + "required": [ + "contentDigest", + "artifactType", + "subjectId" + ], + "type": "object" +} diff --git a/cmd/asyncapi-gen/integration_test.go b/cmd/asyncapi-gen/integration_test.go new file mode 100644 index 0000000..8d9fa72 --- /dev/null +++ b/cmd/asyncapi-gen/integration_test.go @@ -0,0 +1,244 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "bufio" + "fmt" + "os" + "path/filepath" + "strings" + "testing" +) + +// parseGenerateDirective reads the go:generate directive from events.go +// and returns the flag arguments (everything after "go run ../cmd/asyncapi-gen"). +// This ensures the integration test always uses the same metadata as the +// real go:generate invocation — no hardcoded duplication. +func parseGenerateDirective(t *testing.T, eventsPath string) []string { + t.Helper() + f, err := os.Open(eventsPath) + if err != nil { + t.Fatalf("opening %s: %v", eventsPath, err) + } + defer f.Close() + + scanner := bufio.NewScanner(f) + // Increase buffer for long go:generate lines. + scanner.Buffer(make([]byte, 0, 64*1024), 64*1024) + const prefix = "//go:generate go run ../cmd/asyncapi-gen " + for scanner.Scan() { + line := scanner.Text() + if strings.HasPrefix(line, prefix) { + argStr := line[len(prefix):] + return splitArgs(argStr) + } + } + if err := scanner.Err(); err != nil { + t.Fatalf("scanning %s: %v", eventsPath, err) + } + t.Fatalf("no //go:generate directive found in %s", eventsPath) + return nil +} + +// splitArgs splits a flag string respecting quoted values (for -description etc). +// Inside quoted segments, \n is interpreted as a real newline to match go generate +// behaviour (go generate processes escape sequences in quoted arguments). +func splitArgs(s string) []string { + var args []string + var current strings.Builder + inQuote := false + for i := 0; i < len(s); i++ { + ch := s[i] + if ch == '"' { + inQuote = !inQuote + continue + } + if ch == ' ' && !inQuote { + if current.Len() > 0 { + args = append(args, current.String()) + current.Reset() + } + continue + } + // Handle escape sequences inside quoted strings. + if inQuote && ch == '\\' && i+1 < len(s) { + next := s[i+1] + switch next { + case 'n': + current.WriteByte('\n') + i++ + continue + case 't': + current.WriteByte('\t') + i++ + continue + case '\\': + current.WriteByte('\\') + i++ + continue + case '"': + current.WriteByte('"') + i++ + continue + } + } + current.WriteByte(ch) + } + if current.Len() > 0 { + args = append(args, current.String()) + } + return args +} + +func TestSplitArgs(t *testing.T) { + tests := []struct { + name string + in string + want []string + }{ + {"simple", "-a foo -b bar", []string{"-a", "foo", "-b", "bar"}}, + {"quoted with spaces", `-title "My API Title"`, []string{"-title", "My API Title"}}, + {"newline escape", `-desc "line1\nline2"`, []string{"-desc", "line1\nline2"}}, + {"tab escape", `-desc "col1\tcol2"`, []string{"-desc", "col1\tcol2"}}, + {"backslash escape", `-path "c:\\dir"`, []string{"-path", "c:\\dir"}}, + {"escaped quote", `-msg "say \"hi\""`, []string{"-msg", `say "hi"`}}, + {"empty input", "", nil}, + {"trailing spaces", "-a foo ", []string{"-a", "foo"}}, + {"adjacent values", `-a "x" -b "y"`, []string{"-a", "x", "-b", "y"}}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got := splitArgs(tt.in) + if len(got) != len(tt.want) { + t.Fatalf("splitArgs(%q) = %v (len %d), want %v (len %d)", tt.in, got, len(got), tt.want, len(tt.want)) + } + for i := range got { + if got[i] != tt.want[i] { + t.Errorf("splitArgs(%q)[%d] = %q, want %q", tt.in, i, got[i], tt.want[i]) + } + } + }) + } +} + +// TestIntegration_GeneratedMatchesCommitted regenerates asyncapi.yaml from +// events/events.go and verifies the output matches the committed file. +// This is the drift detector: it fails if the two are out of sync. +func TestIntegration_GeneratedMatchesCommitted(t *testing.T) { + inputPath := filepath.Join("..", "..", "events", "events.go") + committedPath := filepath.Join("..", "..", "api", "events", "asyncapi.yaml") + + // Parse flags from the go:generate directive — single source of truth. + args := parseGenerateDirective(t, inputPath) + opts := parseFlags(args) + + specs, err := ParseFile(inputPath) + if err != nil { + t.Fatalf("ParseFile: %v", err) + } + + doc := BuildDoc(specs, DocMeta{ + Title: opts.Title, + Version: opts.Version, + Description: opts.Description, + LicenseName: opts.LicenseName, + ContactName: opts.ContactName, + ContactURL: opts.ContactURL, + ServerURL: opts.Server, + }) + + outPath := filepath.Join(t.TempDir(), "asyncapi.yaml") + if err := WriteYAML(doc, outPath); err != nil { + t.Fatalf("WriteYAML: %v", err) + } + + generated, err := os.ReadFile(outPath) + if err != nil { + t.Fatalf("reading generated file: %v", err) + } + committed, err := os.ReadFile(committedPath) + if err != nil { + t.Fatalf("reading committed file: %v", err) + } + + if string(generated) != string(committed) { + t.Errorf("generated asyncapi.yaml does not match committed file.\n"+ + "Run `go generate ./events/...` to update it.\n\n"+ + "--- committed\n+++ generated\n%s", + diffStrings(string(committed), string(generated)), + ) + } +} + +// TestIntegration_JSONSchemasMatchCommitted regenerates JSON Schema files +// and verifies they match the committed versions. +func TestIntegration_JSONSchemasMatchCommitted(t *testing.T) { + inputPath := filepath.Join("..", "..", "events", "events.go") + committedDir := filepath.Join("..", "..", "api", "events", "schemas") + + specs, err := ParseFile(inputPath) + if err != nil { + t.Fatalf("ParseFile: %v", err) + } + + outDir := filepath.Join(t.TempDir(), "schemas") + if err := WriteJSONSchemas(specs, outDir); err != nil { + t.Fatalf("WriteJSONSchemas: %v", err) + } + + for _, spec := range specs { + files := []string{ + spec.StructName + ".schema.json", + envelopeSchemaName(spec.StructName) + ".schema.json", + } + for _, name := range files { + generated, err := os.ReadFile(filepath.Join(outDir, name)) + if err != nil { + t.Fatalf("reading generated %s: %v", name, err) + } + committed, err := os.ReadFile(filepath.Join(committedDir, name)) + if err != nil { + t.Fatalf("reading committed %s: %v", name, err) + } + if string(generated) != string(committed) { + t.Errorf("generated %s does not match committed file.\n"+ + "Run `go generate ./events/...` to update it.\n\n"+ + "--- committed\n+++ generated\n%s", + name, diffStrings(string(committed), string(generated)), + ) + } + } + } +} + +// diffStrings returns a simple line-diff between a and b. +func diffStrings(a, b string) string { + aLines := splitLines(a) + bLines := splitLines(b) + var out []string + max := len(aLines) + if len(bLines) > max { + max = len(bLines) + } + for i := 0; i < max; i++ { + var al, bl string + if i < len(aLines) { + al = aLines[i] + } + if i < len(bLines) { + bl = bLines[i] + } + if al != bl { + out = append(out, fmt.Sprintf("line %d:\n committed: %q\n generated: %q", i+1, al, bl)) + } + } + if len(out) == 0 { + return "(no line differences found — may be whitespace)" + } + return strings.Join(out, "\n") +} + +func splitLines(s string) []string { + return strings.Split(s, "\n") +} diff --git a/cmd/asyncapi-gen/jsonschema.go b/cmd/asyncapi-gen/jsonschema.go new file mode 100644 index 0000000..8ac3712 --- /dev/null +++ b/cmd/asyncapi-gen/jsonschema.go @@ -0,0 +1,120 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "encoding/json" + "fmt" + "os" + "path/filepath" +) + +// JSONSchema represents a JSON Schema document using ordered map entries +// to produce deterministic output via encoding/json. +type JSONSchema = map[string]any + +// BuildDataJSONSchema builds a standalone JSON Schema document for the +// data payload of an event spec. The schema uses JSON Schema Draft 2020-12. +func BuildDataJSONSchema(spec EventSpec) JSONSchema { + properties := make(JSONSchema) + var required []string + + for _, f := range spec.Fields { + prop := JSONSchema{"type": goTypeToJSONSchema(f.GoType)} + if f.Description != "" { + prop["description"] = f.Description + } + properties[f.JSONName] = prop + if f.Required { + required = append(required, f.JSONName) + } + } + + schema := JSONSchema{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": fmt.Sprintf("%s.schema.json", spec.StructName), + "$comment": "Generated by cmd/asyncapi-gen from events/events.go — do not edit manually; run 'go generate ./events/...' to regenerate.", + "type": "object", + "properties": properties, + } + if spec.DocComment != "" { + schema["description"] = spec.DocComment + } + if len(required) > 0 { + schema["required"] = required + } + + return schema +} + +// BuildEnvelopeJSONSchema builds a standalone JSON Schema document for +// the CloudEvents v1.0 envelope wrapping the data payload. The data +// field references the data schema by relative $ref. +func BuildEnvelopeJSONSchema(spec EventSpec) JSONSchema { + envelopeName := envelopeSchemaName(spec.StructName) + dataSchemaFile := spec.StructName + ".schema.json" + + properties := JSONSchema{ + "specversion": JSONSchema{"type": "string", "const": "1.0"}, + "id": JSONSchema{"type": "string", "format": "uuid"}, + "type": JSONSchema{"type": "string", "const": spec.CEType}, + "source": JSONSchema{"type": "string", "description": "URI identifying the producing service"}, + "subject": JSONSchema{"type": "string", "description": "The compliance subject identifier"}, + "time": JSONSchema{"type": "string", "format": "date-time"}, + "datacontenttype": JSONSchema{"type": "string", "const": "application/json"}, + "data": JSONSchema{"$ref": dataSchemaFile}, + } + + return JSONSchema{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": fmt.Sprintf("%s.schema.json", envelopeName), + "$comment": "Generated by cmd/asyncapi-gen from events/events.go — do not edit manually; run 'go generate ./events/...' to regenerate.", + "type": "object", + "description": fmt.Sprintf("CloudEvents v1.0 envelope for %s", spec.CEType), + "required": []string{"specversion", "id", "type", "source", "subject", "time", "datacontenttype", "data"}, + "properties": properties, + } +} + +// WriteJSONSchemas writes the envelope and data JSON Schema files for +// each event spec to the given directory. Creates the directory if it +// does not exist. +func WriteJSONSchemas(specs []EventSpec, dir string) error { + if err := os.MkdirAll(dir, 0o755); err != nil { + return fmt.Errorf("creating schemas directory: %w", err) + } + + for _, spec := range specs { + dataSchema := BuildDataJSONSchema(spec) + if err := writeJSON(dataSchema, filepath.Join(dir, spec.StructName+".schema.json")); err != nil { + return err + } + + envName := envelopeSchemaName(spec.StructName) + envSchema := BuildEnvelopeJSONSchema(spec) + if err := writeJSON(envSchema, filepath.Join(dir, envName+".schema.json")); err != nil { + return err + } + } + + return nil +} + +// envelopeSchemaName converts "EvidenceIngestedData" to "EvidenceIngestedCloudEvent". +func envelopeSchemaName(structName string) string { + return messageKey(structName) + "CloudEvent" +} + +// writeJSON marshals v to indented JSON and writes it to path. +func writeJSON(v any, path string) error { + b, err := json.MarshalIndent(v, "", " ") + if err != nil { + return fmt.Errorf("marshaling JSON schema: %w", err) + } + b = append(b, '\n') + + if err := os.WriteFile(path, b, 0o644); err != nil { //nolint:gosec // 0o644 is correct for generated schema files (SC-005) + return fmt.Errorf("writing %s: %w", path, err) + } + return nil +} diff --git a/cmd/asyncapi-gen/jsonschema_test.go b/cmd/asyncapi-gen/jsonschema_test.go new file mode 100644 index 0000000..ed1a033 --- /dev/null +++ b/cmd/asyncapi-gen/jsonschema_test.go @@ -0,0 +1,148 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "encoding/json" + "os" + "path/filepath" + "testing" +) + +func TestBuildDataJSONSchema(t *testing.T) { + schema := BuildDataJSONSchema(singleSpec()) + + if schema["$schema"] != "https://json-schema.org/draft/2020-12/schema" { + t.Errorf("$schema = %q, want Draft 2020-12 URI", schema["$schema"]) + } + if schema["$id"] != "WidgetCreatedData.schema.json" { + t.Errorf("$id = %q, want %q", schema["$id"], "WidgetCreatedData.schema.json") + } + if schema["type"] != "object" { + t.Errorf("type = %q, want %q", schema["type"], "object") + } + + required, ok := schema["required"].([]string) + if !ok { + t.Fatal("required is not []string") + } + requiredSet := map[string]bool{} + for _, r := range required { + requiredSet[r] = true + } + if !requiredSet["widgetId"] { + t.Error("widgetId should be required") + } + if !requiredSet["name"] { + t.Error("name should be required") + } + if requiredSet["tag"] { + t.Error("tag should not be required") + } + + props, ok := schema["properties"].(JSONSchema) + if !ok { + t.Fatal("properties is not JSONSchema") + } + widgetID, ok := props["widgetId"].(JSONSchema) + if !ok { + t.Fatal("widgetId property not found") + } + if widgetID["type"] != "string" { + t.Errorf("widgetId type = %q, want %q", widgetID["type"], "string") + } + if widgetID["description"] != "Unique widget identifier" { + t.Errorf("widgetId description = %q, want %q", widgetID["description"], "Unique widget identifier") + } + + // Data schema should have description from doc comment + if schema["description"] != "WidgetCreatedData is the payload for widget.created events." { + t.Errorf("data schema description = %q, want %q", schema["description"], "WidgetCreatedData is the payload for widget.created events.") + } + + // Field without description should have no description key + nameProp, ok := props["name"].(JSONSchema) + if !ok { + t.Fatal("name property not found") + } + if _, hasDesc := nameProp["description"]; hasDesc { + t.Error("name should not have description key") + } +} + +func TestBuildEnvelopeJSONSchema(t *testing.T) { + schema := BuildEnvelopeJSONSchema(singleSpec()) + + if schema["$schema"] != "https://json-schema.org/draft/2020-12/schema" { + t.Errorf("$schema = %q, want Draft 2020-12 URI", schema["$schema"]) + } + if schema["$id"] != "WidgetCreatedCloudEvent.schema.json" { + t.Errorf("$id = %q, want %q", schema["$id"], "WidgetCreatedCloudEvent.schema.json") + } + + props, ok := schema["properties"].(JSONSchema) + if !ok { + t.Fatal("properties is not JSONSchema") + } + + specversion, ok := props["specversion"].(JSONSchema) + if !ok { + t.Fatal("specversion property not found") + } + if specversion["const"] != "1.0" { + t.Errorf("specversion const = %q, want %q", specversion["const"], "1.0") + } + + ceType, ok := props["type"].(JSONSchema) + if !ok { + t.Fatal("type property not found") + } + if ceType["const"] != "dev.example.widget.created" { + t.Errorf("type const = %q, want %q", ceType["const"], "dev.example.widget.created") + } + + data, ok := props["data"].(JSONSchema) + if !ok { + t.Fatal("data property not found") + } + if data["$ref"] != "WidgetCreatedData.schema.json" { + t.Errorf("data $ref = %q, want %q", data["$ref"], "WidgetCreatedData.schema.json") + } +} + +func TestWriteJSONSchemas(t *testing.T) { + dir := filepath.Join(t.TempDir(), "schemas") + + err := WriteJSONSchemas([]EventSpec{singleSpec()}, dir) + if err != nil { + t.Fatalf("WriteJSONSchemas: %v", err) + } + + // Verify data schema file + dataPath := filepath.Join(dir, "WidgetCreatedData.schema.json") + dataBytes, err := os.ReadFile(dataPath) + if err != nil { + t.Fatalf("reading data schema: %v", err) + } + var dataSchema map[string]any + if err := json.Unmarshal(dataBytes, &dataSchema); err != nil { + t.Fatalf("parsing data schema JSON: %v", err) + } + if dataSchema["$schema"] != "https://json-schema.org/draft/2020-12/schema" { + t.Error("data schema missing $schema") + } + + // Verify envelope schema file + envPath := filepath.Join(dir, "WidgetCreatedCloudEvent.schema.json") + envBytes, err := os.ReadFile(envPath) + if err != nil { + t.Fatalf("reading envelope schema: %v", err) + } + var envSchema map[string]any + if err := json.Unmarshal(envBytes, &envSchema); err != nil { + t.Fatalf("parsing envelope schema JSON: %v", err) + } + if envSchema["$schema"] != "https://json-schema.org/draft/2020-12/schema" { + t.Error("envelope schema missing $schema") + } +} diff --git a/cmd/asyncapi-gen/main.go b/cmd/asyncapi-gen/main.go new file mode 100644 index 0000000..418eea9 --- /dev/null +++ b/cmd/asyncapi-gen/main.go @@ -0,0 +1,91 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Command asyncapi-gen generates an AsyncAPI 3.0 document from annotated +// Go event structs. Run via go generate in the events package. +package main + +import ( + "flag" + "fmt" + "io" + "os" +) + +// Options holds the parsed command-line flags for asyncapi-gen. +type Options struct { + Input string + Output string + Title string + Version string + Server string + Description string + LicenseName string + ContactName string + ContactURL string + SchemasDir string +} + +func main() { + opts := parseFlags(os.Args[1:]) + if err := run(opts, os.Stdout, os.Stderr); err != nil { + fmt.Fprintf(os.Stderr, "asyncapi-gen: %v\n", err) + os.Exit(1) + } +} + +// parseFlags parses command-line arguments into Options. +func parseFlags(args []string) Options { + fs := flag.NewFlagSet("asyncapi-gen", flag.ExitOnError) + var opts Options + fs.StringVar(&opts.Input, "input", "", "Path to Go source file containing annotated event structs (required)") + fs.StringVar(&opts.Output, "output", "", "Path to write the generated asyncapi.yaml (required)") + fs.StringVar(&opts.Title, "title", "", "AsyncAPI document title (required)") + fs.StringVar(&opts.Version, "version", "", "AsyncAPI document version (required)") + fs.StringVar(&opts.Server, "server", "", "NATS server URL, e.g. nats://localhost:4222 (required)") + fs.StringVar(&opts.Description, "description", "", "AsyncAPI document description (optional)") + fs.StringVar(&opts.LicenseName, "license", "", "License name, e.g. Apache-2.0 (optional)") + fs.StringVar(&opts.ContactName, "contact-name", "", "Contact name (optional)") + fs.StringVar(&opts.ContactURL, "contact-url", "", "Contact URL (optional)") + fs.StringVar(&opts.SchemasDir, "schemas-dir", "", "Directory to write JSON Schema files (optional)") + _ = fs.Parse(args) + return opts +} + +// run executes the asyncapi-gen pipeline with the given options. +func run(opts Options, stdout, stderr io.Writer) error { + if opts.Input == "" || opts.Output == "" || opts.Title == "" || opts.Version == "" || opts.Server == "" { + return fmt.Errorf("required flags missing: -input -output -title -version -server") + } + + specs, err := ParseFile(opts.Input) + if err != nil { + return fmt.Errorf("parse error: %w", err) + } + if len(specs) == 0 { + return fmt.Errorf("no annotated structs found in %s", opts.Input) + } + + doc := BuildDoc(specs, DocMeta{ + Title: opts.Title, + Version: opts.Version, + Description: opts.Description, + LicenseName: opts.LicenseName, + ContactName: opts.ContactName, + ContactURL: opts.ContactURL, + ServerURL: opts.Server, + }) + + if err := WriteYAML(doc, opts.Output); err != nil { + return fmt.Errorf("write error: %w", err) + } + + if opts.SchemasDir != "" { + if err := WriteJSONSchemas(specs, opts.SchemasDir); err != nil { + return fmt.Errorf("schema write error: %w", err) + } + fmt.Fprintf(stdout, "asyncapi-gen: wrote JSON schemas to %s\n", opts.SchemasDir) + } + + fmt.Fprintf(stdout, "asyncapi-gen: wrote %s (%d event(s))\n", opts.Output, len(specs)) + return nil +} diff --git a/cmd/asyncapi-gen/main_test.go b/cmd/asyncapi-gen/main_test.go new file mode 100644 index 0000000..0e73e73 --- /dev/null +++ b/cmd/asyncapi-gen/main_test.go @@ -0,0 +1,209 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "bytes" + "os" + "path/filepath" + "strings" + "testing" +) + +func TestRun_MissingRequiredFlags(t *testing.T) { + tests := []struct { + name string + opts Options + }{ + {"missing input", Options{Output: "out.yaml", Title: "T", Version: "1.0", Server: "nats://x"}}, + {"missing output", Options{Input: "in.go", Title: "T", Version: "1.0", Server: "nats://x"}}, + {"missing title", Options{Input: "in.go", Output: "out.yaml", Version: "1.0", Server: "nats://x"}}, + {"missing version", Options{Input: "in.go", Output: "out.yaml", Title: "T", Server: "nats://x"}}, + {"missing server", Options{Input: "in.go", Output: "out.yaml", Title: "T", Version: "1.0"}}, + {"all empty", Options{}}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + var stdout, stderr bytes.Buffer + err := run(tt.opts, &stdout, &stderr) + if err == nil { + t.Fatal("expected error for missing required flags") + } + if !strings.Contains(err.Error(), "required flags missing") { + t.Errorf("error = %q, want mention of required flags", err) + } + }) + } +} + +func TestRun_ParseError(t *testing.T) { + var stdout, stderr bytes.Buffer + opts := Options{ + Input: "nonexistent_file.go", + Output: "out.yaml", + Title: "Test", + Version: "1.0", + Server: "nats://localhost:4222", + } + err := run(opts, &stdout, &stderr) + if err == nil { + t.Fatal("expected error for nonexistent file") + } + if !strings.Contains(err.Error(), "parse error") { + t.Errorf("error = %q, want mention of parse error", err) + } +} + +func TestRun_NoAnnotatedStructs(t *testing.T) { + dir := t.TempDir() + input := filepath.Join(dir, "empty.go") + content := "package testdata\n\ntype Plain struct {\n\tName string `json:\"name\"`\n}\n" + if err := os.WriteFile(input, []byte(content), 0o644); err != nil { //nolint:gosec // 0o644 is correct for test fixture files (SC-005) + t.Fatalf("writing test file: %v", err) + } + + var stdout, stderr bytes.Buffer + opts := Options{ + Input: input, + Output: filepath.Join(dir, "out.yaml"), + Title: "Test", + Version: "1.0", + Server: "nats://localhost:4222", + } + err := run(opts, &stdout, &stderr) + if err == nil { + t.Fatal("expected error for no annotated structs") + } + if !strings.Contains(err.Error(), "no annotated structs") { + t.Errorf("error = %q, want mention of no annotated structs", err) + } +} + +func TestRun_HappyPath(t *testing.T) { + dir := t.TempDir() + var stdout, stderr bytes.Buffer + opts := Options{ + Input: "testdata/fixture.go", + Output: filepath.Join(dir, "asyncapi.yaml"), + Title: "Test API", + Version: "1.0.0", + Server: "nats://localhost:4222", + } + err := run(opts, &stdout, &stderr) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if !strings.Contains(stdout.String(), "wrote") { + t.Errorf("stdout = %q, want mention of wrote", stdout.String()) + } +} + +func TestRun_HappyPathWithSchemas(t *testing.T) { + dir := t.TempDir() + var stdout, stderr bytes.Buffer + opts := Options{ + Input: "testdata/fixture.go", + Output: filepath.Join(dir, "asyncapi.yaml"), + Title: "Test API", + Version: "1.0.0", + Server: "nats://localhost:4222", + SchemasDir: filepath.Join(dir, "schemas"), + } + err := run(opts, &stdout, &stderr) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + output := stdout.String() + if !strings.Contains(output, "wrote JSON schemas") { + t.Errorf("stdout = %q, want mention of wrote JSON schemas", output) + } + if !strings.Contains(output, "wrote") { + t.Errorf("stdout = %q, want mention of wrote", output) + } +} + +func TestRun_WriteError(t *testing.T) { + var stdout, stderr bytes.Buffer + opts := Options{ + Input: "testdata/fixture.go", + Output: "/nonexistent/deeply/nested/dir/out.yaml", + Title: "Test API", + Version: "1.0.0", + Server: "nats://localhost:4222", + } + err := run(opts, &stdout, &stderr) + if err == nil { + t.Fatal("expected error for invalid output path") + } + if !strings.Contains(err.Error(), "write error") { + t.Errorf("error = %q, want mention of write error", err) + } +} + +func TestRun_SchemaWriteError(t *testing.T) { + dir := t.TempDir() + var stdout, stderr bytes.Buffer + opts := Options{ + Input: "testdata/fixture.go", + Output: filepath.Join(dir, "asyncapi.yaml"), + Title: "Test API", + Version: "1.0.0", + Server: "nats://localhost:4222", + SchemasDir: "/nonexistent/deeply/nested/dir/schemas", + } + err := run(opts, &stdout, &stderr) + if err == nil { + t.Fatal("expected error for invalid schemas directory") + } + if !strings.Contains(err.Error(), "schema write error") { + t.Errorf("error = %q, want mention of schema write error", err) + } +} + +func TestParseFlags_AllFlags(t *testing.T) { + args := []string{ + "-input", "events.go", + "-output", "out.yaml", + "-title", "My API", + "-version", "2.0.0", + "-server", "nats://host:4222", + "-description", "A description", + "-license", "MIT", + "-contact-name", "Alice", + "-contact-url", "https://example.com", + "-schemas-dir", "/tmp/schemas", + } + opts := parseFlags(args) + + checks := []struct { + name string + got string + want string + }{ + {"Input", opts.Input, "events.go"}, + {"Output", opts.Output, "out.yaml"}, + {"Title", opts.Title, "My API"}, + {"Version", opts.Version, "2.0.0"}, + {"Server", opts.Server, "nats://host:4222"}, + {"Description", opts.Description, "A description"}, + {"LicenseName", opts.LicenseName, "MIT"}, + {"ContactName", opts.ContactName, "Alice"}, + {"ContactURL", opts.ContactURL, "https://example.com"}, + {"SchemasDir", opts.SchemasDir, "/tmp/schemas"}, + } + for _, c := range checks { + if c.got != c.want { + t.Errorf("%s = %q, want %q", c.name, c.got, c.want) + } + } +} + +func TestParseFlags_Defaults(t *testing.T) { + opts := parseFlags([]string{}) + if opts.Input != "" { + t.Errorf("Input default = %q, want empty", opts.Input) + } + if opts.SchemasDir != "" { + t.Errorf("SchemasDir default = %q, want empty", opts.SchemasDir) + } +} diff --git a/cmd/asyncapi-gen/parser.go b/cmd/asyncapi-gen/parser.go new file mode 100644 index 0000000..7053663 --- /dev/null +++ b/cmd/asyncapi-gen/parser.go @@ -0,0 +1,253 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "fmt" + "go/ast" + "go/parser" + "go/token" + "os" + "reflect" + "strings" +) + +// EventSpec holds the generator metadata extracted from one event data struct. +type EventSpec struct { + StructName string + Channel string + Params map[string]string // param name → description + Stream string + CEType string + SendSummary string + RecvSummary string + ChannelDescription string // from asyncapi tag description: key + DocComment string // Go doc comment on the struct + Fields []FieldSpec +} + +// FieldSpec describes one data field extracted from a struct. +type FieldSpec struct { + JSONName string + GoType string // e.g. "string", "*string", "int" + Required bool // false when omitempty or pointer type + Description string // from asyncapi-field:"description:..." tag +} + +// ParseFile parses the Go source file at path and returns one EventSpec +// per annotated struct. Returns an error if the file cannot be parsed or +// a required tag key is missing. +func ParseFile(path string) ([]EventSpec, error) { + src, err := os.ReadFile(path) + if err != nil { + return nil, fmt.Errorf("reading file: %w", err) + } + + fset := token.NewFileSet() + f, err := parser.ParseFile(fset, path, src, parser.ParseComments) + if err != nil { + return nil, fmt.Errorf("parsing file: %w", err) + } + + var specs []EventSpec + for _, decl := range f.Decls { + genDecl, ok := decl.(*ast.GenDecl) + if !ok { + continue + } + for _, spec := range genDecl.Specs { + typeSpec, ok := spec.(*ast.TypeSpec) + if !ok { + continue + } + structType, ok := typeSpec.Type.(*ast.StructType) + if !ok { + continue + } + + // Extract doc comment: prefer TypeSpec.Doc, fall back to GenDecl.Doc + // (single-spec type declarations attach comments to GenDecl). + var docComment string + switch { + case typeSpec.Doc != nil: + docComment = strings.TrimSpace(typeSpec.Doc.Text()) + case genDecl.Doc != nil: + docComment = strings.TrimSpace(genDecl.Doc.Text()) + } + + es, ok, err := extractEventSpec(typeSpec.Name.Name, structType) + if err != nil { + return nil, err + } + if ok { + es.DocComment = docComment + specs = append(specs, es) + } + } + } + return specs, nil +} + +// extractEventSpec extracts an EventSpec from a struct type if it has a +// sentinel blank field with an asyncapi tag. Returns ok=false if the struct +// is not annotated. +func extractEventSpec(name string, st *ast.StructType) (EventSpec, bool, error) { + var asyncapiTag string + var dataFields []FieldSpec + + for _, field := range st.Fields.List { + // Sentinel blank field: unnamed or named "_", type struct{} + isSentinel := false + if len(field.Names) == 0 { + isSentinel = true + } else if len(field.Names) == 1 && field.Names[0].Name == "_" { + isSentinel = true + } + + if isSentinel { + if field.Tag == nil { + continue + } + raw := strings.Trim(field.Tag.Value, "`") + tag := reflect.StructTag(raw) + val := tag.Get("asyncapi") + if val != "" { + asyncapiTag = val + } + continue + } + + // Data field + if field.Tag == nil { + continue + } + raw := strings.Trim(field.Tag.Value, "`") + tag := reflect.StructTag(raw) + jsonVal := tag.Get("json") + if jsonVal == "" || jsonVal == "-" { + continue + } + parts := strings.SplitN(jsonVal, ",", 2) + jsonName := parts[0] + omitempty := len(parts) > 1 && strings.Contains(parts[1], "omitempty") + + goType := fieldGoType(field.Type) + required := !omitempty && !strings.HasPrefix(goType, "*") + + description := extractFieldDescription(tag) + + dataFields = append(dataFields, FieldSpec{ + JSONName: jsonName, + GoType: goType, + Required: required, + Description: description, + }) + } + + if asyncapiTag == "" { + return EventSpec{}, false, nil + } + + es, err := parseAsyncAPITag(name, asyncapiTag) + if err != nil { + return EventSpec{}, false, err + } + es.Fields = dataFields + return es, true, nil +} + +// fieldGoType returns a string representation of the Go type expression. +func fieldGoType(expr ast.Expr) string { + switch t := expr.(type) { + case *ast.Ident: + return t.Name + case *ast.StarExpr: + return "*" + fieldGoType(t.X) + case *ast.ArrayType: + return "[]" + fieldGoType(t.Elt) + default: + return "interface{}" + } +} + +// extractFieldDescription extracts the description value from an +// asyncapi-field struct tag. The tag format is: +// +// asyncapi-field:"description:some text here" +func extractFieldDescription(tag reflect.StructTag) string { + val := tag.Get("asyncapi-field") + if val == "" { + return "" + } + const prefix = "description:" + if strings.HasPrefix(val, prefix) { + return val[len(prefix):] + } + return "" +} + +// parseAsyncAPITag parses a comma-separated key:value asyncapi tag string. +// Values may contain spaces but not commas. Multiple param entries are +// supported by repeating the param key. +func parseAsyncAPITag(structName, tag string) (EventSpec, error) { + es := EventSpec{ + StructName: structName, + Params: make(map[string]string), + } + + pairs := strings.Split(tag, ",") + for _, pair := range pairs { + pair = strings.TrimSpace(pair) + if pair == "" { + continue + } + idx := strings.IndexByte(pair, ':') + if idx < 0 { + return EventSpec{}, fmt.Errorf("struct %s: asyncapi tag segment %q missing ':'", structName, pair) + } + key := strings.TrimSpace(pair[:idx]) + val := strings.TrimSpace(pair[idx+1:]) + switch key { + case "channel": + es.Channel = val + case "param": + eqIdx := strings.IndexByte(val, '=') + if eqIdx < 0 { + return EventSpec{}, fmt.Errorf("struct %s: param tag %q missing '='", structName, val) + } + es.Params[val[:eqIdx]] = val[eqIdx+1:] + case "stream": + es.Stream = val + case "type": + es.CEType = val + case "send": + es.SendSummary = val + case "receive": + es.RecvSummary = val + case "description": + es.ChannelDescription = val + } + } + + requiredFields := []struct { + key string + value string + }{ + {"channel", es.Channel}, + {"stream", es.Stream}, + {"type", es.CEType}, + {"send", es.SendSummary}, + {"receive", es.RecvSummary}, + } + var missing []string + for _, rf := range requiredFields { + if rf.value == "" { + missing = append(missing, rf.key) + } + } + if len(missing) > 0 { + return EventSpec{}, fmt.Errorf("struct %s: asyncapi tag missing required keys: %s", structName, strings.Join(missing, ", ")) + } + + return es, nil +} diff --git a/cmd/asyncapi-gen/parser_test.go b/cmd/asyncapi-gen/parser_test.go new file mode 100644 index 0000000..f7398b7 --- /dev/null +++ b/cmd/asyncapi-gen/parser_test.go @@ -0,0 +1,172 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +func TestParseFile_ReturnsEventSpec(t *testing.T) { + specs, err := ParseFile("testdata/fixture.go") + if err != nil { + t.Fatalf("ParseFile: %v", err) + } + if len(specs) != 1 { + t.Fatalf("len(specs) = %d, want 1", len(specs)) + } + + s := specs[0] + + if s.StructName != "WidgetCreatedData" { + t.Errorf("StructName = %q, want %q", s.StructName, "WidgetCreatedData") + } + if s.Channel != "core.widget.created.{ownerId}" { + t.Errorf("Channel = %q, want %q", s.Channel, "core.widget.created.{ownerId}") + } + if s.Params["ownerId"] != "The widget owner identifier" { + t.Errorf("Params[ownerId] = %q, want %q", s.Params["ownerId"], "The widget owner identifier") + } + if s.Stream != "WIDGETS" { + t.Errorf("Stream = %q, want %q", s.Stream, "WIDGETS") + } + if s.CEType != "dev.example.widget.created" { + t.Errorf("CEType = %q, want %q", s.CEType, "dev.example.widget.created") + } + if s.SendSummary != "Published when a widget is created" { + t.Errorf("SendSummary = %q, want %q", s.SendSummary, "Published when a widget is created") + } + if s.RecvSummary != "Consume widget-created events" { + t.Errorf("RecvSummary = %q, want %q", s.RecvSummary, "Consume widget-created events") + } + if s.ChannelDescription != "Widget creation pipeline" { + t.Errorf("ChannelDescription = %q, want %q", s.ChannelDescription, "Widget creation pipeline") + } + if s.DocComment != "WidgetCreatedData is the payload for widget.created events." { + t.Errorf("DocComment = %q, want %q", s.DocComment, "WidgetCreatedData is the payload for widget.created events.") + } +} + +func TestParseFile_ReturnsFields(t *testing.T) { + specs, err := ParseFile("testdata/fixture.go") + if err != nil { + t.Fatalf("ParseFile: %v", err) + } + + fields := specs[0].Fields + + // sentinel blank field must be excluded + for _, f := range fields { + if f.JSONName == "" || f.JSONName == "_" { + t.Errorf("sentinel field leaked into Fields: %+v", f) + } + } + + // widgetId — required (no omitempty, not pointer), has description + widgetID := findField(fields, "widgetId") + if widgetID == nil { + t.Fatal("field widgetId not found") + } + if !widgetID.Required { + t.Error("widgetId should be required") + } + if widgetID.GoType != "string" { + t.Errorf("widgetId GoType = %q, want %q", widgetID.GoType, "string") + } + if widgetID.Description != "Unique widget identifier" { + t.Errorf("widgetId Description = %q, want %q", widgetID.Description, "Unique widget identifier") + } + + // tag — optional (omitempty), no description tag + tag := findField(fields, "tag") + if tag == nil { + t.Fatal("field tag not found") + } + if tag.Required { + t.Error("tag should not be required (omitempty)") + } + if tag.Description != "" { + t.Errorf("tag Description = %q, want empty (no asyncapi-field tag)", tag.Description) + } + + // parentId — optional (pointer) + parentID := findField(fields, "parentId") + if parentID == nil { + t.Fatal("field parentId not found") + } + if parentID.Required { + t.Error("parentId should not be required (pointer)") + } + if parentID.GoType != "*string" { + t.Errorf("parentId GoType = %q, want %q", parentID.GoType, "*string") + } +} + +func TestParseFile_MissingRequiredTag_ReturnsError(t *testing.T) { + // Write a temp file with a missing required tag key + content := `package testdata +type BadData struct { + _ struct{} ` + "`" + `asyncapi:"channel:core.bad.{id}"` + "`" + ` + Name string ` + "`" + `json:"name"` + "`" + ` +}` + path := filepath.Join(t.TempDir(), "bad.go") + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { //nolint:gosec // 0o644 is correct for test fixture files (SC-005) + t.Fatalf("WriteFile: %v", err) + } + _, err := ParseFile(path) + if err == nil { + t.Error("expected error for missing required tag keys") + } + if !strings.Contains(err.Error(), "missing required keys") { + t.Errorf("error = %q, want mention of missing required keys", err) + } +} + +func TestParseFile_ColonlessSegment_ReturnsError(t *testing.T) { + content := `package testdata +type BadData struct { + _ struct{} ` + "`" + `asyncapi:"channel:x,ORPHAN,stream:S,type:t,send:s,receive:r"` + "`" + ` + Name string ` + "`" + `json:"name"` + "`" + ` +}` + path := filepath.Join(t.TempDir(), "bad_colon.go") + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { //nolint:gosec // 0o644 is correct for test fixture files (SC-005) + t.Fatalf("WriteFile: %v", err) + } + _, err := ParseFile(path) + if err == nil { + t.Error("expected error for colonless asyncapi tag segment") + } + if !strings.Contains(err.Error(), "missing ':'") { + t.Errorf("error = %q, want mention of missing ':'", err) + } +} + +func TestParseFile_MalformedParam_ReturnsError(t *testing.T) { + content := `package testdata +type BadData struct { + _ struct{} ` + "`" + `asyncapi:"channel:x,param:noeq,stream:S,type:t,send:s,receive:r"` + "`" + ` + Name string ` + "`" + `json:"name"` + "`" + ` +}` + path := filepath.Join(t.TempDir(), "bad_param.go") + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { //nolint:gosec // 0o644 is correct for test fixture files (SC-005) + t.Fatalf("WriteFile: %v", err) + } + _, err := ParseFile(path) + if err == nil { + t.Error("expected error for malformed param tag") + } + if !strings.Contains(err.Error(), "missing '='") { + t.Errorf("error = %q, want mention of missing '='", err) + } +} + +func findField(fields []FieldSpec, jsonName string) *FieldSpec { + for i := range fields { + if fields[i].JSONName == jsonName { + return &fields[i] + } + } + return nil +} diff --git a/cmd/asyncapi-gen/schema.go b/cmd/asyncapi-gen/schema.go new file mode 100644 index 0000000..d954a75 --- /dev/null +++ b/cmd/asyncapi-gen/schema.go @@ -0,0 +1,328 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "fmt" + "net/url" + "strings" +) + +// AsyncAPIDoc is the top-level AsyncAPI 3.0 document model. +type AsyncAPIDoc struct { + AsyncAPI string `yaml:"asyncapi"` + Info Info `yaml:"info"` + DefaultContentType string `yaml:"defaultContentType"` + Servers map[string]Server `yaml:"servers"` + Channels map[string]Channel `yaml:"channels"` + Operations map[string]Operation `yaml:"operations"` + Components Components `yaml:"components"` +} + +// Info holds document metadata. +type Info struct { + Title string `yaml:"title"` + Version string `yaml:"version"` + Description string `yaml:"description,omitempty"` + License *License `yaml:"license,omitempty"` + Contact *Contact `yaml:"contact,omitempty"` +} + +// License holds the license information for the AsyncAPI document. +type License struct { + Name string `yaml:"name"` +} + +// Contact holds the contact information for the AsyncAPI document. +type Contact struct { + Name string `yaml:"name,omitempty"` + URL string `yaml:"url,omitempty"` +} + +// Server describes a NATS server. +type Server struct { + Host string `yaml:"host"` + Protocol string `yaml:"protocol"` + Description string `yaml:"description,omitempty"` +} + +// Channel describes a NATS subject channel. +type Channel struct { + Address string `yaml:"address"` + Description string `yaml:"description,omitempty"` + Parameters map[string]Parameter `yaml:"parameters,omitempty"` + Messages map[string]Ref `yaml:"messages"` +} + +// Parameter describes a channel address parameter. +type Parameter struct { + Description string `yaml:"description"` +} + +// Ref is an AsyncAPI $ref object. +type Ref struct { + Ref string `yaml:"$ref"` +} + +// Operation describes a send or receive operation. +type Operation struct { + Action string `yaml:"action"` + Summary string `yaml:"summary"` + Channel Ref `yaml:"channel"` + Bindings OperationBinding `yaml:"bindings,omitempty"` +} + +// OperationBinding holds protocol-specific operation bindings. +type OperationBinding struct { + NATS NATSOperationBinding `yaml:"nats,omitempty"` +} + +// NATSOperationBinding holds NATS JetStream stream metadata. +// Stream is serialised as the AsyncAPI extension field x-stream because the +// official NATS binding 0.1.0 schema does not define a stream property; +// x-prefixed extensions are accepted by the AsyncAPI validator. +type NATSOperationBinding struct { + Stream string `yaml:"x-stream,omitempty"` + BindingVersion string `yaml:"bindingVersion,omitempty"` +} + +// Components holds reusable AsyncAPI components. +type Components struct { + Messages map[string]Message `yaml:"messages"` + Schemas map[string]Schema `yaml:"schemas"` +} + +// Message describes an AsyncAPI message. +type Message struct { + Name string `yaml:"name"` + Title string `yaml:"title"` + ContentType string `yaml:"contentType"` + Payload Ref `yaml:"payload"` +} + +// Schema is a simplified JSON Schema object for AsyncAPI. +type Schema struct { + Type string `yaml:"type,omitempty"` + Description string `yaml:"description,omitempty"` + Required []string `yaml:"required,omitempty"` + Properties map[string]Schema `yaml:"properties,omitempty"` + Const string `yaml:"const,omitempty"` + Format string `yaml:"format,omitempty"` + Ref string `yaml:"$ref,omitempty"` +} + +// DocMeta holds document-level metadata for the generated AsyncAPI spec. +type DocMeta struct { + Title string + Version string + Description string + LicenseName string + ContactName string + ContactURL string + ServerURL string +} + +// BuildDoc constructs an AsyncAPIDoc from the given specs and document metadata. +func BuildDoc(specs []EventSpec, meta DocMeta) AsyncAPIDoc { + info := Info{Title: meta.Title, Version: meta.Version} + if meta.Description != "" { + info.Description = meta.Description + } + if meta.LicenseName != "" { + info.License = &License{Name: meta.LicenseName} + } + if meta.ContactName != "" || meta.ContactURL != "" { + info.Contact = &Contact{Name: meta.ContactName, URL: meta.ContactURL} + } + + doc := AsyncAPIDoc{ + AsyncAPI: "3.0.0", + Info: info, + DefaultContentType: "application/cloudevents+json", + Servers: buildServers(meta.ServerURL), + Channels: make(map[string]Channel), + Operations: make(map[string]Operation), + Components: Components{ + Messages: make(map[string]Message), + Schemas: make(map[string]Schema), + }, + } + + for _, spec := range specs { + chKey := channelName(spec.StructName) + msgKey := messageKey(spec.StructName) + envSchemaKey := envelopeSchemaName(spec.StructName) + dataSchemaKey := spec.StructName + + // Channel + params := make(map[string]Parameter) + for k, v := range spec.Params { + params[k] = Parameter{Description: v} + } + doc.Channels[chKey] = Channel{ + Address: spec.Channel, + Description: spec.ChannelDescription, + Parameters: params, + Messages: map[string]Ref{ + msgKey: {Ref: fmt.Sprintf("#/components/messages/%s", msgKey)}, + }, + } + + // Send operation + sendKey := "publish" + upperFirst(chKey) + doc.Operations[sendKey] = Operation{ + Action: "send", + Summary: spec.SendSummary, + Channel: Ref{Ref: fmt.Sprintf("#/channels/%s", chKey)}, + Bindings: OperationBinding{ + NATS: NATSOperationBinding{Stream: spec.Stream, BindingVersion: "0.1.0"}, + }, + } + + // Receive operation + recvKey := "consume" + upperFirst(chKey) + doc.Operations[recvKey] = Operation{ + Action: "receive", + Summary: spec.RecvSummary, + Channel: Ref{Ref: fmt.Sprintf("#/channels/%s", chKey)}, + } + + // Message + doc.Components.Messages[msgKey] = Message{ + Name: msgKey, + Title: humanTitle(spec.StructName), + ContentType: "application/cloudevents+json", + Payload: Ref{Ref: fmt.Sprintf("#/components/schemas/%s", envSchemaKey)}, + } + + // CloudEvents envelope schema + doc.Components.Schemas[envSchemaKey] = buildEnvelopeSchema(spec) + + // Data schema + doc.Components.Schemas[dataSchemaKey] = buildDataSchema(spec) + } + + return doc +} + +// buildServers parses the server URL and returns the servers map. +func buildServers(rawURL string) map[string]Server { + u, err := url.Parse(rawURL) + if err != nil || u.Host == "" { + return map[string]Server{"nats": {Host: rawURL, Protocol: "nats"}} + } + return map[string]Server{ + "nats": { + Host: u.Host, + Protocol: u.Scheme, + }, + } +} + +// buildEnvelopeSchema returns the CloudEvents envelope schema for a spec. +func buildEnvelopeSchema(spec EventSpec) Schema { + return Schema{ + Type: "object", + Description: fmt.Sprintf("CloudEvents v1.0 envelope for %s", spec.CEType), + Required: []string{"specversion", "id", "type", "source", "subject", "time", "datacontenttype", "data"}, + Properties: map[string]Schema{ + "specversion": {Type: "string", Const: "1.0"}, + "id": {Type: "string", Format: "uuid"}, + "type": {Type: "string", Const: spec.CEType}, + "source": {Type: "string", Description: "URI identifying the producing service"}, + "subject": {Type: "string", Description: "The compliance subject identifier"}, + "time": {Type: "string", Format: "date-time"}, + "datacontenttype": {Type: "string", Const: "application/json"}, + "data": {Ref: fmt.Sprintf("#/components/schemas/%s", spec.StructName)}, + }, + } +} + +// buildDataSchema builds the data payload schema from struct fields. +func buildDataSchema(spec EventSpec) Schema { + props := make(map[string]Schema) + var required []string + + for _, f := range spec.Fields { + s := Schema{Type: goTypeToJSONSchema(f.GoType)} + if f.Description != "" { + s.Description = f.Description + } + props[f.JSONName] = s + if f.Required { + required = append(required, f.JSONName) + } + } + + s := Schema{ + Type: "object", + Required: required, + Properties: props, + } + if spec.DocComment != "" { + s.Description = spec.DocComment + } + return s +} + +// goTypeToJSONSchema maps Go type strings to JSON Schema type strings. +// Slice types are mapped to "array" and nested struct types to "object". +// Qualified types (e.g. time.Time) and interface{} map to "object". +func goTypeToJSONSchema(goType string) string { + base := strings.TrimPrefix(goType, "*") + if strings.HasPrefix(base, "[]") { + return "array" + } + switch base { + case "string": + return "string" + case "int", "int32", "int64": + return "integer" + case "float32", "float64": + return "number" + case "bool": + return "boolean" + default: + return "object" + } +} + +// channelName converts a struct name like "EvidenceIngestedData" to "evidenceIngested". +func channelName(structName string) string { + name := strings.TrimSuffix(structName, "Data") + if len(name) == 0 { + return structName + } + return strings.ToLower(name[:1]) + name[1:] +} + +// messageKey converts a struct name like "EvidenceIngestedData" to "EvidenceIngested". +func messageKey(structName string) string { + return strings.TrimSuffix(structName, "Data") +} + +// humanTitle converts a struct name like "EvidenceIngestedData" to "Evidence Ingested". +func humanTitle(structName string) string { + name := strings.TrimSuffix(structName, "Data") + if len(name) == 0 { + return structName + } + var parts []string + start := 0 + for i := 1; i < len(name); i++ { + if name[i] >= 'A' && name[i] <= 'Z' { + parts = append(parts, name[start:i]) + start = i + } + } + parts = append(parts, name[start:]) + return strings.Join(parts, " ") +} + +// upperFirst uppercases the first letter of s. +func upperFirst(s string) string { + if s == "" { + return s + } + return strings.ToUpper(s[:1]) + s[1:] +} diff --git a/cmd/asyncapi-gen/schema_test.go b/cmd/asyncapi-gen/schema_test.go new file mode 100644 index 0000000..6d118df --- /dev/null +++ b/cmd/asyncapi-gen/schema_test.go @@ -0,0 +1,371 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "testing" +) + +func testMeta() DocMeta { + return DocMeta{Title: "Test API", Version: "1.0.0", ServerURL: "nats://localhost:4222"} +} + +func singleSpec() EventSpec { + return EventSpec{ + StructName: "WidgetCreatedData", + Channel: "core.widget.created.{ownerId}", + Params: map[string]string{"ownerId": "The widget owner identifier"}, + Stream: "WIDGETS", + CEType: "dev.example.widget.created", + SendSummary: "Published when a widget is created", + RecvSummary: "Consume widget-created events", + ChannelDescription: "Widget creation pipeline", + DocComment: "WidgetCreatedData is the payload for widget.created events.", + Fields: []FieldSpec{ + {JSONName: "widgetId", GoType: "string", Required: true, Description: "Unique widget identifier"}, + {JSONName: "name", GoType: "string", Required: true}, + {JSONName: "tag", GoType: "string", Required: false}, + {JSONName: "parentId", GoType: "*string", Required: false, Description: "Parent widget ID"}, + }, + } +} + +func TestBuildDoc_InfoFields(t *testing.T) { + doc := BuildDoc([]EventSpec{singleSpec()}, testMeta()) + + if doc.AsyncAPI != "3.0.0" { + t.Errorf("AsyncAPI = %q, want %q", doc.AsyncAPI, "3.0.0") + } + if doc.Info.Title != "Test API" { + t.Errorf("Title = %q, want %q", doc.Info.Title, "Test API") + } + if doc.Info.Version != "1.0.0" { + t.Errorf("Version = %q, want %q", doc.Info.Version, "1.0.0") + } + if doc.DefaultContentType != "application/cloudevents+json" { + t.Errorf("DefaultContentType = %q, want %q", doc.DefaultContentType, "application/cloudevents+json") + } +} + +func TestBuildDoc_ServerURL(t *testing.T) { + doc := BuildDoc([]EventSpec{singleSpec()}, testMeta()) + + if len(doc.Servers) != 1 { + t.Fatalf("len(Servers) = %d, want 1", len(doc.Servers)) + } + srv := doc.Servers["nats"] + if srv.Host != "localhost:4222" { + t.Errorf("Host = %q, want %q", srv.Host, "localhost:4222") + } + if srv.Protocol != "nats" { + t.Errorf("Protocol = %q, want %q", srv.Protocol, "nats") + } +} + +func TestBuildDoc_Channel(t *testing.T) { + doc := BuildDoc([]EventSpec{singleSpec()}, testMeta()) + + ch, ok := doc.Channels["widgetCreated"] + if !ok { + t.Fatal("channel widgetCreated not found") + } + if ch.Address != "core.widget.created.{ownerId}" { + t.Errorf("Address = %q, want %q", ch.Address, "core.widget.created.{ownerId}") + } + param, ok := ch.Parameters["ownerId"] + if !ok { + t.Fatal("parameter ownerId not found") + } + if param.Description != "The widget owner identifier" { + t.Errorf("param description = %q, want %q", param.Description, "The widget owner identifier") + } +} + +func TestBuildDoc_ChannelDescription(t *testing.T) { + doc := BuildDoc([]EventSpec{singleSpec()}, testMeta()) + + ch, ok := doc.Channels["widgetCreated"] + if !ok { + t.Fatal("channel widgetCreated not found") + } + if ch.Description != "Widget creation pipeline" { + t.Errorf("channel description = %q, want %q", ch.Description, "Widget creation pipeline") + } +} + +func TestBuildDoc_ChannelDescription_Empty(t *testing.T) { + spec := singleSpec() + spec.ChannelDescription = "" + doc := BuildDoc([]EventSpec{spec}, testMeta()) + + ch := doc.Channels["widgetCreated"] + if ch.Description != "" { + t.Errorf("channel description = %q, want empty", ch.Description) + } +} + +func TestBuildDoc_DataSchemaDescription(t *testing.T) { + doc := BuildDoc([]EventSpec{singleSpec()}, testMeta()) + + dataSchema, ok := doc.Components.Schemas["WidgetCreatedData"] + if !ok { + t.Fatal("schema WidgetCreatedData not found") + } + if dataSchema.Description != "WidgetCreatedData is the payload for widget.created events." { + t.Errorf("data schema description = %q, want %q", dataSchema.Description, "WidgetCreatedData is the payload for widget.created events.") + } +} + +func TestBuildDoc_DataSchemaDescription_Empty(t *testing.T) { + spec := singleSpec() + spec.DocComment = "" + doc := BuildDoc([]EventSpec{spec}, testMeta()) + + dataSchema := doc.Components.Schemas["WidgetCreatedData"] + if dataSchema.Description != "" { + t.Errorf("data schema description = %q, want empty", dataSchema.Description) + } +} + +func TestBuildDoc_Operations(t *testing.T) { + doc := BuildDoc([]EventSpec{singleSpec()}, testMeta()) + + sendOp, ok := doc.Operations["publishWidgetCreated"] + if !ok { + t.Fatal("operation publishWidgetCreated not found") + } + if sendOp.Action != "send" { + t.Errorf("send action = %q, want %q", sendOp.Action, "send") + } + if sendOp.Summary != "Published when a widget is created" { + t.Errorf("send summary = %q, want %q", sendOp.Summary, "Published when a widget is created") + } + + recvOp, ok := doc.Operations["consumeWidgetCreated"] + if !ok { + t.Fatal("operation consumeWidgetCreated not found") + } + if recvOp.Action != "receive" { + t.Errorf("receive action = %q, want %q", recvOp.Action, "receive") + } + if recvOp.Bindings.NATS.Stream != "" { + t.Errorf("receive op NATS stream = %q, want empty (no binding)", recvOp.Bindings.NATS.Stream) + } +} + +func TestBuildDoc_DataSchemaFields(t *testing.T) { + doc := BuildDoc([]EventSpec{singleSpec()}, testMeta()) + + dataSchema, ok := doc.Components.Schemas["WidgetCreatedData"] + if !ok { + t.Fatal("schema WidgetCreatedData not found") + } + + widgetIDProp, ok := dataSchema.Properties["widgetId"] + if !ok { + t.Fatal("property widgetId not found") + } + if widgetIDProp.Type != "string" { + t.Errorf("widgetId type = %q, want %q", widgetIDProp.Type, "string") + } + if widgetIDProp.Description != "Unique widget identifier" { + t.Errorf("widgetId description = %q, want %q", widgetIDProp.Description, "Unique widget identifier") + } + + // name has no description tag — should be empty + nameProp, ok := dataSchema.Properties["name"] + if !ok { + t.Fatal("property name not found") + } + if nameProp.Description != "" { + t.Errorf("name description = %q, want empty", nameProp.Description) + } + + // widgetId and name must be in required list + required := map[string]bool{} + for _, r := range dataSchema.Required { + required[r] = true + } + if !required["widgetId"] { + t.Error("widgetId should be required") + } + if !required["name"] { + t.Error("name should be required") + } + if required["tag"] { + t.Error("tag should not be required") + } + if required["parentId"] { + t.Error("parentId should not be required") + } +} + +func TestBuildDoc_CloudEventsEnvelope(t *testing.T) { + doc := BuildDoc([]EventSpec{singleSpec()}, testMeta()) + + env, ok := doc.Components.Schemas["WidgetCreatedCloudEvent"] + if !ok { + t.Fatal("schema WidgetCreatedCloudEvent not found") + } + + specversion, ok := env.Properties["specversion"] + if !ok { + t.Fatal("specversion property not found") + } + if specversion.Const != "1.0" { + t.Errorf("specversion const = %q, want %q", specversion.Const, "1.0") + } + + ceType, ok := env.Properties["type"] + if !ok { + t.Fatal("type property not found") + } + if ceType.Const != "dev.example.widget.created" { + t.Errorf("type const = %q, want %q", ceType.Const, "dev.example.widget.created") + } +} + +func TestBuildDoc_NATSBinding(t *testing.T) { + doc := BuildDoc([]EventSpec{singleSpec()}, testMeta()) + + op, ok := doc.Operations["publishWidgetCreated"] + if !ok { + t.Fatal("operation publishWidgetCreated not found") + } + if op.Bindings.NATS.Stream != "WIDGETS" { + t.Errorf("NATS stream = %q, want %q", op.Bindings.NATS.Stream, "WIDGETS") + } +} + +func TestGoTypeToJSONSchema(t *testing.T) { + tests := []struct { + goType string + want string + }{ + {"string", "string"}, + {"*string", "string"}, + {"int", "integer"}, + {"int32", "integer"}, + {"int64", "integer"}, + {"float32", "number"}, + {"float64", "number"}, + {"bool", "boolean"}, + {"SomeStruct", "object"}, + {"*int", "integer"}, + {"*bool", "boolean"}, + {"[]string", "array"}, + {"[]*int", "array"}, + {"interface{}", "object"}, + } + for _, tt := range tests { + t.Run(tt.goType, func(t *testing.T) { + got := goTypeToJSONSchema(tt.goType) + if got != tt.want { + t.Errorf("goTypeToJSONSchema(%q) = %q, want %q", tt.goType, got, tt.want) + } + }) + } +} + +func TestBuildServers_ValidURL(t *testing.T) { + servers := buildServers("nats://myhost:4222") + srv, ok := servers["nats"] + if !ok { + t.Fatal("nats server not found") + } + if srv.Host != "myhost:4222" { + t.Errorf("Host = %q, want %q", srv.Host, "myhost:4222") + } + if srv.Protocol != "nats" { + t.Errorf("Protocol = %q, want %q", srv.Protocol, "nats") + } +} + +func TestBuildServers_MalformedURL(t *testing.T) { + // URL without scheme — Host will be empty, falls back to raw string + servers := buildServers("host:4222") + srv, ok := servers["nats"] + if !ok { + t.Fatal("nats server not found") + } + // Fallback: raw URL used as host + if srv.Host != "host:4222" { + t.Errorf("Host = %q, want %q", srv.Host, "host:4222") + } + if srv.Protocol != "nats" { + t.Errorf("Protocol = %q, want %q", srv.Protocol, "nats") + } +} + +func TestBuildServers_EmptyURL(t *testing.T) { + servers := buildServers("") + srv, ok := servers["nats"] + if !ok { + t.Fatal("nats server not found") + } + if srv.Host != "" { + t.Errorf("Host = %q, want empty", srv.Host) + } + if srv.Protocol != "nats" { + t.Errorf("Protocol = %q, want %q", srv.Protocol, "nats") + } +} + +func TestChannelName(t *testing.T) { + tests := []struct { + in, want string + }{ + {"EvidenceIngestedData", "evidenceIngested"}, + {"WidgetCreatedData", "widgetCreated"}, + // edge: name becomes empty after trim, returns original + {"Data", "Data"}, + {"SimpleData", "simple"}, + } + for _, tt := range tests { + t.Run(tt.in, func(t *testing.T) { + got := channelName(tt.in) + if got != tt.want { + t.Errorf("channelName(%q) = %q, want %q", tt.in, got, tt.want) + } + }) + } +} + +func TestHumanTitle(t *testing.T) { + tests := []struct { + in, want string + }{ + {"EvidenceIngestedData", "Evidence Ingested"}, + {"WidgetCreatedData", "Widget Created"}, + {"SimpleData", "Simple"}, + // edge: name becomes empty after trim, returns original + {"Data", "Data"}, + } + for _, tt := range tests { + t.Run(tt.in, func(t *testing.T) { + got := humanTitle(tt.in) + if got != tt.want { + t.Errorf("humanTitle(%q) = %q, want %q", tt.in, got, tt.want) + } + }) + } +} + +func TestUpperFirst(t *testing.T) { + tests := []struct { + in, want string + }{ + {"hello", "Hello"}, + {"Hello", "Hello"}, + {"", ""}, + {"a", "A"}, + } + for _, tt := range tests { + t.Run(tt.in, func(t *testing.T) { + got := upperFirst(tt.in) + if got != tt.want { + t.Errorf("upperFirst(%q) = %q, want %q", tt.in, got, tt.want) + } + }) + } +} diff --git a/cmd/asyncapi-gen/testdata/fixture.go b/cmd/asyncapi-gen/testdata/fixture.go new file mode 100644 index 0000000..ab8e184 --- /dev/null +++ b/cmd/asyncapi-gen/testdata/fixture.go @@ -0,0 +1,13 @@ +// SPDX-License-Identifier: Apache-2.0 + +package testdata + +// WidgetCreatedData is the payload for widget.created events. +type WidgetCreatedData struct { + _ struct{} `asyncapi:"channel:core.widget.created.{ownerId},param:ownerId=The widget owner identifier,stream:WIDGETS,type:dev.example.widget.created,send:Published when a widget is created,receive:Consume widget-created events,description:Widget creation pipeline"` + + WidgetID string `json:"widgetId" asyncapi-field:"description:Unique widget identifier"` + Name string `json:"name"` + Tag string `json:"tag,omitempty"` + ParentID *string `json:"parentId,omitempty" asyncapi-field:"description:Parent widget ID"` +} diff --git a/cmd/asyncapi-gen/writer.go b/cmd/asyncapi-gen/writer.go new file mode 100644 index 0000000..fe0a191 --- /dev/null +++ b/cmd/asyncapi-gen/writer.go @@ -0,0 +1,31 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "fmt" + "os" + "path/filepath" + + "gopkg.in/yaml.v3" +) + +// WriteYAML marshals doc to YAML and writes it to path, prepending the +// SPDX license header. The file is created with 0o644 permissions. +func WriteYAML(doc AsyncAPIDoc, path string) error { + b, err := yaml.Marshal(doc) + if err != nil { + return fmt.Errorf("marshaling AsyncAPI document: %w", err) + } + + header := "# SPDX-License-Identifier: Apache-2.0\n" + out := append([]byte(header), b...) + + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { //nolint:gosec // 0o755 is correct for output directories (SC-005) + return fmt.Errorf("creating output directory: %w", err) + } + if err := os.WriteFile(path, out, 0o644); err != nil { //nolint:gosec // 0o644 is correct for generated YAML output files (SC-005) + return fmt.Errorf("writing output file: %w", err) + } + return nil +} diff --git a/cmd/asyncapi-gen/writer_test.go b/cmd/asyncapi-gen/writer_test.go new file mode 100644 index 0000000..e0cf87d --- /dev/null +++ b/cmd/asyncapi-gen/writer_test.go @@ -0,0 +1,59 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +func TestWriteYAML_CreatesFile(t *testing.T) { + doc := BuildDoc([]EventSpec{singleSpec()}, testMeta()) + out := filepath.Join(t.TempDir(), "asyncapi.yaml") + + if err := WriteYAML(doc, out); err != nil { + t.Fatalf("WriteYAML: %v", err) + } + + b, err := os.ReadFile(out) + if err != nil { + t.Fatalf("ReadFile: %v", err) + } + content := string(b) + + if !strings.Contains(content, "asyncapi: 3.0.0") { + t.Error("output missing 'asyncapi: 3.0.0'") + } + if !strings.Contains(content, "application/cloudevents+json") { + t.Error("output missing defaultContentType") + } + if !strings.Contains(content, "core.widget.created.{ownerId}") { + t.Error("output missing channel address") + } + if !strings.Contains(content, "WIDGETS") { + t.Error("output missing NATS stream name") + } + if !strings.Contains(content, "specversion") { + t.Error("output missing CloudEvents envelope field") + } +} + +func TestWriteYAML_SpdxHeader(t *testing.T) { + doc := BuildDoc([]EventSpec{singleSpec()}, testMeta()) + out := filepath.Join(t.TempDir(), "asyncapi.yaml") + + if err := WriteYAML(doc, out); err != nil { + t.Fatalf("WriteYAML: %v", err) + } + + b, err := os.ReadFile(out) + if err != nil { + t.Fatalf("ReadFile: %v", err) + } + + if !strings.HasPrefix(string(b), "# SPDX-License-Identifier: Apache-2.0") { + t.Error("output missing SPDX header") + } +} diff --git a/docs/versioning.md b/docs/versioning.md new file mode 100644 index 0000000..ba9543e --- /dev/null +++ b/docs/versioning.md @@ -0,0 +1,139 @@ + +# Event Versioning Strategy + +This document defines how ComplyTime event contracts evolve without breaking +existing subscribers. It tells producers which field to change for a given kind +of change, and tells consumers what they must handle. Read it before making any +change to a type in [`events/events.go`](../events/events.go). + +The strategy follows the accepted ADRs in `complytime/complytime`: +[ADR-0019](https://github.com/complytime/complytime/blob/main/docs/ADRs/0019-event-driven-ingestion.md) +(event-driven ingestion), +[ADR-0020](https://github.com/complytime/complytime/blob/main/docs/ADRs/0020-nats-jetstream-event-bus.md) +(NATS JetStream), +[ADR-0021](https://github.com/complytime/complytime/blob/main/docs/ADRs/0021-cloudevents-envelope.md) +(CloudEvents envelope), and +[ADR-0022](https://github.com/complytime/complytime/blob/main/docs/ADRs/0022-asyncapi-interface-docs.md) +(AsyncAPI contract). + +## The four layers + +Each layer versions a different thing. Changing the wrong one either breaks +subscribers or signals nothing. + +| Layer | Field | Example | Versions | Reacts | +|-------|-------|---------|----------|--------| +| NATS subject | channel address | `core.evidence.ingested.{subjectId}` | Nothing — routing address only | Broker (routing) | +| CloudEvents type | `type` | `dev.complytime.evidence.ingested` | The event's payload contract (breaking changes) | Consumer per-message dispatch | +| AsyncAPI contract | `info.version` | `0.1.0` | The whole published contract | Humans, codegen tooling | +| CloudEvents envelope | `specversion` | `1.0` | The CNCF envelope format — not ours | Nobody (hands off) | + +### NATS subject — frozen routing address + +The subject is where a subscriber listens (ADR-0020, `domain.action.entity` +hierarchy). It MUST NOT carry a version segment. Version information belongs in +the CloudEvents `type` (see below), per ADR-0021, which routes and filters on +`type` and `source`. + +Keeping version out of the subject means subscribers bind once, with a wildcard, +and never re-subscribe: + +``` +core.evidence.ingested.* # every subjectId +core.evidence.ingested.> # every subjectId and any deeper segments +``` + +That single binding survives both a new `subjectId` and a new payload version, +because neither changes the subject. + +### CloudEvents `type` — the breaking-change signal + +The `type` attribute identifies the event and its payload contract. A breaking +payload change bumps `type`; consumers switch on it: + +```go +switch e.Type() { +case "dev.complytime.evidence.ingested": // v1 + handleV1(e) +case "dev.complytime.evidence.ingested.v2": // v2 + handleV2(e) +} +``` + +Both versions flow on the same subject. A v1-only consumer keeps matching the v1 +`type` and ignores v2 messages; a v2 consumer handles both. No coordinated +cutover (ADR-0019: the API must not need to change when a consumer is added or +removed). + +### AsyncAPI `info.version` — the human and codegen contract + +The [AsyncAPI spec](../api/events/asyncapi.yaml) is the public, machine-readable +contract (ADR-0022). Consumers integrate from it and pin codegen to +`info.version`. It is generated from the Go types, so bump it in the +`//go:generate` directive in [`events/events.go`](../events/events.go), then +run `go generate ./events/...`. + +### CloudEvents `specversion` — do not touch + +`specversion` is the CNCF CloudEvents envelope version (currently `1.0`). It +changes only if CloudEvents itself releases a new envelope format. It is never +your schema-change signal. Do not conflate it with `type`. + +## Pre-1.0 stability + +While `info.version` is below `1.0.0`, the contract is considered unstable and +may change in place without a version bump. No downstream consumers should pin +to a pre-1.0 contract version for codegen stability. Once the contract reaches +`1.0.0`, the decision rules below become mandatory. + +## Decision rule + +Classify the change, then act: + +### Additive / non-breaking + +Adding an optional field, a new enum value, or a new event type. + +1. Bump `info.version` minor (`0.1.0` → `0.2.0`) in the `//go:generate` + directive in [`events/events.go`](../events/events.go). +2. Leave `type`, `specversion`, and the subject unchanged. +3. Run `go generate ./events/...` and commit the regenerated artifacts. + +Old consumers ignore the new field (tolerant JSON reader). Nothing breaks. + +### Breaking + +Removing or renaming a field, changing a field's type, or changing the required +set. + +1. Bump the CloudEvents `type` with a version suffix + (`dev.complytime.evidence.ingested` → `dev.complytime.evidence.ingested.v2`). + Update **both** the `Type…` constant and the `type:` key inside the `asyncapi` + sentinel tag on the corresponding `*Data` struct in + [`events/events.go`](../events/events.go). The generated `const` in + `api/events/asyncapi.yaml` and `api/events/schemas/*.schema.json` comes from + the tag, not the constant. +2. Bump `info.version` major (`1.0.0` → `2.0.0`) in the `//go:generate` + directive. +3. Keep the subject unchanged. +4. Producer emits the new `type` on the same subject. Retire the old `type` only + after consumers have migrated. +5. Run `go generate ./events/...` and commit the regenerated artifacts. + +## Consumer contract + +To stay compatible across versions, a subscriber MUST: + +- Bind the subject with a wildcard, not a fixed `subjectId`. +- Dispatch on the CloudEvents `type`, not on subject text. +- Treat unknown fields as ignorable (tolerant reader). +- Treat an unknown `type` as skip-or-log, never as a fatal error. + +## Reference + +- Source of truth: [`events/events.go`](../events/events.go) +- Generated contract: [`api/events/asyncapi.yaml`](../api/events/asyncapi.yaml), + [`api/events/schemas/`](../api/events/schemas/) +- CI validation: [`.github/workflows/ci_asyncapi.yml`](../.github/workflows/ci_asyncapi.yml) +- [CloudEvents specification](https://cloudevents.io/) +- [AsyncAPI specification](https://www.asyncapi.com/) diff --git a/events/events.go b/events/events.go index 0341641..41d7454 100644 --- a/events/events.go +++ b/events/events.go @@ -4,6 +4,8 @@ // evidence lifecycle. package events +//go:generate go run ../cmd/asyncapi-gen -input ./events.go -output ../api/events/asyncapi.yaml -schemas-dir ../api/events/schemas -title "ComplyTime API Events" -version 0.1.0 -description "Event contract for the ComplyTime evidence lifecycle.\n\nAll public events use CloudEvents v1.0 envelope (JSON format).\nThis spec is generated from Go types in the events package via cmd/asyncapi-gen.\nDo not edit manually — run 'go generate ./events/...' to regenerate." -license Apache-2.0 -contact-name ComplyTime -contact-url https://github.com/complytime/complyapi -server nats://localhost:4222 + import ( "errors" "time" @@ -18,11 +20,14 @@ const TypeEvidenceIngested = "dev.complytime.evidence.ingested" // EvidenceIngestedData is the CloudEvents data payload for // evidence.ingested events. type EvidenceIngestedData struct { - ContentDigest string `json:"contentDigest"` - ArtifactType string `json:"artifactType"` - StorageRef string `json:"storageRef,omitempty"` - SubjectID string `json:"subjectId"` - ShardID *string `json:"shardId,omitempty"` + //nolint:unused + _ struct{} `asyncapi:"channel:core.evidence.ingested.{subjectId},param:subjectId=The compliance subject identifier,stream:EVIDENCE,type:dev.complytime.evidence.ingested,send:Published when evidence is accepted for processing; before sealing,receive:Consume evidence-ingested events,description:Evidence ingestion pipeline for compliance artifacts"` + + ContentDigest string `json:"contentDigest" asyncapi-field:"description:SHA-256 digest of the evidence artifact"` + ArtifactType string `json:"artifactType" asyncapi-field:"description:Gemara artifact type"` + StorageRef string `json:"storageRef,omitempty" asyncapi-field:"description:Internal storage reference"` + SubjectID string `json:"subjectId" asyncapi-field:"description:Compliance subject identifier"` + ShardID *string `json:"shardId,omitempty" asyncapi-field:"description:Subject shard identifier (null when sharding is not configured)"` } // NewEvidenceIngestedEvent constructs a CloudEvents v1.0 event with diff --git a/go.mod b/go.mod index 3270174..6df5f29 100644 --- a/go.mod +++ b/go.mod @@ -5,6 +5,7 @@ go 1.26.5 require ( github.com/cloudevents/sdk-go/v2 v2.16.2 github.com/google/uuid v1.6.0 + gopkg.in/yaml.v3 v3.0.1 ) require ( diff --git a/go.sum b/go.sum index 9a620ca..b578de5 100644 --- a/go.sum +++ b/go.sum @@ -10,11 +10,15 @@ github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M= github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= +github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs= +github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= @@ -31,5 +35,8 @@ go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8= go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E= golang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE= golang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f h1:BLraFXnmrev5lT+xlilqcH8XK9/i0At2xKjWk4p6zsU= +gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=