diff --git a/.agents/architecture.md b/.agents/architecture.md new file mode 100644 index 00000000..45ce6c84 --- /dev/null +++ b/.agents/architecture.md @@ -0,0 +1,74 @@ +# Architecture + +Read this for language, compiler, runtime, dependency, or project-boundary changes. This describes the checked-out implementation; [ADRs](../docs/adr/README.md) explain its history. + +## Purpose and entry points + +HydraScript is a TypeScript/Go-inspired interpreter with strong static structural typing. The JavaScript-like syntax is its own language: do not assume JavaScript coercion, class semantics, or complete ECMAScript support. + +- [Readme](../Readme.md): language behavior and examples. +- [TokenTypes](../src/Domain/HydraScript.Domain.Constants/TokenTypes.cs): lexical definitions. +- [grammar.txt](../docs/grammar.txt): grammar. +- [Program](../src/HydraScript/Program.cs) and [ExecuteCommand](../src/HydraScript/ExecuteCommand.cs): CLI composition and the input path/`--dump` options. +- [Executor](../src/Infrastructure/HydraScript.Infrastructure/Executor.cs): application execution and exit codes. + +## Execution pipeline + +```text +Program / ExecuteCommand + -> per-invocation ServiceProvider + -> Executor.Invoke + -> ISourceCodeProvider.GetText + -> TopDownParser.Parse -> RegexLexer -> AST + -> CodeGenerator.GetInstructions + -> StaticAnalyzer.Analyze + -> SymbolTableInitializer + -> TypeSystemLoader + -> DeclarationVisitor + -> SemanticChecker + -> InstructionProvider / ExpressionInstructionProvider + -> AddressedInstructions + -> VirtualMachine.Run -> instruction.Execute -> next address +``` + +The pre-analysis order is defined by registrations in [AddStaticAnalysis](../src/Application/HydraScript.Application.StaticAnalysis/ServiceCollectionExtensions.cs). Code generation calls static analysis before emitting instructions; preserve that sequencing. + +## Ownership and dependency rules + +[ADR 0001: OOP, DDD, and Clean Architecture](../docs/adr/0001-oop-ddd-clean-architecture.md#project-boundaries) owns the project-boundary table. Read it before adding references or moving responsibilities. Domain stays independent of Application/Infrastructure/CLI; Application connects the compiler subdomains, and Infrastructure supplies adapters and composition. + +## Constraints to preserve + +- AST nodes describe syntax and expose [Visitor.NET dispatch](../docs/adr/0002-visitor-net-adoption.md); each visitor controls its traversal. Put analysis/emission logic in the corresponding visitors. Review parent/scope propagation and visitors when adding nodes. +- [Structural type rules](../docs/adr/0006-structural-type-system.md) govern compatibility and operator checking. Type declarations are built before references are resolved; overload lookup uses typed symbol IDs and signatures. +- [Addressed instructions and the VM](../docs/adr/0003-addressed-instructions-and-virtual-machine.md) separate instruction identity from position; control flow follows returned addresses. Review jump targets and address identity when editing instruction collections. +- Runtime `IValue.Get()` still returns `object?`. Do not describe the backend as fully typed. Numeric length results were normalized to `double` in PR #242; type-system changes must agree with runtime values and operators. +- Compound assignments are desugared by the parser. Check evaluation order and side effects when changing member access or assignment lowering. +- `PatternGenerator` produces `PatternContainer.g.cs`; `GeneratedRegexContainer` consumes the constant. Change token definitions/generator inputs, not files under `obj/`. See [ADR 0004](../docs/adr/0004-generated-lexer-pattern.md). The generator links Constants sources and is referenced with `OutputItemType="Analyzer"`, `ReferenceOutputAssembly="false"`, and `PrivateAssets="all"`. +- DI registrations are predominantly singleton within a service provider. The CLI creates/disposes a provider for each invocation; tests and benchmarks can reuse state differently. Inspect storage/frame lifetime before introducing repeated execution or concurrency. +- Dumping wraps lexer, parser, and VM using keyed services. Preserve ordinary execution behavior and verify dump behavior for affected changes. +- `Executor` returns 0 on success, 1 for lexer/parser/semantic errors, and 2 for other .NET runtime errors. Tests should distinguish these paths. + +## Build and platform + +[Directory.Build.props](../Directory.Build.props) defines `net10.0`, latest C#, nullable references, implicit usings, warnings as errors, AOT compatibility, and currently `BuildInParallel=false`. Constants and the lexer generator override to `netstandard2.0`; tests/generator/constants opt out of AOT compatibility. Preserve child props' parent imports. + +[Directory.Packages.props](../Directory.Packages.props) owns package versions, enables transitive pinning, and disables version overrides. Do not put independent package versions into project files. + +[global.json](../global.json) selects Microsoft.Testing.Platform but does not pin the SDK. CI requests .NET 10.x; check the installed SDK before running commands. + +The CLI publishes Native AOT executables for `win-x64`, `linux-x64`, and `osx-arm64`. Tool packaging also has an `any` managed fallback. Preserve trim/AOT compatibility and explicit DI; assess new reflection-dependent libraries against [ADR 0005](../docs/adr/0005-native-aot-and-performance.md). + +## Change routing + +| Change | Inspect together | +| --- | --- | +| Syntax or operator | TokenTypes, grammar, parser/AST, semantic operator rules, emission/runtime, language docs, regression tests | +| Type, overload, scope, return | IR types/symbol IDs, static-analysis visitors/storages, runtime representation, error-program tests | +| Instructions or calls | AddressedInstructions, code-generation visitors, frames/values/VM, recursion and nested-call tests | +| Host I/O or dumping | Infrastructure adapters, DI, Executor, TestHostFixture, input/dump tests | +| Build, dependencies, publishing | All inherited props, generator analyzer reference, CI workflows, ADRs 0004/0005 | + +[Master CI](../.github/workflows/master.yml) derives version tags with [GitVersion](../GitVersion.yml). The [release-branch workflow](../.github/workflows/release.yml) uses GitReleaseManager and matching milestones to publish releases and packages. Tags are not themselves published releases; treat release-branch pushes as publishing actions requiring authorization. + +Use GitHub MCP to search for relevant implementation context when working on one of these areas. \ No newline at end of file diff --git a/.agents/testing.md b/.agents/testing.md new file mode 100644 index 00000000..58d45949 --- /dev/null +++ b/.agents/testing.md @@ -0,0 +1,92 @@ +# Testing + +Read this before changing executable code/tests or choosing validation commands. Use the applicable Rider and .NET skills required by [AGENTS.md](../AGENTS.md). + +## Testing model + +Use component tests to isolate compiler/runtime contracts, generator tests to verify build-time output, and interpreter regressions to verify language behavior across the real pipeline. These suites complement one another: a script sample's successful exit is a smoke check, not a substitute for output/error assertions. + +NSubstitute is the shared test-double library; use the existing AutoFixture and assertion helpers where applicable. xUnit provides fixture/test-context support, while `TestHostFixture` explicitly composes interpreter services. The current setup does not use Moq or XUnit.DependencyInjection; see the [test-double choice](https://github.com/Stepami/hydrascript/pull/75) and [runner/fixture migration](https://github.com/Stepami/hydrascript/pull/176) for implementation context. + +## Detect the runner + +The checked-in configuration is SDK-style .NET 10, **xUnit on Microsoft.Testing.Platform (MTP)**, in native MTP `dotnet test` mode. [global.json](../global.json) selects the runner; [tests/Directory.Build.props](../tests/Directory.Build.props) enables the xUnit MTP runner and executable test projects. The current package is `xunit.v3.mtp-v2`; its version is centrally managed and should be read from [Directory.Packages.props](../Directory.Packages.props). + +Use `dotnet-test:platform-detection` and `dotnet-test:run-tests` before selecting commands; use `dotnet-test:filter-syntax` for unfamiliar filters. Recheck configuration after dependency/SDK changes. + +## Where tests live + +| Project | Responsibility | Conventions | +| --- | --- | --- | +| [HydraScript.UnitTests](../tests/HydraScript.UnitTests) | Domain and Application behavior | Assembly-wide `Category=Unit`; xUnit, AwesomeAssertions, NSubstitute, AutoFixture | +| [LexerRegexGenerator.UnitTests](../tests/HydraScript.Infrastructure.LexerRegexGenerator.UnitTests) | Generated pattern source and ordering | Explicit `Category=Unit`; Roslyn GeneratorDriver and xUnit assertions | +| [HydraScript.IntegrationTests](../tests/HydraScript.IntegrationTests) | Real interpreter pipeline, success/error programs, console/input/dumps | TestHostFixture composes production services and substitutes external dependencies | + +Before locating C# tests, use `rider-skills:finding-tests` and its documented Rider `findTests` route. Read the returned tests and fixture before adding coverage. Follow the skill's fallback rules when tooling is unavailable. + +## Writing tests + +- Use `dotnet-test:code-testing-agent` for test implementation and `dotnet-test:assertion-quality` for assertions; use the relevant gap-analysis skill when the task is specifically about missing coverage. +- Follow `MethodName_Scenario_ExpectedBehavior` (Roy Osherove). The happy-path convention requested in issue #205 is `MethodName_Always_Success`; prefer a specific scenario/outcome when it adds information. +- Add a minimal regression at the stage that owns the defect. For language behavior, also verify the interpreter's observable result where unit tests cannot establish it. +- Reuse [AutoHydraScriptDataAttribute](../tests/HydraScript.UnitTests/AutoHydraScriptDataAttribute.cs) for applicable unit fixtures and [TestHostFixture](../tests/HydraScript.IntegrationTests/TestHostFixture.cs) for pipeline tests. Keep semantic inputs explicit when randomized data would hide the case. +- Use a disposable runner from `fixture.GetRunner(new TestHostFixture.Options(InMemoryScript: script))`. Assert the appropriate `Executor.ExitCodes` and relevant output/diagnostics, as in [ArithmeticTests](../tests/HydraScript.IntegrationTests/SuccessPrograms/ArithmeticTests.cs) and [FunctionWithoutReturnStatementTests](../tests/HydraScript.IntegrationTests/ErrorPrograms/FunctionWithoutReturnStatementTests.cs). +- [SuccessfulProgramsTests](../tests/HydraScript.IntegrationTests/SuccessPrograms/SuccessfulProgramsTests.cs) automatically enumerates files directly in `Samples/`; those cases assert successful exit only. A new sample joins that smoke suite, but does not establish expected output. Add a focused assertion for behavior-sensitive regressions. Samples are copied to test output with `PreserveNewest`. +- Fixture defaults mock files and environment variables; sample tests explicitly use real ones. Avoid introducing machine-dependent state, input waits, or real environment mutation into new cases. +- New diagnostics need targeted negative cases; do not assume the existing sample suite covers them. + +## Fixture ownership and isolation + +- `TestHostFixture.GetRunner` begins with production `AddDomain`, `AddApplication`, and `AddInfrastructure` registrations. Keep the interpreter stages real in integration tests; substitute external dependencies rather than the behavior under test. +- Each returned runner owns a new service provider. Dispose the runner after use; do not assume invoking the same runner again resets mutable interpreter state. +- Use `configureTestServices` for scenario-specific replacements after the fixture defaults. File-system and environment doubles are enabled by default; an in-memory script replaces source loading. +- A class fixture can outlive individual tests and theory rows. `LogMessages` is cleared only when the fixture is disposed, not when a runner is created or disposed. Restrict assertions to messages captured for the current invocation, or use a separate fixture when isolation is required. +- Captured messages come from the fixture's fake logger, with error messages including exception text. The production CLI uses its own console logging setup; tests of captured messages are not byte-for-byte CLI-format tests. + +## Commands from the repository root + +Prefer Rider build/run tools for development checks and focused tests. Use Rider's terminal for the following CLI/CI commands when needed. A build start is not a result: poll its returned session; record test exit status and actual executed counts. + +```powershell +dotnet --version +dotnet restore ExtendedJavaScriptSubset.slnx +dotnet build ExtendedJavaScriptSubset.slnx --no-restore -c Debug +dotnet test --solution ExtendedJavaScriptSubset.slnx -c Debug --no-build +``` + +Run unit-category tests (includes generator tests): + +```powershell +dotnet test --solution ExtendedJavaScriptSubset.slnx -c Debug --no-build --filter-trait "Category=Unit" +``` + +Run one class or the integration project: + +```powershell +dotnet test --project tests/HydraScript.UnitTests -c Debug --filter-class "HydraScript.UnitTests.Application.TypeDeclarationsResolverTests" +dotnet test --project tests/HydraScript.IntegrationTests -c Debug +``` + +Native MTP uses `--project`/`--solution` and passes MTP flags directly. xUnit filters use `--filter-class`, `--filter-method`, or `--filter-trait`. VSTest's `--filter FullyQualifiedName=...`, `--logger trx`, and `--collect` are not the commands for this setup. + +Only use `--no-build` after matching outputs were built. The shared test props ignore MTP exit code 8 (no tests); therefore a successful exit alone does not prove a selected test ran. Verify a nonzero count for the intended project/class; an empty nonmatching project in a solution-wide trait run can be expected. + +## Coverage and completion + +The integration project references the MTP CodeCoverage extension. Do not pass `--coverage` to the whole solution: the other test projects do not register that extension. + +```powershell +dotnet test --project tests/HydraScript.IntegrationTests -c Debug --no-build --coverage --coverage-output-format cobertura --coverage-output coverage.cobertura.xml --coverage-settings tests/coverage-exclude.xml +``` + +[PR CI](../.github/workflows/pr.yml) runs unit-category tests and integration coverage on Ubuntu, then requires **80% changed-line coverage** against `origin/master` using `diff-cover`. It reads `TestResults/coverage.cobertura.xml`. The threshold is not an overall project percentage. [Push CI](../.github/workflows/push.yml) runs all tests on Windows; [master CI](../.github/workflows/master.yml) collects integration coverage on Windows. + +Treat the unit suite and integration coverage gate as separate checks: an integration coverage report does not establish that component or generator tests passed. + +Respect [coverage-exclude.xml](../tests/coverage-exclude.xml). Do not expand exclusions simply to pass the gate. Local coverage collection alone does not verify the CI diff threshold. + +For behavior changes, run the focused regression first, then affected project tests; run the full suite for changes crossing pipeline stages or shared infrastructure. Check Rider diagnostics and build where applicable. Pure documentation changes need link/path, structure, and formatting validation, without running the interpreter suite. For an isolated semantic refactor, follow the Rider skill's validation rules; behavior changes still need the relevant checks. + +For performance tasks, use [HydraScript.Benchmarks](../benchmarks/HydraScript.Benchmarks) and report configuration, SDK/runtime, workload, timing, and allocations. The current harness shuffles samples and processes a subset while reusing services; control those variables before attributing a performance difference. Benchmark numbers are not correctness tests. + +Report checks actually run, passed/failed counts, skipped or unmatched tests, and any limitations. \ No newline at end of file diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..191b8bcb --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,26 @@ +# Agent entry point + +HydraScript is a statically typed scripting-language interpreter in C#. Keep this file short; load the follow-ups needed for the task. + +## Route the task + +- Read [architecture](.agents/architecture.md) before changing language behavior, dependencies, or project boundaries. +- Read [testing](.agents/testing.md) before changing executable code or tests, or choosing validation commands. +- Read the [ADR index](docs/adr/README.md) and relevant records before making architectural decisions. Keep checked-in guidance focused on the current implementation. +- Follow [.editorconfig](.editorconfig), [CONTRIBUTING.md](CONTRIBUTING.md), and the existing local conventions. New guidance and ADRs use English. + +## Required tools and skills + +- **Use Rider MCP for repository development:** inspect the open solution and worktree first; use its symbol navigation, usages, diagnostics, refactoring, build, and run capabilities where applicable. Run CLI-only operations through Rider's terminal when available. +- **Read and use the applicable Rider skills:** `rider-skills:refactoring-code` for semantic refactoring, `rider-skills:debugging-code` for runtime investigation needing debugger evidence, and `rider-skills:finding-tests` before locating tests for C# changes. Equivalent unprefixed Rider skills are valid. Follow their documented `execute_tool` routing. +- **Read and use applicable .NET skills before acting:** `dotnet-test:platform-detection` and `dotnet-test:run-tests` for test commands; `code-testing-agent` and `assertion-quality` from `dotnet-test` when authoring tests; relevant `dotnet-msbuild`, `dotnet-nuget`, `dotnet-upgrade`, or `dotnet-advanced` skills for their respective tasks. Resolve names from the installed skill catalog; read referenced instructions as required. +- Skill use is mandatory when applicable, not a requirement to run unrelated workflows. Prose changes do not require a debugger, refactoring, or test generation. +- If Rider MCP or a required skill is unavailable, report the exact limitation and use an available documented fallback. Never silently skip the requirement or claim a tool ran. + +## Work and handoff + +1. Establish acceptance criteria, inspect existing changes, and identify affected stages and tests. +2. Use GitHub MCP to search for context relevant to the task, affected symbols, or linked issue/PR. Read matching discussions, reviews, and release/milestone context as needed. Keep only concise evidence links for implemented decisions in ADRs; do not copy GitHub inventories or planned work into Markdown. +3. Implement the scoped change, preserving unrelated work. Use semantic refactoring tools for symbol changes and patch-based edits for other content. +4. Validate using [testing](.agents/testing.md). Update affected follow-ups and add or supersede a consistently structured ADR for a material decision. +5. Report the result, checks actually completed, and remaining limitations. Keep GitHub writes and release actions within the user's authorization. \ No newline at end of file diff --git a/Readme.md b/Readme.md index efdbdc08..0f9b32fd 100644 --- a/Readme.md +++ b/Readme.md @@ -2,6 +2,10 @@ ![logo](hydrascript-logo.jpg) +## Agent development + +Start with [AGENTS.md](AGENTS.md) for required tooling and task routing, then load the linked architecture, testing, and decision-history guidance. + ## Installation Interpreter executable is built during release for 3 following platforms: @@ -25,7 +29,7 @@ It started as a bachelor thesis "Extended JavaScript Subset". Now it's named "Hy I took [ECMA-262 standard](https://www.ecma-international.org/publications-and-standards/standards/ecma-262/) as a basis and made: - [Lexical structure](src/Domain/HydraScript.Domain.Constants/TokenTypes.cs) -- [Grammar](src/Domain/HydraScript.Domain.FrontEnd/Parser/grammar.txt) +- [Grammar](docs/grammar.txt) [Working samples can be found here.](tests/HydraScript.IntegrationTests/Samples) I use them for integration tests. @@ -183,6 +187,40 @@ array = array ++ [5, 7] // concatenation | :: | binary | [] and number | void | | ~ | unary | [] or string | number | +### Compound assignments + +Combine a binary operation with assignment to an existing mutable target. For example, `x += 2` is parsed as `x = x + 2`. + +Supported operators are `+=`, `-=`, `*=`, `/=`, `%=`, `++=`, `&&=`, and `||=`. The same operand-type rules apply as for the corresponding binary operator: arithmetic uses numbers, `+=` also concatenates strings, `++=` concatenates arrays, and `&&=` / `||=` operate on booleans. + +```text +let count = 2 +count += 3 // 5 +count *= 4 // 20 + +let text = "Hydra" +text += "Script" // "HydraScript" + +let values = [1, 2] +values ++= [3] // [1, 2, 3] + +let enabled = false +enabled ||= true // true +enabled &&= false // false +``` + +Writable object properties and array elements also support compound assignment: + +```text +let point = { x: 1; } +point.x += 2 // 3 + +let numbers = [2, 4] +numbers[0] *= 3 // [6, 4] +``` + +These forms follow the usual assignment and type checks; they do not make read-only targets writable. Logical assignments do not short-circuit: their right-hand expressions are evaluated even when the current target value already determines the boolean result. For member targets, keep index expressions free of side effects: parser expansion can evaluate them more than once. + ### Conditionals The language supports classic `if-else`: @@ -349,11 +387,14 @@ Default: HydraScript file.js ``` -Dumping debug info as files (tokens, ast, ir code): +Dumping debug info as files (tokens, AST, and VM instructions): + ``` HydraScript file.js --dump ``` +This writes `file.tokens`, `file.dot`, and `file.tac` beside the script while still executing it. See the [dump guide](docs/dump.md) for file formats, examples, and AST rendering. + ## Sources: 1. "Compilers Construction" and "Optimized Code Generation" courses of [@bmstu-iu9](https://www.github.com/bmstu-iu9) @@ -362,4 +403,4 @@ HydraScript file.js --dump 4. [Stanford CS143 Lectures](https://web.stanford.edu/class/archive/cs/cs143/cs143.1128/) 5. [Simple Virtual Machine](https://github.com/parrt/simple-virtual-machine) 6. Ахо А., Ульман Дж. Теория синтаксического анализа, перевода и компиляции -7. Свердлов С. З. Языки программирования и методы трансляции +7. Свердлов С. З. Языки программирования и методы трансляции \ No newline at end of file diff --git a/docs/adr/0001-oop-ddd-clean-architecture.md b/docs/adr/0001-oop-ddd-clean-architecture.md new file mode 100644 index 00000000..3640d454 --- /dev/null +++ b/docs/adr/0001-oop-ddd-clean-architecture.md @@ -0,0 +1,56 @@ +# ADR 0001: Organize domain objects with DDD and Clean Architecture + +- Status: Accepted +- Decision date: 2024-07-28 +- Recorded: 2026-09-07 +- Evidence basis: Retrospective; linked GitHub records and current source. Consequences include implementation-derived guidance. +- Supersedes: None +- Superseded by: None + +## Context + +HydraScript has distinct language concerns: recognizing syntax, understanding symbols and types, and executing instructions. The original AST also exposed instruction generation, making syntax responsible for application operations. This violated single responsibility and created dependencies between compiler stages that should be independently understandable. + +The architecture discussions explicitly called for DDD subdomains and onion/Clean Architecture. OOP is the implementation model visible in the source, not a separately recorded historical vote. + +## Decision + +Use domain objects and interfaces for AST nodes, scopes, symbols, types, instructions, addresses, values, and frames. Put behavior behind the contract that owns it, using composition and polymorphism rather than a single interpreter class. + +Apply DDD to the compiler's vocabulary and FrontEnd, IR, and BackEnd subdomains. This does not impose business-application patterns such as ORM repositories or aggregate roots on a compiler. + +Enforce Clean Architecture boundaries through separate projects, not just folders. Domain owns representations and contracts; Application owns analysis and emission operations across domains; Infrastructure supplies concrete adapters, pipeline orchestration, and DI registration. The CLI selects options and composes the executable application. AST operations use [Visitor.NET](0002-visitor-net-adoption.md). + +### Project boundaries + +Paths below are under `src/`. The references column describes direct production project dependencies, not NuGet dependencies. + +| Project | Owns | Direct production references | +| --- | --- | --- | +| `Domain/HydraScript.Domain.Constants` | Shared token definitions | None | +| `Domain/HydraScript.Domain.FrontEnd` | Lexer, coordinates, parser, AST, scopes | Constants | +| `Domain/HydraScript.Domain.IR` | Symbols, signatures, structural types, operator metadata | None | +| `Domain/HydraScript.Domain.BackEnd` | Addresses, executable instructions, values, frames, VM contracts | None | +| `Application/HydraScript.Application.StaticAnalysis` | Declaration resolution, type inference, semantic and return checks | FrontEnd, IR | +| `Application/HydraScript.Application.CodeGeneration` | Instruction visitors and value construction | FrontEnd, BackEnd | +| `Infrastructure/HydraScript.Infrastructure` | Pipeline adapters, files, console, environment, dumping, DI | Both Application projects; lexer generator as an analyzer | +| `Infrastructure/HydraScript.Infrastructure.LexerRegexGenerator` | Build-time lexer pattern generation | Linked Constants source files, not a normal project reference | +| `HydraScript` | CLI, executable composition, package metadata | Infrastructure | + +## Consequences + +- Domain must not depend on Application, Infrastructure, or the CLI. Domain projects can have appropriate NuGet dependencies; they are not dependency-free. +- StaticAnalysis joins FrontEnd with IR; CodeGeneration joins FrontEnd with BackEnd. BackEnd does not reference IR. Preserve this distinction when moving responsibilities. +- New language behavior may span several projects. Change the owning model, application passes, runtime implementation, and relevant tests together without moving application operations back into AST nodes. +- DI lifetimes are a separate concern from layering: most services are singleton within a provider, while the CLI creates a provider per invocation. Layer separation does not make mutable services reusable or thread-safe. + +## Alternatives + +The previous CLI plus combined library, with AST-owned analysis/emission and concrete visitor dependencies, was replaced. The records justify separating responsibilities and enforcing boundaries; they do not establish a comparative evaluation of every architectural style. + +## Evidence + +- [Issue #31](https://github.com/Stepami/hydrascript/issues/31): AST-owned instruction generation and single responsibility. +- [Issue #51](https://github.com/Stepami/hydrascript/issues/51) and its [boundary discussion](https://github.com/Stepami/hydrascript/issues/51#issuecomment-2254190545): DDD/Clean Architecture and the intended domain pairings. +- [PR #72](https://github.com/Stepami/hydrascript/pull/72) and [PR #73](https://github.com/Stepami/hydrascript/pull/73): domain separation and project-enforced layers; the latter supplies the decision date. +- Current [solution](../../ExtendedJavaScriptSubset.slnx), [service composition](../../src/Infrastructure/HydraScript.Infrastructure/ServiceCollectionExtensions.cs), and [architecture follow-up](../../.agents/architecture.md). \ No newline at end of file diff --git a/docs/adr/0002-visitor-net-adoption.md b/docs/adr/0002-visitor-net-adoption.md new file mode 100644 index 00000000..83b6c892 --- /dev/null +++ b/docs/adr/0002-visitor-net-adoption.md @@ -0,0 +1,43 @@ +# ADR 0002: Adopt Visitor.NET for AST operations + +- Status: Accepted +- Decision date: 2024-07-20 +- Recorded: 2026-09-07 +- Evidence basis: Retrospective; linked GitHub records and current source. Consequences include implementation-derived guidance. +- Supersedes: None +- Superseded by: None + +## Context + +The same syntax tree needs symbol initialization, type loading, semantic checking, and instruction generation. Keeping these operations on nodes, or making nodes depend on each concrete visitor and its result type, couples FrontEnd to Application and BackEnd. The project needed extensible AST operations without reversing the dependency direction. + +## Decision + +Use Visitor.NET's acyclic generic visitor contracts and generated visitable dispatch. + +- `IAbstractSyntaxTreeNode` implements `IVisitable` and exposes its children through `IReadOnlyList`. +- Concrete AST nodes are partial types marked with `[AutoVisitable]`. `Visitor.NET.AutoVisitableGen` generates their dispatch code. +- Application passes over AST nodes derive from `VisitorBase` or `VisitorNoReturnBase`, implementing `IVisitor` or its no-return form for the node kinds they handle. +- Recursive processing uses `child.Accept(This)`. Nodes depend on the shared visitable contract, not on `SemanticChecker`, `InstructionProvider`, or their concrete result types. + +The same approach serves a separate type-syntax hierarchy: `TypeValue : IVisitable`, concrete `[AutoVisitable]` records, and `TypeBuilder : VisitorBase`. + +Generated dispatch selects the node-specific operation; it does not decide traversal. Each pass controls which children it visits and in what order. The static-analysis pre-pass order remains explicit in DI registration, and analysis runs before emission. + +## Consequences + +- A new analysis or emission pass can be added without adding a visitor-specific `Accept` overload to every AST node. +- A new node kind still requires checking generated visitability, child enumeration, parent/scope propagation, and the relevant visitors. A default visitor implementation can accept a node without performing the required work. +- Traversal ordering and pass ordering are semantic contracts. Do not replace them with indiscriminate traversal just because dispatch is generated. +- Keep the generator a build-time dependency. Edit node declarations and visitor implementations, not generated output. + +## Alternatives + +The code previously placed operations on the AST, then used hand-written `Accept` overloads tied to concrete visitors and backend result types. The generic/generated integration replaced those dependencies. There is no recorded comparison against Roslyn visitors or a universal pattern-matching dispatcher. + +## Evidence + +- [PR #4](https://github.com/Stepami/hydrascript/pull/4): initial visitor adoption. +- [PR #69](https://github.com/Stepami/hydrascript/pull/69): acyclic generic contracts and generated visitability replace concrete visitor dependencies; [issue #51](https://github.com/Stepami/hydrascript/issues/51) connects that change to layer isolation. +- Current [node contract](../../src/Domain/HydraScript.Domain.FrontEnd/Parser/IAbstractSyntaxTreeNode.cs), [example node](../../src/Domain/HydraScript.Domain.FrontEnd/Parser/Impl/Ast/Nodes/Expressions/BinaryExpression.cs), and [FrontEnd package references](../../src/Domain/HydraScript.Domain.FrontEnd/HydraScript.Domain.FrontEnd.csproj). +- Current [type syntax](../../src/Domain/HydraScript.Domain.FrontEnd/Parser/Impl/Ast/Nodes/Declarations/TypeValue.cs), [type visitor](../../src/Application/HydraScript.Application.StaticAnalysis/Visitors/TypeBuilder.cs), [analysis registration](../../src/Application/HydraScript.Application.StaticAnalysis/ServiceCollectionExtensions.cs), and [instruction visitor](../../src/Application/HydraScript.Application.CodeGeneration/Visitors/InstructionProvider.cs). \ No newline at end of file diff --git a/docs/adr/0003-addressed-instructions-and-virtual-machine.md b/docs/adr/0003-addressed-instructions-and-virtual-machine.md new file mode 100644 index 00000000..be8fbe5b --- /dev/null +++ b/docs/adr/0003-addressed-instructions-and-virtual-machine.md @@ -0,0 +1,43 @@ +# ADR 0003: Execute addressed instructions through a virtual machine + +- Status: Accepted +- Decision date: 2022-12-23 +- Recorded: 2026-09-07 +- Evidence basis: Retrospective; linked GitHub records and current source. Consequences include implementation-derived guidance. +- Supersedes: None +- Superseded by: None + +## Context + +HydraScript lowers source programs into executable instruction objects. When addresses were array positions, removing an instruction shifted later positions and broke control flow. Instruction identity needed to survive collection edits independently of the instruction's position. + +## Decision + +Keep code generation separate from execution. Application visitors emit `AddressedInstructions`; the VM follows addresses and delegates behavior to each instruction's `Execute` method. + +- `AddressedInstructions` maintains instruction order with a linked list of addresses and dictionaries for address/node/instruction lookup. +- `IAddress.Next` represents fall-through. Instructions return the next `IAddress?`; jumps, calls, and returns can choose a different address, and `null` terminates execution. +- Replacing an instruction preserves its address. `HashAddress` equality includes a GUID as well as its seed; labels instead use their label names. +- `ExecuteParams` carries console access, frame context, the call stack, and the argument queue. Instructions encapsulate runtime operations, and `IValue` encapsulates value access. +- Calls enter a frame and retain the calling address and return destination. Returns read the callee result, leave its frame, write the result into the caller context, and resume at the saved address's successor. + +The emitted program is an in-memory VM instruction model. The [TAC dump](../dump.md) is a diagnostic listing, not a stable serialized bytecode format. + +## Consequences + +- Preserve address identity for retained/replaced instructions and keep fall-through links and branch targets valid. +- Removal rewires a predecessor's successor; it does not automatically retarget every incoming jump. Deletion still requires control-flow analysis of the affected references. +- Expression emission often obtains its result from the last `Simple.Left`, and assignment emission can redirect that destination. New instructions must respect their callers' result conventions. +- Instructions, frames, and execution parameters are mutable. Provider reuse, recursion, nested calls, and repeated execution require explicit lifetime checks. +- `IValue.Get()` returns `object?`. Static language checking precedes execution, but the CLR payloads and runtime operator implementations must still agree; the backend is not a fully typed CLR instruction system. + +## Alternatives + +Array indices as instruction addresses were replaced after removal invalidated jumps. Seed-only address identity was later strengthened to distinguish otherwise colliding instructions. The records do not establish a formal comparison against native code emission or another bytecode VM design. + +## Evidence + +- [Issue #18](https://github.com/Stepami/hydrascript/issues/18) and [PR #21](https://github.com/Stepami/hydrascript/pull/21): position-independent, linked addresses. +- [Issue #29](https://github.com/Stepami/hydrascript/issues/29) and [PR #65](https://github.com/Stepami/hydrascript/pull/65): collision-safe instruction identity. +- [PR #215](https://github.com/Stepami/hydrascript/pull/215) and [PR #214](https://github.com/Stepami/hydrascript/pull/214): instruction destinations and encapsulated frame/value access. +- Current [instruction collection](../../src/Domain/HydraScript.Domain.BackEnd/AddressedInstructions.cs), [VM](../../src/Domain/HydraScript.Domain.BackEnd/Impl/VirtualMachine.cs), [call](../../src/Domain/HydraScript.Domain.BackEnd/Impl/Instructions/WithAssignment/CallFunction.cs), and [return](../../src/Domain/HydraScript.Domain.BackEnd/Impl/Instructions/Return.cs). \ No newline at end of file diff --git a/docs/adr/0004-generated-lexer-pattern.md b/docs/adr/0004-generated-lexer-pattern.md new file mode 100644 index 00000000..99cb25c8 --- /dev/null +++ b/docs/adr/0004-generated-lexer-pattern.md @@ -0,0 +1,42 @@ +# ADR 0004: Generate the lexer pattern from shared token definitions + +- Status: Accepted +- Decision date: 2024-08-03 +- Recorded: 2026-09-07 +- Evidence basis: Retrospective; linked GitHub records and current source. Consequences include implementation-derived guidance. +- Supersedes: None +- Superseded by: None + +## Context + +Token spellings and patterns are known at build time. Building their combined regex at interpreter startup adds runtime work; maintaining an independent handwritten regex risks disagreement with token metadata. The generated implementation must also remain outside the Domain layer that consumes it. + +## Decision + +Generate the combined lexer pattern from a single set of shared definitions, then use .NET regex source generation for the concrete matcher. + +1. `TokenTypes.Stream` in Constants defines each token's tag, regex pattern, priority, and ignore flag. +2. `PatternGenerator` orders those definitions by priority, forms named alternatives, appends the fallback `ERROR` pattern, and emits the constant in `PatternContainer.g.cs`. +3. Infrastructure's `GeneratedRegexContainer` consumes that constant through `[GeneratedRegex]`. +4. Domain's `Structure` accesses the matcher through the static-abstract `IGeneratedRegexContainer` contract. Runtime token metadata is built from the same shared definitions. + +The custom generator targets `netstandard2.0` and links Constants source files. Infrastructure references it as an analyzer with `OutputItemType="Analyzer"`, `ReferenceOutputAssembly="false"`, and `PrivateAssets="all"`; it is compiler infrastructure, not a runtime service. + +## Consequences + +- Add or change lexical definitions in Constants; do not hand-edit generated files or copy the resulting regex into another source file. +- Overlapping tokens depend on priority. Matching groups, runtime token tags, ignore behavior, and the terminal error fallback must remain aligned. +- Check both generated source and runtime tokenization when changing the pipeline. Successful generation alone does not establish correct token boundaries. +- Keep the Domain contract independent of Infrastructure's concrete generated class. Preserve the generator's target framework, linked sources, and analyzer-only reference semantics. +- This supports the project's [AOT and performance approach](0005-native-aot-and-performance.md) without requiring the Domain to know how the matcher was generated. + +## Alternatives + +Runtime pattern construction and a separately maintained handwritten regex were avoided. The first generator iteration still required copying output manually; the later fully automatic integration removed that intermediate maintenance step. That old copying workflow is not part of the current design. + +## Evidence + +- [Issue #57](https://github.com/Stepami/hydrascript/issues/57) and its [architecture discussion](https://github.com/Stepami/hydrascript/issues/57#issuecomment-2263958010): build-time pattern knowledge and the Domain/Infrastructure contract. +- [PR #77](https://github.com/Stepami/hydrascript/pull/77) and [PR #115](https://github.com/Stepami/hydrascript/pull/115): initial generation and removal of manual copying. +- [Issue #236](https://github.com/Stepami/hydrascript/issues/236) and [PR #237](https://github.com/Stepami/hydrascript/pull/237): linked Constants sources and generator references. +- Current [token definitions](../../src/Domain/HydraScript.Domain.Constants/TokenTypes.cs), [pattern generator](../../src/Infrastructure/HydraScript.Infrastructure.LexerRegexGenerator/PatternGenerator.cs), [generator project](../../src/Infrastructure/HydraScript.Infrastructure.LexerRegexGenerator/HydraScript.Infrastructure.LexerRegexGenerator.csproj), and [regex container](../../src/Infrastructure/HydraScript.Infrastructure/GeneratedRegexContainer.cs). \ No newline at end of file diff --git a/docs/adr/0005-native-aot-and-performance.md b/docs/adr/0005-native-aot-and-performance.md new file mode 100644 index 00000000..7660a4e3 --- /dev/null +++ b/docs/adr/0005-native-aot-and-performance.md @@ -0,0 +1,56 @@ +# ADR 0005: Preserve Native AOT and prefer measured, allocation-conscious implementations + +- Status: Accepted +- Decision date: 2025-04-03 +- Recorded: 2026-09-07 +- Evidence basis: Retrospective; linked GitHub records and current source. Consequences include implementation-derived guidance. +- Supersedes: None +- Superseded by: None + +## Context + +HydraScript should remain a maintainable C# interpreter while being distributed as a small native executable without a separately installed .NET runtime. Native compilation and binary size shaped dependency selection and service composition. Recurring interpreter work also motivated lower-allocation query, text, and logging implementations. + +These goals are related but not interchangeable: AOT compatibility, executable size, startup, throughput, and allocations have different tradeoffs. + +## Decision + +Publish Native AOT binaries for `win-x64`, `linux-x64`, and `osx-arm64`. The .NET tool also supplies native RID packages and an `any` managed fallback built with `PublishAot=false`. + +Preserve AOT/trimming compatibility and prefer lower-allocation implementations in recurring paths when they preserve semantics and their benefit can be demonstrated. The current implementation makes the following choices: + +| Area | Adopted approach | +| --- | --- | +| Distribution policy | `PublishAot=true`; Release favors size, disables debug symbols/stack-trace support, and uses invariant globalization | +| Composition | Explicit service registration and keyed dump decorators instead of runtime discovery/decorator infrastructure | +| Source generation | Generated lexer/regex, Visitor.NET visitable dispatch, and a source-generated JSON context for supported runtime values | +| Queries | ZLinq's `AsValueEnumerable()` in analysis, emission, type handling, and traversal; ordinary LINQ remains where it is still used | +| Text and logging | ZString builders/join/concat and ZLogger's interpolated console logging | +| Small operations and traversal | Selected structs/record structs and a reusable traversal queue with explicit disposal | +| Scanning and lookup | Span-based newline scanning with `SearchValues`, read-heavy frozen token lookup, and lazy token enumeration | + +Value types are selective: token-definition DTOs, operation descriptors, operator implementations, and the AST traversal enumerator use them. They do not establish an allocation-free contract; storing an operator struct as `IOperator` can box it. Token and coordinate records are still reference types. + +Spans are used, but the current source does not use `stackalloc`. Stack allocation is not a historical adoption to claim or a blanket requirement for future edits. Its suitability depends on buffer size, lifetime, safety, and measured benefit. + +## Consequences + +- Evaluate new runtime dependencies and reflection-dependent code against AOT/trimming constraints. Preserve generated serializer coverage when changing runtime value shapes. +- Native binaries remain platform-specific, and the managed fallback still has runtime requirements. Preserve the fallback override rather than treating every package as a native executable. +- Size-oriented Release settings reduce diagnostic information and change globalization assumptions. Compare performance under the relevant configuration rather than assuming Debug and native Release behave alike. +- Prefer existing ZLinq/ZString/ZLogger patterns in their established paths, but do not replace every LINQ query or class solely on the assumption that another API or a struct must be faster. +- Respect builder and reusable-enumerator lifetimes, disposal, nesting, and reentrancy. Lower allocation is not useful if it changes behavior; dumping deliberately materializes tokens for inspection. +- Use the BenchmarkDotNet managed/AOT jobs and memory diagnostics for comparable measurements. The current harness shuffles samples, executes a subset, and reuses a provider; control workload and state before attributing a difference. Historical numbers are not universal speedup guarantees. + +## Alternatives + +Framework-dependent execution preceded native distribution. Hosting integration and Scrutor decoration were removed during AOT preparation. Conventional logging, string-building, and LINQ implementations preceded the measured library changes; they are not universally prohibited. + +## Evidence + +- [Issue #146](https://github.com/Stepami/hydrascript/issues/146) and [PR #166](https://github.com/Stepami/hydrascript/pull/166): native compilation and binary-size work; [PR #156](https://github.com/Stepami/hydrascript/pull/156) / [PR #159](https://github.com/Stepami/hydrascript/pull/159): composition changes. +- [PR #181](https://github.com/Stepami/hydrascript/pull/181), [PR #182](https://github.com/Stepami/hydrascript/pull/182), and [PR #183](https://github.com/Stepami/hydrascript/pull/183): ZLogger, ZString, and ZLinq adoption with historical comparisons. +- [Issue #81](https://github.com/Stepami/hydrascript/issues/81) and [PR #211](https://github.com/Stepami/hydrascript/pull/211): read-heavy frozen lookup and lazy lexing. +- [PR #244](https://github.com/Stepami/hydrascript/pull/244) / [PR #246](https://github.com/Stepami/hydrascript/pull/246): hybrid native/managed tool distribution. +- Current [build policy](../../Directory.Build.props), [CLI packaging](../../src/HydraScript/HydraScript.csproj), [JSON source generation](../../src/Domain/HydraScript.Domain.BackEnd/Impl/Instructions/WithAssignment/ExplicitCast/AsString.cs), and [operator storage](../../src/Domain/HydraScript.Domain.IR/Types/Type.cs); see also [Visitor.NET](0002-visitor-net-adoption.md) and [lexer generation](0004-generated-lexer-pattern.md). +- Current [traversal enumerator](../../src/Domain/HydraScript.Domain.FrontEnd/Parser/Impl/Ast/TraverseEnumerator.cs), [span-based scanning](../../src/Domain/HydraScript.Domain.FrontEnd/Lexer/Impl/TextCoordinateSystemComputer.cs), [benchmark harness](../../benchmarks/HydraScript.Benchmarks/InvokeBenchmark.cs), and [measurement guidance](../../.agents/testing.md). \ No newline at end of file diff --git a/docs/adr/0006-structural-type-system.md b/docs/adr/0006-structural-type-system.md new file mode 100644 index 00000000..c0da5e73 --- /dev/null +++ b/docs/adr/0006-structural-type-system.md @@ -0,0 +1,90 @@ +# ADR 0006: Use structural static types with type-owned operator rules + +- Status: Accepted +- Decision date: Unknown +- Recorded: 2026-09-07 +- Evidence basis: Retrospective; linked GitHub records and current source. Consequences include implementation-derived guidance. +- Supersedes: None +- Superseded by: None + +## Context + +HydraScript combines JavaScript-like syntax with strong static structural typing. The language needs to reject incompatible operations before execution without introducing classes, inheritance, or nominal identities for every user-defined type. + +Structural typing predates the later overload, cast, and operator refactorings linked below; those PR dates are not treated as the original adoption date. + +## Decision + +Resolve declarations, signatures, and expression types before emitting instructions. Keep type descriptions and operator metadata in Domain.IR and their construction/checking in Application.StaticAnalysis. `IHydraScriptTypesService` centralizes built-in types, defaults, and explicit conversion rules. Type declarations are built before their references are resolved. + +### Structural equality, not nominal identity + +A type alias names a type description; it does not create a distinct nominal type. Resolved object types compare their complete property-name/type shape, and arrays compare their element types. Declared and inferred object properties are canonicalized before comparison; differing declaration order does not define a different shape. Methods are not included in object type equality. + +```text +type Point = { x: number; y: number; } +type Vector = { x: number; y: number; } + +let p: Point = { x: 3; y: 4; } +let v: Vector = p +``` + +The assignment is allowed because the property shapes match, not because the aliases have the same name. Object matching is exact-shape equality, not a general rule that any object with extra properties is assignable. + +Compatibility is broader than a single symmetric equality function: nullable/object/null rules can make `Type.Equals` directional. Assignment, equality-operator validation, and explicit-cast checks use `CommutativeTypeEqualityComparer` to accept either comparison direction; other checks still call `Equals` directly. Preserve the intended context instead of globally substituting CLR identity, alias names, or one comparer. + +### Go-inspired methods + +A method is an ordinary function whose first parameter is an explicit receiver. Registration requires that parameter to resolve to an object type and be written using a named type alias: + +```text +function lengthSquared(self: Point): number { + return self.x * self.x + self.y * self.y +} + +>>> p.lengthSquared() +``` + +The receiver is inserted as the first call argument. An explicitly annotated `p: Point` retains the object-type instance and method metadata associated with that alias. An inferred lookalike object does not automatically acquire that method list. This binding role for aliases does not make structural type equality nominal. Overload lookup uses function names and parameter-type signatures. + +### Operators and scripting syntax + +`IOperator` describes static operator capabilities, not execution: + +- `Values` lists the recognized spellings. +- `TryGetResultType(OperationDescriptor, out Type)` checks the operand types and determines the result type. +- `OperationDescriptor` carries the spelling and operand types; `Type.TryGetOperator` selects the `IOperator` rule used by semantic checking. +- Backend instructions implement the actual operation separately. A new operator must align lexical recognition, parsing, type metadata, emission, and execution. + +| Operand family | Examples | +| --- | --- | +| Numbers | Arithmetic, unary `-`, ordering | +| Booleans | `!`, `&&`, `\|\|` | +| Strings | `+` / `++` concatenation, `~` length, `[]` indexing returning a string | +| Arrays | `++` concatenation, `~` length, `[]` element access, `::` removal returning `void` | +| Compatible types | `==` and `!=`, registered through the base type's operator metadata | + +Assignments, `as`, and `with` have dedicated syntax and checks rather than being `IOperator` implementations. Compound assignments lower into assignment plus an existing binary operator; their user-facing behavior is documented in the [README](../../Readme.md#compound-assignments). + +Shell-like facilities are also statically checked: `$NAME` accesses a string-valued environment variable, `<<< destination` reads a line into a string destination, and `>>> expression` produces console output. These are references/statements with their own visitors and runtime instructions, not `IOperator` descriptors or an interactive shell. + +## Consequences + +- Preserve structural compatibility across aliases, canonical property construction, nullable handling, and signature lookup. Do not infer nominal typing from the use of named aliases for method registration. +- Structural **type** equality does not imply deep runtime **value** equality. The VM's `==` uses CLR `object.Equals`; separate list/dictionary payloads do not become deeply equal because their static types match. +- Keep static result types consistent with CLR payloads returned through `IValue.Get(): object?`. Length results use `double` to participate consistently in numeric operations. +- Adding a node/operator/type rule requires positive and negative semantic cases plus observable runtime regressions. A metadata-only change is not a complete language feature. +- The language exposes built-in operator metadata, not a syntax for users to define arbitrary operator overloads. + +## Alternatives + +Nominal alias identity is not the implemented model; Go-inspired receiver syntax does not import Go's entire type system. The available history does not establish a formal comparison of typing paradigms. One-phase declaration resolution and dispersed operation checks were replaced by the current staged resolution and centralized type/operator rules. + +## Evidence + +- [PR #151](https://github.com/Stepami/hydrascript/pull/151): typed symbol IDs and overload signatures; [PR #217](https://github.com/Stepami/hydrascript/pull/217): explicit casts and type equality. +- [PR #226](https://github.com/Stepami/hydrascript/pull/226): centralized types, symmetric assignment checking, and operators as type metadata. +- [PR #240](https://github.com/Stepami/hydrascript/pull/240) / [issue #231](https://github.com/Stepami/hydrascript/issues/231): two-phase type resolution; [PR #242](https://github.com/Stepami/hydrascript/pull/242): numeric length representation. +- [Issue #201](https://github.com/Stepami/hydrascript/issues/201) / [PR #214](https://github.com/Stepami/hydrascript/pull/214) and [issue #200](https://github.com/Stepami/hydrascript/issues/200) / [PR #218](https://github.com/Stepami/hydrascript/pull/218): environment and input syntax. +- Current [object equality](../../src/Domain/HydraScript.Domain.IR/Types/ObjectType.cs), [type construction](../../src/Application/HydraScript.Application.StaticAnalysis/Visitors/TypeBuilder.cs), [method registration](../../src/Application/HydraScript.Application.StaticAnalysis/Visitors/DeclarationVisitor.cs), and [compatibility comparer](../../src/Domain/HydraScript.Domain.IR/Types/CommutativeTypeEqualityComparer.cs). +- Current [IOperator](../../src/Domain/HydraScript.Domain.IR/Types/IOperator.cs), [semantic checking](../../src/Application/HydraScript.Application.StaticAnalysis/Visitors/SemanticChecker.cs), and [runtime operators](../../src/Domain/HydraScript.Domain.BackEnd/Impl/Instructions/WithAssignment/Simple.cs). \ No newline at end of file diff --git a/docs/adr/README.md b/docs/adr/README.md new file mode 100644 index 00000000..aba46688 --- /dev/null +++ b/docs/adr/README.md @@ -0,0 +1,27 @@ +# Architecture decision records + +These records describe the six lasting design pillars that shaped HydraScript. They explain the implemented choices, their rationale, and the constraints agents should preserve. They are retrospective records, not a release log or a list of individual refactorings. + +| ADR | Project-shaping decision | +| --- | --- | +| [0001](0001-oop-ddd-clean-architecture.md) | OOP, DDD, Clean Architecture, and project boundaries | +| [0002](0002-visitor-net-adoption.md) | Visitor.NET for extensible AST operations | +| [0003](0003-addressed-instructions-and-virtual-machine.md) | Addressed instructions and virtual-machine execution | +| [0004](0004-generated-lexer-pattern.md) | Generated lexer pattern from shared token definitions | +| [0005](0005-native-aot-and-performance.md) | Native AOT and measured, allocation-conscious implementation | +| [0006](0006-structural-type-system.md) | Structural static types, Go-inspired methods, and operator rules | + +## What belongs here + +A record should explain a lasting choice that shapes several parts of the system. Supporting implementation details belong under the relevant pillar; ordinary fixes, package updates, CI changes, and test-tool migrations do not need their own ADR. + +Use [architecture](../../.agents/architecture.md) for task routing, [testing](../../.agents/testing.md) for test tools, fixtures, commands, and coverage, and the [README](../../Readme.md) / [dump guide](../dump.md) for language and CLI usage. + +## Keep records consistent + +1. Use [template.md](template.md), retaining its metadata keys and the five sections: Context, Decision, Consequences, Alternatives, Evidence. +2. Record only implemented choices. Use concise GitHub evidence and source links, not copied inventories or planned work. Search GitHub MCP for task-relevant discussions, reviews, commits, releases, and milestones when more context is needed. +3. Distinguish direct historical rationale from current-source observations and inferred consequences. Do not invent rejected alternatives or claim an optimization is universally faster. +4. The decision date identifies the principal documented adoption; later evolution belongs in the evidence. Use `Unknown` when the original choice cannot be dated, and keep the documentation date separate. +5. Update the relevant pillar when clarifying its current implementation. A genuinely new project-shaping decision uses the next unused ID; an actual replacement links `Supersedes` / `Superseded by` and preserves the superseded rationale. +6. Update the index and affected follow-ups together. Keep one authoritative project-boundary table in ADR 0001 and operational testing guidance in `.agents/testing.md`. \ No newline at end of file diff --git a/docs/adr/template.md b/docs/adr/template.md new file mode 100644 index 00000000..82a59fba --- /dev/null +++ b/docs/adr/template.md @@ -0,0 +1,31 @@ +# ADR NNNN: Decision title + +- Status: Accepted +- Decision date: YYYY-MM-DD or Unknown +- Recorded: YYYY-MM-DD +- Evidence basis: Contemporary or Retrospective; identify direct evidence and inference. +- Supersedes: None or linked ADR +- Superseded by: None or linked ADR + +## Context + +Describe the concrete problem and constraints. Link the issue that motivates the decision. + +## Decision + +State the implemented approach and scope, supported by source and implementation evidence. + +## Consequences + +Describe benefits, costs, compatibility impact, and verification needed. Label inferred implications. + +## Alternatives + +Summarize documented alternatives and why they were not selected. If none were recorded, say so. + +## Evidence + +- Issues: linked issues, relevant comments, and relationships, or None recorded. +- Pull requests: linked implementation/review evidence, merge state/date, or None recorded. +- Releases and milestones: linked release/tag and milestone with state, or Not yet released / None recorded. +- Implementation: relative links to code/tests; distinguish current code from historical state. \ No newline at end of file diff --git a/docs/dump.md b/docs/dump.md new file mode 100644 index 00000000..487a31f7 --- /dev/null +++ b/docs/dump.md @@ -0,0 +1,97 @@ +# Interpreter dumps + +Use `--dump` to save the intermediate representations of a script while running it. This helps distinguish tokenization, parsing, and instruction-generation problems. Dumping is not a dry run: the script still executes and can perform its usual input/output. + +## Generate the files + +Save this as `sample.js`: + +```text +let x = 1 +x += 2 +>>> x +``` + +Run it with: + +```shell +hydrascript sample.js --dump +``` + +The script prints `3`. The option also accepts `-d` and `/d`. + +| File | Contents | Written when | +| --- | --- | --- | +| `sample.tokens` | Plain-text lexer token stream | Tokenization finishes successfully | +| `sample.dot` | Abstract syntax tree (AST) in Graphviz DOT format | Parsing finishes, before static analysis | +| `sample.tac` | Plain-text three-address-code (TAC) instruction listing | Static analysis and code generation finish, just before VM execution | + +Files are written beside the source script, not necessarily in the terminal's working directory. For example, `hydrascript scripts/sample.js --dump` writes `scripts/sample.tokens`, `scripts/sample.dot`, and `scripts/sample.tac`. The source directory must be writable. + +## `.tokens`: what the lexer recognized + +Each token includes its kind, source span, and source spelling. Line and column numbers start at 1; the end coordinate is exclusive. For the example's second line: + +```text +Ident (2, 1)-(2, 2): x +Assign (2, 3)-(2, 5): += +IntegerLiteral (2, 6)-(2, 7): 2 +``` + +Here, `+=` is one assignment token, not separate `+` and `=` tokens. Comments and whitespace are omitted; the stream ends with `EOP` (end of program). Use this file to check token boundaries, operator recognition, and source coordinates. + +## `.dot`: how the parser grouped the syntax + +DOT is a graph-description language, not an image format. The dump begins with `digraph ast`; nodes have syntax labels, and arrows connect parents to children. It represents the parsed tree before type checking, not a control-flow graph. + +The parser already expands `x += 2` into `x = x + 2`. This excerpt shows that subtree, with node IDs shortened and unrelated nodes omitted: + +```dot +digraph ast { + 1 [label="="] + 2 [label="MemberExpression"] + 3 [label="+"] + 4 [label="x"] + 5 [label="x"] + 6 [label="2"] + 1 -> 2 + 1 -> 3 + 2 -> 4 + 3 -> 5 + 3 -> 6 +} +``` + +With [Graphviz](https://graphviz.org/doc/info/command.html) installed and its `dot` command on your path, render the full dump as SVG: + +```shell +dot -Tsvg sample.dot -o sample.svg +``` + +Open `sample.svg` in a browser or image viewer. Graphviz is only needed for rendering; the interpreter creates the DOT file without it. Generated node IDs can change between runs, so compare tree structure and labels rather than IDs. + +## `.tac`: what the VM will execute + +TAC means three-address code: complex expressions are lowered into simple operations, alongside instructions for control flow, calls, and input/output. This is HydraScript's VM instruction listing, not native assembly or .NET IL. + +For the sample, the listing looks like this, with the generated temporary name shortened to `_t1`: + +```text +x = 1 +x = x + 2 +_t1 = x as String +Output _t1 +End +``` + +The output path converts the value to a string before printing it. Larger programs also contain temporaries, labels, jumps, and function calls. Temporary names can vary between runs. + +The listing is written before execution: it does not contain the resulting value of every variable, visited branches, or an execution trace. Having a `.tac` file does not prove the program ran successfully. + +## Failed runs and existing files + +Each stage overwrites its own dump when reached. A lexer error prevents a fresh token dump; a parser error can leave only a fresh `.tokens` file; a static-analysis error can leave fresh `.tokens` and `.dot` files but no fresh `.tac`. Runtime errors can occur after all three have been written. + +Old dumps from earlier runs are not automatically removed. Check timestamps or remove the old generated files before retrying, so a stale file is not mistaken for output from a failed run. Dumps can contain source literals and other sensitive script content; review them before sharing or committing. + +See the [architecture guide](../.agents/architecture.md) for the pipeline and [dumping adapters](../src/Infrastructure/HydraScript.Infrastructure/Dumping) for the implementation. \ No newline at end of file diff --git a/src/Domain/HydraScript.Domain.FrontEnd/Parser/grammar.txt b/docs/grammar.txt similarity index 100% rename from src/Domain/HydraScript.Domain.FrontEnd/Parser/grammar.txt rename to docs/grammar.txt