Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/workflows/cli-release-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ jobs:
persist-credentials: false
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 24
node-version: 24.18.0
package-manager-cache: false
- name: Install pinned Rust toolchain
shell: pwsh
Expand All @@ -111,6 +111,10 @@ jobs:
- name: Install Rust target
shell: pwsh
run: rustup target add '${{ matrix.target }}'
- name: Install pinned TypeScript oracle
if: matrix.label == 'win32-x64'
shell: pwsh
run: npm ci
- name: Test Windows cache guard
if: matrix.label == 'win32-x64'
shell: pwsh
Expand Down
32 changes: 32 additions & 0 deletions crates/bamts-cli/tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# CLI execution and chaos tests

Use the repository-pinned Node 24.18.0 on `PATH`, then install the locked TypeScript 7.0.2 oracle from the repository root:

```sh
npm ci
cargo test --locked -p bamts-cli
```

Run the bounded chaos suite alone:

```sh
cargo test --locked -p bamts-cli --test cli chaos::
```

The oracle helpers verify both versions, even when a single differential test is selected. They compile the original `.ts` files with `tsc --strict`, execute the emitted JavaScript in Node, and compare it with the original TypeScript executed through real `bamts --api` processes in JIT and AOT modes. Compiler internals and transport dispatch are not mocked.

## Coverage

`runtime-workload.ts` exercises typed record processing, loop closure capture, destructuring defaults, Map iteration, generators, labeled control flow with `finally`, and asynchronous completion. `optional-chain-continuations.ts` covers skipped computed keys and arguments, optional-call continuations, non-null assertions, grouped method receivers, and deletion.

The generated-program test uses eight fixed seeds and sixteen bounded functions per seed. Each function executes arithmetic through `continue` and `finally`. Expected results are calculated independently in Rust and checked against Node before comparison with JIT and AOT. Failures print the seed and complete generated source. No source depends on wall time or host randomness.

The transport test interleaves 96 malformed JSON/request frames with valid version requests across eight sessions. Seeded fragments split headers and bodies into 1–17-byte writes. Every malformed frame must produce the expected typed error, and every subsequent request must receive its own response in order.

Additional tests cover exact header limits, unterminated headers with the peer still connected, duplicate lengths, truncated frames, LSP edit recovery, 512 KiB TypeScript output, and simultaneous 256 KiB stdout/stderr capture. The process harness drains both pipes while the child runs, retains a 120-second child deadline, and kills/reaps a timed-out child. The capture regression has a five-second deadline; the watchdog regression uses 200 milliseconds.

## Limits

These tests provide bounded regression evidence, not exhaustive fuzzing, formal gate admission, full TypeScript compatibility, or performance benchmark evidence. Socket-specific process tests are Unix-only. The deterministic framing unit tests remain portable. The watchdog controls the direct child, not arbitrary descendant process trees.

The checker still diagnoses some valid optional-chain indexed accesses with `BAMTS-C064`; the optional-chain fixture separately establishes corrected execution semantics. No reference implementation source is copied into these fixtures.
102 changes: 84 additions & 18 deletions crates/bamts-cli/tests/cli.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use std::fs;
use std::io::Write;
use std::io::{Read, Write};
#[cfg(unix)]
use std::os::unix::io::OwnedFd;
#[cfg(unix)]
Expand Down Expand Up @@ -73,6 +73,39 @@ vm.runInThisContext("import('node:util').then(function(ns) { process.stdout.writ
"#;
static NEXT_DIRECTORY: AtomicU32 = AtomicU32::new(0);

#[path = "cli/chaos.rs"]
mod chaos;

#[test]
fn process_capture_drains_stdout_and_stderr_before_waiting() {
let child = Command::new("node")
.args([
"-e",
"process.stdout.write('x'.repeat(262144)); process.stderr.write('y'.repeat(262144));",
])
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.expect("Node is required for CLI E2E oracle tests");
let output =
wait_for_output_with_timeout(child, "large two-pipe output", Duration::from_secs(5));
assert_success(&output, "large two-pipe output");
assert_eq!(output.stdout, vec![b'x'; 262144]);
assert_eq!(output.stderr, vec![b'y'; 262144]);
}

#[test]
#[should_panic(expected = "watchdog probe exceeded")]
fn process_capture_still_terminates_and_reaps_a_hung_child() {
let child = Command::new("node")
.args(["-e", "setInterval(() => {}, 1000);"])
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.expect("Node watchdog probe starts");
wait_for_output_with_timeout(child, "watchdog probe", Duration::from_millis(200));
}

#[test]
fn optional_chain_continuations_skip_side_effects_and_preserve_receivers() {
let project = ScratchDirectory::new();
Expand Down Expand Up @@ -1669,24 +1702,57 @@ fn assert_execution_success(output: &ExecutionOutput, command: &str) {
);
}

fn wait_for_output(mut child: Child, command: &str) -> Output {
let deadline = Instant::now() + Duration::from_secs(120);
loop {
match child.try_wait() {
Ok(Some(_)) => return child.wait_with_output().expect("finished child output"),
Ok(None) if Instant::now() < deadline => thread::sleep(Duration::from_millis(10)),
Ok(None) => {
child.kill().expect("timed-out child is killed");
let output = child.wait_with_output().expect("timed-out child output");
panic!(
"{command} exceeded 120 seconds\nstdout:\n{}\nstderr:\n{}",
String::from_utf8_lossy(&output.stdout),
String::from_utf8_lossy(&output.stderr)
);
fn wait_for_output(child: Child, command: &str) -> Output {
wait_for_output_with_timeout(child, command, Duration::from_secs(120))
}

fn wait_for_output_with_timeout(mut child: Child, command: &str, timeout: Duration) -> Output {
let deadline = Instant::now() + timeout;
drop(child.stdin.take());
thread::scope(|scope| {
let stdout = child
.stdout
.take()
.map(|pipe| scope.spawn(move || read_output(pipe)));
let stderr = child
.stderr
.take()
.map(|pipe| scope.spawn(move || read_output(pipe)));
let (status, timed_out) = loop {
match child.try_wait() {
Ok(Some(status)) => break (status, false),
Ok(None) if Instant::now() < deadline => thread::sleep(Duration::from_millis(10)),
Ok(None) => {
child.kill().expect("timed-out child is killed");
break (child.wait().expect("timed-out child is reaped"), true);
}
Err(error) => {
child.kill().expect("failed child wait is terminated");
child.wait().expect("failed child wait is reaped");
panic!("could not wait for {command}: {error}");
}
}
Err(error) => panic!("could not wait for {command}: {error}"),
}
}
};
let output = Output {
status,
stdout: stdout.map_or_else(Vec::new, |reader| reader.join().expect("stdout reader")),
stderr: stderr.map_or_else(Vec::new, |reader| reader.join().expect("stderr reader")),
};
assert!(
!timed_out,
"{command} exceeded {timeout:?}\nstdout:\n{}\nstderr:\n{}",
String::from_utf8_lossy(&output.stdout),
String::from_utf8_lossy(&output.stderr)
);
output
})
}

fn read_output(mut pipe: impl Read) -> Vec<u8> {
let mut bytes = Vec::new();
pipe.read_to_end(&mut bytes)
.expect("child pipe is readable");
bytes
}

fn framed(payload: &[u8]) -> Vec<u8> {
Expand Down
Loading
Loading