Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -237,9 +237,25 @@ Scale characterization with 4,101 synthetic Agent Todos still hits the existing
repair for File/SQLite. The repaired contract API can read that collection;
this does not qualify the remaining whole-command payload boundary.

The next B work remains history artifact lookup and remaining public payload/
cold-path costs, preserving file-change freshness, full decision inputs and
corruption rejection. Contract checks and attention now share one request-local, validated canonical
The next B work is SQLite admission on a frozen source/runtime profile: rerun
the existing reference capacity axes, reconcile concurrency/recovery/consumer-lag
evidence, and verify the applicability of retained natural-time soak results.
The comparison runner's former conflict expectation contradicted merged #5169:
an identical historical intent must return its original applied revision/cursor.
The runner now checks that result, independently rejects projection/event/receipt
drift, and walks the complete history before and after retries without retaining
all expected snapshots. A failing invariant prevents report publication; checks
stay outside the unchanged timing windows. This repairs the qualification tool,
not a provider defect or a D2/default pass. #4224 already reports a soak started
on September 14 at `e98191faa`; its final result and applicability to the current
candidate still need evidence. Do not call it unstarted or restart its clock
solely because an unrelated source revision changed.

Last-caller Python decision retirement can proceed independently where the TS
replacement and affected real callers are proven. Whole Markdown writer removal
still requires C's new-Goal/upgrade/recovery exits. Complete consumer metadata,
freshness and decision inputs remain acceptance requirements. Contract checks
and attention now share one request-local, validated canonical
Todo snapshot per runtime/Goal. Standalone checks and subsequent requests read
afresh; lease and projection-writeback reads do not participate. Consumer edits
cannot mutate retained input, and a failed first read cannot recover midway
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -183,8 +183,18 @@ Todo 写入时的业务校验,也不重审完成/deferred 历史的授权。
仍触及既有 `todo.succession.project` RPC 响应预算;修复后的合同 API 能读取该集合,
不代表剩余整命令包体边界已完成验收。

B 下一步仍是历史 artifact 查找和剩余公共包体/冷路径,保留文件变化 freshness、
完整决策输入及损坏拒绝。合同检查与 attention 现在按 runtime/Goal 共享请求内已校验的完整
B 下一步聚焦冻结 source/runtime profile 下的 SQLite 准入:重新跑已有 reference
容量轴,对齐并发/恢复/consumer lag 证据,并核对保留的自然时间 soak 适用性。
比较 runner 原先要求历史重试返回 conflict,与已合并 #5169 矛盾:相同完整意图应
返回原 applied revision/cursor。现在核对原结果,分别拒绝 projection/event/receipt
漂移,在重试前后分页验证全部历史,不保留所有预期快照。不变量失败就不发布成功
报告;这些检查放在既有计时窗口之外。这修复的是验证工具,不代表 provider 故障、
D2 通过或默认切换。#4224 已报告在 `e98191faa` 上于 9 月 14 日开始 soak,仍需最终
结果及对当前候选的适用性证据,不能称为未开始,也不能仅因无关 source 修订就重启计时。

已有 TS 替代且真实受影响调用方验证完成的 Python 重复决策,可以按最后调用方独立
退役;整条 Markdown writer 删除仍需 C 的新 Goal/升级/恢复出口。消费者完整
metadata、freshness 和决策输入继续验收。合同检查与 attention 现在按 runtime/Goal 共享请求内已校验的完整
canonical Todo 快照。独立检查和下一次请求重新读取;租约与投影写回读取不参与。消费者修改
不会污染保留输入,首次读取失败不会在请求中途恢复成功。这不代表 registry、Markdown、
历史或多个 Goal 之间的原子快照。集成后继续核对安装态消费者,A/C 与 D2
Expand Down
12 changes: 9 additions & 3 deletions docs/reference/sqlite-authority-store.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,9 +85,15 @@ node --experimental-sqlite --experimental-strip-types \
```

The runner creates and removes its own temporary store, checks complete
projections (including Todo metadata), original receipts and reopened state,
and records the source revision, runtime, runner hash and tracked source diff
hash. It does not open a selected live Goal. RSS includes fixture/checking
projections (including Todo metadata), events, original receipts and reopened
state, and records the source revision, runtime, runner hash and tracked source
diff hash. After timing, it verifies that an exact historical retry returns the
original applied revision/cursor, even after later commits; projection-, event-
or receipt-only drift must conflict. The later head, original receipt and entire
paged history must remain unchanged. `historical_replay_and_conflict_checks`
is reported only after these checks pass; a failed check prevents a successful
report. These are storage guarantees, not Goal-instance isolation or permission
to repeat external effects. It does not open a selected live Goal. RSS includes fixture/checking
allocations; File publication bytes are application bytes, not physical disk
writes. Use the existing SQLite capacity runner for WAL traffic and D2 history
sizes. Keep performance experiments separate from concurrent test suites.
Expand Down
80 changes: 69 additions & 11 deletions examples/coordination/local-provider-comparison.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ import {performance} from "node:perf_hooks";
import {parseArgs} from "node:util";
import {fileURLToPath} from "node:url";
import type {JsonObject} from "../../loopx/control_plane/effect_program.ts";
import type {AuthorityStore, AuthorityStoreCommit} from "../../loopx/control_plane/coordination/authority_store.ts";
import type {AuthorityStore, AuthorityStoreCommit, AuthorityStoreCommitResult} from "../../loopx/control_plane/coordination/authority_store.ts";
import {canonicalAuthorityBytes} from "../../loopx/control_plane/coordination/authority_store_codec.ts";
import {FileAuthorityStore} from "../../loopx/control_plane/coordination/file_authority_store.ts";
import {SqliteAuthorityStore} from "../../loopx/control_plane/coordination/sqlite_authority_store.ts";
import {sqliteRuntimeIdentity} from "../../loopx/control_plane/coordination/sqlite_runtime.ts";
Expand Down Expand Up @@ -84,6 +85,9 @@ async function measure(root: string) {
const finalProjection = projectionAt(count - 1);
let revision: string | null = null;
let first: AuthorityStoreCommit | undefined;
let firstResult: Extract<AuthorityStoreCommitResult, {status: "applied"}> | undefined;
const receiptsAt = (index: number) => [{operation_id: `op-${index}`, index,
metadata: {checked: true, labels: ["synthetic", "保留"]}}];
let filePublicationBytes = 0;
const fileBytes = (): number => readdirSync(root).reduce((sum, name) => sum + statSync(join(root, name)).size, 0);
const timed = async <T>(action: () => Promise<T>, into: number[]): Promise<T> => {
Expand All @@ -93,11 +97,11 @@ async function measure(root: string) {
for (let index = 0; index < count; index++) {
const input: AuthorityStoreCommit = {expected_provider_revision: revision, operation_id: `op-${index}`,
next_projection: projectionAt(index), events: [{kind: "observation", index}],
receipts: [{operation_id: `op-${index}`, index, metadata: {checked: true, labels: ["synthetic", "保留"]}}]};
receipts: receiptsAt(index)};
const result = await timed(() => store.commitAuthority(input), commits);
assert.equal(result.status, "applied"); if (result.status !== "applied") throw new Error("commit rejected");
revision = result.provider_revision;
if (index === 0) first = input;
if (index === 0) { first = input; firstResult = result; }
if (values.provider === "file") filePublicationBytes += statSync((store as FileAuthorityStore).path).size;
if ((index + 1) % 128 === 0) process.stderr.write(`${values.provider} ${values.workload}: ${index + 1}/${count}\n`);
}
Expand All @@ -109,6 +113,10 @@ async function measure(root: string) {
const receiptIndex = Math.floor(index * (count - 1) / (samples - 1));
const receipt = await timed(() => store.readReceipt(`op-${receiptIndex}`), reads);
assert.equal(receipt.status, "found");
if (receipt.status === "found") {
assert.equal(receipt.cursor, String(receiptIndex + 1));
assert.deepEqual(receipt.receipts, receiptsAt(receiptIndex));
}
const page = await timed(() => store.scanCommitted(String(count - 100), 100), scans);
assert.equal(page.status, "page");
if (page.status === "page") {
Expand All @@ -117,8 +125,8 @@ async function measure(root: string) {
const ordinal = count - 100 + offset;
assert.equal(row.operation_id, `op-${ordinal}`);
assert.deepEqual(row.projection, projectionAt(ordinal));
assert.deepEqual(row.receipts, [{operation_id: `op-${ordinal}`, index: ordinal,
metadata: {checked: true, labels: ["synthetic", "保留"]}}]);
assert.deepEqual(row.events, [{kind: "observation", index: ordinal}]);
assert.deepEqual(row.receipts, receiptsAt(ordinal));
}
}
}
Expand All @@ -132,13 +140,62 @@ async function measure(root: string) {
cold.push(performance.now() - started); assert.equal(child.status, 0, child.stderr);
assert.deepEqual(JSON.parse(child.stdout).head, finalProjection);
}
const reopened = openStore(root), replay = await reopened.commitAuthority(first!);
assert.equal(replay.status, "conflict"); // Current store contract reconciles via readReceipt.
const original = await reopened.readReceipt(first!.operation_id);
// Qualification is outside the timings. Walk bounded pages without retaining
// N full projections, checking independently generated input and exact history.
const reopened = openStore(root);
const historyDigest = async () => {
const digest = createHash("sha256");
let cursor: string | null = null, checked = 0;
for (;;) {
const page = await reopened.scanCommitted(cursor, 100);
assert.equal(page.status, "page"); if (page.status !== "page") throw new Error("history read rejected");
assert(page.transactions.length > 0);
for (const row of page.transactions) {
assert(checked < count, "history includes an unexpected transaction");
assert.equal(row.operation_id, `op-${checked}`);
assert.equal(row.cursor, String(checked + 1));
assert.deepEqual(row.projection, projectionAt(checked));
assert.deepEqual(row.events, [{kind: "observation", index: checked}]);
assert.deepEqual(row.receipts, receiptsAt(checked));
digest.update(canonicalAuthorityBytes(row)); digest.update("\n"); checked++;
}
if (!page.has_more) break;
assert.equal(page.next_cursor, String(checked));
cursor = page.next_cursor;
}
assert.equal(checked, count);
return digest.digest("hex");
};
assert(first && firstResult);
const before = await reopened.loadAuthority(), original = await reopened.readReceipt(first.operation_id);
assert.equal(before.status, "loaded");
if (before.status === "loaded") {
assert.equal(before.provider_revision, revision); assert.equal(before.cursor, String(count));
assert.deepEqual(before.head, finalProjection);
}
assert.equal(original.status, "found");
if (original.status === "found") assert.deepEqual(original.receipts, first!.receipts);
const after = await reopened.loadAuthority();
assert.equal(after.status, "loaded"); if (after.status === "loaded") assert.equal(after.provider_revision, revision);
if (original.status === "found") {
assert.equal(original.provider_revision, firstResult.provider_revision);
assert.equal(original.cursor, firstResult.cursor); assert.deepEqual(original.receipts, first.receipts);
}
const retainedHistory = await historyDigest();
// An identical historical intent returns its original result despite a stale
// basis. Projection-, event- and receipt-only drift must conflict, not append.
for (const change of ["none", "projection", "events", "receipts"] as const) {
const input = structuredClone(first);
if (change === "projection") input.next_projection.replay_marker = "different";
if (change === "events") input.events = [{kind: "observation", index: -1}];
if (change === "receipts") input.receipts = [{...receiptsAt(0)[0], replay_marker: "different"}];
const replay = await reopened.commitAuthority(input);
if (change === "none") assert.deepEqual(replay, firstResult);
else {
assert.equal(replay.status, "conflict", `${change}-only drift was accepted`);
if (replay.status === "conflict") assert.equal(replay.conflict_kind, "operation_id_exists");
}
assert.deepEqual(await reopened.loadAuthority(), before, `${change} changed the later head`);
assert.deepEqual(await reopened.readReceipt(first.operation_id), original, `${change} changed the original receipt`);
}
assert.equal(await historyDigest(), retainedHistory, "replay attempts changed retained history");
assert.deepEqual(sourceIdentity(), source, "measurement source changed while running");
return {schema_version: "loopx_local_provider_comparison_v0", provider: values.provider, workload: values.workload,
source, node: process.version, sqlite: sqliteRuntimeIdentity(), platform: process.platform, arch: process.arch,
Expand All @@ -148,5 +205,6 @@ async function measure(root: string) {
post_fill_rss_bytes: postFillRss,
final_store_bytes: fileBytes(), file_document_publication_bytes: values.provider === "file" ? filePublicationBytes : null,
complete_record_and_receipt_checks: "passed", original_receipt_recovery_after_reopen: "passed",
historical_replay_and_conflict_checks: "passed",
limits: "bounded sequential store experiment; cold process includes module loading, not cold OS cache; RSS includes fixture and verification allocations, not a steady-state qualification; no CLI, concurrent writers, crash, soak or formal D2 qualification; File publication bytes are application bytes, not physical writes; SQLite WAL traffic is measured by the separate capacity runner"};
}
30 changes: 30 additions & 0 deletions tests/control_plane_ts/local_provider_comparison.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import assert from "node:assert/strict";
import {spawnSync} from "node:child_process";
import {mkdir, mkdtemp, readFile, readdir, rm} from "node:fs/promises";
import {tmpdir} from "node:os";
import {join} from "node:path";
import {fileURLToPath} from "node:url";
import test from "node:test";

for (const provider of ["file", "sqlite"]) {
test(`${provider} comparison verifies historical replay after later commits and cleans its store`,
{timeout: 120000}, async t => {
const directory = await mkdtemp(join(tmpdir(), "local-provider-report-"));
t.after(() => rm(directory, {recursive: true, force: true}));
const data = join(directory, "tmp"), output = join(directory, "report.json");
await mkdir(data);
const child = spawnSync(process.execPath, ["--no-warnings", "--experimental-sqlite", "--experimental-strip-types",
fileURLToPath(new URL("../../examples/coordination/local-provider-comparison.ts", import.meta.url)),
"--provider", provider, "--workload", "mixed", "--commits", "128", "--samples", "3", "--output", output],
{encoding: "utf8", timeout: 110000, env: {...process.env, TMPDIR: data, TMP: data, TEMP: data}});
assert.equal(child.status, 0, child.stderr);
const report = JSON.parse(await readFile(output, "utf8"));
assert.equal(report.provider, provider);
assert.equal(report.commits, 128);
assert.equal(report.complete_record_and_receipt_checks, "passed");
assert.equal(report.original_receipt_recovery_after_reopen, "passed");
assert.equal(report.historical_replay_and_conflict_checks, "passed");
assert.equal(report.warm_head.n, 3);
assert.deepEqual(await readdir(data), []);
});
}
2 changes: 2 additions & 0 deletions tsconfig.control-plane.json
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,10 @@
"loopx/control_plane/work_items/task_lease_acquire_cli.ts",
"examples/nokv-authority-store/live-qualification.ts",
"examples/coordination/sqlite-capacity.ts",
"examples/coordination/local-provider-comparison.ts",
"examples/coordination/sqlite-authority-migration.ts",
"tests/control_plane_ts/sqlite_capacity.test.ts",
"tests/control_plane_ts/local_provider_comparison.test.ts",
"tests/control_plane_ts/effect_program.test.ts",
"tests/control_plane_ts/monitor_metadata.test.ts",
"tests/control_plane_ts/effect_runtime_errors.test.ts",
Expand Down
Loading