Skip to content

fix: correct IR Conv2D and Reshape execution and tighten accuracy validation - #108

Open
2813183274-cloud wants to merge 1 commit into
ScratchV-Compiler:mainfrom
2813183274-cloud:fix/issue-55-ir-precision
Open

2813183274-cloud wants to merge 1 commit into
ScratchV-Compiler:mainfrom
2813183274-cloud:fix/issue-55-ir-precision

Conversation

@2813183274-cloud

@2813183274-cloud 2813183274-cloud commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

问题与修复

1. 修复 Conv2D 的 im2col 排列错误

原实现中,输入展开后的排列顺序与卷积权重的展平顺序不一致,导致多输入通道卷积将错误的输入元素与权重相乘;多 batch 时也存在排列问题。

本次统一 im2col 与 NCHW 权重的展平顺序,并正确保留 batch 与输出位置的对应关系。

2. 修复 Reshape 的 IR 执行

原解释器未正确使用 IR 中记录的目标形状,经常回退为二维展平。

本次改为按照 IR 的目标 shape 执行:

  • 0:复制输入对应维度;
  • -1:自动推导维度;
  • 非法形状直接报错,不再通过回退掩盖问题。

最新主分支已经修复 ONNX Reshape 目标值的解析,因此本 PR 保留上游解析器实现,不重复修改。

3. 修正 IR 精度验证

原测试将 IR 解释结果称为“编译输出”,使用 MSE < 1.0、MAE < 1.0 的宽松阈值,并要求输出与参考结果不一致,无法可靠验证正确性。

本次:

  • 明确测试验证的是 IR 语义执行结果;
  • 按模型输出名称选择 ONNX Runtime 参考结果;
  • 要求输出 shape 完全一致;
  • 使用 rtol=1e-4、atol=1e-5 比较数值;
  • 删除“输出必须不一致”的断言。

验证结果

原修复分支:

  • 相关测试 123 项通过,无跳过。
  • Conv2D 直接验证 10 组,覆盖多通道、多 batch、stride、padding、1×1 和偏置,最大绝对误差不超过 1.91×10⁻⁶。
  • Reshape 直接验证 4 组,与 ONNX Runtime 完全一致。
  • 当前 CNN 在 seed=42 下,IR 输出与 ONNX Runtime 一致,最大绝对误差、MAE、MSE 均为 0。
  • 全量测试 943 项通过、1 项失败;失败原因是指令计数器测试未关闭临时文件,Windows 拒绝删除,与本次修复无关。

适配最新主分支后:

  • 模拟器与伪指令回归测试 95 项通过。
  • CNN 测试在上游解析器调用 onnx.shape_inference.infer_shapes 时发生 Windows 原生访问异常,尚未进入 IR 执行。该分支的 CNN 精度复测仍待完成,因此暂以草稿 PR 提交。

范围与后续工作

本 PR 仅修改 IR 解释器和 CNN 精度测试,不包含 standalone 后端改动、测试运行日志或规划文档。

主后端 Conv2D、GEMM、MaxPool、Sigmoid 的完整 lowering,以及主后端真实二进制精度验证仍需后续处理。

关联 #55,本 PR 不关闭该 Issue。

…idation

Align im2col rows with NCHW weights and preserve batch ordering. Read ONNX reshape target values and apply the target shape during IR execution. Validate IR output shapes and values against ONNX Runtime with strict tolerances.

Refs ScratchV-Compiler#55
@github-actions

Copy link
Copy Markdown

🤖 AI Code Review

共审查 1 个变更文件

📁 tests/test_cnn_pipeline.py

Code Review: tests/test_cnn_pipeline.py

Overall solid improvement — the shift from loose MSE/MAE thresholds to strict shape + assert_allclose verification is the right call. A few issues:


🟡 Redundant skipif condition — The __import__("importlib").util.find_spec("onnxruntime") pattern is repeated 3 times (lines ~330, ~339, ~347). Extract to a module-level constant to avoid drift and improve readability:

_ONNXRT_AVAILABLE = __import__("importlib").util.find_spec("onnxruntime") is not None

Or better, use pytest.importorskip("onnxruntime") inside the onnxrt_output fixture so all dependent tests skip automatically.


🟡 test_output_names_match_model — weak assertion message — set(onnxrt_output) == expected_names gives no diff on failure. Consider:

assert set(onnxrt_output) == expected_names, (
    f"Output name mismatch: runtime={set(onnxrt_output)}, "
    f"model={expected_names}"
)

🟡 Hardcoded single-output assumption — test_ir_output_matches_onnxruntime asserts len(output_names) == 1 and fails with a message. If IRTraceExecutor ever supports multi-output models, this test becomes a silent landmine. A pytest.skip() or pytest.xfail() with a clearer message would signal intent better than a bare assert.


💭 Tolerance constants lack rationale — IR_RTOL = 1e-4 / IR_ATOL = 1e-5 are reasonable for float32 IR vs. ONNX Runtime, but a one-line comment explaining the choice (e.g., "accounts for single-precision accumulation order differences") would help future readers tune them confidently.


💭 test_output_shape / test_output_range_reasonable — Both hardcode assumptions specific to this one CNN model (size == 1, sigmoid range). Fine for now, but adding a comment like # TODO: parameterize when testing additional models signals this is model-specific.


No blockers. The core verification logic (shape check + assert_allclose with named output lookup) is correct and a meaningful improvement over the previous MSE/MAE approach.


@2813183274-cloud
2813183274-cloud marked this pull request as ready for review October 15, 2026 07:19

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant