feat(eval): Phase 13.5 classification framework freeze + true end-to-end evaluator - #12
Merged
Conversation
added 5 commits
August 20, 2026 00:47
…lassification freeze doc (phase 13.5)
…il traces (pers_info)
…>prompt visibility mapping
…vel/grading declared not-frozen (pending clarification)
…isional reward coefficient; grading additive-extension; RL-init wording (no specific checkpoint frozen)
xianyu9n
force-pushed
the
phase13-5-e2e-freeze
branch
from
August 19, 2026 16:50
6303b65 to
6a11561
Compare
…trict parser; grading resolved by a new joint parser reusing answer-choice decode
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 13.5 — Classification Framework Freeze + true end-to-end evaluator
Commits(rebase 后)
da8252dfeat(eval): true end-to-end choice-protocol evaluator + unit tests + classification freeze doc7ed073ddocs(freeze): 各阶段真实 I/O 示例 + true-E2E 成功/失败 trace2f653f2docs(freeze): Field metadata 标注为待分类数据 + raw→prompt 可见性映射6b8714cdocs(freeze): 冻结范围收窄为 classification pipeline;data_level/grading not-frozen6a11561docs(freeze): PR review —— 合成候选 fixture / provisional reward 系数移出 frozen;grading additive 扩展;RL-init 措辞c8e38c9docs(freeze): PR review ——check_stage2_choices保持 frozen classification-only 严格 parser;grading 由新增 joint/grading-aware parser(复用 answer-choice decode)处理Changed files(相对 master)
script/verl/sft/evaluate_true_e2e.py— true E2E(Stage2 由 Stage1 预测 Top-5 动态构建;复用 build_stage1/2_prompt + check_stage{1,2}_choices + corpus)tests/evaluation/test_evaluate_true_e2e.py— Stage1 miss / hit+correct / hit+wrong / malformed 不 crash / candidates==预测 Top-5docs/design/framework-freeze.md— Classification Framework FreezeFreeze 文档口径(本 PR 修改摘要)
check_stage2_choices(及check_stage1_choices)是 frozen 的 classification-only 严格 parser;未来加入 grading 不修改它,而是新增 joint/grading-aware parser,复用answer choice → canonical category_id分类语义build_candidates为 fixture 非生产检索策略)、负采样 / hard-negative 策略、未来候选采样改进 ——build_candidates()本身不改stage2_partial=0.5数值)、后续任务定义确认的 reward weighting ——reward.py数值不动data_levelremains provenance only{"answer":"3","data_level":"L2"});冻结的是「answer choice → canonical category_id」,不是「只能有 answer 一个字段」Qwen/Qwen2.5-7B-Instruct;Phase 13 checkpoint = reproducible baseline / candidate downstream-init artifact,不冻结任何具体 checkpoint;不得用 test 选 checkpoint验证
proxy_e2e(PR11)与true_e2e明确区分bash -n/py_compile(PR11 文件)已在 PR feat(sft): Phase 13 正式 7B SFT/LoRA baseline(launcher + evaluator + merge tool) #11 验证data_level未纳入 supervised target