Skip to content

docs: prepare independent App first-use and repeat-use evaluation - #5329

Merged
huangruiteng merged 1 commit into
mainfrom
codex/first-use-evaluation-0930
Sep 30, 2026
Merged

huangruiteng merged 1 commit into
mainfrom
codex/first-use-evaluation-0930

Conversation

@huangruiteng

Copy link
Copy Markdown
Collaborator

The roadmap asks for five independent first-use attempts and later repeat use, but its targets have no shared participant handout or observation sheet. This prepares one App study around GQ01: a short cash-flow request, pinned-package/source/budget freeze, independent report checks, and five unrun slots that retain failures and withdrawals.

The protocol reuses existing first-run and usage-story feedback forms. It distinguishes complete onboarding, configuration, accepted-result availability, recovery, attention and measured/unknown cost. Preparing it does not qualify an installed journey, recruit participants, or replace the routing and two-cycle team acceptance gates.

Scope and continuation

  • Complete within this scope: the reusable study protocol and its EN/ZH roadmap, golden-query and adoption-loop references.
  • Remaining: fill the freeze sheet after the installed product journey is qualified, then run with five consenting first-time users; repeat use stays voluntary. No package, model or participant trial ran here.
  • This is a documentation-only preparation slice. Runtime, inbox, routing, telemetry and UI behavior remain outside the diff.

Validation

  • Tested revision: a68d5bf
  • Run state: finished
  • Input classes: synthetic (repository smokes); no participant input
Check kind Result Evidence / limitation
static passed All 10 introduced Markdown links and anchors resolve, including both existing GitHub feedback forms.
static passed uv run --extra test python examples/public-adoption-loop-doc-smoke.py; existing workflow copy and boundary intact.
static passed git diff --check and changed-document public-boundary scan; five files clean.
manual passed Unrun ledger, fixed denominators, no maintainer-rescue pass, optional public submission/citation, unknown subscription cost, and exact accepted-version timing reviewed.
static passed uv run --extra test loopx canary premerge --from-git-diff; 11 selected checks and 3 direct diff checks passed; no failed checks, required skips or manual holds. Validated candidate bytes were committed without further edits.
real_entrypoint not_applicable No runtime or UI change; installed App and participant results remain explicitly unqualified.

Initial canary setup missed repository npm development dependencies. After installing them with npm ci --ignore-scripts, the selected semantic check was rerun; it and the full selected premerge set passed. No paid model evaluations were run.

Quality receipt: cqr_960fffc67e1c42c8e64e. One bounded review repair clarified subscription-cost coverage and excluded retrospective scoring delay. No additional runtime abstraction was needed.

Frontend / visual evidence

UI impact: none. No first viewport, navigation, documentation chrome or packaged UI changes.

Boundary

Documentation update; public docs surface. Existing roadmap S1/S4/S12/S13 and G5 acceptance are referenced without adding a milestone. Shared-authority/TS fixture impact: N/A. No private state, credentials, raw traces, internal links or machine paths are included. Every commit is DCO signed off.

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

English verdict: APPROVE. No blocking finding in this documentation preparation slice.

Reviewed head: a68d5bf

动机

既有 roadmap 已有五人首次使用与次日复用目标,但缺少共同的参与者请求、冻结表和观察规则。本次提供可直接准备试用的完整协议,属于有价值的阶段增量;实际发行包验收和参与者试用仍未完成。

改动思路

沿用 GQ01、既有 G5 目标和 First run / Usage story 表单。先固定包、公开输入、独立答案依据、预算和五个样本,再使用普通 App 请求。详细观察留在私有研究记录,公开反馈和匿名引用分别自愿选择,不引入执行器、遥测或另一套收件箱。

具体改动

关键内容讲解

冻结表和简短双语请求让参与者不用记 Agent ID 或内部协议。观察表分别记录完整引导、就绪后配置、独立验收版本的可用时间、人工补位和已知/未知成本。五行均为未运行;失败、未参与与退出不替换。订阅成本未知不等于零,早期错误报告也不能提前停止计时。英文与中文 roadmap、golden queries 和 adoption loop 都指向同一协议。

对主干的风险

主要风险是把协议准备误读成产品已经通过验收。文档明确固定包、真实恢复与后续五人试用的缺口;维护者终端救援不计首次独立成功。实际安装、模型和参与者评测未运行,不能从文档检查推导这些结果。现有 adoption smoke、10 个新增链接/锚点、边界扫描以及风险选择的 11 项 premerge 检查和 3 项 diff 检查通过,无未解决失败、必需跳过或手工 hold。首次环境缺少 npm 开发依赖,安装后失败项与整组选中检查均通过;未查询或等待 GitHub CI。

我的整体评价

长期执行与已安装用户体验保持不变,本次只完善其评估准备。没有运行时默认、权限、兼容解码或首屏变更;状态分类、领域中立和指导/强制规则的运行时检查不适用。采用一个人工观察表,避免新增通用评分框架;有界修订已明确计时与未知成本。最终提交后的质量回执 cqr_960fffc67e1c42c8e64e 有效,远端补丁内容与本地最终 diff 一致。批准此文档范围,保持真实产品与试用验收开放;本次不合并。

@huangruiteng
huangruiteng merged commit 40852df into main Sep 30, 2026
21 of 22 checks passed
@huangruiteng
huangruiteng deleted the codex/first-use-evaluation-0930 branch September 30, 2026 03:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant