Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/native-conformance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ on:
workflow_dispatch:
inputs:
mode:
description: Candidate uses a temporary build; supported tests the production manifest.
description: Supported tests the production manifest; candidate rejects reviewed tuples.
type: choice
options:
- candidate-linux-x64
- supported
default: candidate-linux-x64
default: supported
required: true

permissions:
Expand Down
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@
namespace owner's account setup and a successful real publication.
- Add opt-in candidate native conformance, keeping experimental combinations
separate from the production support manifest.
- Extend Codex 0.166.0 native capture to Linux x64 after actual candidate
conformance on Ubuntu 24.04 with Node 22.20.0 and 24.8.0. Preserve the sanitized
run evidence and check the promoted production manifest separately.
- Prepare maintainer interviews, public-source pilot candidates, and reviewed
outreach drafts. External responses and useful-case outcomes remain pending.
- Add weekly public GitHub aggregate reports and a separate manual evidence
Expand Down
44 changes: 26 additions & 18 deletions docs/compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,22 @@
SkillWhy's offline comparison, demo, and sharing use Node.js 22 or later. Native
capture is a separate compatibility surface.

The initial adapter targets **Codex CLI 0.166.0 on macOS arm64**. Other versions or
platforms produce an `unsupported` snapshot instead of silently guessing. This is
an early alpha, not a promise that every configuration on that combination works.
Native capture admits **Codex CLI 0.166.0 on macOS arm64 and Linux x64**. Linux
evidence covers **Ubuntu 24.04 x64**, not every Linux distribution. Other CLI
versions or platform/architecture combinations produce an `unsupported` snapshot
instead of silently guessing. This is an early alpha, not a promise that every
configuration on an admitted combination works.

Actual alpha.2 metadata conformance runs passed on this combination with Node.js
Actual alpha.2 metadata conformance runs passed on macOS arm64 with Node.js
**22.20.0** and **24.8.0**. They covered duplicate names, an explicitly disabled
temporary skill, a missing-description error, and capture → diff → share.
See the [validation record](validation.md) for exact fixtures, timestamps,
reproduction commands, and limits. This evidence does not add another supported
Codex version or operating system.
reproduction commands, and limits. Linux x64 was admitted after reviewing both
Node versions in the live Ubuntu 24.04
[candidate run](https://github.com/hyscodebase/skillwhy/actions/runs/41456035667).
The original Linux build still returned `unsupported` in that run; the ten cases
passed through its explicitly labeled temporary candidate copy. A subsequent
`supported` run checks the admitted, unmodified production build.

The adapter requests a fresh process's `skills/list` response. It does not attach
to an existing Desktop chat or read a model transcript. The native result does not
Expand Down Expand Up @@ -48,24 +54,26 @@ the current Codex environment and is not a network-isolation or managed-policy
test. A passing default CI job validates offline and fake-process behavior;
**it is not evidence of live Codex support on each CI operating system**.

## Candidate platform checks
## Native verification and candidate history

Production decisions are centralized in `src/compatibility.ts`. Codex
0.166.0/Linux x64 is currently an unapproved candidate; normal capture still
returns `unsupported` there.
Production decisions are centralized in `src/compatibility.ts`. The two verified
Linux candidate summaries are preserved in
[the validation record](validation.md#alpha3-native-support-investigation).

The source-checkout script accepts `--candidate linux-x64` only on Linux x64.
It checks the original CLI's unsupported result, then runs the live fixture suite
against a temporary copy of the build with one compatibility-predicate override.
Before admission, the source-checkout script's `--candidate linux-x64` mode
checked the original CLI's unsupported result and then ran the fixture suite
against a temporary copy with one compatibility-predicate override. Now that the
tuple is reviewed, that experimental mode refuses it with
`CANDIDATE_ALREADY_SUPPORTED` on Linux x64. Use the ordinary `--live` command.
There is no candidate flag or environment-variable bypass in the shipped CLI.
The report labels this as `temporary-candidate`; it must not be presented as a
production compatibility result.
Historical `temporary-candidate` reports remain labeled as such.

The manually dispatched `Native conformance (opt-in)` workflow installs exactly
`@openai/codex@0.166.0` on Ubuntu 24.04 x64 and runs Node 22.20.0 and 24.8.0. Its
`candidate-linux-x64` mode tests this temporary copy. The `supported` mode instead
tests the unmodified production manifest and should fail on unapproved hosts.
Only sanitized successful summaries are uploaded, with the source revision.
default `supported` mode tests the unmodified production manifest and should fail
on unapproved hosts. The historical candidate mode is retained but rejects this
already reviewed tuple. Only sanitized successful summaries are uploaded, with
the source revision.

Review both native job results, their ten fixture outcomes, and any limitations
before adding a production tuple. Keep unsupported versions and architectures
Expand Down
12 changes: 8 additions & 4 deletions docs/execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ Preparing materials is not equivalent to completing an external validation gate.
| 2. External pilots | Self-serve guide and report form available | Three external attempts; at least two useful comparisons without author assistance |
| 3. Correct repeated pilot friction | Engineering review added explicit next steps for unsupported, partial, and unassociated results | Two repeated pilot blockers corrected and rechecked by participants |
| 4. npm distribution | Public alpha metadata and an optional packaging/publication workflow prepared | Namespace owner authenticates, first package publishes, clean registry install passes; then trusted publisher setup |
| 5. Broader native compatibility | A separate Linux x64 candidate suite is prepared; macOS production checks pass | Actual x64 conformance evidence, reviewed promotion, and production-manifest checks |
| 5. Broader native compatibility | Actual Ubuntu 24.04 x64 candidate checks passed on Node 22/24; macOS production checks pass | Production-manifest recheck after reviewed promotion |
| 6. Two useful external cases | Interview and evidence-register formats prepared | Two participant-confirmed, approved public accounts of help with an actual problem |
| 7. Demonstration and README | Actual synthetic CLI output rendered as a 42-second GIF/MP4, with transcript | Published assets and README links verified after release |
| 8. Maintenance automation | Shared test workflow, pinned actionlint, regression probes, and weekly dependency proposals prepared | Hosted CI passes and dependency configuration is present on the default branch |
| 9. Targeted introduction and weekly evidence | Two channel drafts, rules research, and public-aggregate reporting prepared | Approved account-based publication, response handling, and actual report execution |
| 7. Demonstration and README | 42-second GIF/MP4 and transcript published; public assets were fetched and matched their local hashes | Complete for the synthetic demonstration; actual user cases remain task 6 |
| 8. Maintenance automation | Shared CI and actionlint passed on the default branch; weekly dependency scans completed | Complete for the configured maintenance checks |
| 9. Targeted introduction and weekly evidence | Two channel drafts prepared; first hosted public-aggregate report succeeded | Approved account-based publication and response handling |

The initial external interview, completed-pilot, and approved-case registers are
empty. npm authentication was unavailable during preparation. These are pending
Expand All @@ -24,3 +24,7 @@ dependencies, not successful outcomes. Keep [the interview record](interviews.md
The weekly [metrics record](metrics.md) keeps repository interest signals separate
from actual use and participant-reviewed outcomes. The 1,000-star ambition remains
a growth objective, not a result established by tests or publication.

Engineering evidence: [integrated CI](https://github.com/hyscodebase/skillwhy/actions/runs/41456032940),
[Linux candidate checks](https://github.com/hyscodebase/skillwhy/actions/runs/41456035667),
and [the first weekly report](https://github.com/hyscodebase/skillwhy/actions/runs/41456039961).
2 changes: 1 addition & 1 deletion docs/interviews.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
## 파일럿 종료와 성공을 따로 판정하기

실행 명령은 중복 복사하지 않고 [pilot.md](pilot.md)를 따른다. 현재 native capture는
**Codex CLI 0.166.0 / macOS arm64**에 한정된다. 다른 환경에서는 합성 demo 피드백으로
**Codex CLI 0.166.0 / macOS arm64·Linux x64**를 지원한다. Linux 검증 환경은 Ubuntu 24.04다. 다른 환경에서는 합성 demo 피드백으로
끝낸다. 파일럿 때문에 Codex 설치·업그레이드·다운그레이드를 요구하지 않는다.

| 분류 | 완료 기준 | 유용한 실제 사례로 집계? |
Expand Down
9 changes: 9 additions & 0 deletions docs/metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,3 +70,12 @@ stay visible as unknown; the script does not increase token permissions.
Sources: GitHub's [repository API](https://docs.github.com/en/rest/repos/repos#get-a-repository),
[issues API](https://docs.github.com/en/rest/issues/issues#list-repository-issues), and
[release assets API](https://docs.github.com/en/rest/releases/assets#list-release-assets).

## First hosted observation

The [first manual workflow run](https://github.com/hyscodebase/skillwhy/actions/runs/41456039961)
completed successfully. Its observation window was 2026-11-02 13:47:44–13:47:45 UTC.
The downloaded JSON artifact recorded 0 stars, 0 forks, 0 external pilot issues,
and 3 release-tarball download requests across two prereleases. The manual
register contained 0 records; actual use, repeat use, and helpfulness remained
unknown. These are a dated baseline, not current totals or adoption outcomes.
28 changes: 28 additions & 0 deletions docs/native-evidence/candidate-run-41456035667-node-22.20.0.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"suite": "skillwhy-native-conformance-v1",
"status": "passed",
"checkedAt": "2026-11-02T13:47:57.652Z",
"hostVersion": "0.166.0",
"adapterVersion": "0.1.0-alpha.3",
"platform": "linux",
"arch": "x64",
"nodeVersion": "22.20.0",
"buildMode": "temporary-candidate",
"candidateTarget": "linux-x64",
"productionSupported": false,
"sourceRevision": "edba794da48015dfff0c31c87a8ab9423c6ac32e",
"cases": [
"enabled-project-fixture",
"duplicate-names-preserved",
"changed-skill-digest",
"cli-capture-diff-share",
"minimal-selected-share",
"session-disabled-fixture",
"enabled-state-diff",
"missing-description-partial",
"partial-diff-inconclusive",
"session-override-not-persisted"
],
"scope": "Fresh CLI metadata for temporary fixtures only; no model turns or skill execution.",
"privacy": "No raw inventory, user configuration, paths, or stderr are included in this report."
}
28 changes: 28 additions & 0 deletions docs/native-evidence/candidate-run-41456035667-node-24.8.0.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"suite": "skillwhy-native-conformance-v1",
"status": "passed",
"checkedAt": "2026-11-02T13:47:53.486Z",
"hostVersion": "0.166.0",
"adapterVersion": "0.1.0-alpha.3",
"platform": "linux",
"arch": "x64",
"nodeVersion": "24.8.0",
"buildMode": "temporary-candidate",
"candidateTarget": "linux-x64",
"productionSupported": false,
"sourceRevision": "edba794da48015dfff0c31c87a8ab9423c6ac32e",
"cases": [
"enabled-project-fixture",
"duplicate-names-preserved",
"changed-skill-digest",
"cli-capture-diff-share",
"minimal-selected-share",
"session-disabled-fixture",
"enabled-state-diff",
"missing-description-partial",
"partial-diff-inconclusive",
"session-override-not-persisted"
],
"scope": "Fresh CLI metadata for temporary fixtures only; no model turns or skill execution.",
"privacy": "No raw inventory, user configuration, paths, or stderr are included in this report."
}
8 changes: 4 additions & 4 deletions docs/outreach.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,11 @@

### 한국어

> 안녕하세요. SkillWhy를 만드는 홍유석입니다. [프로젝트의 문서/이슈 링크]에서 [직접 확인한 discovery 관련 내용]을 읽었습니다. 저는 fresh Codex CLI가 반환하는 skill 정보를 비교하고, 선택한 한 항목의 최소 진단 파일을 만드는 작은 OSS를 개발하고 있습니다. 이 방식이 실제 지원 과정에 필요한지 10분 정도 의견을 듣고 싶습니다. 현재 live capture는 Codex CLI 0.166.0 / macOS arm64만 지원하며, 다른 환경에서는 API 키 없이 합성 데모만 보셔도 됩니다. 원본 설정이나 snapshot을 보내실 필요는 없습니다. 관심이 있으시면 편한 방식으로 알려주세요. [파일럿 안내](https://github.com/hyscodebase/skillwhy/blob/main/docs/pilot.md)
> 안녕하세요. SkillWhy를 만드는 홍유석입니다. [프로젝트의 문서/이슈 링크]에서 [직접 확인한 discovery 관련 내용]을 읽었습니다. 저는 fresh Codex CLI가 반환하는 skill 정보를 비교하고, 선택한 한 항목의 최소 진단 파일을 만드는 작은 OSS를 개발하고 있습니다. 이 방식이 실제 지원 과정에 필요한지 10분 정도 의견을 듣고 싶습니다. 현재 live capture는 Codex CLI 0.166.0 / macOS arm64·Linux x64를 지원하며(Linux 검증은 Ubuntu 24.04), 다른 환경에서는 API 키 없이 합성 데모만 보셔도 됩니다. 원본 설정이나 snapshot을 보내실 필요는 없습니다. 관심이 있으시면 편한 방식으로 알려주세요. [파일럿 안내](https://github.com/hyscodebase/skillwhy/blob/main/docs/pilot.md)

### English

> Hi, I'm Yuseok, the maintainer of SkillWhy. I read [specific discovery detail] in [your project's document/issue]. I'm building a small OSS tool to compare the skill metadata returned by fresh Codex CLI processes and prepare a minimal support file for one selected entry. I'd value ten minutes of feedback on whether that helps your support workflow. Live capture currently supports only Codex CLI 0.166.0 on macOS arm64; the synthetic demo needs no Codex or API key. You don't need to send any raw configuration or snapshots. If this is relevant, you're welcome to try the [pilot](https://github.com/hyscodebase/skillwhy/blob/main/docs/pilot.md) or reply in whichever format suits you.
> Hi, I'm Yuseok, the maintainer of SkillWhy. I read [specific discovery detail] in [your project's document/issue]. I'm building a small OSS tool to compare the skill metadata returned by fresh Codex CLI processes and prepare a minimal support file for one selected entry. I'd value ten minutes of feedback on whether that helps your support workflow. Live capture currently supports Codex CLI 0.166.0 on macOS arm64 and Linux x64 (native Linux checks use Ubuntu 24.04); the synthetic demo needs no Codex or API key. You don't need to send any raw configuration or snapshots. If this is relevant, you're welcome to try the [pilot](https://github.com/hyscodebase/skillwhy/blob/main/docs/pilot.md) or reply in whichever format suits you.

후보별 첫 문장은 실제 관찰에 한정한다. C01에는 이미 존재하는 native verifier를
인정하고 작은 비교·공유 흐름의 필요성을 묻는다. C02·C05에는 Desktop 문제를
Expand Down Expand Up @@ -50,7 +50,7 @@ CLI 도구가 해결했다고 연결하지 않는다. C03에는 installer 문제
>
> 새 Codex CLI 프로세스가 반환한 목록을 저장하고, 활성 상태·수집한 SKILL.md fingerprint 등의 차이를 보여줍니다. 지원 요청에는 전체 설정 대신 직접 고른 한 항목의 최소 파일을 만들 수 있습니다. 기본 공유 파일에는 이름·경로·본문·fingerprint가 제외되며 자동 업로드는 없습니다.
>
> 현재 live capture는 **Codex CLI 0.166.0 / macOS arm64**만 지원합니다. 기존 Desktop 세션이나 모델의 실제 skill 선택·실행은 검사하지 않습니다. Node.js 22+와 Git이 있으면 다음 합성 데모를 실행할 수 있습니다. 최초 실행에는 다운로드와 빌드가 필요하고, 데모에는 Codex나 API 키가 필요하지 않습니다.
> 현재 live capture는 **Codex CLI 0.166.0 / macOS arm64·Linux x64**를 지원합니다. Linux에서 실제 검증한 환경은 Ubuntu 24.04입니다. 기존 Desktop 세션이나 모델의 실제 skill 선택·실행은 검사하지 않습니다. Node.js 22+와 Git이 있으면 다음 합성 데모를 실행할 수 있습니다. 최초 실행에는 다운로드와 빌드가 필요하고, 데모에는 Codex나 API 키가 필요하지 않습니다.
>
> `npx --yes github:hyscodebase/skillwhy#v0.1.0-alpha.3 demo`
>
Expand All @@ -74,7 +74,7 @@ CLI 도구가 해결했다고 연결하지 않는다. C03에는 installer 문제
>
> The implementation is TypeScript with no runtime dependencies. One deliberate boundary is that discovery does not prove a model selected, read, or executed a skill. SkillWhy also does not inspect an existing Desktop session or choose a winner among same-name copies.
>
> Live capture currently supports **Codex CLI 0.166.0 on macOS arm64 only**. For a synthetic demo, you need Node.js 22+ and Git, but no Codex installation or API key:
> Live capture currently supports **Codex CLI 0.166.0 on macOS arm64 and Linux x64**, with Linux native checks on Ubuntu 24.04. For a synthetic demo, you need Node.js 22+ and Git, but no Codex installation or API key:
>
> `npx --yes github:hyscodebase/skillwhy#v0.1.0-alpha.3 demo`
>
Expand Down
3 changes: 2 additions & 1 deletion docs/pilot.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ from live native observations.** Note your approximate time to first success.

## 2. Try your own snapshot, if already supported (about 5 minutes)

Live capture supports **Codex CLI 0.166.0 on macOS arm64 only**. Check your existing
Live capture supports **Codex CLI 0.166.0 on macOS arm64 and Linux x64**
(Linux native checks use Ubuntu 24.04). Check your existing
environment with `codex --version`; use `node --version` and
`node -p "process.platform + '/' + process.arch"` for the other runtime details.
If unsupported or absent, finish with demo feedback. Do not install, upgrade, or
Expand Down
Loading
Loading