diff --git a/.agents/skills/accept/SKILL.md b/.agents/skills/accept/SKILL.md index 761e6e4..97165f7 100644 --- a/.agents/skills/accept/SKILL.md +++ b/.agents/skills/accept/SKILL.md @@ -11,9 +11,10 @@ Read `LORE.md` (if present), `docs/spec.md` (`/accept` requirements), and `CONTE ## Step 0 — Bootstrap (mandatory) Read and execute `.agents/skills/matstudylab-bootstrap/SKILL.md` before any other step. -Do not proceed until bootstrap completes (sync, skip, or failed-with-continue). +Do not proceed until bootstrap completes (skip, setup/update, or warn-and-continue). +On `NODE_MISSING` or `SYNC_FAILED_CONTINUE`: **warn + continue** with vendored skills only — no three-option HITL menu (that menu is `/build` only). -**Completion criterion:** bootstrap outcome identified (fresh, synced, or failed-with-continue). +**Completion criterion:** bootstrap outcome identified (skip, setup/update OK, or warn-and-continue). ## Step 1 — Parse variant diff --git a/.agents/skills/ask-matt/PHASE-BOUNDARIES.md b/.agents/skills/ask-matt/PHASE-BOUNDARIES.md new file mode 100644 index 0000000..cb31e6a --- /dev/null +++ b/.agents/skills/ask-matt/PHASE-BOUNDARIES.md @@ -0,0 +1,55 @@ +# Phase boundaries + +A **phase** is a chunk of work inside a session — the grilling, the implementation, the QA. The definition is fuzzy on purpose: a phase ends when you think *"ok, we're done with that"*. + +The **phase boundary** is the gap between two phases, and it is the only place this decision belongs. Mid-phase there is no decision to make — continue, or split the work that's left into subagents. Compacting mid-phase makes the agent lose the thread. + +## The five options + +| Option | What it does | +| ------------ | --------------------------------------------------------------- | +| **Continue** | Stay in the session. No context switch at all. | +| **`/clear`** | Empty the context window and start from nothing. | +| **`/handoff`** | Write a portable markdown file and seed a session anywhere with it. | +| **Subagent** | Send the task to its own context window and get a report back. | +| **`/compact`** | Compress this context and seed a fresh session with the summary. | + +## The tree + +Work top to bottom at the boundary. The first **yes** wins. + +**1. Can you continue in this session?** Two things make the answer yes: the next phase needs this phase as a **primary source**, or you have enough [smart zone](https://www.aihero.dev/ai-coding-dictionary/smart-zone) left (~150k tokens) for the next phase to fit. Grilling → implementation is the standard yes: the implementation wants the reasoning verbatim, not a summary of it. Continue costs nothing and loses nothing, so rule it out before anything else. + +**2. Is the context irrelevant to what comes next?** Is everything in this session — the exploration, the decisions, the dead ends — disposable? If so, **`/clear`**. It is the cheapest move on the board: it takes no time and hands back the whole window. `/clear` also isn't terminal — the old session stays resumable. + +The cost of getting this wrong is one-way. Clear a *relevant* context and you lose the **why** behind what you built, and no amount of reading the diff back gets it returned. + +**3. Do you need to hand off?** `/handoff` is narrow. You need it only when you are: + +- swapping to a **new harness** (Claude → Codex), +- moving to a **new directory** or repo, +- sending the work to a **colleague**, +- or forking a side task you found **mid-phase** without derailing what you're doing. + +That list is the whole clause. What `/handoff` buys is **portability** — a file that travels. If nothing is travelling, you don't need it. + +**4. Can the task be done AFK?** Is it scoped tightly enough to run with you away from the keyboard, no steering? Then send it to a **subagent** and leave this session untouched. Automated review is the standard case: the agent reads the diff and reports, and you aren't needed while it does. + +**5. Otherwise, `/compact`.** Relevant context, same harness, same directory, and you need to stay in the loop — this is where the tree lands, and it lands here often. Pass it an instruction (`/compact we're going to QA this area`) so the summary keeps what the next phase needs. + +`/compact` is the **default, not the first reach**. It sits at the bottom because the four questions above it are all cheaper or more precise. The failure mode when people start here is a fresh session that is confidently wrong about a decision the summary flattened. + +## Primary and secondary sources + +Every move except **Continue** turns a **primary source** into a **secondary source** — the session as it happened, replaced by a summary of it. The trade is always the same shape: + +| Source | Information | Noise | Room to move | +| --------------------------------- | ----------- | ----- | ------------ | +| Primary (Continue) | Full | Lots | Little | +| Secondary (`/compact`, `/handoff`) | Lossy | Less | Lots | + +This is why question 1 comes first. You only pay the lossiness when staying costs more than it saves. + +## These are judgement calls + +The questions are not objective — each has taste in it, and the same boundary can go two ways on two days. The value is in asking them **in order**, at the boundary rather than in the middle of the work. diff --git a/.agents/skills/ask-matt/SKILL.md b/.agents/skills/ask-matt/SKILL.md index cc1866a..7f3ab78 100644 --- a/.agents/skills/ask-matt/SKILL.md +++ b/.agents/skills/ask-matt/SKILL.md @@ -14,13 +14,13 @@ A **flow** is a path through the skills. Most paths run along one **main flow**, The route most work travels. You have an idea and want it built. -1. **`/grill-with-docs`** — sharpen the idea by interview. Start here when you **have a codebase**: it's stateful, retaining what it learns in `CONTEXT.md` and ADRs. (No codebase? Use `/grill-me` — see Standalone. Both run the same `/grilling` primitive; `grill-with-docs` is the one that leaves a paper trail.) -2. **Branch — can you settle every question in conversation?** If a question needs a runnable answer (state, business logic, a UI you have to see), detour through a prototype, bridged by **`/handoff`** in both directions (see Crossing sessions): +1. **`/grill-with-docs`** — sharpen the idea by interview. Start here whenever you are **working in a working directory**: it's stateful, retaining what it learns in `CONTEXT.md` and ADRs. (No working directory? Use `/grill-me` — see Standalone. Both run the same `/grilling` primitive; `grill-with-docs` is the one that leaves a paper trail, which makes it the better of the two whenever a repo is there to leave it in.) +2. **Branch — can you settle every question in conversation?** If a question needs a runnable answer (state, business logic, a UI you have to see), detour through a prototype, bridged by **`/handoff`** in both directions (a prototype lives in its own directory, which is exactly what `/handoff` is for — see Phase boundaries): - **`/handoff`** out, then open a fresh session against that file, - **`/prototype`** to answer the question with throwaway code, - **`/handoff`** back what you learned, and reference it from the original idea thread. 3. **Branch — is this a multi-session build?** - - **Yes** → **`/to-spec`** (turn the thread into a spec), then **`/to-tickets`** to split it into tracer-bullet tickets, each declaring its **blocking edges**. On a local tracker that's an ordered `tickets.md` you work by hand; on a real tracker the edges become native blocking links, so any ticket whose blockers are done can be grabbed — kick off **`/implement`** per ticket, **clearing context between each one**. + - **Yes** → **`/to-spec`** (turn the thread into a spec), then **`/to-tickets`** to split it into tracer-bullet tickets, each declaring its **blocking edges**. On a local tracker that's one file per ticket under `.scratch//issues/`, worked blockers-first by hand; on a real tracker the edges become native blocking links, so any ticket whose blockers are done can be grabbed — kick off **`/implement`** per ticket, **`/clear`ing context between each one**. Each ticket is self-contained, so the last one's context is disposable. - **No** → **`/implement`** right here, in the same context window. Either way, **`/implement`** builds each issue by driving **`/tdd`** internally — one red-green slice at a time — then closes out by running **`/code-review`**, a two-axis review (Standards + Spec) of the diff, before committing. Reach for **`/tdd`** on its own when you just want to build a concrete behaviour test-first without a full spec, and **`/code-review`** on its own whenever you want to review a branch or PR against a fixed point. @@ -29,7 +29,7 @@ The route most work travels. You have an idea and want it built. Keep steps 1–3 in **one unbroken context window** — don't compact or clear until after `/to-tickets` — so the grilling, spec, and tickets all build on the same thinking. Each `/implement` then starts fresh, working from the ticket. -The limit on this is the **[smart zone](https://www.aihero.dev/ai-coding-dictionary/smart-zone)**: the window (~120k tokens on state-of-the-art models) within which the model still reasons sharply. If a session approaches it before `/to-tickets`, don't push on degraded — `/handoff` and continue in a fresh thread. +The limit on this is the **[smart zone](https://www.aihero.dev/ai-coding-dictionary/smart-zone)**: the window (~150k tokens on state-of-the-art models) within which the model still reasons sharply. If a session approaches it before `/to-tickets`, don't push on degraded — `/compact` at the nearest phase boundary and carry on (see Phase boundaries). ## On-ramps @@ -41,7 +41,9 @@ A starting situation that generates work, then merges onto the main flow. - **Something's broken** → **`/diagnosing-bugs`**. For the hard ones: the bug that resists a first glance, the intermittent flake, the regression that crept in between two known-good states. It refuses to theorise until it has a **tight feedback loop** — one command that already goes red on *this* bug — then fixes with a regression test. Its post-mortem hands off to **`/improve-codebase-architecture`** when the real finding is that there's no good seam to lock the bug down. -- **A huge, foggy effort — a greenfield project or a huge feature build, too big for one session** → **`/wayfinder`**. When the way from here to the destination isn't visible yet, it charts a **shared map** of investigation tickets on the issue tracker and resolves them one at a time — producing **decisions, not deliverables** — until the fog is pushed back and the way is clear. Then it merges onto the main flow at **`/to-spec`** (or, if the effort turned out small enough, straight to **`/implement`**). Where **`/grill-with-docs`** sharpens an idea you can hold in one session, wayfinder is for the idea you can't. +- **A huge, foggy effort — a greenfield project or a huge feature build, too big for one session** → **`/wayfinder`**, the most cognitively demanding flow here. When the way from here to the destination isn't visible yet, it charts a **shared map** of **decision tickets** on the issue tracker and resolves them one at a time — producing **decisions, not deliverables** — until the fog is pushed back and the way is clear. Where **`/grill-with-docs`** sharpens an idea you can hold in one session, wayfinder is for the idea you can't — and it's slower and denser, so save it for exactly that, never a well-scoped feature. + + When the map clears, **it hands off, it doesn't build**: merge onto the main flow at **`/to-spec`**, which collapses the map's linked decisions into a buildable plan, then `/to-tickets` and `/implement` as usual. Looping the map straight into `/implement` skips that collapse and throws the linked detail away — go straight to `/implement` only when the effort turned out genuinely small. ## Codebase health @@ -56,20 +58,32 @@ Two model-invoked references that run *beneath* the other skills — each the si - **`/domain-modeling`** — sharpen the project's *domain* language: challenge a fuzzy term, resolve an overloaded word ("account" doing three jobs), record a hard-to-reverse decision as an ADR. It's the active discipline `/grill-with-docs` drives to keep `CONTEXT.md` a clean glossary. - **`/codebase-design`** — the deep-module vocabulary (module, interface, depth, seam, adapter, leverage, locality) for designing a module's *shape*: a lot of behaviour behind a small interface at a clean seam. `/tdd` and `/improve-codebase-architecture` both speak it. -## Crossing sessions +## Phase boundaries + +A **phase** is a chunk of work inside a session — the grilling, the implementation, the QA. At the **boundary** between two of them you have five options, and picking between them is the fuzziest decision in this whole map: + +- **Continue** — stay put. Costs nothing, loses nothing. +- **`/clear`** — empty the window, when nothing here matters to what's next. +- **`/handoff`** — write a portable markdown file. Narrow: only for a **new harness**, a **new directory**, a **colleague**, or forking a side task **mid-phase**. What it buys is portability. +- **Subagent** — send a tightly-scoped task to its own window and get a report back. +- **`/compact`** — compress this context and seed a fresh session with it. The **default**, at the bottom of the tree rather than the first reach. -- **`/handoff`** — when a thread is full or you need to branch off (e.g. into a `/prototype` session), this compacts the conversation into a markdown file. You don't continue in place — you **open a new session and reference that file** to carry the context across. It's the bridge between context windows, in either direction. Use it when you want a **fresh session** but need the **current conversation preserved**. -- **`/compact`** (built-in) — stay in the **same conversation**, letting the earlier turns be summarized. Use it at **intentional breaks between phases**, when you don't mind losing the verbatim history. Don't compact mid-phase — the agent can lose its way. `/handoff` forks; `/compact` continues. +Read [PHASE-BOUNDARIES.md](PHASE-BOUNDARIES.md) for the ordered tree — the five questions, the reasoning behind each branch, and why the primary-source cost makes **Continue** the one to rule out first. Make the decision **at** a boundary; mid-phase, continue or split the rest into subagents. ## Standalone Off the main flow entirely. -- **`/grill-me`** — the same relentless interview as `/grill-with-docs`, but for when you have **no codebase**. Stateless: it saves nothing locally, builds no `CONTEXT.md`. Reach for it to sharpen any plan or design that doesn't live in a repo. -- **`/prototype`** — a small, throwaway program that answers one design question: does this state model feel right, or what should this UI look like. Throwaway from day one — keep the answer, delete the code. It's the detour in step 2 of the main flow, but reach for it any time a design question is hard to settle on paper. +- **`/grill-me`** — the same relentless interview as `/grill-with-docs`, but **stateless**: it saves nothing locally and builds no `CONTEXT.md`. Reach for it when you are **not working in a working directory** — sharpening a plan, a design, a piece of writing, anything with no repo under it. If you are in a working directory, use `/grill-with-docs` instead: it runs the same interview and leaves a paper trail, so it is strictly the better one. +- **`/grilling`** — the interview primitive itself: rounds, the frontier, facts are the agent's job and decisions are yours. `/grill-me` and `/grill-with-docs` are the two named ways in, and `/triage`, `/wayfinder` and `/improve-codebase-architecture` all run it internally. Reach for it directly only when you want the interview with no wrapper around it. +- **`/resolving-merge-conflicts`** — work an in-progress merge or rebase conflict hunk by hunk, resolving by **intent** traced to each side's primary source rather than by picking lines, then finish the operation. It never runs `--abort`. Standalone and off every flow: reach for it when you are already mid-conflict. +- **`/prototype`** — a small, throwaway program that answers one design question: does this state model feel right, or what should this UI look like. Throwaway is a constraint on how the code is written, not a promise to destroy it: the answer folds into the real code, and the prototype itself is kept as a **primary source** on a `prototype/` branch out of main, pointed at from the implementation issue. It's the detour in step 2 of the main flow, but reach for it any time a design question is hard to settle on paper. - **`/research`** — delegate reading legwork to a **background agent**: it investigates a question against **primary sources**, then leaves a cited Markdown file in the repo. Keep working while it reads. The file it produces is something to take *into* the main flow at `/grill-with-docs` — research feeds the thinking, it doesn't replace it. +- **`/to-questionnaire`** — when the thing blocking you isn't in your head or the codebase but in **someone else's**, this writes them a questionnaire to fill in. It's the inverse of `/grill-me`: instead of interviewing you about the subject, it interviews you about the **send** — who it's going to, what you need back — and aims the questions at the gap. What comes back is material for `/grill-with-docs` or `/to-spec`. +- **`/wizard`** — for the steps only a **human** can take: provisioning infrastructure, setting up credentials or CI secrets, clicking through an unfamiliar third-party dashboard, running a one-off migration or cutover. It generates an interactive bash script that opens each URL, captures each value, and writes it into `.env` and GitHub secrets — so the procedure stops being something you re-explain to an agent every time. Model-invoked, so the agent reaches for it the moment it hits a wall only you can pass. If the agent could just do it itself, it should; this is for where a human is genuinely in the loop. +- **`/wait-what`** — the corrective for a message that didn't land. Use it mid-conversation, inside any other skill, and the agent re-pitches what it just said with the context you were missing, in plain English, using the `CONTEXT.md` vocabulary. It works after the fact; `/grill-with-docs` is the upfront cure, because a shared language agreed early is what stops the jargon arriving at all. - **`/teach`** — learn a concept over multiple sessions, using the current directory as a stateful workspace. -- **`/writing-great-skills`** — reference for writing and editing skills well. +- **`/writing-for-agents`** — reference for writing documents agents consume: skills, AGENTS.md, pointed-at docs. ## Precondition diff --git a/.agents/skills/ask-matt/agents/openai.yaml b/.agents/skills/ask-matt/agents/openai.yaml new file mode 100644 index 0000000..5c60d51 --- /dev/null +++ b/.agents/skills/ask-matt/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Ask Matt" + short_description: "Find the right skill or workflow" +policy: + allow_implicit_invocation: false diff --git a/.agents/skills/build/SKILL.md b/.agents/skills/build/SKILL.md index 7ecda13..724b2a2 100644 --- a/.agents/skills/build/SKILL.md +++ b/.agents/skills/build/SKILL.md @@ -8,20 +8,83 @@ Onboard lab scripts from `import/` into `codes///` with user confi Read `LORE.md`, `AGENTS.md`, `docs/spec.md`, and `CONTEXT.md` before acting. +English is the default for HITL copy below. Localize only when `LORE.md` or a `/build` language choice says so. + ## Step 0 — Bootstrap (mandatory) Read and execute `.agents/skills/matstudylab-bootstrap/SKILL.md` before any other step. -Do not proceed until bootstrap completes (sync, skip, or failed-with-continue). +Do not proceed until bootstrap completes (skip, setup/update, or warn-and-continue). + +**Completion criterion:** bootstrap outcome identified (skip, setup/update OK, or warn-and-continue). + +## Step 1 — Skills HITL + +After Step 0, branch on the bootstrap outcome. Canonical English strings live in `.scratch/skills-auto-provisioning-v2/assets/prototype-node-gate-messages.md` (Scenarios A–G). Prefer those exact texts. + +Preference sidecar: `.matstudylab/skills-pref.json` (`mode`: `auto` | `local-only`, `last_synced_at`). Absence ⇒ treat as `auto`. Never auto-install Node. + +### Helper — write preference mode + +Preserve `last_synced_at` unless the bootstrap script just stamped it. From repo root: + +```bash +python3 - <<'PY' +from pathlib import Path +import sys +sys.path.insert(0, "scripts/lib") +from skills_bootstrap import load_pref, write_pref +path = Path(".matstudylab/skills-pref.json") +data = load_pref(path) +data["mode"] = "auto" # or "local-only" +write_pref(path, data) +PY +``` + +### Branch A — `local-only` (`bootstrap: SKIPPED_LOCAL_ONLY`) + +Show Scenario A. Do **not** probe Node or sync. + +| Choice | Action | +|--------|--------| +| Continue with /build | Proceed to Step 2 | +| Re-enable sync (switch to auto mode) | Write `mode: auto`, re-run Step 0 bootstrap, then re-enter Step 1 on the new outcome | + +### Branch B — Node/`npx` missing (`bootstrap: NODE_MISSING`) + +Show Scenario B (three options). + +| Choice | Action | +|--------|--------| +| Continue with vendored skills (this session only) | Proceed to Step 2 (no pref write) | +| Continue and remember: do not sync (local-only) | Write `mode: local-only`, then Step 2 | +| Open setup — how to install Node and sync skills | Detect OS; print **only** the matching block from `docs/agents/skills-node-os-checklist.md`. Never auto-install. When the user has Node/`npx`, re-run Step 0; on `SETUP_OK` / `UPDATE_OK` go to Branch E | + +### Branch C — Sync failed (`bootstrap: SYNC_FAILED_CONTINUE`) + +Show Scenario C (include a short stderr / exit detail). Same three-option table as Branch B, except option 3 is **Retry setup/update** (re-run Step 0 when Node is present). + +### Branch D — Incomplete setup (no successful sync yet) + +Trigger when preference is `auto`/absent **and** setup is still incomplete after Step 0: empty/missing `skills` map in `skills-lock.json`, **or** `last_synced_at` is null, **and** you did **not** just get `SETUP_OK` / `UPDATE_OK` / `SKIPPED_FRESH`. Prefer Branch B or C when those markers also apply. + +Show Scenario D. Same three options; option 3 is **Open setup — sync now** (OS checklist if no Node; else re-run Step 0). + +### Branch E — Quiet success + +| Outcome | Message | +|---------|---------| +| `SETUP_OK` / `UPDATE_OK` | Scenario G. Ensure `mode: auto` (script stamps `last_synced_at`). Continue to Step 2 — no menu. | +| `SKIPPED_FRESH` | Scenario E one-liner. Continue to Step 2 — no menu. | -**Completion criterion:** bootstrap outcome identified (fresh, synced, or failed-with-continue). +**Completion criterion:** HITL branch resolved — user choice applied (pref written when required), quiet success acknowledged, or setup path waiting on Node install / re-bootstrap; then proceed to Step 2 only when continuing `/build` this turn. -## Step 1 — LORE onboarding (first run) +## Step 2 — LORE onboarding (first run) If `LORE.md` is missing or lacks companion `.md` language / git workflow, ask the user and write answers to `LORE.md` from `docs/templates/LORE.md`. **Completion criterion:** companion language and git workflow are recorded (or user declined onboarding). -## Step 2 — Scan `import/` +## Step 3 — Scan `import/` ```bash ./scripts/build-import.sh scan @@ -31,7 +94,7 @@ Recursively list bundle proposals grouped by folder under `import/`. **Completion criterion:** the user sees proposed bundles and optical magnitude (`/`) for each. -## Step 3 — Grill-me (mandatory before any write) +## Step 4 — Grill-me (mandatory before any write) Run `.agents/skills/grill-me/SKILL.md` (→ `/grilling`) on the catalog plan: types, bundle names, 1:1 vs N:1, files to move, and what stays in `import/` for later sessions. @@ -39,7 +102,7 @@ Run `.agents/skills/grill-me/SKILL.md` (→ `/grilling`) on the catalog plan: ty **Completion criterion:** user confirmed the catalog plan. -## Step 4 — Homonym check +## Step 5 — Homonym check For each bundle: @@ -54,7 +117,7 @@ For each bundle: **Completion criterion:** every bundle has a resolved homonym decision. -## Step 5 — Catalog confirmed bundles +## Step 6 — Catalog confirmed bundles Move only user-confirmed bundles (directly to `codes/`, not via `new/`): @@ -68,7 +131,7 @@ New `codes//` folders require user confirmation; append folder map + decis **Completion criterion:** each confirmed bundle exists under `codes/`; only cataloged files were removed from `import/`. -## Step 6 — Post-build handoff +## Step 7 — Post-build handoff - Recommend `/explain` for priority scripts. - If LORE git workflow is `own-repo`, propose a commit after user OK (`catalog: /`). @@ -80,9 +143,14 @@ New `codes//` folders require user confirmation; append folder map + decis - Never modify `codes/` without explicit user confirmation per bundle. - Never delete from `import/` except files cataloged in this session. - Never delete or overwrite measurement data outside the confirmed bundle folder move. +- Never auto-install Node or run package managers for the user; OS checklists are display-only. ## Reference - Wayfinder spec: `.scratch/matstudylab/issues/11-grilling-comando-build.md` - Companion template: `docs/templates/script-companion.md` - Homonym + incremental rules: `docs/spec.md` (`/build`, US32–33) +- HITL decisions: `.scratch/skills-auto-provisioning-v2/issues/02-grilling-build-hitl-dialogue.md` +- Canonical English HITL copy: `.scratch/skills-auto-provisioning-v2/assets/prototype-node-gate-messages.md` +- OS install checklists: `docs/agents/skills-node-os-checklist.md` +- Manual HITL verification: `docs/agents/build-skills-hitl-checklist.md` diff --git a/.agents/skills/claude-handoff/SKILL.md b/.agents/skills/claude-handoff/SKILL.md new file mode 100644 index 0000000..69b26bc --- /dev/null +++ b/.agents/skills/claude-handoff/SKILL.md @@ -0,0 +1,18 @@ +--- +name: claude-handoff +description: Hand the current conversation off to a fresh background agent that picks up the work immediately. +argument-hint: "What will the next session be used for?" +disable-model-invocation: true +--- + +Write a handoff summary of the current conversation so a fresh agent can continue the work. Instead of saving it, launch a background agent seeded with the summary as its prompt: `claude --bg --name "" ""`. It starts in the current working directory and returns immediately; the user manages it with `claude agents`. + +Always pass `-n`/`--name` with a descriptive name (e.g. `--name "Fix login bug"`) — it sets the display name shown in the job list, session picker, and terminal title. + +Include a "suggested skills" section in the summary, which suggests skills that the agent should invoke. + +Do not duplicate content already captured in other artifacts (specs, plans, ADRs, issues, commits, diffs). Reference them by path or URL instead. + +Redact any sensitive information, such as API keys, passwords, or personally identifiable information — the summary becomes the agent's prompt. + +If the user passed arguments, treat them as a description of what the next session will focus on and tailor the summary accordingly. diff --git a/.agents/skills/claude-handoff/agents/openai.yaml b/.agents/skills/claude-handoff/agents/openai.yaml new file mode 100644 index 0000000..0a7aa5d --- /dev/null +++ b/.agents/skills/claude-handoff/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Claude Handoff" + short_description: "Hand off to a background agent" +policy: + allow_implicit_invocation: false diff --git a/.agents/skills/code-review/SKILL.md b/.agents/skills/code-review/SKILL.md index 2a0b524..2d276fe 100644 --- a/.agents/skills/code-review/SKILL.md +++ b/.agents/skills/code-review/SKILL.md @@ -1,12 +1,12 @@ --- name: code-review -description: Review the changes since a fixed point (commit, branch, tag, or merge-base) along two axes — Standards (does the code follow this repo's documented coding standards?) and Spec (does the code match what the originating issue/PRD asked for?). Runs both reviews in parallel sub-agents and reports them side by side. Use when the user wants to review a branch, a PR, work-in-progress changes, or asks to "review since X". +description: Review the changes since a fixed point (commit, branch, tag, or merge-base) along two axes — Standards (does the code follow this repo's documented coding standards?) and Spec (does the code match what the originating issue/spec asked for?). Runs both reviews in parallel sub-agents and reports them side by side. Use when the user wants to review a branch, a PR, work-in-progress changes, or asks to "review since X". --- Two-axis review of the diff between `HEAD` and a fixed point the user supplies: - **Standards** — does the code conform to this repo's documented coding standards? -- **Spec** — does the code faithfully implement the originating issue / PRD / spec? +- **Spec** — does the code faithfully implement the originating issue / spec? Both axes run as **parallel sub-agents** so they don't pollute each other's context, then this skill aggregates their findings. @@ -28,7 +28,7 @@ Look for the originating spec, in this order: 1. Issue references in the commit messages (`#123`, `Closes #45`, GitLab `!67`, etc.) — fetch via the workflow in `docs/agents/issue-tracker.md`. 2. A path the user passed as an argument. -3. A PRD/spec file under `docs/`, `specs/`, or `.scratch/` matching the branch name or feature. +3. A spec file under `docs/`, `specs/`, or `.scratch/` matching the branch name or feature. 4. If nothing is found, ask the user where the spec is. If they say there isn't one, the **Spec** sub-agent will skip and report "no spec available". ### 3. Identify the standards sources @@ -57,8 +57,6 @@ Each smell reads *what it is* → *how to fix*; match it against the diff: ### 4. Spawn both sub-agents in parallel -Send a single message with two `Agent` tool calls. Use the `general-purpose` subagent for both. - **Standards sub-agent prompt** — include: - The full diff command and commit list. diff --git a/.agents/skills/code-review/agents/openai.yaml b/.agents/skills/code-review/agents/openai.yaml new file mode 100644 index 0000000..9076774 --- /dev/null +++ b/.agents/skills/code-review/agents/openai.yaml @@ -0,0 +1,3 @@ +interface: + display_name: "Code Review" + short_description: "Review a diff on standards and spec" diff --git a/.agents/skills/codebase-design/DESIGN-IT-TWICE.md b/.agents/skills/codebase-design/DESIGN-IT-TWICE.md index 49a7c42..8419ad6 100644 --- a/.agents/skills/codebase-design/DESIGN-IT-TWICE.md +++ b/.agents/skills/codebase-design/DESIGN-IT-TWICE.md @@ -18,7 +18,7 @@ Show this to the user, then immediately proceed to Step 2. The user reads and th ### 2. Spawn sub-agents -Spawn 3+ sub-agents in parallel using the Agent tool. Each must produce a **radically different** interface for the deepened module. +Spawn 3+ sub-agents in parallel. Each must produce a **radically different** interface for the deepened module. Prompt each sub-agent with a separate technical brief (file paths, coupling details, dependency category from [DEEPENING.md](DEEPENING.md), what sits behind the seam). The brief is independent of the user-facing problem-space explanation in Step 1. Give each agent a different design constraint: diff --git a/.agents/skills/codebase-design/agents/openai.yaml b/.agents/skills/codebase-design/agents/openai.yaml new file mode 100644 index 0000000..3180715 --- /dev/null +++ b/.agents/skills/codebase-design/agents/openai.yaml @@ -0,0 +1,3 @@ +interface: + display_name: "Codebase Design" + short_description: "Vocabulary for deep-module design" diff --git a/.agents/skills/diagnosing-bugs/SKILL.md b/.agents/skills/diagnosing-bugs/SKILL.md index f400de7..7f8acf7 100644 --- a/.agents/skills/diagnosing-bugs/SKILL.md +++ b/.agents/skills/diagnosing-bugs/SKILL.md @@ -9,6 +9,12 @@ A discipline for hard bugs. Skip phases only when explicitly justified. When exploring the codebase, read `CONTEXT.md` (if it exists) to get a clear mental model of the relevant modules, and check ADRs in the area you're touching. +## Redact + +This skill has you show commands, outputs and captured artifacts. **Redact every secret first** — write `` in its place. Build loops against env vars, so the credential stays in the environment rather than in what you show. Captured artifacts carry auth headers: quote only the lines that carry the signal. + +If the redacted output is not enough to diagnose the bug, say so and ask the user. + ## Phase 1 — Build a feedback loop **This is the skill.** Everything else is mechanical. If you have a **tight** pass/fail signal for the bug — one that goes red on _this_ bug — you will find the cause; bisection, hypothesis-testing, and instrumentation all just consume it. If you don't have one, no amount of staring at code will save you. @@ -46,11 +52,11 @@ The goal is not a clean repro but a **higher reproduction rate**. Loop the trigg ### When you genuinely cannot build a loop -Stop and say so explicitly. List what you tried. Ask the user for: (a) access to whatever environment reproduces it, (b) a captured artifact (HAR file, log dump, core dump, screen recording with timestamps), or (c) permission to add temporary production instrumentation. Do **not** proceed to hypothesise without a loop. +Stop and say so explicitly. List what you tried. Ask the user for: (a) access to whatever environment reproduces it, (b) a redacted captured artifact (HAR file, log dump, core dump, screen recording with timestamps), or (c) permission to add temporary production instrumentation. Do **not** proceed to hypothesise without a loop. ### Completion criterion — a tight loop that goes red -Phase 1 is done when the loop is **tight** and **red-capable**: you can name **one command** — a script path, a test invocation, a curl — that you have **already run at least once** (paste the invocation and its output), and that is: +Phase 1 is done when the loop is **tight** and **red-capable**: you can name **one command** — a script path, a test invocation, a curl — that you have **already run at least once** (show the invocation and its output, redacted), and that is: - [ ] **Red-capable** — it drives the actual bug code path and asserts the **user's exact symptom**, so it can go red on this bug and green once fixed. Not "runs without erroring" — it must be able to _catch this specific bug_. - [ ] **Deterministic** — same verdict every run (flaky bugs: a pinned, high reproduction rate, per above). diff --git a/.agents/skills/diagnosing-bugs/agents/openai.yaml b/.agents/skills/diagnosing-bugs/agents/openai.yaml new file mode 100644 index 0000000..a13a755 --- /dev/null +++ b/.agents/skills/diagnosing-bugs/agents/openai.yaml @@ -0,0 +1,3 @@ +interface: + display_name: "Diagnosing Bugs" + short_description: "Diagnose hard bugs and regressions" diff --git a/.agents/skills/diagnosing-bugs/scripts/hitl-loop.template.sh b/.agents/skills/diagnosing-bugs/scripts/hitl-loop.template.sh index 40afc46..43daedd 100644 --- a/.agents/skills/diagnosing-bugs/scripts/hitl-loop.template.sh +++ b/.agents/skills/diagnosing-bugs/scripts/hitl-loop.template.sh @@ -11,6 +11,9 @@ # capture VAR "" → show question, read response into VAR # # At the end, captured values are printed as KEY=VALUE for the agent to parse. +# +# `capture` prints its value back to the terminal, where the agent reads it — so +# capture observations, and leave signing in to the user as a `step`. set -euo pipefail diff --git a/.agents/skills/domain-modeling/agents/openai.yaml b/.agents/skills/domain-modeling/agents/openai.yaml new file mode 100644 index 0000000..7f1522d --- /dev/null +++ b/.agents/skills/domain-modeling/agents/openai.yaml @@ -0,0 +1,3 @@ +interface: + display_name: "Domain Modeling" + short_description: "Build and sharpen a domain model" diff --git a/.agents/skills/explain/SKILL.md b/.agents/skills/explain/SKILL.md index b315213..fb871d2 100644 --- a/.agents/skills/explain/SKILL.md +++ b/.agents/skills/explain/SKILL.md @@ -11,9 +11,9 @@ Read `LORE.md`, `AGENTS.md`, `docs/spec.md`, and `CONTEXT.md` before acting. Use ## Step 0 — Bootstrap (mandatory) Read and execute `.agents/skills/matstudylab-bootstrap/SKILL.md` before any other step. -Do not proceed until bootstrap completes (sync, skip, or failed-with-continue). +Do not proceed until bootstrap completes (skip, setup/update, or warn-and-continue). -**Completion criterion:** bootstrap outcome identified (fresh, synced, or failed-with-continue). +**Completion criterion:** bootstrap outcome identified (skip, setup/update OK, or warn-and-continue). ## Step 1 — Parse invocation diff --git a/.agents/skills/git-guardrails-claude-code/SKILL.md b/.agents/skills/git-guardrails-claude-code/SKILL.md new file mode 100644 index 0000000..d943c68 --- /dev/null +++ b/.agents/skills/git-guardrails-claude-code/SKILL.md @@ -0,0 +1,95 @@ +--- +name: git-guardrails-claude-code +description: Set up Claude Code hooks to block dangerous git commands (push, reset --hard, clean, branch -D, etc.) before they execute. Use when user wants to prevent destructive git operations, add git safety hooks, or block git push/reset in Claude Code. +--- + +# Setup Git Guardrails + +Sets up a PreToolUse hook that intercepts and blocks dangerous git commands before Claude executes them. + +## What Gets Blocked + +- `git push` (all variants including `--force`) +- `git reset --hard` +- `git clean -f` / `git clean -fd` +- `git branch -D` +- `git checkout .` / `git restore .` + +When blocked, Claude sees a message telling it that it does not have authority to access these commands. + +## Steps + +### 1. Ask scope + +Ask the user: install for **this project only** (`.claude/settings.json`) or **all projects** (`~/.claude/settings.json`)? + +### 2. Copy the hook script + +The bundled script is at: [scripts/block-dangerous-git.sh](scripts/block-dangerous-git.sh) + +Copy it to the target location based on scope: + +- **Project**: `.claude/hooks/block-dangerous-git.sh` +- **Global**: `~/.claude/hooks/block-dangerous-git.sh` + +Make it executable with `chmod +x`. + +### 3. Add hook to settings + +Add to the appropriate settings file: + +**Project** (`.claude/settings.json`): + +```json +{ + "hooks": { + "PreToolUse": [ + { + "matcher": "Bash", + "hooks": [ + { + "type": "command", + "command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/block-dangerous-git.sh" + } + ] + } + ] + } +} +``` + +**Global** (`~/.claude/settings.json`): + +```json +{ + "hooks": { + "PreToolUse": [ + { + "matcher": "Bash", + "hooks": [ + { + "type": "command", + "command": "~/.claude/hooks/block-dangerous-git.sh" + } + ] + } + ] + } +} +``` + +If the settings file already exists, merge the hook into existing `hooks.PreToolUse` array — don't overwrite other settings. + +### 4. Ask about customization + +Ask if user wants to add or remove any patterns from the blocked list. Edit the copied script accordingly. + +### 5. Verify + +Run a quick test: + +```bash +echo '{"tool_input":{"command":"git push origin main"}}' | +``` + +Should exit with code 2 and print a BLOCKED message to stderr. diff --git a/.agents/skills/git-guardrails-claude-code/agents/openai.yaml b/.agents/skills/git-guardrails-claude-code/agents/openai.yaml new file mode 100644 index 0000000..3f5d756 --- /dev/null +++ b/.agents/skills/git-guardrails-claude-code/agents/openai.yaml @@ -0,0 +1,3 @@ +interface: + display_name: "Git Guardrails for Claude Code" + short_description: "Block dangerous git commands" diff --git a/.agents/skills/git-guardrails-claude-code/scripts/block-dangerous-git.sh b/.agents/skills/git-guardrails-claude-code/scripts/block-dangerous-git.sh new file mode 100755 index 0000000..c40b59c --- /dev/null +++ b/.agents/skills/git-guardrails-claude-code/scripts/block-dangerous-git.sh @@ -0,0 +1,25 @@ +#!/bin/bash + +INPUT=$(cat) +COMMAND=$(echo "$INPUT" | jq -r '.tool_input.command') + +DANGEROUS_PATTERNS=( + "git push" + "git reset --hard" + "git clean -fd" + "git clean -f" + "git branch -D" + "git checkout \." + "git restore \." + "push --force" + "reset --hard" +) + +for pattern in "${DANGEROUS_PATTERNS[@]}"; do + if echo "$COMMAND" | grep -qE "$pattern"; then + echo "BLOCKED: '$COMMAND' matches dangerous pattern '$pattern'. The user has prevented you from doing this." >&2 + exit 2 + fi +done + +exit 0 diff --git a/.agents/skills/grill-me/agents/openai.yaml b/.agents/skills/grill-me/agents/openai.yaml new file mode 100644 index 0000000..4d6fb0c --- /dev/null +++ b/.agents/skills/grill-me/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Grill Me" + short_description: "Sharpen a plan through interview" +policy: + allow_implicit_invocation: false diff --git a/.agents/skills/grill-with-docs/agents/openai.yaml b/.agents/skills/grill-with-docs/agents/openai.yaml new file mode 100644 index 0000000..5dbe278 --- /dev/null +++ b/.agents/skills/grill-with-docs/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Grill with Docs" + short_description: "Grill a design and write its docs" +policy: + allow_implicit_invocation: false diff --git a/.agents/skills/grilling/SKILL.md b/.agents/skills/grilling/SKILL.md index 219930f..95bd01e 100644 --- a/.agents/skills/grilling/SKILL.md +++ b/.agents/skills/grilling/SKILL.md @@ -1,12 +1,22 @@ --- name: grilling -description: Grill the user relentlessly about a plan or design. Use when the user wants to stress-test a plan before building, or uses any 'grill' trigger phrases. +description: Grill the user relentlessly about a plan, decision, or idea. Use when the user wants to stress-test their thinking, or uses any 'grill' trigger phrases. --- -Interview me relentlessly about every aspect of this plan until we reach a shared understanding. Walk down each branch of the design tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer. +Interview the user relentlessly until you reach a shared understanding. Map this as a **design tree**: every decision branches into the decisions that hang off it. -Ask the questions one at a time, waiting for feedback on each question before continuing. Asking multiple questions at once is bewildering. +Work the tree in **rounds**. The **frontier** is every decision whose prerequisites are already settled — the questions you can ask _now_ without guessing at answers you haven't heard yet. Ask the whole frontier in one round: number each question and give your recommended answer. Then wait for the user's answers before the next round. -If a *fact* can be found by exploring the codebase, look it up rather than asking me. The *decisions*, though, are mine — put each one to me and wait for my answer. +Each question should be formatted like so: -Do not enact the plan until I confirm we have reached a shared understanding. +``` +❓ **Q1** - ****: + +➡️ +``` + +Each round the user answers reshapes the tree — settled decisions push the frontier outward and unblock questions that depended on them. Recompute the frontier and ask the next round. A question whose answer depends on another question still open in this round belongs to a _later_ round, not this one. + +Finding _facts_ is your job, never the user's. When a frontier question needs a fact from the environment (filesystem, tools, etc.), dispatch a sub-agent to find it — don't ask the user for anything you could look up yourself. Don't block on it: a running exploration is an unsettled prerequisite, so only the questions downstream of it wait for the sub-agent to report — ask the rest of the frontier now. The _decisions_ are the user's — put each to them and wait. + +The session is done when the frontier is empty: every branch of the design tree visited, nothing left silently assumed. Do not act on it until the user confirms you have reached a shared understanding. diff --git a/.agents/skills/grilling/agents/openai.yaml b/.agents/skills/grilling/agents/openai.yaml new file mode 100644 index 0000000..ddbdb96 --- /dev/null +++ b/.agents/skills/grilling/agents/openai.yaml @@ -0,0 +1,3 @@ +interface: + display_name: "Grilling" + short_description: "Stress-test thinking a round of questions at a time" diff --git a/.agents/skills/handoff/agents/openai.yaml b/.agents/skills/handoff/agents/openai.yaml new file mode 100644 index 0000000..6e1d8da --- /dev/null +++ b/.agents/skills/handoff/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Handoff" + short_description: "Compact a conversation into a handoff" +policy: + allow_implicit_invocation: false diff --git a/.agents/skills/implement/agents/openai.yaml b/.agents/skills/implement/agents/openai.yaml new file mode 100644 index 0000000..f8794dc --- /dev/null +++ b/.agents/skills/implement/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Implement" + short_description: "Build work from a spec or tickets" +policy: + allow_implicit_invocation: false diff --git a/.agents/skills/improve-codebase-architecture/SKILL.md b/.agents/skills/improve-codebase-architecture/SKILL.md index a79b493..529761a 100644 --- a/.agents/skills/improve-codebase-architecture/SKILL.md +++ b/.agents/skills/improve-codebase-architecture/SKILL.md @@ -17,9 +17,14 @@ This command is _informed_ by the project's domain model and built on a shared d ### 1. Explore +**Scope before you scan — YAGNI.** Deepening a module pays off by making future changes to it easier, so put extra weight on the parts of the codebase that have recently changed. Decide *where* to look before you look: + +- If the user named a direction — a module, a subsystem, a pain point — take it, and skip the inference below. +- Otherwise, walk back a good stretch of the commit history (`git log --oneline`) to find the codebase's hot spots — the files and areas that keep coming up — and let those paths pull your attention first. If the changes are scattered with no clear hot spot, widen the net. + Read the project's domain glossary (`CONTEXT.md`) and any ADRs in the area you're touching first. -Then use the Agent tool with `subagent_type=Explore` to walk the codebase. Don't follow rigid heuristics — explore organically and note where you experience friction: +Then spawn a sub-agent to walk the codebase. Don't follow rigid heuristics — explore organically and note where you experience friction: - Where does understanding one concept require bouncing between many small modules? - Where are modules **shallow** — interface nearly as complex as the implementation? @@ -56,7 +61,7 @@ Do NOT propose interfaces yet. After the file is written, ask the user: "Which o ### 3. Grilling loop -Once the user picks a candidate, run the `/grilling` skill to walk the design tree with them — constraints, dependencies, the shape of the deepened module, what sits behind the seam, what tests survive. +Once the user picks a candidate, run the `/grilling` skill to walk the decision tree with them — constraints, dependencies, the shape of the deepened module, what sits behind the seam, what tests survive. Side effects happen inline as decisions crystallize — run the `/domain-modeling` skill to keep the domain model current as you go: diff --git a/.agents/skills/improve-codebase-architecture/agents/openai.yaml b/.agents/skills/improve-codebase-architecture/agents/openai.yaml new file mode 100644 index 0000000..706fdca --- /dev/null +++ b/.agents/skills/improve-codebase-architecture/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Improve Codebase Architecture" + short_description: "Find and grill architecture improvements" +policy: + allow_implicit_invocation: false diff --git a/.agents/skills/loop-me/SKILL.md b/.agents/skills/loop-me/SKILL.md new file mode 100644 index 0000000..c408efa --- /dev/null +++ b/.agents/skills/loop-me/SKILL.md @@ -0,0 +1,32 @@ +--- +name: loop-me +description: Grill me about specs for the workflows I want to build, within this workspace. +disable-model-invocation: true +argument-hint: "A workflow to design, or nothing to go find one" +--- + +Run a stateful `/grilling` session whose only output is **workflow** specs. Use the grilling discipline — relentless, a round of questions at a time, a recommended answer attached to each — aimed at the vocabulary and goal below. Create, edit, and delete specs as the grilling resolves things. + +## The loop lens + +A **loop** is a recurring pattern in the user's life: their career, their week, their morning, a single repeated activity. Picturing a life as loops within loops reveals how predictable its activities really are — which is what makes them worth **delegating**. Use the lens to find loops worth specifying, and propose ones the user hasn't noticed. + +A **workflow** is the spec of one loop, made real. You run a workflow on a loop — the loop is its running instantiation. Workflows live in `workflows/*.md` and are the source of truth. + +## Vocabulary + +A shared language, reached for only when a workflow calls for it — never a checklist. **Mandate nothing structural**: a workflow needs no AI, no checkpoint, and no schedule unless the grilling shows it does. + +- **Trigger** — what fires each run: an **event** (a new email, a new issue) or a **schedule** (every morning). Event-triggering is usually the more efficient. +- **Checkpoint** — a human-in-the-loop point where the user is asked to verify or decide. Some workflows have none and run autonomously; some use no AI at all. +- **Push right** — defer the checkpoint as far as it will go. Do maximal work before involving the human, so they are asked once, late, with everything prepared. +- **Brief** — what a checkpoint presents: a tight, decision-ready summary — what was produced, why, and a link down to the asset itself — never the raw output. The user reads a brief, not a draft. Speed of review is imperative. + +## Definition of done + +A workflow spec is done when an implementer agent could build it without asking a single question. Grill until then; nothing is done while a question remains. + +## The workspace + +- `workflows/*.md` — one spec per workflow. +- `NOTES.md` — raw notes on the user's world: the tools they use, the channels they process, and their own terminology for both. When it is empty or thin, interview them about their world before specifying anything. Sharpen fuzzy terms into canonical ones as they surface, and record them here. diff --git a/.agents/skills/loop-me/agents/openai.yaml b/.agents/skills/loop-me/agents/openai.yaml new file mode 100644 index 0000000..1a4f411 --- /dev/null +++ b/.agents/skills/loop-me/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Loop Me" + short_description: "Spec the workflows you want to build" +policy: + allow_implicit_invocation: false diff --git a/.agents/skills/matlab/SKILL.md b/.agents/skills/matlab/SKILL.md index 006047b..9469060 100644 --- a/.agents/skills/matlab/SKILL.md +++ b/.agents/skills/matlab/SKILL.md @@ -1,371 +1,274 @@ --- name: matlab -description: MATLAB and GNU Octave numerical computing for matrix operations, data analysis, visualization, and scientific computing. Use when writing MATLAB/Octave scripts for linear algebra, signal processing, image processing, differential equations, optimization, statistics, or creating scientific visualizations. Also use when the user needs help with MATLAB syntax, functions, or wants to convert between MATLAB and Python code. Scripts can be executed with MATLAB or the open-source GNU Octave interpreter. -license: For MATLAB (https://www.mathworks.com/pricing-licensing.html) and for Octave (GNU General Public License version 3) -compatibility: Requires either MATLAB or Octave to be installed for testing, but not required for just generating scripts. -metadata: {"version": "1.0", "skill-author": "K-Dense Inc."} +description: Build, review, migrate, and safely plan MATLAB or GNU Octave numerical workflows, including arrays, tabular/time data, tests, projects, graphics, MAT files, and explicit Python interoperability. +license: MIT +compatibility: >- + Documentation is pinned where noted to proprietary MATLAB R2026a and free + GNU Octave 11.3.0. Bundled Python CLIs require Python 3.11+ and run locally + without MATLAB or Octave; optional MAT inventory uses scipy and/or h5py. +allowed-tools: Read Write Bash Glob Python +metadata: + version: "1.1" + skill-author: "K-Dense Inc." + last-reviewed: "2026-07-23" --- -# MATLAB/Octave Scientific Computing - -MATLAB is a numerical computing environment optimized for matrix operations and scientific computing. GNU Octave is a free, open-source alternative with high MATLAB compatibility. - -## Quick Start - -**Running MATLAB scripts:** -```bash -# MATLAB (commercial) -matlab -nodisplay -nosplash -r "run('script.m'); exit;" - -# GNU Octave (free, open-source) -octave script.m -``` - -**Install GNU Octave:** -```bash -# macOS -brew install octave - -# Ubuntu/Debian -sudo apt install octave - -# Windows - download from https://octave.org/download -``` - -## Core Capabilities - -### 1. Matrix Operations - -MATLAB operates fundamentally on matrices and arrays: - -```matlab -% Create matrices -A = [1 2 3; 4 5 6; 7 8 9]; % 3x3 matrix -v = 1:10; % Row vector 1 to 10 -v = linspace(0, 1, 100); % 100 points from 0 to 1 - -% Special matrices -I = eye(3); % Identity matrix -Z = zeros(3, 4); % 3x4 zero matrix -O = ones(2, 3); % 2x3 ones matrix -R = rand(3, 3); % Random uniform -N = randn(3, 3); % Random normal - -% Matrix operations -B = A'; % Transpose -C = A * B; % Matrix multiplication -D = A .* B; % Element-wise multiplication -E = A \ b; % Solve linear system Ax = b -F = inv(A); % Matrix inverse -``` - -For complete matrix operations, see [references/matrices-arrays.md](references/matrices-arrays.md). - -### 2. Linear Algebra +# MATLAB and GNU Octave + +Use this skill to design or review numerical code, migrate MATLAB releases, +prepare reproducible projects, and plan trusted execution. MATLAB and GNU +Octave are distinct products: compatibility is partial, not a license or +behavior guarantee. + +## Product and license gate + +- **MATLAB R2026a is proprietary.** Do not assume MATLAB, MATLAB Online, a + named toolbox, MATLAB Test, MATLAB Compiler, MATLAB Coder, Parallel Computing + Toolbox, or an add-on is installed, licensed, or available to the user. +- **MATLAB Runtime is not MATLAB.** It runs compatible applications produced + with MATLAB Compiler; it cannot run arbitrary source or host MATLAB Engine + for Python. Building artifacts needs the applicable licensed compiler and + every product used by the source. +- **GNU Octave 11.3.0 is free software under GPLv3+.** Octave packages are not + MATLAB toolboxes. Similar names do not imply API, numerical, graphics, or + licensing equivalence. +- Ask which runtime, release, platform, installed products, and license context + the user actually has. Treat availability as `unknown` until confirmed. + +See [Octave compatibility](references/octave-compatibility.md) and +[execution/product boundaries](references/executing-scripts.md). + +## Nonnegotiable safety boundary + +Never run an untrusted `.m`, `.mlx`, MEX binary, MAT file, project startup or +shutdown action, package installer, or generated artifact. Static review does +not prove safety. + +Treat these as execution or code-loading surfaces: + +- `eval`, `evalin`, `assignin`, text-derived `feval`, `str2func`, callbacks, + timers, app callbacks, and dynamically modified paths; +- `system`, `unix`, `dos`, shell escape `!`, Java, .NET, Python (`py.*`, + `pyrun`, `pyrunfile`), MEX, and native libraries; +- `mex`, `codegen`, MATLAB Compiler, build tasks, package/project startup, and + generated code; +- `load`, object deserialization (`loadobj`, custom serialization), function + handles, Java/System objects, and class code reachable from MAT files. + +`.mlx` is an opaque archive for this toolkit and MEX is native executable code. +Do not use Python pickle for exchange. Inspect first, isolate when appropriate, +obtain explicit approval, then invoke a user-confirmed executable and license. +Bundled scripts are static or dry-run tools: none launches MATLAB, Octave, +Python Engine, a compiler, or a subprocess. + +## Default workflow + +1. **Clarify target.** Record MATLAB release or Octave version, OS/architecture, + base product versus required toolboxes/packages, expected inputs/outputs, + numerical tolerances, and whether execution is authorized. +2. **Inventory statically.** Scan `.m` files, opaque artifacts, project paths, + required products, and MAT headers before any runtime loads them. +3. **Choose code form.** Prefer functions with an `arguments` block for + automation. Use scripts only for controlled orchestration and live scripts + for reviewed interactive narratives. +4. **Make semantics explicit.** Record shapes, classes, units, missing-value + rules, indexing, implicit expansion, RNG algorithm/seed, tolerances, and + output formats. +5. **Test without hidden state.** Keep fixtures synthetic, paths project-local, + graphics deterministic, and tests independent of base-workspace residue. +6. **Plan execution.** Generate an argv plan, review startup/path effects and + licenses, and launch only after explicit approval outside these helpers. +7. **Capture provenance.** Hash named inputs/code and record release, products, + RNG policy, tolerances, and command plan without dumping the environment. + +## Language and data checklist + +### Scripts, functions, and live scripts + +- Scripts share the caller/base workspace and leave variables behind. + Functions have local workspaces and explicit inputs/outputs. +- Live scripts (`.mlx`) mix code and rich output but are not plain-text + review artifacts. Export reviewed code to `.m` for static inspection. +- Avoid `clear all`, broad `addpath(genpath(...))`, dependence on `pwd`, global + variables, and silent name shadowing. Use project roots and `fullfile`. +- Validate sizes, classes, and values in `arguments` blocks. Remember that + type declarations can convert inputs; validators check without converting. +- A main function file should match the main function name. Local functions + are private to the file; since R2024a they can appear anywhere in a script + outside conditional contexts. ```matlab -% Eigenvalues and eigenvectors -[V, D] = eig(A); % V: eigenvectors, D: diagonal eigenvalues - -% Singular value decomposition -[U, S, V] = svd(A); - -% Matrix decompositions -[L, U] = lu(A); % LU decomposition -[Q, R] = qr(A); % QR decomposition -R = chol(A); % Cholesky (symmetric positive definite) - -% Solve linear systems -x = A \ b; % Preferred method -x = linsolve(A, b); % With options -x = inv(A) * b; % Less efficient -``` - -For comprehensive linear algebra, see [references/mathematics.md](references/mathematics.md). - -### 3. Plotting and Visualization - -```matlab -% 2D Plots -x = 0:0.1:2*pi; -y = sin(x); -plot(x, y, 'b-', 'LineWidth', 2); -xlabel('x'); ylabel('sin(x)'); -title('Sine Wave'); -grid on; - -% Multiple plots -hold on; -plot(x, cos(x), 'r--'); -legend('sin', 'cos'); -hold off; - -% 3D Surface -[X, Y] = meshgrid(-2:0.1:2, -2:0.1:2); -Z = X.^2 + Y.^2; -surf(X, Y, Z); -colorbar; - -% Save figures -saveas(gcf, 'plot.png'); -print('-dpdf', 'plot.pdf'); -``` - -For complete visualization guide, see [references/graphics-visualization.md](references/graphics-visualization.md). - -### 4. Data Import/Export - -```matlab -% Read tabular data -T = readtable('data.csv'); -M = readmatrix('data.csv'); - -% Write data -writetable(T, 'output.csv'); -writematrix(M, 'output.csv'); - -% MAT files (MATLAB native) -save('data.mat', 'A', 'B', 'C'); % Save variables -load('data.mat'); % Load all -S = load('data.mat', 'A'); % Load specific - -% Images -img = imread('image.png'); -imwrite(img, 'output.jpg'); -``` - -For complete I/O guide, see [references/data-import-export.md](references/data-import-export.md). - -### 5. Control Flow and Functions - -```matlab -% Conditionals -if x > 0 - disp('positive'); -elseif x < 0 - disp('negative'); -else - disp('zero'); +function y = scaleSignal(x, options) +arguments + x (:,1) double {mustBeFinite} + options.Scale (1,1) double {mustBeFinite, mustBeNonzero} = 1 end - -% Loops -for i = 1:10 - disp(i); +y = x .* options.Scale; end - -while x > 0 - x = x - 1; -end - -% Functions (in separate .m file or same file) -function y = myfunction(x, n) - y = x.^n; -end - -% Anonymous functions -f = @(x) x.^2 + 2*x + 1; -result = f(5); % 36 -``` - -For complete programming guide, see [references/programming.md](references/programming.md). - -### 6. Statistics and Data Analysis - -```matlab -% Descriptive statistics -m = mean(data); -s = std(data); -v = var(data); -med = median(data); -[minVal, minIdx] = min(data); -[maxVal, maxIdx] = max(data); - -% Correlation -R = corrcoef(X, Y); -C = cov(X, Y); - -% Linear regression -p = polyfit(x, y, 1); % Linear fit -y_fit = polyval(p, x); - -% Moving statistics -y_smooth = movmean(y, 5); % 5-point moving average -``` - -For statistics reference, see [references/mathematics.md](references/mathematics.md). - -### 7. Differential Equations - -```matlab -% ODE solving -% dy/dt = -2y, y(0) = 1 -f = @(t, y) -2*y; -[t, y] = ode45(f, [0 5], 1); -plot(t, y); - -% Higher-order: y'' + 2y' + y = 0 -% Convert to system: y1' = y2, y2' = -2*y2 - y1 -f = @(t, y) [y(2); -2*y(2) - y(1)]; -[t, y] = ode45(f, [0 10], [1; 0]); -``` - -For ODE solvers guide, see [references/mathematics.md](references/mathematics.md). - -### 8. Signal Processing - -```matlab -% FFT -Y = fft(signal); -f = (0:length(Y)-1) * fs / length(Y); -plot(f, abs(Y)); - -% Filtering -b = fir1(50, 0.3); % FIR filter design -y_filtered = filter(b, 1, signal); - -% Convolution -y = conv(x, h, 'same'); -``` - -For signal processing, see [references/mathematics.md](references/mathematics.md). - -## Common Patterns - -### Pattern 1: Data Analysis Pipeline - -```matlab -% Load data -data = readtable('experiment.csv'); - -% Clean data -data = rmmissing(data); % Remove missing values - -% Analyze -grouped = groupsummary(data, 'Category', 'mean', 'Value'); - -% Visualize -figure; -bar(grouped.Category, grouped.mean_Value); -xlabel('Category'); ylabel('Mean Value'); -title('Results by Category'); - -% Save -writetable(grouped, 'results.csv'); -saveas(gcf, 'results.png'); -``` - -### Pattern 2: Numerical Simulation - -```matlab -% Parameters -L = 1; N = 100; T = 10; dt = 0.01; -x = linspace(0, L, N); -dx = x(2) - x(1); - -% Initial condition -u = sin(pi * x); - -% Time stepping (heat equation) -for t = 0:dt:T - u_new = u; - for i = 2:N-1 - u_new(i) = u(i) + dt/(dx^2) * (u(i+1) - 2*u(i) + u(i-1)); - end - u = u_new; -end - -plot(x, u); ``` -### Pattern 3: Batch Processing +Read [programming](references/programming.md). + +### Arrays, indexing, and numerics + +- MATLAB uses 1-based, column-major indexing. `A(i,j)`, `A(k)`, `A(:,j)`, + `A{...}`, and `A.(name)` have different semantics. +- `*`, `/`, `\`, and `^` are matrix operations; dotted forms are + element-wise. Use `A\b`, not `inv(A)*b`. +- Since R2016b, compatible dimensions expand implicitly. Assert intended shape + before operations that could accidentally form an outer result. +- Preallocate when output size is known, but do not vectorize at the cost of + huge temporaries or unreadable code. Measure with `timeit` or the profiler. +- Compare floating-point results with domain-chosen absolute and relative + tolerances, not blanket `==` or a magic multiple of `eps`. +- Pin both random algorithm and seed. Use named `RandStream` substreams for + independent parallel work; do not use time-based `rng("shuffle")` for a + reproducibility claim. + +Read [arrays](references/matrices-arrays.md) and +[mathematics](references/mathematics.md). + +### Tables, timetables, and missing values + +- A `table` has named, equal-height variables that may have different types. + `T(rows,vars)` returns a table; `T{rows,vars}` extracts contents; `T.Var` + selects one variable. +- A `timetable` additionally has row times. Sort, validate time zones and + uniqueness, then use `retime`/`synchronize` intentionally. +- Missing sentinels are type-specific: `NaN`, `NaT`, ``, + ``, and empty character vectors. Integer and logical arrays have + no standard missing sentinel. +- Define import options rather than relying on inference for production data. + Preserve units, time zones, variable names, encodings, and missing rules. + +Read [data import/export](references/data-import-export.md). + +## Graphics and export + +Use explicit figure/axes handles and `tiledlayout`; label units; set limits, +color scales, font sizes, and colormaps deliberately. Prefer `exportgraphics` +over `saveas` for publication output. In R2026a it exports raster, PDF/EPS/EMF, +SVG, GIF, and interactive HTML; format capabilities differ. Specify +`ContentType="vector"` for suitable PDF/SVG-style output and `Resolution` for +raster output. Review accessibility and embedded-raster behavior. + +Read [graphics and export](references/graphics-visualization.md). + +## MAT files and exchange + +- Version 7 is the normal `save` default; `matfile` creates 7.3 by default. + Versions 4/6/7/7.3 differ in types, compression, and per-variable limits. +- Version 7.3 is HDF5-based, not an arbitrary HDF5 interchange contract. + Partial access and chunking can help large arrays. +- Never load an untrusted MAT file. Inventory headers/datasets first. Objects + can invoke class deserialization behavior; opaque/function/native content + requires escalation. +- Prefer CSV/JSON/Parquet/HDF5 with a documented schema for simple exchange. + Do not rename pickle payloads as MAT files and do not deserialize pickle. + +Read [data import/export](references/data-import-export.md). + +## Projects, analysis, and tests + +- Use MATLAB Projects for controlled paths, startup/shutdown tasks, + dependencies, source control, and reproducible entry points. Review project + actions before opening an untrusted project. +- `matlab.codetools.requiredFilesAndProducts` and Dependency Analyzer are + static approximations; dynamic dispatch can cause misses or false positives. + A required-product report does not prove a license is available. +- Use Code Analyzer (`codeIssues`; legacy text workflows can use `checkcode`) + and `codeCompatibilityReport` before migration. +- Base MATLAB includes script-, function-, and class-based + `matlab.unittest` workflows. Parallel runs require Parallel Computing + Toolbox. Dependency-based selection, richer quality dashboards, generated + tests, and advanced coverage/equivalence features can require MATLAB Test or + other products. +- R2026a `runtests` automatically opens and later closes a project when target + tests belong to a project that is not already open. Account for startup and + shutdown actions before using this behavior. + +Read [programming](references/programming.md) and +[execution/testing](references/executing-scripts.md). + +## Python integration, pinned to R2026a + +- R2026a supports 64-bit CPython 3.9-3.13 for MATLAB Interface to Python, + MATLAB Engine for Python, and MATLAB Compiler SDK for Python. +- The current R2026a PyPI package reviewed here is + `matlabengine==26.1.12` (released 2026-05-08). It requires an installed + R2026a; MATLAB Runtime alone is insufficient. R2026a also ships a + preinstalled Engine distribution under one named `matlabroot` path. +- Package installation does not grant MATLAB or toolbox licenses. Configure + one named interpreter/executable; do not print the full environment, + `PATH`, `PYTHONPATH`, or credentials. +- `pyenv` controls MATLAB-to-Python interpreter selection. In-process Python + generally requires restarting MATLAB to switch; out-of-process Python can + be terminated and reconfigured. +- Starting Engine is an explicit execution action: + `matlab.engine.start_matlab()` starts a MATLAB process and can check out a + license. Never call it merely to probe availability. +- Verify conversion semantics for NumPy arrays, pandas DataFrames, + tables/timetables, strings/missing values, datetime/duration, dictionaries, + shape/order, and unsupported sparse/object/categorical cases. + +Read [Python integration](references/python-integration.md). + +## Local helper CLIs + +Every helper is network-free, bounded, symlink-rejecting, and nonexecuting. +Run from this skill directory with Python 3.11+. Bash is allowed only to invoke +these Python CLIs and validation commands; never use it to execute a generated +MATLAB/Octave argv plan or untrusted artifact. + +| Helper | Purpose | +|---|---| +| `scripts/plan_batch_command.py` | Produce reviewed MATLAB/Octave argv; never execute | +| `scripts/scan_m_code.py` | Scan `.m` text and flag opaque `.mlx`/MEX risks | +| `scripts/validate_project_manifest.py` | Validate paths and declared product/license status | +| `scripts/inventory_mat_file.py` | Header/metadata inventory; never call `loadmat` | +| `scripts/plan_python_compatibility.py` | Check R2026a CPython/Engine compatibility | +| `scripts/reproducibility_report.py` | Hash named local artifacts and emit a bounded report | +| `scripts/generate_function_scaffold.py` | Dry-run or create function and unit-test scaffolds | -```matlab -% Process multiple files -files = dir('data/*.csv'); -results = cell(length(files), 1); - -for i = 1:length(files) - data = readtable(fullfile(files(i).folder, files(i).name)); - results{i} = analyze(data); % Custom analysis function -end - -% Combine results -all_results = vertcat(results{:}); +```bash +python scripts/scan_m_code.py path/to/source --root path/to/project +python scripts/plan_batch_command.py matlab script path/to/main.m --root path/to/project +python scripts/validate_project_manifest.py project-manifest.json --root path/to/project +python scripts/inventory_mat_file.py data.mat --root path/to/project +python scripts/plan_python_compatibility.py --python-version 3.13 +python scripts/reproducibility_report.py --root path/to/project --file src/analyze.m +python scripts/generate_function_scaffold.py analyzeSignal --root path/to/project ``` -## Reference Files - -- **[matrices-arrays.md](references/matrices-arrays.md)** - Matrix creation, indexing, manipulation, and operations -- **[mathematics.md](references/mathematics.md)** - Linear algebra, calculus, ODEs, optimization, statistics -- **[graphics-visualization.md](references/graphics-visualization.md)** - 2D/3D plotting, customization, export -- **[data-import-export.md](references/data-import-export.md)** - File I/O, tables, data formats -- **[programming.md](references/programming.md)** - Functions, scripts, control flow, OOP -- **[python-integration.md](references/python-integration.md)** - Calling Python from MATLAB and vice versa -- **[octave-compatibility.md](references/octave-compatibility.md)** - Differences between MATLAB and GNU Octave -- **[executing-scripts.md](references/executing-scripts.md)** - Executing generated scripts and for testing - -## GNU Octave Compatibility - -GNU Octave is highly compatible with MATLAB. Most scripts work without modification. Key differences: - -- Use `#` or `%` for comments (MATLAB only `%`) -- Octave allows `++`, `--`, `+=` operators -- Some toolbox functions unavailable in Octave -- Use `pkg load` for Octave packages - -For complete compatibility guide, see [references/octave-compatibility.md](references/octave-compatibility.md). - -## Best Practices - -1. **Vectorize operations** - Avoid loops when possible: - ```matlab - % Slow - for i = 1:1000 - y(i) = sin(x(i)); - end - - % Fast - y = sin(x); - ``` - -2. **Preallocate arrays** - Avoid growing arrays in loops: - ```matlab - % Slow - for i = 1:1000 - y(i) = i^2; - end - - % Fast - y = zeros(1, 1000); - for i = 1:1000 - y(i) = i^2; - end - ``` - -3. **Use appropriate data types** - Tables for mixed data, matrices for numeric: - ```matlab - % Numeric data - M = readmatrix('numbers.csv'); - - % Mixed data with headers - T = readtable('mixed.csv'); - ``` - -4. **Comment and document** - Use function help: - ```matlab - function y = myfunction(x) - %MYFUNCTION Brief description - % Y = MYFUNCTION(X) detailed description - % - % Example: - % y = myfunction(5); - y = x.^2; - end - ``` - -## Additional Resources - -- MATLAB Documentation: https://www.mathworks.com/help/matlab/ -- GNU Octave Manual: https://docs.octave.org/latest/ -- MATLAB Onramp (free course): https://www.mathworks.com/learn/tutorials/matlab-onramp.html -- File Exchange: https://www.mathworks.com/matlabcentral/fileexchange/ \ No newline at end of file +The scaffold generator defaults to dry-run; writing requires `--write` and +refuses collisions. SciPy and h5py are optional inventory backends; if +authorized, add exact reviewed versions to the caller's project lockfile. +They are not required for `--help` or header-only inventory, and this skill +does not perform package installation. + +## References + +- [Programming, workspaces, projects, analysis, tests](references/programming.md) +- [Matrices, indexing, types, missingness, performance](references/matrices-arrays.md) +- [Numerical methods, tolerances, RNG, toolbox boundaries](references/mathematics.md) +- [Graphics and `exportgraphics`](references/graphics-visualization.md) +- [Import/export, tables/timetables, MAT semantics and safety](references/data-import-export.md) +- [MATLAB/Octave command-line execution and migration](references/executing-scripts.md) +- [MATLAB and Python interoperability](references/python-integration.md) +- [GNU Octave 11.3.0 compatibility differences](references/octave-compatibility.md) + +Bundled JSON assets are the [project manifest](assets/project_manifest_template.json), +[reproducibility manifest](assets/reproducibility_manifest_template.json), and +[R2026a Python table](assets/python_compatibility_r2026a.json). There is no +`templates/` directory and no Markdown file is loaded from `assets/`; +local-link tests enforce this package contract. + +## Primary sources (verified 2026-07-23) + +- [MATLAB R2026a documentation](https://www.mathworks.com/help/matlab/) +- [MATLAB R2026a release notes](https://www.mathworks.com/help/matlab/release-notes.html) +- [R2026a system requirements](https://www.mathworks.com/support/requirements/matlab-system-requirements.html) +- [Python compatibility by release](https://www.mathworks.com/support/requirements/python-compatibility.html) +- [MATLAB Engine installation](https://www.mathworks.com/help/matlab/matlab_external/install-the-matlab-engine-for-python.html) +- [GNU Octave 11.3.0 release](https://octave.org/) +- [GNU Octave current manual](https://docs.octave.org/latest/) diff --git a/.agents/skills/matlab/assets/project_manifest_template.json b/.agents/skills/matlab/assets/project_manifest_template.json new file mode 100644 index 0000000..dd79f3d --- /dev/null +++ b/.agents/skills/matlab/assets/project_manifest_template.json @@ -0,0 +1,33 @@ +{ + "schema_version": "1.0", + "project_name": "example-project", + "runtime": "matlab", + "matlab_release": "R2026a", + "octave_version": null, + "entry_points": [ + { + "kind": "function", + "path": "src/analyzeSignal.m" + } + ], + "test_paths": [ + "tests" + ], + "required_products": [ + { + "license_status": "unknown", + "minimum_release": "R2026a", + "name": "MATLAB", + "purpose": "Base language and numerical runtime" + } + ], + "optional_products": [], + "octave_packages": [], + "startup_actions": [], + "shutdown_actions": [], + "external_interfaces": [], + "generated_artifacts": [], + "notes": [ + "unknown means availability and entitlement have not been confirmed" + ] +} diff --git a/.agents/skills/matlab/assets/python_compatibility_r2026a.json b/.agents/skills/matlab/assets/python_compatibility_r2026a.json new file mode 100644 index 0000000..3962c44 --- /dev/null +++ b/.agents/skills/matlab/assets/python_compatibility_r2026a.json @@ -0,0 +1,27 @@ +{ + "schema_version": "1.0", + "as_of": "2026-07-23", + "matlab_release": "R2026a", + "python_implementation": "CPython", + "python_architecture_bits": 64, + "supported_python_versions": [ + "3.9", + "3.10", + "3.11", + "3.12", + "3.13" + ], + "matlab_engine_package": { + "name": "matlabengine", + "version": "26.1.12", + "release_date": "2026-05-08" + }, + "installed_matlab_required": true, + "matlab_runtime_is_sufficient": false, + "preinstalled_engine_relative_path": "extern/engines/python/dist", + "official_sources": [ + "https://www.mathworks.com/support/requirements/python-compatibility.html", + "https://www.mathworks.com/help/matlab/matlab_external/install-the-matlab-engine-for-python.html", + "https://pypi.org/project/matlabengine/26.1.12/" + ] +} diff --git a/.agents/skills/matlab/assets/reproducibility_manifest_template.json b/.agents/skills/matlab/assets/reproducibility_manifest_template.json new file mode 100644 index 0000000..7f927ce --- /dev/null +++ b/.agents/skills/matlab/assets/reproducibility_manifest_template.json @@ -0,0 +1,32 @@ +{ + "schema_version": "1.0", + "runtime": "matlab", + "runtime_version": "R2026a", + "platform": { + "architecture": "confirm-explicitly", + "operating_system": "confirm-explicitly" + }, + "products_manifest": "project-manifest.json", + "randomness": { + "algorithm": "twister", + "seed": 1729, + "substream": null + }, + "numeric_policy": { + "absolute_tolerance": 1e-12, + "relative_tolerance": 1e-9, + "rationale": "replace with a domain-specific justification" + }, + "named_files": [ + { + "path": "src/analyzeSignal.m", + "sha256": "populate-with-reproducibility-report" + } + ], + "data_contracts": [], + "graphics_exports": [], + "external_interfaces": [], + "notes": [ + "Record named facts only; do not dump the environment or credentials." + ] +} diff --git a/.agents/skills/matlab/references/data-import-export.md b/.agents/skills/matlab/references/data-import-export.md index b53db94..4d340ae 100644 --- a/.agents/skills/matlab/references/data-import-export.md +++ b/.agents/skills/matlab/references/data-import-export.md @@ -1,479 +1,221 @@ -# Data Import and Export Reference +# Data Import, Tables, Timetables, and MAT Files -## Table of Contents -1. [Text and CSV Files](#text-and-csv-files) -2. [Spreadsheets](#spreadsheets) -3. [MAT Files](#mat-files) -4. [Images](#images) -5. [Tables and Data Types](#tables-and-data-types) -6. [Low-Level File I/O](#low-level-file-io) +This reference targets MATLAB R2026a. Treat every external file as untrusted +until its provenance, size, structure, and parser risk are reviewed. -## Text and CSV Files +## Safe import workflow -### Reading Text Files +1. Accept one named local path under a confirmed root. +2. Reject URLs, traversal, symlinks, device files, and unexpected extensions. +3. Bound compressed and uncompressed size, rows, columns, variables, nesting, + strings, and HDF5 objects. +4. Inventory format and metadata before loading values. +5. Define schema, classes, units, encoding, missing sentinels, time zones, and + duplicate policy. +6. Import the narrowest columns/ranges needed. +7. Validate before computation. +8. Write to a new local output; refuse accidental overwrite. -```matlab -% Recommended high-level functions -T = readtable('data.csv'); % Read as table (mixed types) -M = readmatrix('data.csv'); % Read as numeric matrix -C = readcell('data.csv'); % Read as cell array -S = readlines('data.txt'); % Read as string array (lines) -str = fileread('data.txt'); % Read entire file as string - -% With options -T = readtable('data.csv', 'ReadVariableNames', true); -T = readtable('data.csv', 'Delimiter', ','); -T = readtable('data.csv', 'NumHeaderLines', 2); -M = readmatrix('data.csv', 'Range', 'B2:D100'); - -% Detect import options -opts = detectImportOptions('data.csv'); -opts.VariableNames = {'Col1', 'Col2', 'Col3'}; -opts.VariableTypes = {'double', 'string', 'double'}; -opts.SelectedVariableNames = {'Col1', 'Col3'}; -T = readtable('data.csv', opts); -``` +Do not use a broad directory scan or environment dump to find data. Remote +imports add network, redirect, credential, and changing-content risks; download +them through a separately approved, checksum-recorded workflow. -### Writing Text Files - -```matlab -% High-level functions -writetable(T, 'output.csv'); -writematrix(M, 'output.csv'); -writecell(C, 'output.csv'); -writelines(S, 'output.txt'); - -% With options -writetable(T, 'output.csv', 'Delimiter', '\t'); -writetable(T, 'output.csv', 'WriteVariableNames', false); -writematrix(M, 'output.csv', 'Delimiter', ','); -``` +## High-level text and spreadsheet import -### Tab-Delimited Files +Choose the output model intentionally: ```matlab -% Reading -T = readtable('data.tsv', 'Delimiter', '\t'); -T = readtable('data.txt', 'FileType', 'text', 'Delimiter', '\t'); - -% Writing -writetable(T, 'output.tsv', 'Delimiter', '\t'); -writetable(T, 'output.txt', 'FileType', 'text', 'Delimiter', '\t'); +options = detectImportOptions("measurements.csv", ... + TextType="string"); +options.SelectedVariableNames = ... + ["SampleID" "Timestamp" "Value" "Quality"]; +options = setvartype(options, "SampleID", "string"); +T = readtable("measurements.csv", options); ``` -## Spreadsheets +- `readtable`: mixed, named column-oriented data. +- `readmatrix`: homogeneous numeric data. +- `readcell`: heterogeneous cells when a table schema is inappropriate. +- `readlines`/`fileread`: bounded text, with explicit encoding expectations. +- `readtimetable`: time-indexed data when row-time semantics are known. -### Reading Excel Files - -```matlab -% Basic reading -T = readtable('data.xlsx'); -M = readmatrix('data.xlsx'); -C = readcell('data.xlsx'); - -% Specific sheet -T = readtable('data.xlsx', 'Sheet', 'Sheet2'); -T = readtable('data.xlsx', 'Sheet', 2); - -% Specific range -M = readmatrix('data.xlsx', 'Range', 'B2:D100'); -M = readmatrix('data.xlsx', 'Sheet', 2, 'Range', 'A1:F50'); - -% With options -opts = detectImportOptions('data.xlsx'); -opts.Sheet = 'Data'; -opts.DataRange = 'A2'; -preview(opts.VariableNames) % Check column names -T = readtable('data.xlsx', opts); - -% Get sheet names -[~, sheets] = xlsfinfo('data.xlsx'); -``` +Use `writetable`, `writematrix`, `writecell`, `writelines`, or +`writetimetable` for corresponding exports. Text/spreadsheet round trips can +change formatting, precision, names, multidimensional variables, empty values, +or types. If exact MATLAB structure matters and the file is trusted, a MAT file +can preserve it—but MAT files have object/code risks and are not a universal +interchange format. + +R2026a adds JSON read/write support for tables and timetables. Define the JSON +orientation/schema and test consumers; "JSON" alone does not specify table +shape, time representation, or missing semantics. -### Writing Excel Files +## Tables and timetables ```matlab -% Basic writing -writetable(T, 'output.xlsx'); -writematrix(M, 'output.xlsx'); -writecell(C, 'output.xlsx'); - -% Specific sheet and range -writetable(T, 'output.xlsx', 'Sheet', 'Results'); -writetable(T, 'output.xlsx', 'Sheet', 'Data', 'Range', 'B2'); -writematrix(M, 'output.xlsx', 'Sheet', 2, 'Range', 'A1'); - -% Append to existing sheet (use Range to specify start position) -writetable(T2, 'output.xlsx', 'Sheet', 'Data', 'WriteMode', 'append'); +required = ["SampleID" "Timestamp" "Value"]; +assert(all(ismember(required, string(T.Properties.VariableNames)))); +assert(isstring(T.SampleID)); +assert(isdatetime(T.Timestamp)); +assert(isnumeric(T.Value)); ``` -## MAT Files +Table rules: -### Saving Variables +- all variables have the same row count; +- variables may differ in class and width; +- `T(rows,vars)` preserves a table; +- `T{rows,vars}` extracts/concatenates contents; +- `T.Var` extracts one variable; +- properties can store units and descriptions but are not always preserved by + external formats. -```matlab -% Save all workspace variables -save('data.mat'); +Timetable rules: -% Save specific variables -save('data.mat', 'x', 'y', 'results'); +- row times are distinct metadata, not an ordinary variable; +- sort and validate row times; +- preserve or normalize `TimeZone`; +- define duplicates before `retime` or `synchronize`; +- choose interpolation/aggregation and union/intersection deliberately; +- validate missing row times separately from `ismissing(TT)`. -% Save with options -save('data.mat', 'x', 'y', '-v7.3'); % Large files (>2GB) -save('data.mat', 'x', '-append'); % Append to existing file -save('data.mat', '-struct', 's'); % Save struct fields as variables +## Missing values -% Compression options -save('data.mat', 'x', '-v7'); % Compressed (default) -save('data.mat', 'x', '-v6'); % Uncompressed, faster -``` +Standard indicators: -### Loading Variables +| Class | Standard missing | +|---|---| +| `double`, `single`, `duration`, `calendarDuration` | `NaN` | +| `datetime` | `NaT` | +| `string` | `` | +| `categorical` | `` | +| cell array of character vectors | empty character vector | +| integer/logical | none | -```matlab -% Load all variables -load('data.mat'); +Use `standardizeMissing` when source sentinels are documented. Include +`missing` in a custom indicator list when you intend to preserve standard +indicators too. `Inf` is not missing by default. -% Load specific variables -load('data.mat', 'x', 'y'); +Never call `rmmissing` as generic cleaning without reporting what rows, +variables, groups, or time coverage were removed. -% Load into structure -S = load('data.mat'); -S = load('data.mat', 'x', 'y'); -x = S.x; -y = S.y; +## MAT file versions -% List contents without loading -whos('-file', 'data.mat'); -vars = who('-file', 'data.mat'); -``` +MAT files are MATLAB binary workspace containers: -### MAT-File Object (Large Files) +| Version | `save` option | Compression | Key capability/limit | +|---|---|---|---| +| 4 | `"-v4"` | no | 2-D double, character, sparse; legacy | +| 6 | `"-v6"` | no | N-D, cell, structure; under 2 GiB per variable | +| 7 | `"-v7"` | yes | Unicode and v6 features; under 2 GiB per variable | +| 7.3 | `"-v7.3"` | yes/chunked | HDF5-based, partial access, variables at least 2 GiB on 64-bit | -```matlab -% Create MAT-file object for partial access -m = matfile('data.mat'); -m.Properties.Writable = true; +Normal `save` operations default to version 7. Creating a new file with +`matfile` defaults to version 7.3. File-system limits still apply. Version 7.3 +adds HDF5 metadata/chunk overhead and can be larger for heterogeneous +containers. -% Read partial data -x = m.bigArray(1:100, :); % First 100 rows only +Do not label arbitrary HDF5 as MATLAB v7.3. The format is HDF5-based but has +MATLAB conventions, references, metadata, and type encodings. GNU Octave 11 +cannot save MATLAB v7.3 and has only limited HDF5-based read support. -% Write partial data -m.bigArray(1:100, :) = newData; +## MAT safety -% Get variable info -sz = size(m, 'bigArray'); -``` +Never `load` an untrusted MAT file, even if selecting one variable. A MAT file +can contain: -## Images +- MATLAB objects whose classes customize deserialization with `loadobj` or + custom element serialization; +- constructors, listeners, or System object load hooks reachable from class + restoration; +- function handles and opaque values; +- Java objects and data interpreted by installed code; +- deeply nested/compressed structures that exhaust resources. -### Reading Images +`whos("-file", path)` is useful inside an already approved MATLAB environment, +but invoking MATLAB is itself execution. The bundled +`scripts/inventory_mat_file.py` never launches MATLAB and: -```matlab -% Read image -img = imread('image.png'); -img = imread('image.jpg'); -img = imread('image.tiff'); - -% Get image info -info = imfinfo('image.png'); -info.Width -info.Height -info.ColorType -info.BitDepth - -% Read specific frames (multi-page TIFF, GIF) -img = imread('animation.gif', 3); % Frame 3 -[img, map] = imread('indexed.gif'); % Indexed image with colormap -``` +- identifies the header/version; +- optionally uses `scipy.io.whosmat` for Level-5 metadata only; +- optionally uses `h5py` for bounded HDF5 names, shapes, dtypes, links, and + attribute names; +- never calls `scipy.io.loadmat`; +- never reads dataset values or follows soft/external HDF5 links; +- never deserializes objects or Python pickle. -### Writing Images +An inventory is triage, not a safety certificate. Object-like, opaque, +function, external-link, malformed, or unsupported content requires +quarantine and expert review. -```matlab -% Write image -imwrite(img, 'output.png'); -imwrite(img, 'output.jpg'); -imwrite(img, 'output.tiff'); - -% With options -imwrite(img, 'output.jpg', 'Quality', 95); -imwrite(img, 'output.png', 'BitDepth', 16); -imwrite(img, 'output.tiff', 'Compression', 'lzw'); - -% Write indexed image with colormap -imwrite(X, map, 'indexed.gif'); - -% Append to multi-page TIFF -imwrite(img1, 'multipage.tiff'); -imwrite(img2, 'multipage.tiff', 'WriteMode', 'append'); -``` +## Partial access -### Image Formats +For a trusted version 7.3 file: ```matlab -% Supported formats (partial list) -% BMP - Windows Bitmap -% GIF - Graphics Interchange Format -% JPEG - Joint Photographic Experts Group -% PNG - Portable Network Graphics -% TIFF - Tagged Image File Format -% PBM, PGM, PPM - Portable bitmap formats - -% Check supported formats -formats = imformats; +file = matfile("trusted-large.mat"); +shape = size(file, "measurements"); +block = file.measurements(1:1000, :); ``` -## Tables and Data Types - -### Creating Tables - -```matlab -% From variables -T = table(var1, var2, var3); -T = table(var1, var2, 'VariableNames', {'Col1', 'Col2'}); - -% From arrays -T = array2table(M); -T = array2table(M, 'VariableNames', {'A', 'B', 'C'}); - -% From cell array -T = cell2table(C); -T = cell2table(C, 'VariableNames', {'Name', 'Value'}); +`matfile` avoids loading an entire variable, but it still processes a MAT file +and can expose class/content risks. Partial read performance depends on HDF5 +chunk layout. Do not use it as a security sandbox. -% From struct -T = struct2table(S); -``` +## Low-level I/O -### Accessing Table Data +Use `onCleanup` to close reviewed files: ```matlab -% By variable name -col = T.VariableName; -col = T.('VariableName'); -col = T{:, 'VariableName'}; - -% By index -row = T(5, :); % Row 5 -col = T(:, 3); % Column 3 as table -data = T{:, 3}; % Column 3 as array -subset = T(1:10, 2:4); % Subset as table -data = T{1:10, 2:4}; % Subset as array - -% Logical indexing -subset = T(T.Value > 5, :); +[fid, message] = fopen("trusted-input.bin", "rb"); +assert(fid >= 0, message); +cleanup = onCleanup(@() fclose(fid)); +values = fread(fid, [4 1000], "single=>single"); ``` -### Modifying Tables +Specify byte order, element type, dimensions, record framing, and maximum +length. Validate `fread` counts and check arithmetic for overflow before +allocating. -```matlab -% Add variable -T.NewVar = newData; -T = addvars(T, newData, 'NewName', 'Col4'); -T = addvars(T, newData, 'Before', 'ExistingCol'); - -% Remove variable -T.OldVar = []; -T = removevars(T, 'OldVar'); -T = removevars(T, {'Col1', 'Col2'}); - -% Rename variable -T = renamevars(T, 'OldName', 'NewName'); -T.Properties.VariableNames{'OldName'} = 'NewName'; - -% Reorder variables -T = movevars(T, 'Col3', 'Before', 'Col1'); -T = T(:, {'Col2', 'Col1', 'Col3'}); -``` +HDF5, netCDF, CDF, FITS, Parquet, audio, video, images, databases, and +spreadsheets each have format/library/product/platform constraints. Use their +official current documentation and enforce parser-specific bounds. -### Table Operations +## Export and provenance -```matlab -% Sorting -T = sortrows(T, 'Column'); -T = sortrows(T, 'Column', 'descend'); -T = sortrows(T, {'Col1', 'Col2'}, {'ascend', 'descend'}); - -% Unique rows -T = unique(T); -T = unique(T, 'rows'); - -% Join tables -T = join(T1, T2); % Inner join on common keys -T = join(T1, T2, 'Keys', 'ID'); -T = innerjoin(T1, T2); -T = outerjoin(T1, T2); - -% Stack/unstack -T = stack(T, {'Var1', 'Var2'}); -T = unstack(T, 'Values', 'Keys'); - -% Group operations -G = groupsummary(T, 'GroupVar', 'mean', 'ValueVar'); -G = groupsummary(T, 'GroupVar', {'mean', 'std'}, 'ValueVar'); -``` +Record: -### Cell Arrays +- source and output checksums; +- schema/version, encoding, delimiter, locale, and numeric precision; +- variable names, classes, units, dimensions, missing rules; +- timestamp/time-zone representation; +- sort/group order; +- MAT version or external format/library; +- MATLAB release and required products. -```matlab -% Create cell array -C = {1, 'text', [1 2 3]}; -C = cell(m, n); % Empty m×n cell array - -% Access contents -contents = C{1, 2}; % Contents of cell (1,2) -subset = C(1:2, :); % Subset of cells (still cell array) - -% Convert -A = cell2mat(C); % To matrix (if compatible) -T = cell2table(C); % To table -S = cell2struct(C, fields); % To struct -``` +Prefer a documented language-neutral format for exchange: -### Structures +- CSV/TSV for simple rectangular values with a sidecar schema; +- JSON for bounded structured data with an explicit schema; +- Parquet for typed tabular interchange when all consumers agree; +- HDF5/netCDF for scientific arrays with documented conventions; +- MAT only for trusted MATLAB-oriented storage. -```matlab -% Create structure -S.field1 = value1; -S.field2 = value2; -S = struct('field1', value1, 'field2', value2); - -% Access fields -val = S.field1; -val = S.('field1'); - -% Field names -names = fieldnames(S); -tf = isfield(S, 'field1'); - -% Structure arrays -S(1).name = 'Alice'; -S(2).name = 'Bob'; -names = {S.name}; % Extract all names -``` - -## Low-Level File I/O - -### Opening and Closing Files - -```matlab -% Open file -fid = fopen('file.txt', 'r'); % Read -fid = fopen('file.txt', 'w'); % Write (overwrite) -fid = fopen('file.txt', 'a'); % Append -fid = fopen('file.bin', 'rb'); % Read binary -fid = fopen('file.bin', 'wb'); % Write binary - -% Check for errors -if fid == -1 - error('Could not open file'); -end - -% Close file -fclose(fid); -fclose('all'); % Close all files -``` - -### Text File I/O - -```matlab -% Read formatted data -data = fscanf(fid, '%f'); % Read floats -data = fscanf(fid, '%f %f', [2 Inf]); % Two columns -C = textscan(fid, '%f %s %f'); % Mixed types - -% Read lines -line = fgetl(fid); % One line (no newline) -line = fgets(fid); % One line (with newline) - -% Write formatted data -fprintf(fid, '%d, %f, %s\n', intVal, floatVal, strVal); -fprintf(fid, '%6.2f\n', data); - -% Read/write strings -str = fscanf(fid, '%s'); -fprintf(fid, '%s', str); -``` - -### Binary File I/O - -```matlab -% Read binary data -data = fread(fid, n, 'double'); % n doubles -data = fread(fid, [m n], 'int32'); % m×n int32s -data = fread(fid, Inf, 'uint8'); % All bytes - -% Write binary data -fwrite(fid, data, 'double'); -fwrite(fid, data, 'int32'); - -% Data types: 'int8', 'uint8', 'int16', 'uint16', 'int32', 'uint32', -% 'int64', 'uint64', 'single', 'double', 'char' -``` +Python pickle is executable deserialization, not a scientific interchange +format. Never create, load, or recommend pickle for MATLAB exchange. -### File Position +## Sources (verified 2026-07-23) -```matlab -% Get position -pos = ftell(fid); - -% Set position -fseek(fid, 0, 'bof'); % Beginning of file -fseek(fid, 0, 'eof'); % End of file -fseek(fid, offset, 'cof'); % Current position + offset - -% Rewind to beginning -frewind(fid); - -% Check end of file -tf = feof(fid); -``` - -### File and Directory Operations - -```matlab -% Check existence -tf = exist('file.txt', 'file'); -tf = exist('folder', 'dir'); -tf = isfile('file.txt'); -tf = isfolder('folder'); - -% List files -files = dir('*.csv'); % Struct array -files = dir('folder/*.mat'); -names = {files.name}; - -% File info -info = dir('file.txt'); -info.name -info.bytes -info.date -info.datenum - -% File operations -copyfile('src.txt', 'dst.txt'); -movefile('src.txt', 'dst.txt'); -delete('file.txt'); - -% Directory operations -mkdir('newfolder'); -rmdir('folder'); -rmdir('folder', 's'); % Remove with contents -cd('path'); -pwd % Current directory -``` - -### Path Operations - -```matlab -% Construct paths -fullpath = fullfile('folder', 'subfolder', 'file.txt'); -fullpath = fullfile(pwd, 'file.txt'); - -% Parse paths -[path, name, ext] = fileparts('/path/to/file.txt'); -% path = '/path/to', name = 'file', ext = '.txt' - -% Temporary files/folders -tmpfile = tempname; -tmpdir = tempdir; -``` +- [Data Import and Export](https://www.mathworks.com/help/matlab/data-import-and-export.html) +- [`detectImportOptions`](https://www.mathworks.com/help/matlab/ref/detectimportoptions.html) +- [`readtable`](https://www.mathworks.com/help/matlab/ref/readtable.html) +- [`writetable`](https://www.mathworks.com/help/matlab/ref/writetable.html) +- [Tables](https://www.mathworks.com/help/matlab/tables.html) +- [Timetables](https://www.mathworks.com/help/matlab/timetables.html) +- [`ismissing`](https://www.mathworks.com/help/matlab/ref/ismissing.html) +- [MAT File Versions](https://www.mathworks.com/help/matlab/import_export/mat-file-versions.html) +- [`MatFile`](https://www.mathworks.com/help/matlab/ref/matlab.io.matfile.html) +- [Object Save and Load](https://www.mathworks.com/help/matlab/save-and-load.html) +- [`loadobj`](https://www.mathworks.com/help/matlab/ref/loadobj.html) +- [HDF5 Files](https://www.mathworks.com/help/matlab/hdf5-files.html) +- [MATLAB R2026a release notes](https://www.mathworks.com/help/matlab/release-notes.html) diff --git a/.agents/skills/matlab/references/executing-scripts.md b/.agents/skills/matlab/references/executing-scripts.md index df71832..3f5f6b2 100644 --- a/.agents/skills/matlab/references/executing-scripts.md +++ b/.agents/skills/matlab/references/executing-scripts.md @@ -1,444 +1,213 @@ -````md -# Running MATLAB and GNU Octave Scripts from Bash - -This document shows common ways to execute MATLAB-style `.m` scripts from a Bash environment using both MATLAB (MathWorks) and GNU Octave. It covers interactive use, non-interactive batch runs, passing arguments, capturing output, and practical patterns for automation and CI. - -## Contents - -- Requirements -- Quick comparisons -- Running MATLAB scripts from Bash - - Interactive mode - - Run a script non-interactively - - Run a function with arguments - - Run one-liners - - Working directory and path handling - - Capturing output and exit codes - - Common MATLAB flags for scripting -- Running Octave scripts from Bash - - Interactive mode - - Run a script non-interactively - - Run a function with arguments - - Run one-liners - - Making `.m` files executable (shebang) - - Working directory and path handling - - Capturing output and exit codes - - Common Octave flags for scripting -- Cross-compatibility tips (MATLAB + Octave) -- Example: a portable runner script -- Troubleshooting - -## Requirements - -### MATLAB -- MATLAB must be installed. -- The `matlab` executable must be on your PATH, or you must reference it by full path. -- A valid license is required to run MATLAB. - -Check: -```bash -matlab -help | head -```` - -### GNU Octave - -* Octave must be installed. -* The `octave` executable must be on your PATH. - -Check: - -```bash -octave --version -``` - -## Quick comparison - -| Task | MATLAB | Octave | -| ----------------------------- | --------------------------------- | ------------------------ | -| Interactive shell | `matlab` (GUI by default) | `octave` | -| Headless run (CI) | `matlab -batch "cmd"` (preferred) | `octave --eval "cmd"` | -| Run script file | `matlab -batch "run('file.m')"` | `octave --no-gui file.m` | -| Exit with code | `exit(n)` | `exit(n)` | -| Make `.m` directly executable | uncommon | common via shebang | - -## Running MATLAB scripts from Bash - -### 1) Interactive mode - -Starts MATLAB. Depending on your platform and install, this may launch a GUI. +# Command-Line Execution, Products, and Migration -```bash -matlab -``` - -For terminal-only use, prefer `-nodesktop` and optionally `-nosplash`: +This reference explains reviewed execution plans. Bundled helpers never launch +MATLAB, GNU Octave, MATLAB Engine, MEX, a compiler, or any subprocess. -```bash -matlab -nodesktop -nosplash -``` +## Authorization gate -### 2) Run a script non-interactively +Before execution, confirm all of the following: -Recommended modern approach: `-batch`. It runs the command and exits when finished. +1. every `.m` file is trusted and statically reviewed; +2. no unreviewed `.mlx`, `.fig`, `.mlapp`, MEX, MAT object, project action, + startup file, package, or generated artifact is reachable; +3. inputs and outputs are strict local paths with bounds and overwrite policy; +4. runtime, exact release, architecture, required products, and license are + confirmed; +5. shell/native/Java/.NET/Python/code-generation surfaces are approved; +6. network, credentials, displays, and external services are understood; +7. the planned argv is shown to the user and execution is explicitly approved. -Run a script with `run()`: +Static scan findings are not proof of safety. Never execute a file solely to +discover what it does. -```bash -matlab -batch "run('myscript.m')" -``` +## MATLAB R2026a `-batch` -If the script relies on being run from its directory, set the working directory first: +MathWorks recommends `-batch` for noninteractive command-line workflows. +Conceptually, an approved plan looks like: -```bash -matlab -batch "cd('/path/to/project'); run('myscript.m')" +```text +["matlab", "-batch", "run('/reviewed/project/main.m')"] ``` -Alternative older pattern: `-r` (less robust for automation because you must ensure MATLAB exits): - -```bash -matlab -nodisplay -nosplash -r "run('myscript.m'); exit" -``` +This is argv, not an instruction to run untrusted code. -### 3) Run a function with arguments +Official R2026a behavior: -If your file defines a function, call it directly. Prefer `-batch`: +- starts without the desktop or splash screen; +- executes the quoted statement noninteractively; +- logs text to standard output/error; +- disables settings changes and toolbox caching; +- can display figures unless paired with `-noFigureWindows` or `-nodisplay`; +- exits automatically with code 0 on success and nonzero on failure; +- errors if code requests interactive dialog input (except supported app-test + fixtures); +- must not be combined with `-r`; +- requires the target to be in the startup folder or on the MATLAB path. -```bash -matlab -batch "myfunc(123, 'abc')" -``` - -To pass values from Bash variables: - -```bash -matlab -batch "myfunc(${N}, '${NAME}')" -``` - -If arguments may contain quotes or spaces, consider writing a small MATLAB wrapper function that reads environment variables. - -### 4) Run one-liners - -```bash -matlab -batch "disp(2+2)" -``` - -Multiple statements: - -```bash -matlab -batch "a=1; b=2; fprintf('%d\n', a+b)" -``` - -### 5) Working directory and path handling - -Common options: - -* Change directory at startup: - -```bash -matlab -batch "cd('/path/to/project'); myfunc()" -``` - -* Add code directories to MATLAB path: - -```bash -matlab -batch "addpath('/path/to/lib'); myfunc()" -``` - -To include subfolders: - -```bash -matlab -batch "addpath(genpath('/path/to/project')); myfunc()" -``` - -### 6) Capturing output and exit codes - -Capture stdout/stderr: - -```bash -matlab -batch "run('myscript.m')" > matlab.out 2>&1 -``` - -Check exit code: - -```bash -matlab -batch "run('myscript.m')" -echo $? -``` - -To explicitly fail a pipeline, use `exit(1)` on error. Example pattern: - -```matlab -try - run('myscript.m'); -catch ME - disp(getReport(ME)); - exit(1); -end -exit(0); -``` - -Run it: - -```bash -matlab -batch "try, run('myscript.m'); catch ME, disp(getReport(ME)); exit(1); end; exit(0);" -``` - -### 7) Common MATLAB flags for scripting - -Commonly useful options: - -* `-batch "cmd"`: run command, return a process exit code, then exit -* `-nodisplay`: no display (useful on headless systems) -* `-nodesktop`: no desktop GUI -* `-nosplash`: no startup splash -* `-r "cmd"`: run command; must include `exit` if you want it to terminate - -Exact availability varies by MATLAB release, so use `matlab -help` for your version. - -## Running GNU Octave scripts from Bash - -### 1) Interactive mode - -```bash -octave -``` - -Quieter: - -```bash -octave --quiet -``` - -### 2) Run a script non-interactively - -Run a file and exit: - -```bash -octave --no-gui myscript.m -``` - -Quieter: - -```bash -octave --quiet --no-gui myscript.m -``` - -Some environments use: - -```bash -octave --no-window-system myscript.m -``` - -### 3) Run a function with arguments - -If `myfunc.m` defines a function `myfunc`, call it via `--eval`: - -```bash -octave --quiet --eval "myfunc(123, 'abc')" -``` - -If your function is not on the Octave path, add paths first: - -```bash -octave --quiet --eval "addpath('/path/to/project'); myfunc()" -``` - -### 4) Run one-liners - -```bash -octave --quiet --eval "disp(2+2)" -``` - -Multiple statements: - -```bash -octave --quiet --eval "a=1; b=2; printf('%d\n', a+b);" -``` - -### 5) Making `.m` files executable (shebang) - -This is a common "standalone script" pattern in Octave. - -Create `myscript.m`: - -```matlab -#!/usr/bin/env octave -disp("Hello from Octave"); -``` - -Make executable: - -```bash -chmod +x myscript.m -``` - -Run: - -```bash -./myscript.m -``` - -If you need flags (quiet, no GUI), use a wrapper script instead, because the shebang line typically supports limited arguments across platforms. - -### 6) Working directory and path handling - -Change directory from the shell before running: - -```bash -cd /path/to/project -octave --quiet --no-gui myscript.m -``` - -Or change directory within Octave: - -```bash -octave --quiet --eval "cd('/path/to/project'); run('myscript.m');" -``` - -Add paths: - -```bash -octave --quiet --eval "addpath('/path/to/lib'); run('myscript.m');" -``` - -### 7) Capturing output and exit codes - -Capture stdout/stderr: - -```bash -octave --quiet --no-gui myscript.m > octave.out 2>&1 -``` - -Exit code: - -```bash -octave --quiet --no-gui myscript.m -echo $? -``` - -To force non-zero exit on error, wrap execution: - -```matlab -try - run('myscript.m'); -catch err - disp(err.message); - exit(1); -end -exit(0); -``` - -Run it: - -```bash -octave --quiet --eval "try, run('myscript.m'); catch err, disp(err.message); exit(1); end; exit(0);" -``` - -### 8) Common Octave flags for scripting - -Useful options: - -* `--eval "cmd"`: run a command string -* `--quiet`: suppress startup messages -* `--no-gui`: disable GUI -* `--no-window-system`: similar headless mode on some installs -* `--persist`: keep Octave open after running commands (opposite of batch behavior) - -Check: - -```bash -octave --help | head -n 50 -``` - -## Cross-compatibility tips (MATLAB and Octave) - -1. Prefer functions over scripts for automation - Functions give cleaner parameter passing and namespace handling. - -2. Avoid toolbox-specific calls if you need portability - Many MATLAB toolboxes have no Octave equivalent. - -3. Be careful with strings and quoting - MATLAB and Octave both support `'single quotes'`, and newer MATLAB supports `"double quotes"` strings. For maximum compatibility, prefer single quotes unless you know your Octave version supports double quotes the way you need. - -4. Use `fprintf` or `disp` for output - For CI logs, keep output simple and deterministic. - -5. Ensure exit codes reflect success or failure - In both environments, `exit(0)` indicates success, `exit(1)` indicates failure. - -## Example: a portable Bash runner - -This script tries MATLAB first if available, otherwise Octave. - -Create `run_mfile.sh`: - -```bash -#!/usr/bin/env bash -set -euo pipefail - -FILE="${1:?Usage: run_mfile.sh path/to/script_or_function.m}" -CMD="${2:-}" # optional command override - -if command -v matlab >/dev/null 2>&1; then - if [[ -n "$CMD" ]]; then - matlab -batch "$CMD" - else - matlab -batch "run('${FILE}')" - fi -elif command -v octave >/dev/null 2>&1; then - if [[ -n "$CMD" ]]; then - octave --quiet --no-gui --eval "$CMD" - else - octave --quiet --no-gui "$FILE" - fi -else - echo "Neither matlab nor octave found on PATH" >&2 - exit 127 -fi -``` - -Make executable: - -```bash -chmod +x run_mfile.sh -``` - -Run: - -```bash -./run_mfile.sh myscript.m -``` - -Or run a function call: - -```bash -./run_mfile.sh myfunc.m "myfunc(1, 'abc')" -``` +Use `-sd ` to set the initial folder. Do not embed untrusted +text in a MATLAB statement. Prefer a fixed function name and JSON-validated +scalar/list arguments converted by the planner. -## Troubleshooting +MATLAB startup still matters. On Linux, the launcher processes +`.matlab7rc.sh`; MATLAB also runs `matlabrc.m` and the first executable +`startup` on its path. `finish.m` can run at normal exit. A MATLAB Project can +add paths and run startup/shutdown actions. `-sd` is not a security sandbox. -### MATLAB: command not found +`-r` is for interactive workflows and has not been recommended for +noninteractive use since R2019a. Older `-r "...; exit"` patterns are easier to +hang or mask errors. -* Add MATLAB to PATH, or invoke it by full path, for example: +## Nonexecuting batch planner ```bash -/Applications/MATLAB_R202x?.app/bin/matlab -batch "disp('ok')" +python scripts/plan_batch_command.py matlab script src/main.m --root . +python scripts/plan_batch_command.py matlab function src/analyze.m \ + --root . --arg-json '{"value": 3}' +python scripts/plan_batch_command.py matlab tests tests/TestAnalyze.m --root . ``` -### Octave: GUI issues on servers +The planner: -* Use `--no-gui` or `--no-window-system`. +- validates a single `.m` target under `--root`; +- rejects symlinks, URLs, traversal, `.mlx`, MEX, and oversized paths; +- validates MATLAB identifiers and JSON values; +- returns argv, a MATLAB statement, assumptions, and warnings; +- marks `executes=false`; +- never checks `PATH`, calls a runtime, reads credentials, or spawns a process. -### Scripts depend on relative paths +JSON object arguments are represented as a MATLAB `struct`; arrays and scalar +JSON values use bounded literal conversion. Review semantics and shape before +approval. -* `cd` into the script directory before launching, or do `cd()` within MATLAB/Octave before calling `run()`. +## GNU Octave 11.3.0 plans -### Quoting problems when passing strings +The current manual documents: -* Avoid complex quoting in `--eval` or `-batch`. -* Use environment variables and read them inside MATLAB/Octave when inputs are complicated. +- `--eval`/`-e` to evaluate code and exit; +- a filename argument to execute a script and exit; +- `--no-gui`, `--quiet`, and `--no-history`; +- `--no-init-all`/`--norc` to skip system and user initialization; +- `--path` to add a narrow function path; +- `--no-window-system` to disable graphics entirely. -### Different behavior between MATLAB and Octave +For deterministic reviewed plans, prefer `--no-init-all --no-history +--quiet --no-gui`. Use `--no-window-system` only when graphics are not needed. +Octave also has site, version, user, local `.octaverc`, and MATLAB-compatible +`startup.m` files; skipping them changes expected user configuration and must +be a conscious choice. -* Check for unsupported functions or toolbox calls. -* Run minimal repro steps using `--eval` or `-batch` to isolate incompatibilities. +Octave does not implement MATLAB `-batch`, Projects, or +`matlab.unittest`. Its BIST `test` function and `%!test` blocks are different. +Do not use an Octave result as proof that MATLAB code, graphics, toolboxes, or +deployment will behave identically. + +## Functions, scripts, and test entry points + +For automation: + +- prefer a main function with explicit inputs/outputs; +- keep scripts free of base-workspace assumptions; +- avoid current-folder dependence and broad path mutation; +- return status through tests/errors rather than calling `exit` inside library + code; +- place all output under a reviewed output root; +- do not request interactive input. + +R2026a `runtests` automatically opens and closes a project when tests belong +to a project not already open. Review project startup/shutdown behavior before +using it. + +Base MATLAB has `matlab.unittest`; parallel execution requires Parallel +Computing Toolbox. Advanced dependency selection, dashboards, generated tests, +coverage/equivalence features can require MATLAB Test or other products. + +## Required products and license boundaries + +Separate four questions: + +1. **Static dependency:** Which products might code reference? +2. **Installation:** Which products/add-ons are installed? +3. **Entitlement:** Which licenses may this user/system use? +4. **Checkout:** Which licenses are available for this run? + +`matlab.codetools.requiredFilesAndProducts` and Dependency Analyzer address +the first question imperfectly. `license("inuse")` observes only products used +on executed paths and itself requires launching MATLAB. None grants a license. + +Do not automatically install MATLAB or a toolbox. Downloads, installers, +network-license configuration, and unattended automation are governed by the +user's MathWorks account, administrator, and license terms. The R2026a Program +Offering Guide has specific automation-server and external-application terms; +do not paraphrase it as legal permission. + +## Compiler and generated-code boundaries + +- **MATLAB Compiler** creates standalone/web applications that run with a + release-compatible MATLAB Runtime. +- **MATLAB Compiler SDK** creates components for external languages. +- **MATLAB Coder** generates C/C++ source from supported MATLAB. +- **GPU Coder, Simulink Coder, Embedded Coder**, support packages, and target + toolchains are separate products/capabilities. +- A platform C/C++/Fortran compiler may also be required and must appear in + the current supported-compiler table. + +Building requires MATLAB plus the compiler/code-generation product and all +products used by the source. Deployed applications can use MATLAB Runtime +under applicable terms, but Runtime does not execute arbitrary `.m` code and +cannot host MATLAB Engine for Python. Generated code must be verified; compiler +success is not scientific validation. + +Never compile untrusted MATLAB, MEX, C/C++, model, or package input. + +## CI design + +A safe CI design uses: + +- a pinned supported MATLAB release/update and platform; +- an administrator-approved license configuration; +- a reviewed project with no hidden startup action; +- immutable source and hashed inputs; +- a nonexecuting plan checked before the actual runner; +- bounded time/memory/output and no interactive dialogs; +- test results and logs that avoid environment/credential dumps; +- product and license failures distinguished from test failures; +- release notes and bug reports checked for the exact products. + +MathWorks provides CI integrations, but their presence does not include MATLAB +or grant a license. + +## Migration to R2026a + +1. Run `codeCompatibilityReport` and Project Upgrade on reviewed code. +2. Run Code Analyzer and dependency analysis. +3. Review base MATLAB and every required product's R2026a release notes, + compatibility considerations, supported platforms, compilers, Python, and + bug reports. +4. Record reference outputs from the old release using justified tolerances. +5. Test startup/path behavior, data import, MAT files, graphics, Python, + external interfaces, and deployment separately. +6. Check R2026a platform changes such as no new Intel Mac release. +7. Pilot before broad migration; retain rollback and provenance. + +Notable base changes relevant to this skill include Python 3.13 support and +environment management, Python string conversion, JSON table/timetable I/O, +interactive HTML export, faster startup and selected kernels, and project-aware +`runtests`. Read the release notes rather than assuming this list is complete. + +## Sources (verified 2026-07-23) + +- [`matlab` on Linux and `-batch`](https://www.mathworks.com/help/matlab/ref/matlablinux.html) +- [Startup Options](https://www.mathworks.com/help/matlab/matlab_env/startup-options.html) +- [Exit MATLAB](https://www.mathworks.com/help/matlab/matlab_env/exit-matlab.html) +- [Run Unit Tests](https://www.mathworks.com/help/matlab/run-unit-tests.html) +- [`runtests` R2026a behavior](https://www.mathworks.com/help/matlab/ref/runtests.html) +- [Analyze Project Dependencies](https://www.mathworks.com/help/matlab/matlab_prog/analyze-project-dependencies.html) +- [`requiredFilesAndProducts`](https://www.mathworks.com/help/matlab/ref/matlab.codetools.requiredfilesandproducts.html) +- [MATLAB Compiler](https://www.mathworks.com/products/compiler.html) +- [MATLAB Runtime](https://www.mathworks.com/products/compiler/matlab-runtime.html) +- [Supported Compilers](https://www.mathworks.com/support/requirements/supported-compilers.html) +- [R2026a System Requirements](https://www.mathworks.com/support/requirements/matlab-system-requirements.html) +- [R2026a Program Offering Guide](https://www.mathworks.com/help/pdf_doc/offering/offering.pdf) +- [R2026a Release Notes](https://www.mathworks.com/help/matlab/release-notes.html) +- [Octave Command-Line Options](https://docs.octave.org/latest/Command-Line-Options.html) +- [Octave Startup Files](https://docs.octave.org/latest/Startup-Files.html) diff --git a/.agents/skills/matlab/references/graphics-visualization.md b/.agents/skills/matlab/references/graphics-visualization.md index f8e8ca1..761017d 100644 --- a/.agents/skills/matlab/references/graphics-visualization.md +++ b/.agents/skills/matlab/references/graphics-visualization.md @@ -1,579 +1,181 @@ -# Graphics and Visualization Reference +# Graphics and Export -## Table of Contents -1. [2D Plotting](#2d-plotting) -2. [3D Plotting](#3d-plotting) -3. [Specialized Plots](#specialized-plots) -4. [Figure Management](#figure-management) -5. [Customization](#customization) -6. [Exporting and Saving](#exporting-and-saving) +This reference targets MATLAB R2026a. Rendering and property support differ in +GNU Octave and across MATLAB releases/platforms. -## 2D Plotting +## Build figures with explicit ownership -### Line Plots +Use handles instead of relying on `gcf`/`gca` in reusable code: ```matlab -% Basic line plot -plot(y); % Plot y vs index -plot(x, y); % Plot y vs x -plot(x, y, 'r-'); % Red solid line -plot(x, y, 'b--o'); % Blue dashed with circles - -% Line specification: [color][marker][linestyle] -% Colors: r g b c m y k w (red, green, blue, cyan, magenta, yellow, black, white) -% Markers: o + * . x s d ^ v > < p h -% Lines: - -- : -. - -% Multiple datasets -plot(x1, y1, x2, y2, x3, y3); -plot(x, [y1; y2; y3]'); % Columns as separate lines - -% With properties -plot(x, y, 'LineWidth', 2, 'Color', [0.5 0.5 0.5]); -plot(x, y, 'Marker', 'o', 'MarkerSize', 8, 'MarkerFaceColor', 'r'); - -% Get handle for later modification -h = plot(x, y); -h.LineWidth = 2; -h.Color = 'red'; -``` - -### Scatter Plots +fig = figure(Color="white"); +layout = tiledlayout(fig, 2, 1, ... + TileSpacing="compact", ... + Padding="compact"); -```matlab -scatter(x, y); % Basic scatter -scatter(x, y, sz); % With marker size -scatter(x, y, sz, c); % With color -scatter(x, y, sz, c, 'filled'); % Filled markers +ax1 = nexttile(layout); +plot(ax1, time, signal, LineWidth=1.5); +xlabel(ax1, "Time (s)"); +ylabel(ax1, "Amplitude (V)"); +title(ax1, "Measured signal"); +grid(ax1, "on"); -% sz: scalar or vector (marker sizes) -% c: color spec, scalar, vector (colormap), or RGB matrix - -% Properties -scatter(x, y, 'MarkerEdgeColor', 'b', 'MarkerFaceColor', 'r'); +ax2 = nexttile(layout); +histogram(ax2, residual, Normalization="pdf"); +xlabel(ax2, "Residual (V)"); +ylabel(ax2, "Density"); ``` -### Bar Charts +Explicit handles make tests, nested layouts, apps, and exports predictable. +Set limits, aspect ratio, color limits, and view deliberately when comparison +across figures matters. -```matlab -bar(y); % Vertical bars -bar(x, y); % At specified x positions -barh(y); % Horizontal bars +## Scientific communication checklist -% Grouped and stacked -bar(Y); % Each column is a group -bar(Y, 'stacked'); % Stacked bars +- Include quantities and units in labels. +- State transformations, normalization, aggregation, and uncertainty. +- Use colorblind-aware, perceptually ordered palettes; do not use color alone + for categories. +- Keep data and annotations distinguishable in grayscale when required. +- Match marker/line width, font size, and panel size to final publication size. +- Avoid misleading axis truncation or 3-D effects. +- Set deterministic sorting/group order before plotting categorical data. +- Add alternative text/caption information in the surrounding document. +- Inspect embedded raster content even when the container format is vector. -% Properties -bar(y, 'FaceColor', 'b', 'EdgeColor', 'k', 'LineWidth', 1.5); -bar(y, 0.5); % Bar width (0 to 1) -``` - -### Area Plots +Graphics functions can belong to separate products. For example, basic +`plot`, `scatter`, `histogram`, `imagesc`, `surf`, and `tiledlayout` are base +MATLAB, while domain-specific statistical, mapping, image, signal, or medical +visualizations can require named toolboxes. -```matlab -area(y); % Filled area under curve -area(x, y); -area(Y); % Stacked areas -area(Y, 'FaceAlpha', 0.5); % Transparent -``` +## Export with `exportgraphics` -### Histograms +Prefer `exportgraphics` for current workflows: ```matlab -histogram(x); % Automatic bins -histogram(x, nbins); % Number of bins -histogram(x, edges); % Specified edges -histogram(x, 'BinWidth', w); % Bin width - -% Normalization -histogram(x, 'Normalization', 'probability'); -histogram(x, 'Normalization', 'pdf'); -histogram(x, 'Normalization', 'count'); % default - -% 2D histogram -histogram2(x, y); -histogram2(x, y, 'DisplayStyle', 'tile'); -histogram2(x, y, 'FaceColor', 'flat'); +exportgraphics(fig, "overview.pdf", ContentType="vector"); +exportgraphics(ax1, "signal.png", Resolution=300); ``` -### Error Bars +R2026a-supported output includes: -```matlab -errorbar(x, y, err); % Symmetric error -errorbar(x, y, neg, pos); % Asymmetric error -errorbar(x, y, yneg, ypos, xneg, xpos); % X and Y errors +- raster: PNG, JPEG, TIFF, GIF; +- vector-capable: PDF, SVG, EPS, and Windows-only EMF; +- interactive HTML web canvas (new in R2026a). -% Horizontal -errorbar(x, y, err, 'horizontal'); +SVG support was added in R2025a. `Append=true` is supported for PDF and GIF, +not every format. `ContentType="vector"` applies where supported, but some plot +content can still be rasterized. `Resolution` is for raster output. R2025a +added dimensions/padding controls; verify exact option and unit support in the +target release. -% With line style -errorbar(x, y, err, 'o-', 'LineWidth', 1.5); -``` +Interactive HTML is active web content, not a static image. Review its +embedded assets and distribution context; do not open an untrusted exported +HTML file automatically. -### Logarithmic Plots +### Which export API? -```matlab -semilogy(x, y); % Log y-axis -semilogx(x, y); % Log x-axis -loglog(x, y); % Both axes log -``` +| API | Prefer for | Notes | +|---|---|---| +| `exportgraphics` | axes, layouts, figures, publication files | current default; crop/padding, vector/raster, multipage PDF | +| `copygraphics` | clipboard | interactive transfer; not reproducible file output | +| `exportapp` | app/UI capture | UI-focused behavior | +| `print` | legacy/device-specific workflows | behavior and UI support differ | +| `savefig` | editable MATLAB figure | MATLAB object artifact, not archival interchange | +| `saveas` | simple legacy save | less control than `exportgraphics` | +| `imwrite` | image arrays/animated GIF construction | not a general figure renderer | -### Polar Plots +Never treat `.fig` as passive. It stores MATLAB graphics objects and should be +handled as an untrusted MATLAB object artifact unless its provenance is known. -```matlab -polarplot(theta, rho); % Polar coordinates -polarplot(theta, rho, 'r-o'); % With line spec +## Headless and batch behavior -% Customize polar axes -pax = polaraxes; -pax.ThetaDir = 'clockwise'; -pax.ThetaZeroLocation = 'top'; -``` +`matlab -batch` starts without the desktop but can still display figure windows +unless `-noFigureWindows` or `-nodisplay` is added. Rendering may depend on +graphics hardware, fonts, installed system support, and platform. A planner +should distinguish: -## 3D Plotting +- **compute-only**: no figures; +- **off-screen export**: figures created but not shown; +- **interactive graphics**: requires a display and user; +- **web-canvas export**: generates active HTML. -### Line and Scatter +The bundled command planner only returns argv and never starts MATLAB. +Review trusted code, fonts, output paths, overwrite policy, and license before +an approved run. -```matlab -% 3D line plot -plot3(x, y, z); -plot3(x, y, z, 'r-', 'LineWidth', 2); +For deterministic export: -% 3D scatter -scatter3(x, y, z); -scatter3(x, y, z, sz, c, 'filled'); -``` +1. create a new explicit figure; +2. set size/units, axes limits, color limits, and fonts; +3. avoid dependence on desktop defaults and current objects; +4. set RNG before randomized jitter/layout; +5. export to a new local path and refuse unintended overwrite; +6. inventory output dimensions, file type, fonts, and embedded raster content; +7. compare images with an appropriate visual tolerance, not byte equality. -### Surface Plots +## Color and layout ```matlab -% Create grid first -[X, Y] = meshgrid(-2:0.1:2, -2:0.1:2); -Z = X.^2 + Y.^2; - -% Surface plot -surf(X, Y, Z); % Surface with edges -surf(Z); % Use indices as X, Y - -% Surface properties -surf(X, Y, Z, 'FaceColor', 'interp', 'EdgeColor', 'none'); -surf(X, Y, Z, 'FaceAlpha', 0.5); % Transparent - -% Mesh plot (wireframe) -mesh(X, Y, Z); -mesh(X, Y, Z, 'FaceColor', 'none'); - -% Surface with contour below -surfc(X, Y, Z); -meshc(X, Y, Z); +colororder(ax1, orderedColors); +colormap(ax2, "parula"); +clim(ax2, [lowerLimit upperLimit]); +axis(ax2, "tight"); ``` -### Contour Plots +Use a sequential map for ordered magnitude, a diverging map around a meaningful +center, and distinct categorical colors for unordered groups. Avoid `jet` for +quantitative interpretation. Keep a shared color scale when panels are meant +to be compared. -```matlab -contour(X, Y, Z); % 2D contour -contour(X, Y, Z, n); % n contour levels -contour(X, Y, Z, levels); % Specific levels -contourf(X, Y, Z); % Filled contours - -[C, h] = contour(X, Y, Z); -clabel(C, h); % Add labels +Use `tiledlayout`/`nexttile` rather than new `subplot` code. Legends and +colorbars can belong to an axes or layout; make ownership explicit. -% 3D contour -contour3(X, Y, Z); -``` +## Time, table, and categorical plots -### Other 3D Plots +Many plotting functions accept tables directly. This preserves variable-name +selection but does not remove the need to validate types and missing data. ```matlab -% Bar3 -bar3(Z); % 3D bar chart -bar3(Z, 'stacked'); - -% Pie3 -pie3(X); % 3D pie chart - -% Waterfall -waterfall(X, Y, Z); % Like mesh with no back lines - -% Ribbon -ribbon(Y); % 3D ribbon - -% Stem3 -stem3(x, y, z); % 3D stem plot +plot(T, "Time", ["Observed" "Predicted"]); +legend(["Observed" "Predicted"], Location="best"); ``` -### View and Lighting - -```matlab -% Set view angle -view(az, el); % Azimuth, elevation -view(2); % Top-down (2D view) -view(3); % Default 3D view -view([1 1 1]); % View from direction - -% Lighting -light; % Add light source -light('Position', [1 0 1]); -lighting gouraud; % Smooth lighting -lighting flat; % Flat shading -lighting none; % No lighting - -% Material properties -material shiny; -material dull; -material metal; - -% Shading -shading flat; % One color per face -shading interp; % Interpolated colors -shading faceted; % With edges (default) -``` +Sort time values and define duplicate/missing handling before plotting. +Categorical order controls axis/group order. Avoid silently dropping missing +values without reporting the count. -## Specialized Plots +## 3-D, transparency, and large data -### Statistical Plots +3-D surfaces, transparency, lighting, and very dense primitives can force +rasterization or produce platform-specific output. For large data: -```matlab -% Box plot -boxplot(data); -boxplot(data, groups); % Grouped -boxplot(data, 'Notch', 'on'); % With notches +- decimate only with a documented visual/statistical rule; +- preserve extremes and events; +- distinguish display reduction from analysis data; +- record the displayed sample count and aggregation; +- test export memory and file size. -% Violin plot (R2023b+) -violinplot(data); +## Review checklist -% Heatmap -heatmap(data); -heatmap(xLabels, yLabels, data); -heatmap(T, 'XVariable', 'Col1', 'YVariable', 'Col2', 'ColorVariable', 'Val'); +- [ ] Every object has an explicit parent handle. +- [ ] Data transformations and missing-value counts are documented. +- [ ] Axes, units, limits, and color scale are intentional. +- [ ] Product/toolbox requirements are declared. +- [ ] Output path is local, new, and reviewed. +- [ ] Vector versus raster intent is explicit. +- [ ] HTML and `.fig` outputs are treated as active/object artifacts. +- [ ] Fonts and embedded raster content are inspected. +- [ ] Batch mode and display requirements are compatible. +- [ ] Accessibility and final-size readability were reviewed. -% Parallel coordinates -parallelplot(data); -``` +## Sources (verified 2026-07-23) -### Image Display - -```matlab -% Display image -imshow(img); % Auto-scaled -imshow(img, []); % Scale to full range -imshow(img, [low high]); % Specify display range - -% Image as plot -image(C); % Direct indexed colors -imagesc(data); % Scaled colors -imagesc(data, [cmin cmax]); % Specify color limits - -% Colormap for imagesc -imagesc(data); -colorbar; -colormap(jet); -``` - -### Quiver and Stream - -```matlab -% Vector field -[X, Y] = meshgrid(-2:0.5:2); -U = -Y; -V = X; -quiver(X, Y, U, V); % 2D arrows -quiver3(X, Y, Z, U, V, W); % 3D arrows - -% Streamlines -streamline(X, Y, U, V, startx, starty); -``` - -### Pie and Donut - -```matlab -pie(X); % Pie chart -pie(X, explode); % Explode slices (logical) -pie(X, labels); % With labels - -% Donut (using patch or workaround) -pie(X); -% Add white circle in center for donut effect -``` - -## Figure Management - -### Creating Figures - -```matlab -figure; % New figure window -figure(n); % Figure with number n -fig = figure; % Get handle -fig = figure('Name', 'My Figure', 'Position', [100 100 800 600]); - -% Figure properties -fig.Color = 'white'; -fig.Units = 'pixels'; -fig.Position = [left bottom width height]; -``` - -### Subplots - -```matlab -subplot(m, n, p); % m×n grid, position p -subplot(2, 2, 1); % Top-left of 2×2 - -% Spanning multiple positions -subplot(2, 2, [1 2]); % Top row - -% With gap control -tiledlayout(2, 2); % Modern alternative -nexttile; -plot(x1, y1); -nexttile; -plot(x2, y2); - -% Tile spanning -nexttile([1 2]); % Span 2 columns -``` - -### Hold and Overlay - -```matlab -hold on; % Keep existing, add new plots -plot(x1, y1); -plot(x2, y2); -hold off; % Release - -% Alternative -hold(ax, 'on'); -hold(ax, 'off'); -``` - -### Multiple Axes - -```matlab -% Two y-axes -yyaxis left; -plot(x, y1); -ylabel('Left Y'); -yyaxis right; -plot(x, y2); -ylabel('Right Y'); - -% Linked axes -ax1 = subplot(2,1,1); plot(x, y1); -ax2 = subplot(2,1,2); plot(x, y2); -linkaxes([ax1, ax2], 'x'); % Link x-axes -``` - -### Current Objects - -```matlab -gcf; % Current figure handle -gca; % Current axes handle -gco; % Current object handle - -% Set current -figure(fig); -axes(ax); -``` - -## Customization - -### Labels and Title - -```matlab -title('My Title'); -title('My Title', 'FontSize', 14, 'FontWeight', 'bold'); - -xlabel('X Label'); -ylabel('Y Label'); -zlabel('Z Label'); % For 3D - -% With interpreter -title('$$\int_0^1 x^2 dx$$', 'Interpreter', 'latex'); -xlabel('Time (s)', 'Interpreter', 'none'); -``` - -### Legend - -```matlab -legend('Series 1', 'Series 2'); -legend({'Series 1', 'Series 2'}); -legend('Location', 'best'); % Auto-place -legend('Location', 'northeast'); -legend('Location', 'northeastoutside'); - -% With specific plots -h1 = plot(x1, y1); -h2 = plot(x2, y2); -legend([h1, h2], {'Data 1', 'Data 2'}); - -legend('off'); % Remove legend -legend('boxoff'); % Remove box -``` - -### Axis Control - -```matlab -axis([xmin xmax ymin ymax]); % Set limits -axis([xmin xmax ymin ymax zmin zmax]); % 3D -xlim([xmin xmax]); -ylim([ymin ymax]); -zlim([zmin zmax]); - -axis equal; % Equal aspect ratio -axis square; % Square axes -axis tight; % Fit to data -axis auto; % Automatic -axis off; % Hide axes -axis on; % Show axes - -% Reverse direction -set(gca, 'YDir', 'reverse'); -set(gca, 'XDir', 'reverse'); -``` - -### Grid and Box - -```matlab -grid on; -grid off; -grid minor; % Minor grid lines - -box on; % Show box -box off; % Hide box -``` - -### Ticks - -```matlab -xticks([0 1 2 3 4 5]); -yticks(0:0.5:3); - -xticklabels({'A', 'B', 'C', 'D', 'E', 'F'}); -yticklabels({'Low', 'Medium', 'High'}); - -xtickangle(45); % Rotate labels -ytickformat('%.2f'); % Format -xtickformat('usd'); % Currency -``` - -### Colors and Colormaps - -```matlab -% Predefined colormaps -colormap(jet); -colormap(parula); % Default -colormap(hot); -colormap(cool); -colormap(gray); -colormap(bone); -colormap(hsv); -colormap(turbo); -colormap(viridis); - -% Colorbar -colorbar; -colorbar('Location', 'eastoutside'); -caxis([cmin cmax]); % Color limits -clim([cmin cmax]); % R2022a+ syntax - -% Custom colormap -cmap = [1 0 0; 0 1 0; 0 0 1]; % Red, green, blue -colormap(cmap); - -% Color order for lines -colororder(colors); % R2019b+ -``` - -### Text and Annotations - -```matlab -% Add text -text(x, y, 'Label'); -text(x, y, z, 'Label'); % 3D -text(x, y, 'Label', 'FontSize', 12, 'Color', 'red'); -text(x, y, 'Label', 'HorizontalAlignment', 'center'); - -% Annotations -annotation('arrow', [x1 x2], [y1 y2]); -annotation('textarrow', [x1 x2], [y1 y2], 'String', 'Peak'); -annotation('ellipse', [x y w h]); -annotation('rectangle', [x y w h]); -annotation('line', [x1 x2], [y1 y2]); - -% Text with LaTeX -text(x, y, '$$\alpha = \beta^2$$', 'Interpreter', 'latex'); -``` - -### Lines and Shapes - -```matlab -% Reference lines -xline(5); % Vertical line at x=5 -yline(10); % Horizontal line at y=10 -xline(5, '--r', 'Threshold'); % With label - -% Shapes -rectangle('Position', [x y w h]); -rectangle('Position', [x y w h], 'Curvature', [0.2 0.2]); % Rounded - -% Patches (filled polygons) -patch(xv, yv, 'blue'); -patch(xv, yv, zv, 'blue'); % 3D -``` - -## Exporting and Saving - -### Save Figure - -```matlab -saveas(gcf, 'figure.png'); -saveas(gcf, 'figure.fig'); % MATLAB figure file -saveas(gcf, 'figure.pdf'); -saveas(gcf, 'figure.eps'); -``` - -### Print Command - -```matlab -print('-dpng', 'figure.png'); -print('-dpng', '-r300', 'figure.png'); % 300 DPI -print('-dpdf', 'figure.pdf'); -print('-dsvg', 'figure.svg'); -print('-deps', 'figure.eps'); -print('-depsc', 'figure.eps'); % Color EPS - -% Vector formats for publication -print('-dpdf', '-painters', 'figure.pdf'); -print('-dsvg', '-painters', 'figure.svg'); -``` - -### Export Graphics (R2020a+) - -```matlab -exportgraphics(gcf, 'figure.png'); -exportgraphics(gcf, 'figure.png', 'Resolution', 300); -exportgraphics(gcf, 'figure.pdf', 'ContentType', 'vector'); -exportgraphics(gca, 'axes_only.png'); % Just the axes - -% For presentations/documents -exportgraphics(gcf, 'figure.emf'); % Windows -exportgraphics(gcf, 'figure.eps'); % LaTeX -``` - -### Copy to Clipboard - -```matlab -copygraphics(gcf); % Copy current figure -copygraphics(gca); % Copy current axes -copygraphics(gcf, 'ContentType', 'vector'); -``` - -### Paper Size (for Printing) - -```matlab -set(gcf, 'PaperUnits', 'inches'); -set(gcf, 'PaperPosition', [0 0 6 4]); -set(gcf, 'PaperSize', [6 4]); -set(gcf, 'PaperPositionMode', 'auto'); -``` +- [`tiledlayout`](https://www.mathworks.com/help/matlab/ref/tiledlayout.html) +- [`exportgraphics`](https://www.mathworks.com/help/matlab/ref/exportgraphics.html) +- [Compare Ways to Export Graphics](https://www.mathworks.com/help/matlab/creating_plots/compare-ways-to-export-save-graphics-plots-from-figures.html) +- [`copygraphics`](https://www.mathworks.com/help/matlab/ref/copygraphics.html) +- [`exportapp`](https://www.mathworks.com/help/matlab/ref/exportapp.html) +- [MATLAB Graphics](https://www.mathworks.com/help/matlab/graphics.html) +- [R2026a release notes](https://www.mathworks.com/help/matlab/release-notes.html) +- [`matlab -batch` behavior on Linux](https://www.mathworks.com/help/matlab/ref/matlablinux.html) diff --git a/.agents/skills/matlab/references/mathematics.md b/.agents/skills/matlab/references/mathematics.md index ece585a..0e6f272 100644 --- a/.agents/skills/matlab/references/mathematics.md +++ b/.agents/skills/matlab/references/mathematics.md @@ -1,553 +1,208 @@ -# Mathematics Reference +# Numerical Methods, Tolerances, and Reproducibility -## Table of Contents -1. [Linear Algebra](#linear-algebra) -2. [Elementary Math](#elementary-math) -3. [Calculus and Integration](#calculus-and-integration) -4. [Differential Equations](#differential-equations) -5. [Optimization](#optimization) -6. [Statistics](#statistics) -7. [Signal Processing](#signal-processing) -8. [Interpolation and Fitting](#interpolation-and-fitting) +This reference targets MATLAB R2026a. Confirm every non-base product before +using toolbox-specific functions. -## Linear Algebra +## Linear systems and decompositions -### Solving Linear Systems +Solve systems; do not form an inverse as an intermediate: ```matlab -% Ax = b -x = A \ b; % Preferred method (mldivide) -x = linsolve(A, b); % With options -x = inv(A) * b; % Less efficient, avoid - -% Options for linsolve -opts.LT = true; % Lower triangular -opts.UT = true; % Upper triangular -opts.SYM = true; % Symmetric -opts.POSDEF = true; % Positive definite -x = linsolve(A, b, opts); - -% xA = b -x = b / A; % mrdivide - -% Least squares (overdetermined system) -x = A \ b; % Minimum norm solution -x = lsqminnorm(A, b); % Explicit minimum norm - -% Nonnegative least squares -x = lsqnonneg(A, b); % x >= 0 constraint +x = A \ b; +residual = A*x - b; +relativeResidual = norm(residual) / ... + max(norm(A)*norm(x) + norm(b), realmin(class(A))); ``` -### Matrix Decompositions +Check dimensions, rank/conditioning, scaling, symmetry, definiteness, and +sparsity. A small residual does not guarantee a small forward error for an +ill-conditioned problem. -```matlab -% LU decomposition: A = L*U or P*A = L*U -[L, U] = lu(A); % L may not be lower triangular -[L, U, P] = lu(A); % P*A = L*U - -% QR decomposition: A = Q*R -[Q, R] = qr(A); % Full decomposition -[Q, R] = qr(A, 0); % Economy size -[Q, R, P] = qr(A); % Column pivoting: A*P = Q*R - -% Cholesky: A = R'*R (symmetric positive definite) -R = chol(A); % Upper triangular -L = chol(A, 'lower'); % Lower triangular +Common base MATLAB operations include: -% LDL': A = L*D*L' (symmetric) -[L, D] = ldl(A); +- `lu`, `qr`, `chol`, `ldl`, `schur`; +- `eig`, `svd`, `eigs`, `svds`; +- `rank`, `cond`, `rcond`, `norm`, `pinv`; +- `lsqminnorm`, `lsqnonneg`, and backslash least squares. -% Schur decomposition: A = U*T*U' -[U, T] = schur(A); % T is quasi-triangular -[U, T] = schur(A, 'complex'); % T is triangular -``` +Use an economy decomposition where appropriate and request only the spectrum +needed for large/sparse problems. Eigenvector signs/phases and bases in +degenerate subspaces are not unique; compare invariant quantities rather than +raw vectors. -### Eigenvalues and Eigenvectors +## Floating-point comparison -```matlab -% Eigenvalues -e = eig(A); % Eigenvalues only -[V, D] = eig(A); % V: eigenvectors, D: diagonal eigenvalues - % A*V = V*D - -% Generalized eigenvalues: A*v = lambda*B*v -e = eig(A, B); -[V, D] = eig(A, B); - -% Sparse/large matrices (subset of eigenvalues) -e = eigs(A, k); % k largest magnitude -e = eigs(A, k, 'smallestabs'); % k smallest magnitude -[V, D] = eigs(A, k, 'largestreal'); -``` +Binary floating point does not represent most decimal fractions exactly. +Choose tolerances from the model, scale, conditioning, discretization, +measurement uncertainty, and algorithm—not from a universal constant. -### Singular Value Decomposition +A robust scalar/elementwise policy often has the form: ```matlab -% SVD: A = U*S*V' -[U, S, V] = svd(A); % Full decomposition -[U, S, V] = svd(A, 'econ'); % Economy size -s = svd(A); % Singular values only - -% Sparse/large matrices -[U, S, V] = svds(A, k); % k largest singular values - -% Applications -r = rank(A); % Rank (count nonzero singular values) -p = pinv(A); % Pseudoinverse (via SVD) -n = norm(A, 2); % 2-norm = largest singular value -c = cond(A); % Condition number = ratio of largest/smallest +errorMagnitude = abs(actual - expected); +limit = absoluteTolerance + relativeTolerance .* abs(expected); +isAcceptable = errorMagnitude <= limit; ``` -### Matrix Properties - -```matlab -d = det(A); % Determinant -t = trace(A); % Trace (sum of diagonal) -r = rank(A); % Rank -n = norm(A); % 2-norm (default) -n = norm(A, 1); % 1-norm (max column sum) -n = norm(A, inf); % Inf-norm (max row sum) -n = norm(A, 'fro'); % Frobenius norm -c = cond(A); % Condition number -c = rcond(A); % Reciprocal condition (fast estimate) -``` - -## Elementary Math - -### Trigonometric Functions +Handle these explicitly: -```matlab -% Radians -y = sin(x); y = cos(x); y = tan(x); -y = asin(x); y = acos(x); y = atan(x); -y = atan2(y, x); % Four-quadrant arctangent +- expected values near zero need an absolute tolerance; +- large expected values often need a relative tolerance; +- `NaN` equality is a semantic decision (`isequaln` differs from `==`); +- `Inf` signs should match when infinity is expected; +- class, size, sparsity, and complex values are part of the contract. -% Degrees -y = sind(x); y = cosd(x); y = tand(x); -y = asind(x); y = acosd(x); y = atand(x); +R2026a documents `isapprox` alongside equality operations. In +`matlab.unittest`, use `AbsTol`/`RelTol` or +`AbsoluteTolerance`/`RelativeTolerance`. Record why values are scientifically +acceptable. -% Hyperbolic -y = sinh(x); y = cosh(x); y = tanh(x); -y = asinh(x); y = acosh(x); y = atanh(x); +Do not widen tolerances automatically after an upgrade. First investigate RNG, +ordering, reduction order, solver defaults/options, data type, threading, +compiler, library, and release-note changes. -% Secant, cosecant, cotangent -y = sec(x); y = csc(x); y = cot(x); -``` +## Random streams -### Exponentials and Logarithms +Record algorithm and seed, not only a seed: ```matlab -y = exp(x); % e^x -y = log(x); % Natural log (ln) -y = log10(x); % Log base 10 -y = log2(x); % Log base 2 -y = log1p(x); % log(1+x), accurate for small x -[F, E] = log2(x); % F * 2^E = x - -y = sqrt(x); % Square root -y = nthroot(x, n); % Real n-th root -y = realsqrt(x); % Real square root (error if x < 0) - -y = pow2(x); % 2^x -y = x .^ y; % Element-wise power +rng(1729, "twister"); +stateAtStart = rng; +samples = randn(1000, 1); ``` -### Complex Numbers +For local independent streams: ```matlab -z = complex(a, b); % a + bi -z = 3 + 4i; % Direct creation - -r = real(z); % Real part -i = imag(z); % Imaginary part -m = abs(z); % Magnitude -p = angle(z); % Phase angle (radians) -c = conj(z); % Complex conjugate - -[theta, rho] = cart2pol(x, y); % Cartesian to polar -[x, y] = pol2cart(theta, rho); % Polar to Cartesian +stream = RandStream("Threefry", Seed=1729); +stream.Substream = 4; +samples = randn(stream, 1000, 1); ``` -### Rounding and Remainders - -```matlab -y = round(x); % Round to nearest integer -y = round(x, n); % Round to n decimal places -y = floor(x); % Round toward -infinity -y = ceil(x); % Round toward +infinity -y = fix(x); % Round toward zero - -y = mod(x, m); % Modulo (sign of m) -y = rem(x, m); % Remainder (sign of x) -[q, r] = deconv(x, m); % Quotient and remainder - -y = sign(x); % Sign (-1, 0, or 1) -y = abs(x); % Absolute value -``` - -### Special Functions - -```matlab -y = gamma(x); % Gamma function -y = gammaln(x); % Log gamma (avoid overflow) -y = factorial(n); % n! -y = nchoosek(n, k); % Binomial coefficient - -y = erf(x); % Error function -y = erfc(x); % Complementary error function -y = erfcinv(x); % Inverse complementary error function +Generator availability and bitwise sequences can vary by algorithm/release. +Avoid `rng("shuffle")` for reproducible work. On parallel workers, time-based +seeding can collide; use supported independent streams/substreams and record +worker mapping. Parallel computing requires Parallel Computing Toolbox. -y = besselj(nu, x); % Bessel J -y = bessely(nu, x); % Bessel Y -y = besseli(nu, x); % Modified Bessel I -y = besselk(nu, x); % Modified Bessel K +## Integration, roots, and differential equations -y = legendre(n, x); % Legendre polynomials -``` +Base MATLAB provides general numerical methods including: -## Calculus and Integration +- `integral`, `integral2`, `integral3`, `trapz`, `cumtrapz`; +- `gradient`, `diff`; +- `fzero`; +- ODE solvers such as `ode45`, `ode23`, `ode113`, `ode15s`, `ode23s`, + `ode23t`, and `ode23tb`; +- boundary-value solvers such as `bvp4c` and `bvp5c`. -### Numerical Integration +Define tolerances and failure criteria: ```matlab -% Definite integrals -q = integral(fun, a, b); % Integrate fun from a to b -q = integral(@(x) x.^2, 0, 1); % Example: integral of x^2 - -% Options -q = integral(fun, a, b, 'AbsTol', 1e-10); -q = integral(fun, a, b, 'RelTol', 1e-6); - -% Improper integrals -q = integral(fun, 0, Inf); % Integrate to infinity -q = integral(fun, -Inf, Inf); % Full real line - -% Multidimensional -q = integral2(fun, xa, xb, ya, yb); % Double integral -q = integral3(fun, xa, xb, ya, yb, za, zb); % Triple integral - -% From discrete data -q = trapz(x, y); % Trapezoidal rule -q = trapz(y); % Unit spacing -q = cumtrapz(x, y); % Cumulative integral +options = odeset( ... + RelTol=1e-7, ... + AbsTol=1e-10, ... + MaxStep=0.05); +[t, y] = ode45(@rhs, [0 5], 1, options); ``` -### Numerical Differentiation - -```matlab -% Finite differences -dy = diff(y); % First differences -dy = diff(y, n); % n-th differences -dy = diff(y, n, dim); % Along dimension - -% Gradient (numerical derivative) -g = gradient(y); % dy/dx, unit spacing -g = gradient(y, h); % dy/dx, spacing h -[gx, gy] = gradient(Z, hx, hy); % Gradient of 2D data -``` +Solver tolerances control local error estimates, not proof of a globally +correct model. Check conservation laws, event localization, stiffness, +step-size convergence, and an independent formulation. R2026a adds an +automatic-differentiation Jacobian option for the `ode` object; verify the +specific solver/problem and release notes before using it. -## Differential Equations +## Optimization and fitting boundaries -### ODE Solvers +Base MATLAB includes `fminsearch` and `fminbnd`. These do not replace +constrained or specialized solvers. -```matlab -% Standard form: dy/dt = f(t, y) -odefun = @(t, y) -2*y; % Example: dy/dt = -2y -[t, y] = ode45(odefun, tspan, y0); - -% Solver selection: -% ode45 - Nonstiff, medium accuracy (default choice) -% ode23 - Nonstiff, low accuracy -% ode113 - Nonstiff, variable order -% ode15s - Stiff, variable order (try if ode45 is slow) -% ode23s - Stiff, low order -% ode23t - Moderately stiff, trapezoidal -% ode23tb - Stiff, TR-BDF2 - -% With options -options = odeset('RelTol', 1e-6, 'AbsTol', 1e-9); -options = odeset('MaxStep', 0.1); -options = odeset('Events', @myEventFcn); % Stop conditions -[t, y] = ode45(odefun, tspan, y0, options); -``` +Examples of separately licensed boundaries: -### Higher-Order ODEs +| Capability | Representative API | Product to confirm | +|---|---|---| +| constrained/nonlinear optimization | `fmincon`, `fminunc`, `lsqnonlin`, `lsqcurvefit` | Optimization Toolbox | +| global/metaheuristic optimization | `ga`, `particleswarm`, `surrogateopt` | Global Optimization Toolbox | +| curve fitting objects/apps | `fit`, Curve Fitter | Curve Fitting Toolbox | +| statistical modeling/distributions | `fitlm`, `fitdist`, `anova`, many tests | Statistics and Machine Learning Toolbox | +| symbolic algebra | `syms`, `solve`, symbolic differentiation | Symbolic Math Toolbox | +| signal design/analysis | `fir1`, `filtfilt`, `designfilt`, `spectrogram` | Signal Processing Toolbox | +| parallel loops/GPU | `parfor`, `parpool`, `gpuArray` | Parallel Computing Toolbox | -```matlab -% y'' + 2y' + y = 0, y(0) = 1, y'(0) = 0 -% Convert to system: y1 = y, y2 = y' -% y1' = y2 -% y2' = -2*y2 - y1 - -odefun = @(t, y) [y(2); -2*y(2) - y(1)]; -y0 = [1; 0]; % [y(0); y'(0)] -[t, y] = ode45(odefun, [0 10], y0); -plot(t, y(:,1)); % Plot y (first component) -``` - -### Boundary Value Problems - -```matlab -% y'' + |y| = 0, y(0) = 0, y(4) = -2 -solinit = bvpinit(linspace(0, 4, 5), [0; 0]); -sol = bvp4c(@odefun, @bcfun, solinit); +Some base functions have similarly named toolbox alternatives. Check the +function's current product page and the project dependency report; never infer +ownership from a code example. -function dydx = odefun(x, y) - dydx = [y(2); -abs(y(1))]; -end +Optimization reproducibility requires objective/constraint definitions, +starting points, bounds, solver/options, stopping tolerances, gradients, +scaling, RNG state for stochastic methods, and exit diagnostics. Compare +feasibility and optimality measures, not only the objective value. -function res = bcfun(ya, yb) - res = [ya(1); yb(1) + 2]; % y(0) = 0, y(4) = -2 -end -``` +## Statistics and signal processing -## Optimization +Base array summaries include `mean`, `median`, `std`, `var`, `min`, `max`, +`movmean`, `movmedian`, `cov`, `corrcoef`, `histcounts`, and polynomial +`polyfit`/`polyval`. Some distribution, model, hypothesis-test, robust, +classification, and specialized plotting APIs require Statistics and Machine +Learning Toolbox. -### Unconstrained Optimization +For FFT work: ```matlab -% Single variable, bounded -[x, fval] = fminbnd(fun, x1, x2); -[x, fval] = fminbnd(@(x) x.^2 - 4*x, 0, 5); - -% Multivariable, unconstrained -[x, fval] = fminsearch(fun, x0); -options = optimset('TolX', 1e-8, 'TolFun', 1e-8); -[x, fval] = fminsearch(fun, x0, options); - -% Display iterations -options = optimset('Display', 'iter'); +n = numel(x); +Y = fft(x); +frequency = (0:n-1).' * (sampleRate/n); ``` -### Root Finding +Document sample rate, units, window, detrending, normalization, one- versus +two-sided spectrum, zero padding, and endpoint convention. `fft` and `conv` are +base MATLAB; many filter-design and spectral-estimation functions are Signal +Processing Toolbox. -```matlab -% Find where f(x) = 0 -x = fzero(fun, x0); % Near x0 -x = fzero(fun, [x1 x2]); % In interval [x1, x2] -x = fzero(@(x) cos(x) - x, 0.5); - -% Polynomial roots -r = roots([1 0 -4]); % Roots of x^2 - 4 = 0 - % Returns [2; -2] -``` +## Verification patterns -### Least Squares +Use several layers: -```matlab -% Linear least squares: minimize ||Ax - b|| -x = A \ b; % Standard solution -x = lsqminnorm(A, b); % Minimum norm solution +1. **Dimensional/invariant checks**: sizes, units, conservation, monotonicity, + positivity, symmetry. +2. **Analytic cases**: small problems with known solutions. +3. **Refinement studies**: mesh, step, quadrature, or tolerance convergence. +4. **Independent implementation**: alternative solver or formulation. +5. **Condition/sensitivity analysis**: perturb inputs and options. +6. **Release comparison**: compare scientifically meaningful observables with + a documented tolerance. +7. **Performance measurement**: after correctness, measure representative + workloads with `timeit`. -% Nonnegative least squares -x = lsqnonneg(A, b); % x >= 0 +Do not claim bitwise reproducibility across releases, hardware, thread counts, +GPU/CPU, or external libraries unless it was actually tested and documented. -% Nonlinear least squares -x = lsqnonlin(fun, x0); % Minimize sum(fun(x).^2) -x = lsqcurvefit(fun, x0, xdata, ydata); % Curve fitting -``` +## Reproducibility record -## Statistics +At minimum capture: -### Descriptive Statistics +- MATLAB release/update or Octave version; +- OS and architecture, only as named fields; +- required products and license status separately; +- source/input hashes and schema versions; +- numeric classes and shapes; +- RNG algorithm, seed, substream, and parallel mapping; +- solver names/options/tolerances and stopping diagnostics; +- expected invariants and acceptance tolerances; +- output format/version and graphics export settings. + +Use `scripts/reproducibility_report.py` to hash only named local artifacts. It +does not inspect the broad environment. -```matlab -% Central tendency -m = mean(x); % Arithmetic mean -m = mean(x, 'all'); % Mean of all elements -m = mean(x, dim); % Mean along dimension -m = mean(x, 'omitnan'); % Ignore NaN values -gm = geomean(x); % Geometric mean -hm = harmmean(x); % Harmonic mean -med = median(x); % Median -mo = mode(x); % Mode - -% Dispersion -s = std(x); % Standard deviation (N-1) -s = std(x, 1); % Population std (N) -v = var(x); % Variance -r = range(x); % max - min -iqr_val = iqr(x); % Interquartile range - -% Extremes -[minv, mini] = min(x); -[maxv, maxi] = max(x); -[lo, hi] = bounds(x); % Min and max together -``` +## Sources (verified 2026-07-23) -### Correlation and Covariance - -```matlab -% Correlation -R = corrcoef(X, Y); % Correlation matrix -r = corrcoef(x, y); % Correlation coefficient - -% Covariance -C = cov(X, Y); % Covariance matrix -c = cov(x, y); % Covariance - -% Cross-correlation (signal processing) -[r, lags] = xcorr(x, y); % Cross-correlation -[r, lags] = xcorr(x, y, 'coeff'); % Normalized -``` - -### Percentiles and Quantiles - -```matlab -p = prctile(x, [25 50 75]); % Percentiles -q = quantile(x, [0.25 0.5 0.75]); % Quantiles -``` - -### Moving Statistics - -```matlab -y = movmean(x, k); % k-point moving average -y = movmedian(x, k); % Moving median -y = movstd(x, k); % Moving standard deviation -y = movvar(x, k); % Moving variance -y = movmin(x, k); % Moving minimum -y = movmax(x, k); % Moving maximum -y = movsum(x, k); % Moving sum - -% Window options -y = movmean(x, [kb kf]); % kb back, kf forward -y = movmean(x, k, 'omitnan'); % Ignore NaN -``` - -### Histograms and Distributions - -```matlab -% Histogram counts -[N, edges] = histcounts(x); % Automatic binning -[N, edges] = histcounts(x, nbins); % Specify number of bins -[N, edges] = histcounts(x, edges); % Specify edges - -% Probability/normalized -[N, edges] = histcounts(x, 'Normalization', 'probability'); -[N, edges] = histcounts(x, 'Normalization', 'pdf'); - -% 2D histogram -[N, xedges, yedges] = histcounts2(x, y); -``` - -## Signal Processing - -### Fourier Transform - -```matlab -% FFT -Y = fft(x); % 1D FFT -Y = fft(x, n); % n-point FFT (zero-pad/truncate) -Y = fft2(X); % 2D FFT -Y = fftn(X); % N-D FFT - -% Inverse FFT -x = ifft(Y); -X = ifft2(Y); -X = ifftn(Y); - -% Shift zero-frequency to center -Y_shifted = fftshift(Y); -Y = ifftshift(Y_shifted); - -% Frequency axis -n = length(x); -fs = 1000; % Sampling frequency -f = (0:n-1) * fs / n; % Frequency vector -f = (-n/2:n/2-1) * fs / n; % Centered frequency vector -``` - -### Filtering - -```matlab -% 1D filtering -y = filter(b, a, x); % Apply IIR/FIR filter -y = filtfilt(b, a, x); % Zero-phase filtering - -% Simple moving average -b = ones(1, k) / k; -y = filter(b, 1, x); - -% Convolution -y = conv(x, h); % Full convolution -y = conv(x, h, 'same'); % Same size as x -y = conv(x, h, 'valid'); % Valid part only - -% Deconvolution -[q, r] = deconv(y, h); % y = conv(q, h) + r - -% 2D filtering -Y = filter2(H, X); % 2D filter -Y = conv2(X, H, 'same'); % 2D convolution -``` - -## Interpolation and Fitting - -### Interpolation - -```matlab -% 1D interpolation -yi = interp1(x, y, xi); % Linear (default) -yi = interp1(x, y, xi, 'spline'); % Spline -yi = interp1(x, y, xi, 'pchip'); % Piecewise cubic -yi = interp1(x, y, xi, 'nearest'); % Nearest neighbor - -% 2D interpolation -zi = interp2(X, Y, Z, xi, yi); -zi = interp2(X, Y, Z, xi, yi, 'spline'); - -% 3D interpolation -vi = interp3(X, Y, Z, V, xi, yi, zi); - -% Scattered data -F = scatteredInterpolant(x, y, v); -vi = F(xi, yi); -``` - -### Polynomial Fitting - -```matlab -% Polynomial fit -p = polyfit(x, y, n); % Fit degree-n polynomial - % p = [p1, p2, ..., pn+1] - % y = p1*x^n + p2*x^(n-1) + ... + pn+1 - -% Evaluate polynomial -yi = polyval(p, xi); - -% With fit quality -[p, S] = polyfit(x, y, n); -[yi, delta] = polyval(p, xi, S); % delta = error estimate - -% Polynomial operations -r = roots(p); % Find roots -p = poly(r); % Polynomial from roots -q = polyder(p); % Derivative -q = polyint(p); % Integral -c = conv(p1, p2); % Multiply polynomials -[q, r] = deconv(p1, p2); % Divide polynomials -``` - -### Curve Fitting - -```matlab -% Using fit function (Curve Fitting Toolbox or basic forms) -% Linear: y = a*x + b -p = polyfit(x, y, 1); -a = p(1); b = p(2); - -% Exponential: y = a*exp(b*x) -% Linearize: log(y) = log(a) + b*x -p = polyfit(x, log(y), 1); -b = p(1); a = exp(p(2)); - -% Power: y = a*x^b -% Linearize: log(y) = log(a) + b*log(x) -p = polyfit(log(x), log(y), 1); -b = p(1); a = exp(p(2)); - -% General nonlinear fitting with lsqcurvefit -model = @(p, x) p(1)*exp(-p(2)*x); % Example: a*exp(-b*x) -p0 = [1, 1]; % Initial guess -p = lsqcurvefit(model, p0, xdata, ydata); -``` +- [Linear Algebra](https://www.mathworks.com/help/matlab/linear-algebra.html) +- [`mldivide`](https://www.mathworks.com/help/matlab/ref/double.mldivide.html) +- [`eq` floating-point guidance and `isapprox`](https://www.mathworks.com/help/matlab/ref/double.eq.html) +- [`AbsoluteTolerance`](https://www.mathworks.com/help/matlab/ref/matlab.unittest.constraints.absolutetolerance-class.html) +- [`RelativeTolerance`](https://www.mathworks.com/help/matlab/ref/matlab.unittest.constraints.relativetolerance-class.html) +- [`rng`](https://www.mathworks.com/help/matlab/ref/rng.html) +- [`RandStream`](https://www.mathworks.com/help/matlab/ref/randstream.html) +- [ODE Solvers](https://www.mathworks.com/help/matlab/ordinary-differential-equations.html) +- [Optimization](https://www.mathworks.com/help/matlab/optimization.html) +- [MATLAB product list and pricing/licensing](https://www.mathworks.com/pricing-licensing.html) +- [MATLAB R2026a release notes](https://www.mathworks.com/help/matlab/release-notes.html) diff --git a/.agents/skills/matlab/references/matrices-arrays.md b/.agents/skills/matlab/references/matrices-arrays.md index ffddd73..e628751 100644 --- a/.agents/skills/matlab/references/matrices-arrays.md +++ b/.agents/skills/matlab/references/matrices-arrays.md @@ -1,349 +1,228 @@ -# Matrices and Arrays Reference +# Arrays, Indexing, Data Types, and Performance -## Table of Contents -1. [Array Creation](#array-creation) -2. [Indexing and Subscripting](#indexing-and-subscripting) -3. [Array Manipulation](#array-manipulation) -4. [Concatenation and Reshaping](#concatenation-and-reshaping) -5. [Array Information](#array-information) -6. [Sorting and Searching](#sorting-and-searching) +This reference targets MATLAB R2026a. Verify GNU Octave behavior separately. -## Array Creation +## Array model -### Basic Creation +MATLAB is 1-based and column-major. Most numeric literals are `double`. +Orientation and trailing singleton dimensions matter. ```matlab -% Direct specification -A = [1 2 3; 4 5 6; 7 8 9]; % 3x3 matrix (rows separated by ;) -v = [1, 2, 3, 4, 5]; % Row vector -v = [1; 2; 3; 4; 5]; % Column vector - -% Range operators -v = 1:10; % 1 to 10, step 1 -v = 0:0.5:5; % 0 to 5, step 0.5 -v = 10:-1:1; % 10 down to 1 - -% Linearly/logarithmically spaced -v = linspace(0, 1, 100); % 100 points from 0 to 1 -v = logspace(0, 3, 50); % 50 points from 10^0 to 10^3 -``` - -### Special Matrices +row = 1:5; % 1-by-5 +column = (1:5).'; % 5-by-1, nonconjugate transpose +A = reshape(1:12, 3, 4); % values fill down columns -```matlab -% Common patterns -I = eye(n); % n×n identity matrix -I = eye(m, n); % m×n identity matrix -Z = zeros(m, n); % m×n zeros -O = ones(m, n); % m×n ones -D = diag([1 2 3]); % Diagonal matrix from vector -d = diag(A); % Extract diagonal from matrix - -% Random matrices -R = rand(m, n); % Uniform [0,1] -R = randn(m, n); % Normal (mean=0, std=1) -R = randi([a b], m, n); % Random integers in [a,b] -R = randperm(n); % Random permutation of 1:n - -% Logical arrays -T = true(m, n); % All true -F = false(m, n); % All false - -% Grids for 2D/3D -[X, Y] = meshgrid(x, y); % 2D grid from vectors -[X, Y, Z] = meshgrid(x, y, z); % 3D grid -[X, Y] = ndgrid(x, y); % Alternative (different orientation) +sameShape = zeros(size(A), "like", A); +singleData = zeros(100, 1, "single"); +logicalMask = false(size(A)); ``` -### Creating from Existing - -```matlab -A_like = zeros(size(B)); % Same size as B -A_like = ones(size(B), 'like', B); % Same size and type as B -A_copy = A; % Copy (by value, not reference) -``` +Use `.'` for a plain transpose and `'` for a conjugate transpose. Use +`size(A,dim)`, `numel`, and `ndims`; avoid `length` when a specific dimension +is intended. -## Indexing and Subscripting +### Core storage choices -### Basic Indexing +| Type | Use | Caution | +|---|---|---| +| dense numeric/logical array | homogeneous computation | implicit conversion and memory | +| sparse numeric/logical array | low-density 2-D matrices | not every operation preserves sparsity | +| string array | text with missing values | differs from character arrays | +| categorical | finite labels and ordering | undefined category is missing | +| cell array | heterogeneous containers | `{}` versus `()` semantics | +| structure | named heterogeneous fields | structure arrays complicate shape | +| table | named, equal-height variables | `()` versus `{}` versus dot indexing | +| timetable | table with row times | time zone, sorting, duplicates, alignment | +| datetime/duration | time points/elapsed time | time zones and calendar duration differ | -```matlab -% Single element (1-based indexing) -elem = A(2, 3); % Row 2, column 3 -elem = A(5); % Linear index (column-major order) - -% Ranges -row = A(2, :); % Entire row 2 -col = A(:, 3); % Entire column 3 -sub = A(1:2, 2:3); % Rows 1-2, columns 2-3 - -% End keyword -last = A(end, :); % Last row -last3 = A(end-2:end, :); % Last 3 rows -``` +Choose integer classes for storage or exact integer semantics, not as a drop-in +for floating computation. Integer overflow and mixed-class operations need +explicit tests. Preserve units in names or metadata. -### Logical Indexing +## Indexing ```matlab -% Find elements meeting condition -idx = A > 5; % Logical array -elements = A(A > 5); % Extract elements > 5 -A(A < 0) = 0; % Set negative elements to 0 - -% Combine conditions -idx = (A > 0) & (A < 10); % AND -idx = (A < 0) | (A > 10); % OR -idx = ~(A == 0); % NOT -``` - -### Linear Indexing +value = A(2, 3); % row 2, column 3 +linear = A(5); % column-major linear index +row = A(2, :); +lastRows = A(max(1,end-2):end, :); +positive = A(A > 0); +A(A < 0) = 0; -```matlab -% Convert between linear and subscript indices -[row, col] = ind2sub(size(A), linearIdx); -linearIdx = sub2ind(size(A), row, col); - -% Find indices of nonzero/condition -idx = find(A > 5); % Linear indices where A > 5 -idx = find(A > 5, k); % First k indices -idx = find(A > 5, k, 'last'); % Last k indices -[row, col] = find(A > 5); % Subscript indices +[r, c] = ind2sub(size(A), linearIndex); +linearIndex = sub2ind(size(A), r, c); ``` -### Advanced Indexing +Prefer logical indexing for selection and `find` only when numeric indices are +needed. Verify mask shape. Deleting with `A(index)=[]` changes shape and can be +ambiguous for multidimensional arrays. + +### Cell, structure, and table indexing ```matlab -% Index with arrays -rows = [1 3 5]; -cols = [2 4]; -sub = A(rows, cols); % Submatrix +C = {42, "sample"; [1 2], datetime("today")}; +cellContainer = C(1, :); % still a cell array +cellContent = C{1, 1}; % contained value -% Logical indexing with another array -B = A(logical_mask); % Elements where mask is true +S.SampleID = "S01"; +name = S.("SampleID"); -% Assignment with indexing -A(1:2, 1:2) = [10 20; 30 40]; % Assign submatrix -A(:) = 1:numel(A); % Assign all elements (column-major) +tableSlice = T(1:10, ["Time" "Value"]); % table +numericValues = T{:, "Value"}; % underlying content +oneVariable = T.Value; % variable content ``` -## Array Manipulation +Curly extraction from a table succeeds only when selected variable contents +can concatenate. Preserve table form when variable names and metadata matter. -### Element-wise Operations +## Operators and implicit expansion -```matlab -% Arithmetic (element-wise uses . prefix) -C = A + B; % Addition -C = A - B; % Subtraction -C = A .* B; % Element-wise multiplication -C = A ./ B; % Element-wise division -C = A .\ B; % Element-wise left division (B./A) -C = A .^ n; % Element-wise power - -% Comparison (element-wise) -C = A == B; % Equal -C = A ~= B; % Not equal -C = A < B; % Less than -C = A <= B; % Less than or equal -C = A > B; % Greater than -C = A >= B; % Greater than or equal -``` +| Operation | Matrix | Element-wise | +|---|---|---| +| multiply | `A*B` | `A.*B` | +| divide | `A/B`, `A\B` | `A./B`, `A.\B` | +| power | `A^n` | `A.^n` | -### Matrix Operations +Addition, subtraction, comparisons, and many element-wise functions use +compatible-size implicit expansion. Since R2016b, a column and row can create +an outer result: ```matlab -% Matrix arithmetic -C = A * B; % Matrix multiplication -C = A ^ n; % Matrix power -C = A'; % Conjugate transpose -C = A.'; % Transpose (no conjugate) - -% Matrix functions -B = inv(A); % Inverse -B = pinv(A); % Pseudoinverse -d = det(A); % Determinant -t = trace(A); % Trace (sum of diagonal) -r = rank(A); % Rank -n = norm(A); % Matrix/vector norm -n = norm(A, 'fro'); % Frobenius norm - -% Solve linear systems -x = A \ b; % Solve Ax = b -x = b' / A'; % Solve xA = b +x = (1:3).'; +y = 10:10:40; +outerSum = x + y; % 3-by-4 ``` -### Common Functions +Before relying on expansion, assert intended orientation: ```matlab -% Apply to each element -B = abs(A); % Absolute value -B = sqrt(A); % Square root -B = exp(A); % Exponential -B = log(A); % Natural log -B = log10(A); % Log base 10 -B = sin(A); % Sine (radians) -B = sind(A); % Sine (degrees) -B = round(A); % Round to nearest integer -B = floor(A); % Round down -B = ceil(A); % Round up -B = real(A); % Real part -B = imag(A); % Imaginary part -B = conj(A); % Complex conjugate +assert(iscolumn(x)); +assert(isrow(y)); ``` -## Concatenation and Reshaping +Do not use `repmat` solely to emulate supported implicit expansion, but use it +when an explicitly materialized tiled array is actually needed. -### Concatenation +## Missing and nonfinite values -```matlab -% Horizontal (side by side) -C = [A B]; % Concatenate columns -C = [A, B]; % Same as above -C = horzcat(A, B); % Function form -C = cat(2, A, B); % Concatenate along dimension 2 - -% Vertical (stacked) -C = [A; B]; % Concatenate rows -C = vertcat(A, B); % Function form -C = cat(1, A, B); % Concatenate along dimension 1 - -% Block diagonal -C = blkdiag(A, B, C); % Block diagonal matrix -``` +Standard missing values are type-specific: -### Reshaping +- `NaN`: `double`, `single`, `duration`, `calendarDuration` +- `NaT`: `datetime` +- ``: `string` +- ``: `categorical` +- `''` inside a cell array of character vectors -```matlab -% Reshape -B = reshape(A, m, n); % Reshape to m×n (same total elements) -B = reshape(A, [], n); % Auto-compute rows -v = A(:); % Flatten to column vector - -% Transpose and permute -B = A'; % Transpose 2D -B = permute(A, [2 1 3]); % Permute dimensions -B = ipermute(A, [2 1 3]); % Inverse permute - -% Remove/add dimensions -B = squeeze(A); % Remove singleton dimensions -B = shiftdim(A, n); % Shift dimensions - -% Replication -B = repmat(A, m, n); % Tile m×n times -B = repelem(A, m, n); % Repeat elements -``` - -### Flipping and Rotating +Integer and logical arrays have no standard missing value. A sentinel such as +`-99` is a data contract, not a MATLAB default. ```matlab -B = flip(A); % Flip along first non-singleton dimension -B = flip(A, dim); % Flip along dimension dim -B = fliplr(A); % Flip left-right (columns) -B = flipud(A); % Flip up-down (rows) -B = rot90(A); % Rotate 90° counterclockwise -B = rot90(A, k); % Rotate k×90° -B = circshift(A, k); % Circular shift +missingMask = ismissing(T); +anyMissing = anymissing(T); +clean = rmmissing(T); +filled = fillmissing(T, "linear", DataVariables="Value"); ``` -## Array Information +Define whether `Inf` is valid separately; it is not a standard missing +floating-point value. `isfinite` distinguishes finite values. `ismissing` +ignores timetable row times, so validate row times explicitly. + +## Tables and timetables -### Size and Dimensions +Every table variable has the same row count but can have a different type and +width. Timetables add row times. ```matlab -[m, n] = size(A); % Rows and columns -m = size(A, 1); % Number of rows -n = size(A, 2); % Number of columns -sz = size(A); % Size vector -len = length(A); % Largest dimension -num = numel(A); % Total number of elements -ndim = ndims(A); % Number of dimensions +T = table(sampleID, group, value, ... + VariableNames=["SampleID" "Group" "Value"]); + +TT = timetable(time, value, quality, ... + VariableNames=["Value" "Quality"]); +TT = sortrows(TT); +hourly = retime(TT, "hourly", "mean"); +aligned = synchronize(TT1, TT2, "intersection"); ``` -### Type Checking +Before time alignment: -```matlab -tf = isempty(A); % Is empty? -tf = isscalar(A); % Is scalar (1×1)? -tf = isvector(A); % Is vector (1×n or n×1)? -tf = isrow(A); % Is row vector? -tf = iscolumn(A); % Is column vector? -tf = ismatrix(A); % Is 2D matrix? -tf = isnumeric(A); % Is numeric? -tf = isreal(A); % Is real (no imaginary)? -tf = islogical(A); % Is logical? -tf = isnan(A); % Which elements are NaN? -tf = isinf(A); % Which elements are Inf? -tf = isfinite(A); % Which elements are finite? -``` +1. normalize or record time zones; +2. define duplicate-time policy; +3. sort row times; +4. choose union/intersection and interpolation/aggregation deliberately; +5. record daylight-saving and calendar assumptions. -### Comparison +Direct calculations on tables/timetables are supported for compatible +variables, but mixed nonnumeric variables can invalidate an operation. +Selecting numeric variables first is often clearer: ```matlab -tf = isequal(A, B); % Are arrays equal? -tf = isequaln(A, B); % Equal, treating NaN as equal? -tf = all(A); % All nonzero/true? -tf = any(A); % Any nonzero/true? -tf = all(A, dim); % All along dimension -tf = any(A, dim); % Any along dimension +numericT = T(:, vartype("numeric")); ``` -## Sorting and Searching - -### Sorting +## Concatenation and reshaping ```matlab -B = sort(A); % Sort columns ascending -B = sort(A, 'descend'); % Sort descending -B = sort(A, dim); % Sort along dimension -[B, idx] = sort(A); % Also return original indices -B = sortrows(A); % Sort rows by first column -B = sortrows(A, col); % Sort by specific column(s) -B = sortrows(A, col, 'descend'); +wide = [A B]; +tall = [A; B]; +flat = A(:); +B = reshape(A, [], 4); +C = permute(X, [2 1 3]); ``` -### Unique and Set Operations +Concatenated dimensions and classes must be compatible. `squeeze` can remove +different dimensions depending on input shape; avoid it in APIs whose output +rank must be stable. -```matlab -B = unique(A); % Unique elements -[B, ia, ic] = unique(A); % With index information -B = unique(A, 'rows'); % Unique rows - -% Set operations -C = union(A, B); % Union -C = intersect(A, B); % Intersection -C = setdiff(A, B); % A - B (in A but not B) -C = setxor(A, B); % Symmetric difference -tf = ismember(A, B); % Is each element of A in B? -``` +## Performance without folklore -### Min/Max +1. Write the clearest correct array code. +2. Use representative data and `timeit`; use the profiler for call-level + diagnosis. +3. Preallocate when a loop's output shape is known. +4. Vectorize operations that map naturally to array kernels. +5. Keep a loop when vectorization creates large temporaries or obscures logic. +6. Preserve sparsity and data class where appropriate. +7. Benchmark each supported release/platform; R2026a includes implementation + speedups that can change old trade-offs. ```matlab -m = min(A); % Column minimums -m = min(A, [], 'all'); % Global minimum -[m, idx] = min(A); % With indices -m = min(A, B); % Element-wise minimum +y = zeros(size(x), "like", x); +for k = 1:numel(x) + y(k) = localTransform(x(k)); +end +``` -M = max(A); % Column maximums -M = max(A, [], 'all'); % Global maximum -[M, idx] = max(A); % With indices +Avoid growing arrays in a loop. However, do not preallocate the wrong class or +shape. `zeros(size(x),"like",x)` is usually safer than an unqualified `zeros`. -[minVal, minIdx] = min(A(:)); % Global min with linear index -[maxVal, maxIdx] = max(A(:)); % Global max with linear index +Parallel arrays, GPU arrays, tall arrays, `parfor`, and distributed arrays +require specific products and supported functions. They also change ordering, +reduction, RNG, and tolerance concerns. Do not suggest them merely because a +loop exists. -% k smallest/largest -B = mink(A, k); % k smallest elements -B = maxk(A, k); % k largest elements -``` +## Numerical review checklist -### Sum and Product +- [ ] Shapes and orientation are asserted where expansion matters. +- [ ] Matrix versus element-wise operators are intentional. +- [ ] Conjugation behavior is intentional. +- [ ] Indexing preserves expected rank and container type. +- [ ] Missing, nonfinite, and sentinel policies are explicit. +- [ ] Table variable names/types and timetable time zones are preserved. +- [ ] Integer overflow and mixed-class conversion are tested. +- [ ] Sparse inputs remain sparse where required. +- [ ] Preallocation and vectorization are measured, not assumed. +- [ ] Memory estimates include temporaries and expanded outputs. -```matlab -s = sum(A); % Column sums -s = sum(A, 'all'); % Total sum -s = sum(A, dim); % Sum along dimension -s = cumsum(A); % Cumulative sum - -p = prod(A); % Column products -p = prod(A, 'all'); % Total product -p = cumprod(A); % Cumulative product -``` +## Sources (verified 2026-07-23) + +- [Array Indexing](https://www.mathworks.com/help/matlab/math/array-indexing.html) +- [Compatible Array Sizes for Basic Operations](https://www.mathworks.com/help/matlab/matlab_prog/compatible-array-sizes-for-basic-operations.html) +- [MATLAB Data Types](https://www.mathworks.com/help/matlab/data-types.html) +- [Tables](https://www.mathworks.com/help/matlab/tables.html) +- [Timetables](https://www.mathworks.com/help/matlab/timetables.html) +- [`ismissing`](https://www.mathworks.com/help/matlab/ref/ismissing.html) +- [Missing Data in MATLAB](https://www.mathworks.com/help/matlab/data_analysis/missing-data-in-matlab.html) +- [Vectorization](https://www.mathworks.com/help/matlab/matlab_prog/vectorization.html) +- [Preallocation](https://www.mathworks.com/help/matlab/matlab_prog/preallocating-arrays.html) +- [`timeit`](https://www.mathworks.com/help/matlab/ref/timeit.html) +- [Profile MATLAB Code](https://www.mathworks.com/help/matlab/matlab_prog/profiling-for-improving-performance.html) diff --git a/.agents/skills/matlab/references/octave-compatibility.md b/.agents/skills/matlab/references/octave-compatibility.md index 2206529..af54c6b 100644 --- a/.agents/skills/matlab/references/octave-compatibility.md +++ b/.agents/skills/matlab/references/octave-compatibility.md @@ -1,544 +1,212 @@ -# GNU Octave Compatibility Reference +# GNU Octave 11.3.0 Compatibility -## Table of Contents -1. [Overview](#overview) -2. [Syntax Differences](#syntax-differences) -3. [Operator Differences](#operator-differences) -4. [Function Differences](#function-differences) -5. [Features Unique to Octave](#features-unique-to-octave) -6. [Features Missing in Octave](#features-missing-in-octave) -7. [Writing Compatible Code](#writing-compatible-code) -8. [Octave Packages](#octave-packages) +GNU Octave 11.3.0 is the current stable release as of 2026-07-23 (released +2026-06-01). It is free software under GPLv3+. MATLAB R2026a is proprietary. +Do not describe Octave as MATLAB, a licensed toolbox substitute, or a +drop-in guarantee. -## Overview +## Compatibility policy -GNU Octave is a free, open-source alternative to MATLAB with high compatibility. Most MATLAB scripts run in Octave with no or minimal modifications. However, there are some differences to be aware of. +Use three labels: -### Installation +- **portable subset**: tested in both exact target versions; +- **MATLAB-only**: depends on MATLAB syntax, objects, projects, products, or + deployment; +- **Octave-only**: uses Octave syntax, packages, BIST, or runtime behavior. -```bash -# macOS (Homebrew) -brew install octave +Source resemblance is not enough. Compare outputs with scientific tolerances, +edge cases, warnings, graphics, performance, and file round trips. -# Ubuntu/Debian -sudo apt install octave +## Current release changes -# Fedora -sudo dnf install octave +Octave 11 introduced improved `classdef` support, broadcasting for sparse, +diagonal, and permutation matrices, more MATLAB-compatible `nanflag`/`vecdim` +behavior, and many function/performance changes. It is still not fully +compatible. NEWS-11 also records behavior changes that can break older Octave +code, including stricter accepted types for several statistics functions. -# Windows -# Download installer from https://octave.org/download -``` - -### Running Octave - -```bash -# Interactive mode -octave - -# Run script -octave script.m -octave --eval "disp('Hello')" - -# GUI mode -octave --gui - -# Command-line only (no graphics) -octave --no-gui -octave-cli -``` - -## Syntax Differences - -### Comments - -```matlab -% MATLAB style (works in both) -% This is a comment - -# Octave style (Octave only) -# This is also a comment in Octave - -% For compatibility, always use % -``` - -### String Quotes - -```matlab -% MATLAB: Single quotes only (char arrays) -str = 'Hello'; % char array -str = "Hello"; % string (R2017a+) - -% Octave: Both work, but different behavior -str1 = 'Hello'; % char array, no escape sequences -str2 = "Hello\n"; % Interprets \n as newline - -% For compatibility, use single quotes for char arrays -% Avoid double quotes with escape sequences -``` - -### Line Continuation - -```matlab -% MATLAB style (works in both) -x = 1 + 2 + 3 + ... - 4 + 5; - -% Octave also accepts backslash -x = 1 + 2 + 3 + \ - 4 + 5; - -% For compatibility, use ... -``` - -### Block Terminators - -```matlab -% MATLAB style (works in both) -if condition - % code -end - -for i = 1:10 - % code -end +Read both the current manual and NEWS before migrating to 11.3.0. The online +`latest` manual reviewed on this date identifies its generated content as +11.1.0 while the project download/news page identifies 11.3.0 as current; +consult NEWS-11 for the maintenance-release delta. -% Octave also accepts specific terminators -if condition - # code -endif +## Command-line differences -for i = 1:10 - # code -endfor +Reviewed Octave argv usually includes: -while condition - # code -endwhile - -% For compatibility, always use 'end' +```text +["octave", "--no-init-all", "--no-history", "--quiet", "--no-gui", "main.m"] ``` -### Function Definitions - -```matlab -% MATLAB requires function in file with same name -% Octave allows command-line function definitions +The bundled planner returns argv but never executes it. -% Octave command-line function -function y = f(x) - y = x^2; -endfunction +Current manual behavior: -% For compatibility, define functions in .m files -``` +- `--eval`/`-e` evaluates code and exits unless `--persist` is set; +- a filename executes and exits; +- `--no-gui` selects CLI; +- `--no-window-system` disables graphics/window-system use; +- `--no-init-all`/`--norc` skips system and user initialization; +- `--path` adds a function path; +- `--quiet` suppresses greeting; `--no-history` avoids history writes. -## Operator Differences +Octave startup can execute site/version files, user configuration, +project-local `.octaverc`, and MATLAB-compatible `startup.m`. Skipping all +startup files can improve reproducibility but can also remove expected package +or path setup. Review the plan. -### Increment/Decrement Operators +MATLAB uses `-batch`, has different startup processing, and has no Octave +`--no-init-all` option. Never pass the same flags to both. -```matlab -% Octave has C-style operators (MATLAB does not) -x++; % x = x + 1 -x--; % x = x - 1 -++x; % Pre-increment ---x; % Pre-decrement - -% For compatibility, use explicit assignment -x = x + 1; -x = x - 1; -``` +## Syntax: portable versus Octave-only -### Compound Assignment +Prefer: -```matlab -% Octave supports (MATLAB does not) -x += 5; % x = x + 5 -x -= 3; % x = x - 3 -x *= 2; % x = x * 2 -x /= 4; % x = x / 4 -x ^= 2; % x = x ^ 2 - -% Element-wise versions -x .+= y; -x .-= y; -x .*= y; -x ./= y; -x .^= y; - -% For compatibility, use explicit assignment -x = x + 5; -x = x .* y; -``` +- `%` comments; +- `...` continuation; +- `end` block terminators; +- explicit `x = x + 1`; +- intermediate variables before indexing a function result; +- short-circuit `&&`/`||` for scalar conditions; +- ordinary `.m` functions with matching filenames. -### Logical Operators +Avoid these Octave-only extensions in portable code: -```matlab -% Both support -& | ~ && || - -% Short-circuit behavior difference: -% MATLAB: & and | short-circuit in if/while conditions -% Octave: Only && and || short-circuit - -% For predictable behavior, use: -% && || for scalar short-circuit logic -% & | for element-wise operations -``` - -### Indexing After Expression - -```matlab -% Octave allows indexing immediately after expression -result = sin(x)(1:10); % First 10 elements of sin(x) -value = func(arg).field; % Access field of returned struct +- `#` comments; +- `endif`, `endfor`, `endfunction`; +- `++`, `--`, `+=`, and related compound assignment; +- `do ... until`; +- indexing directly into an expression result; +- backslash line continuation; +- Octave package/BIST directives in MATLAB production files unless isolated. -% MATLAB requires intermediate variable -temp = sin(x); -result = temp(1:10); +Both support implicit expansion/broadcasting in many modern cases, but special +classes and edge cases differ. Octave 11 expanded broadcasting for special +matrix types. Test orientation, sparse output, empty dimensions, and mixed +classes in both. -temp = func(arg); -value = temp.field; +## MATLAB features with no assumed Octave equivalent -% For compatibility, use intermediate variables -``` +The current Octave manual does not establish drop-in equivalents for: -## Function Differences +- MATLAB `arguments` blocks and all validators/name-value semantics; +- `table`/`timetable` workflows and their R2026a JSON conversions; +- MATLAB Projects, dependency analyzer, Project Upgrade, or project-aware + `runtests`; +- `matlab.unittest`, MATLAB Test, Code Quality Dashboard; +- live scripts `.mlx`, App Designer `.mlapp`, `.fig` object compatibility; +- Simulink and MathWorks toolbox APIs; +- MATLAB Engine API for Python, MATLAB Compiler/Runtime, MATLAB Coder, or + MathWorks deployment products; +- full MATLAB `classdef`, events/listeners, serialization, and metaclass + behavior. -### Built-in Functions +Use feature detection and separate adapters only after testing. Do not silently +replace a missing MATLAB toolbox function with a similarly named Octave +package function. -Most basic functions are compatible. Some differences: +## Packages versus toolboxes -```matlab -% Function name differences -% MATLAB Octave Alternative -% ------ ------------------ -% inputname (not available) -% inputParser (partial support) -% validateattributes (partial support) - -% Behavior differences in edge cases -% Check documentation for specific functions -``` +Octave packages are distributed separately from core Octave. MATLAB toolboxes +are separately licensed MathWorks products. APIs, algorithms, defaults, +validation, object models, and release schedules differ. -### Random Number Generation +Never run `pkg install`, load an untrusted package, or alter `.octaverc` +automatically. Package installation can fetch/build/execute code. Record exact +package name/version/source/checksum and obtain approval. -```matlab -% Both use Mersenne Twister by default -% Seed setting is similar -rng(42); % MATLAB -rand('seed', 42); % Octave (also accepts rng syntax) +`pkg load` mutates the function path and can shadow core or project functions. +Test `which`/resolution only in a trusted approved runtime. -% For compatibility -rng(42); % Works in modern Octave -``` +## Tests -### Graphics +Octave's built-in self-test system scans `%!` blocks and uses `test`. It is not +`matlab.unittest`. ```matlab -% Basic plotting is compatible -plot(x, y); -xlabel('X'); ylabel('Y'); -title('Title'); -legend('Data'); - -% Some advanced features differ -% - Octave uses gnuplot or Qt graphics -% - Some property names may differ -% - Animation/GUI features vary - -% Test graphics code in both environments +%!test +%! observed = hypot(3, 4); +%! assert(observed, 5, 1e-12); ``` -### File I/O +Portable production functions and runtime-specific test harnesses should be +separate when test syntax differs. The nonexecuting planner can prepare an +Octave BIST argv, but an approved runtime is required to run it. -```matlab -% Basic I/O is compatible -save('file.mat', 'x', 'y'); -load('file.mat'); -dlmread('file.txt'); -dlmwrite('file.txt', data); +## MAT and HDF5 compatibility -% MAT-file versions -save('file.mat', '-v7'); % Compatible format -save('file.mat', '-v7.3'); % HDF5 format (partial Octave support) +Octave 11 supports writing MATLAB v4, v6, and v7 binary formats. It **does not +implement saving MATLAB v7.3**. -% For compatibility, use -v7 or -v6 -``` +Octave can save its own HDF5 representation when built with HDF5. Its `load +-hdf5` has limited ability to read MATLAB v7.3, mainly for supported numeric +content; many types are unsupported. An Octave HDF5 file is not automatically +a MATLAB v7.3 file. + +Octave's current manual states that `classdef` objects are saved as structures +in supporting formats and are not restored as `classdef` objects. This differs +substantially from MATLAB object serialization. + +Never load untrusted MAT/HDF5 files in either runtime. Use the bundled bounded +technical inventory first, then escalate object/opaque/function/external-link +content. + +For portable simple data, use MATLAB v7 only after testing classes, shapes, +text, sparse/complex values, and metadata. Prefer schema-documented +language-neutral formats where feasible. -## Features Unique to Octave +## Graphics -### do-until Loop +Basic calls such as `plot`, labels, legends, images, surfaces, and `print` are +similar, but renderers, fonts, properties, layout, transparency, callbacks, +and export formats differ. + +Do not assume Octave implements R2026a `exportgraphics`, SVG/HTML web canvas, +`tiledlayout`, UI objects, or property behavior. Build a small compatibility +test and compare exported dimensions, font embedding, vector/raster content, +colors, and clipping. + +Octave 11 NEWS documents graphics compatibility changes such as colorbar and +event-field behavior. Review NEWS for each update. -```matlab -% Octave only -do - x = x + 1; -until (x > 10) - -% Equivalent MATLAB/compatible code -x = x + 1; -while x <= 10 - x = x + 1; -end -``` +## Numerical differences + +Even when both runtimes call similarly named LAPACK/BLAS-backed functions, +results can differ because of: + +- linked libraries, versions, threads, and architecture; +- solver implementation/default/tolerance changes; +- sparse ordering and pivot choices; +- random generator algorithms and streams; +- toolbox/package algorithms; +- floating reduction order; +- unsupported or converted classes. -### unwind_protect +Compare residuals, invariants, objective/feasibility, and domain observables. +Do not demand identical eigenvector signs, cluster bases, or bitwise +floating-point output without a justified contract. -```matlab -% Octave only - guaranteed cleanup -unwind_protect - % code that might error - result = risky_operation(); -unwind_protect_cleanup - % always executed (like finally) - cleanup(); -end_unwind_protect - -% MATLAB equivalent -try - result = risky_operation(); -catch -end -cleanup(); % Not guaranteed if error not caught -``` +## Portability checklist -### Built-in Documentation +- [ ] Exact MATLAB and Octave versions recorded. +- [ ] Core versus toolbox/package requirements separated. +- [ ] Only portable syntax used in shared source. +- [ ] Shapes, missing values, strings, and implicit expansion tested. +- [ ] RNG algorithm/seed behavior tested separately. +- [ ] Numerical tolerances justified. +- [ ] MAT/HDF5 round trips cover every used class. +- [ ] Graphics compared from exported files. +- [ ] Runtime-specific tests and deployment kept separate. +- [ ] No package installation or code execution occurred implicitly. -```matlab -% Octave supports Texinfo documentation in functions -function y = myfunction(x) - %% -*- texinfo -*- - %% @deftypefn {Function File} {@var{y} =} myfunction (@var{x}) - %% Description of myfunction. - %% @end deftypefn - y = x.^2; -endfunction -``` - -### Package System +## Sources (verified 2026-07-23) -```matlab -% Octave Forge packages -pkg install -forge control -pkg load control - -% List installed packages -pkg list - -% For MATLAB compatibility, use equivalent toolboxes -% or include package functionality directly -``` - -## Features Missing in Octave - -### Simulink - -```matlab -% No Octave equivalent -% Simulink models (.slx, .mdl) cannot run in Octave -``` - -### MATLAB Toolboxes - -```matlab -% Many toolbox functions not available -% Some have Octave Forge equivalents: - -% MATLAB Toolbox Octave Forge Package -% --------------- -------------------- -% Control System control -% Signal Processing signal -% Image Processing image -% Statistics statistics -% Optimization optim - -% Check pkg list for available packages -``` - -### App Designer / GUIDE - -```matlab -% MATLAB GUI tools not available in Octave -% Octave has basic UI functions: -uicontrol, uimenu, figure properties - -% For cross-platform GUIs, consider: -% - Web-based interfaces -% - Qt (via Octave's Qt graphics) -``` - -### Object-Oriented Programming - -```matlab -% Octave has partial classdef support -% Some features missing or behave differently: -% - Handle class events -% - Property validation -% - Some access modifiers - -% For compatibility, use simpler OOP patterns -% or struct-based approaches -``` - -### Live Scripts - -```matlab -% .mlx files are MATLAB-only -% Use regular .m scripts for compatibility -``` - -## Writing Compatible Code - -### Detection - -```matlab -function tf = isOctave() - tf = exist('OCTAVE_VERSION', 'builtin') ~= 0; -end - -% Use for conditional code -if isOctave() - % Octave-specific code -else - % MATLAB-specific code -end -``` - -### Best Practices - -```matlab -% 1. Use % for comments, not # -% Good -% This is a comment - -% Avoid -# This is a comment (Octave only) - -% 2. Use ... for line continuation -% Good -x = 1 + 2 + 3 + ... - 4 + 5; - -% Avoid -x = 1 + 2 + 3 + \ - 4 + 5; - -% 3. Use 'end' for all blocks -% Good -if condition - code -end - -% Avoid -if condition - code -endif - -% 4. Avoid compound operators -% Good -x = x + 1; - -% Avoid -x++; -x += 1; - -% 5. Use single quotes for strings -% Good -str = 'Hello World'; - -% Avoid (escape sequence issues) -str = "Hello\nWorld"; - -% 6. Use intermediate variables for indexing -% Good -temp = func(arg); -result = temp(1:10); - -% Avoid (Octave only) -result = func(arg)(1:10); - -% 7. Save MAT-files in compatible format -save('data.mat', 'x', 'y', '-v7'); -``` - -### Testing Compatibility - -```bash -# Test in both environments -matlab -nodisplay -nosplash -r "run('test_script.m'); exit;" -octave --no-gui test_script.m - -# Create test script -# test_script.m: -# try -# main_function(); -# disp('Test passed'); -# catch ME -# disp(['Test failed: ' ME.message]); -# end -``` - -## Octave Packages - -### Installing Packages - -```matlab -% Install from Octave Forge -pkg install -forge package_name - -% Install from file -pkg install package_file.tar.gz - -% Install from URL -pkg install 'http://example.com/package.tar.gz' - -% Uninstall -pkg uninstall package_name -``` - -### Using Packages - -```matlab -% Load package (required before use) -pkg load control -pkg load signal -pkg load image - -% Load at startup (add to .octaverc) -pkg load control - -% List loaded packages -pkg list - -% Unload package -pkg unload control -``` - -### Common Packages - -| Package | Description | -|---------|-------------| -| control | Control systems design | -| signal | Signal processing | -| image | Image processing | -| statistics | Statistical functions | -| optim | Optimization algorithms | -| io | Input/output functions | -| struct | Structure manipulation | -| symbolic | Symbolic math (via SymPy) | -| parallel | Parallel computing | -| netcdf | NetCDF file support | - -### Package Management - -```matlab -% Update all packages -pkg update - -% Get package description -pkg describe package_name - -% Check for updates -pkg list % Compare with Octave Forge website -``` +- [GNU Octave home/current release](https://octave.org/) +- [GNU Octave 11 release notes](https://octave.org/NEWS-11.html) +- [GNU Octave current manual](https://docs.octave.org/latest/) +- [Command-Line Options](https://docs.octave.org/latest/Command-Line-Options.html) +- [Startup Files](https://docs.octave.org/latest/Startup-Files.html) +- [Simple File I/O and MAT v7.3 limitation](https://docs.octave.org/latest/Simple-File-I_002fO.html) +- [`classdef` compatibility status](https://docs.octave.org/latest/classdef-Classes.html) +- [Test Functions](https://docs.octave.org/latest/Test-Functions.html) +- [GNU GPL](https://octave.org/license.html) diff --git a/.agents/skills/matlab/references/programming.md b/.agents/skills/matlab/references/programming.md index 94ff26c..827f80d 100644 --- a/.agents/skills/matlab/references/programming.md +++ b/.agents/skills/matlab/references/programming.md @@ -1,672 +1,225 @@ -# Programming Reference - -## Table of Contents -1. [Scripts and Functions](#scripts-and-functions) -2. [Control Flow](#control-flow) -3. [Function Types](#function-types) -4. [Error Handling](#error-handling) -5. [Performance and Debugging](#performance-and-debugging) -6. [Object-Oriented Programming](#object-oriented-programming) - -## Scripts and Functions - -### Scripts - -```matlab -% Scripts are .m files with MATLAB commands -% They run in the base workspace (share variables) - -% Example: myscript.m -% This is a comment -x = 1:10; -y = x.^2; -plot(x, y); -title('My Plot'); - -% Run script -myscript; % Or: run('myscript.m') -``` - -### Functions - -```matlab -% Functions have their own workspace -% Save in file with same name as function - -% Example: myfunction.m -function y = myfunction(x) -%MYFUNCTION Brief description of function -% Y = MYFUNCTION(X) detailed description -% -% Example: -% y = myfunction(5); -% -% See also OTHERFUNCTION - y = x.^2; -end - -% Multiple outputs -function [result1, result2] = multioutput(x) - result1 = x.^2; - result2 = x.^3; -end - -% Variable arguments -function varargout = flexfun(varargin) - % varargin is cell array of inputs - % varargout is cell array of outputs - n = nargin; % Number of inputs - m = nargout; % Number of outputs -end -``` - -### Input Validation - -```matlab -function result = validatedinput(x, options) - arguments - x (1,:) double {mustBePositive} - options.Normalize (1,1) logical = false - options.Scale (1,1) double {mustBePositive} = 1 - end - - result = x * options.Scale; - if options.Normalize - result = result / max(result); - end -end - -% Usage -y = validatedinput([1 2 3], 'Normalize', true, 'Scale', 2); - -% Common validators -% mustBePositive, mustBeNegative, mustBeNonzero -% mustBeInteger, mustBeNumeric, mustBeFinite -% mustBeNonNaN, mustBeReal, mustBeNonempty -% mustBeMember, mustBeInRange, mustBeGreaterThan -``` - -### Local Functions - -```matlab -% Local functions appear after main function -% Only accessible within the same file - -function result = mainfunction(x) - intermediate = helper1(x); - result = helper2(intermediate); -end - -function y = helper1(x) - y = x.^2; -end - -function y = helper2(x) - y = sqrt(x); -end -``` - -## Control Flow - -### Conditional Statements - -```matlab -% if-elseif-else -if condition1 - % statements -elseif condition2 - % statements -else - % statements -end - -% Logical operators -% & - AND (element-wise) -% | - OR (element-wise) -% ~ - NOT -% && - AND (short-circuit, scalars) -% || - OR (short-circuit, scalars) -% == - Equal -% ~= - Not equal -% <, <=, >, >= - Comparisons - -% Example -if x > 0 && y > 0 - quadrant = 1; -elseif x < 0 && y > 0 - quadrant = 2; -elseif x < 0 && y < 0 - quadrant = 3; -else - quadrant = 4; -end -``` - -### Switch Statements - -```matlab -switch expression - case value1 - % statements - case {value2, value3} % Multiple values - % statements - otherwise - % default statements -end - -% Example -switch dayOfWeek - case {'Saturday', 'Sunday'} - dayType = 'Weekend'; - case {'Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday'} - dayType = 'Weekday'; - otherwise - dayType = 'Unknown'; -end -``` - -### For Loops - -```matlab -% Basic for loop -for i = 1:10 - % statements using i -end - -% Custom step -for i = 10:-1:1 - % count down -end - -% Loop over vector -for val = [1 3 5 7 9] - % val takes each value -end - -% Loop over columns of matrix -for col = A - % col is a column vector -end - -% Loop over cell array -for i = 1:length(C) - item = C{i}; -end -``` - -### While Loops - -```matlab -% Basic while loop -while condition - % statements - % Update condition -end - -% Example -count = 0; -while count < 10 - count = count + 1; - % Do something -end -``` - -### Loop Control - -```matlab -% Break - exit loop immediately -for i = 1:100 - if someCondition - break; - end -end - -% Continue - skip to next iteration -for i = 1:100 - if skipCondition - continue; - end - % Process i -end - -% Return - exit function -function y = myfunction(x) - if x < 0 - y = NaN; - return; - end - y = sqrt(x); -end -``` - -## Function Types - -### Anonymous Functions - -```matlab -% Create inline function -f = @(x) x.^2 + 2*x + 1; -g = @(x, y) x.^2 + y.^2; - -% Use -y = f(5); % 36 -z = g(3, 4); % 25 - -% With captured variables -a = 2; -h = @(x) a * x; % Captures current value of a -y = h(5); % 10 -a = 3; % Changing a doesn't affect h -y = h(5); % Still 10 - -% No arguments -now_fn = @() datestr(now); -timestamp = now_fn(); - -% Pass to other functions -result = integral(f, 0, 1); -``` - -### Nested Functions - -```matlab -function result = outerfunction(x) - y = x.^2; % Shared with nested functions - - function z = nestedfunction(a) - z = y + a; % Can access y from outer scope - end - - result = nestedfunction(10); -end -``` - -### Function Handles - -```matlab -% Create handle to existing function -h = @sin; -y = h(pi/2); % 1 - -% From string -h = str2func('cos'); - -% Get function name -name = func2str(h); - -% Get handles to local functions -handles = localfunctions; - -% Function info -info = functions(h); -``` - -### Callbacks - -```matlab -% Using function handles as callbacks - -% Timer example -t = timer('TimerFcn', @myCallback, 'Period', 1); -start(t); - -function myCallback(~, ~) - disp(['Time: ' datestr(now)]); -end - -% With anonymous function -t = timer('TimerFcn', @(~,~) disp('Tick'), 'Period', 1); - -% GUI callbacks -uicontrol('Style', 'pushbutton', 'Callback', @buttonPressed); -``` - -## Error Handling - -### Try-Catch - -```matlab -try - % Code that might error - result = riskyOperation(); -catch ME - % Handle error - disp(['Error: ' ME.message]); - disp(['Identifier: ' ME.identifier]); - - % Optionally rethrow - rethrow(ME); -end - -% Catch specific errors -try - result = operation(); -catch ME - switch ME.identifier - case 'MATLAB:divideByZero' - result = Inf; - case 'MATLAB:nomem' - rethrow(ME); - otherwise - result = NaN; - end -end -``` - -### Throwing Errors - -```matlab -% Simple error -error('Something went wrong'); - -% With identifier -error('MyPkg:InvalidInput', 'Input must be positive'); - -% With formatting -error('MyPkg:OutOfRange', 'Value %f is out of range [%f, %f]', val, lo, hi); - -% Create and throw exception -ME = MException('MyPkg:Error', 'Error message'); -throw(ME); - -% Assertion -assert(condition, 'Message if false'); -assert(x > 0, 'MyPkg:NotPositive', 'x must be positive'); -``` - -### Warnings - -```matlab -% Issue warning -warning('This might be a problem'); -warning('MyPkg:Warning', 'Warning message'); - -% Control warnings -warning('off', 'MyPkg:Warning'); % Disable specific warning -warning('on', 'MyPkg:Warning'); % Enable -warning('off', 'all'); % Disable all -warning('on', 'all'); % Enable all - -% Query warning state -s = warning('query', 'MyPkg:Warning'); - -% Temporarily disable -origState = warning('off', 'MATLAB:nearlySingularMatrix'); -% ... code ... -warning(origState); -``` - -## Performance and Debugging - -### Timing - -```matlab -% Simple timing -tic; -% ... code ... -elapsed = toc; - -% Multiple timers -t1 = tic; -% ... code ... -elapsed1 = toc(t1); - -% CPU time -t = cputime; -% ... code ... -cpuElapsed = cputime - t; - -% Profiler -profile on; -myfunction(); -profile viewer; % GUI to analyze results -p = profile('info'); % Get programmatic results -profile off; -``` - -### Memory - -```matlab -% Memory info -[user, sys] = memory; % Windows only -whos; % Variable sizes - -% Clear variables -clear x y z; -clear all; % All variables (use sparingly) -clearvars -except x y; % Keep specific variables -``` - -### Debugging - -```matlab -% Set breakpoints (in editor or programmatically) -dbstop in myfunction at 10 -dbstop if error -dbstop if warning -dbstop if naninf % Stop on NaN or Inf - -% Step through code -dbstep % Next line -dbstep in % Step into function -dbstep out % Step out of function -dbcont % Continue execution -dbquit % Quit debugging - -% Clear breakpoints -dbclear all - -% Examine state -dbstack % Call stack -whos % Variables -``` - -### Vectorization Tips - -```matlab -% AVOID loops when possible -% Slow: -for i = 1:n - y(i) = x(i)^2; -end - -% Fast: -y = x.^2; - -% Element-wise operations (use . prefix) -y = a .* b; % Element-wise multiply -y = a ./ b; % Element-wise divide -y = a .^ b; % Element-wise power - -% Built-in functions operate on arrays -y = sin(x); % Apply to all elements -s = sum(x); % Sum all -m = max(x); % Maximum - -% Logical indexing instead of find -% Slow: -idx = find(x > 0); -y = x(idx); - -% Fast: -y = x(x > 0); - -% Preallocate arrays -% Slow: -y = []; -for i = 1:n - y(i) = compute(i); -end - -% Fast: -y = zeros(1, n); -for i = 1:n - y(i) = compute(i); -end -``` - -### Parallel Computing - -```matlab -% Parallel for loop -parfor i = 1:n - results(i) = compute(i); -end - -% Note: parfor has restrictions -% - Iterations must be independent -% - Variable classifications (sliced, broadcast, etc.) - -% Start parallel pool -pool = parpool; % Default cluster -pool = parpool(4); % 4 workers - -% Delete pool -delete(gcp('nocreate')); - -% Parallel array operations -spmd - % Each worker executes this block - localData = myData(labindex); - result = process(localData); -end -``` - -## Object-Oriented Programming - -### Class Definition - -```matlab -% In file MyClass.m -classdef MyClass - properties - PublicProp - end - - properties (Access = private) - PrivateProp - end - - properties (Constant) - ConstProp = 42 - end - - methods - % Constructor - function obj = MyClass(value) - obj.PublicProp = value; - end - - % Instance method - function result = compute(obj, x) - result = obj.PublicProp * x; - end - end - - methods (Static) - function result = staticMethod(x) - result = x.^2; - end - end -end -``` - -### Using Classes - -```matlab -% Create object -obj = MyClass(10); - -% Access properties -val = obj.PublicProp; -obj.PublicProp = 20; - -% Call methods -result = obj.compute(5); -result = compute(obj, 5); % Equivalent - -% Static method -result = MyClass.staticMethod(3); - -% Constant property -val = MyClass.ConstProp; -``` - -### Inheritance - -```matlab -classdef DerivedClass < BaseClass - properties - ExtraProp - end - - methods - function obj = DerivedClass(baseVal, extraVal) - % Call superclass constructor - obj@BaseClass(baseVal); - obj.ExtraProp = extraVal; +# Programming, Projects, Analysis, and Tests + +This reference targets MATLAB R2026a. Base MATLAB, separately licensed +products, and GNU Octave must not be conflated. + +## Choose the right code artifact + +| Artifact | Workspace | Best use | Main risk | +|---|---|---|---| +| script `.m` | caller/base workspace | small reviewed orchestration | hidden inputs, leaked variables, path/state dependence | +| function `.m` | local workspace | reusable computation and automation | implicit conversions or undocumented side effects | +| live script `.mlx` | script-like interactive workspace | narrative exploration and teaching | opaque archive, embedded output, weak text review | +| class `.m` | object state and methods | durable abstractions | constructors, listeners, serialization callbacks | +| MEX | native process code | approved performance/interface work | arbitrary native execution | + +Prefer a function with explicit inputs, outputs, and an `arguments` block. +Keep a top-level script thin. Export reviewed live code to plain `.m` before +static inspection. Never open or run an untrusted project, live script, app, +class, MEX file, or package. + +Scripts share the base workspace and leave variables there. Functions, +including local functions, have private workspaces. Since R2024a, local +functions in scripts can appear anywhere in the file except inside conditional +contexts. The main function should match its filename. + +```matlab +function summary = summarizeSignal(signal, options) +%SUMMARIZESIGNAL Return deterministic summary statistics. +arguments + signal (:,1) double {mustBeFinite} + options.Center (1,1) logical = true +end + +if options.Center + signal = signal - mean(signal); +end +summary = struct( ... + "Count", numel(signal), ... + "Mean", mean(signal), ... + "StandardDeviation", std(signal)); +end +``` + +### Argument validation details + +- A size declaration such as `(:,1)` permits a column of any height. +- A class declaration can convert compatible input. Do not mistake conversion + for validation. +- Validators such as `mustBeFinite` check values without changing them. +- A default makes an argument optional. Required positional inputs precede + optional inputs. +- Name-value inputs use a structure name in the signature and dotted fields in + the block. +- Code generation support is a MATLAB Coder capability with additional + restrictions and a separate license; an `arguments` block alone does not + make code generation available. + +## Workspace and path hygiene + +1. Derive files from a confirmed project root, not the user's incidental + current folder. +2. Use `fullfile`; never construct paths by concatenating separators. +3. Add the narrowest reviewed directory. Avoid broad `genpath` because it can + expose hidden, generated, test, private, or malicious files. +4. Do not silently mutate `path`, `userpath`, preferences, startup files, or + Java/native search paths. +5. Avoid `global`, `persistent` cache state without invalidation, and broad + clearing. `clear all` can clear loaded functions and disrupt debugging. +6. Use `onCleanup` for resources such as files and temporary state. +7. Pass loaded data through a structure (`S = load(...)`) rather than injecting + names into a function or base workspace—but only after the MAT file is + trusted. + +MATLAB runs `startup.m` when it is found on the search path and `finish.m` on +normal exit. Projects can add paths and run startup/shutdown actions. Review all +of these before execution. + +## Dynamic and external execution surfaces + +Escalate any use of: + +- `eval`, `evalin`, `assignin`, `str2func`, text-derived `feval`, dynamic + property or callback names; +- shell entry points (`system`, `unix`, `dos`, `!`); +- Java class paths/methods, .NET assemblies, Python (`py.*`, `pyrun`, + `pyrunfile`), MEX, C/C++ libraries, and generated code; +- timers, UI callbacks, listeners, project tasks, build tasks, package startup, + and test fixtures with external effects; +- object loading (`loadobj`, `matlab.mixin.CustomElementSerialization`) and + System object load hooks. + +Function handles are safer than text dispatch only when the handle itself comes +from trusted code. Never pass untrusted text into a dispatch mechanism. Static +scanning is triage, not proof of absence. + +## MATLAB Projects + +A project can track files, control the path, declare references and packages, +run startup/shutdown tasks, integrate source control, and analyze dependencies. +Use project APIs only after reviewing project metadata and tasks. + +Recommended project record: + +- project name and root; +- MATLAB release and architecture; +- entry points and test roots; +- required and optional MathWorks products, each with license status + `unknown`, `confirmed`, or `unavailable`; +- external system dependencies and generated artifacts; +- startup/shutdown actions and path changes; +- RNG/tolerance/data-schema policy. + +Dependency Analyzer and `matlab.codetools.requiredFilesAndProducts` use static +analysis. Dynamic dispatch, overloaded methods, callbacks, generated names, and +conditional paths can cause misses or false positives. Their product lists do +not prove that a license can be checked out. + +## Code Analyzer and compatibility checks + +Use these only on trusted text: + +- `codeIssues(path)` returns a structured Code Analyzer result and supports + programmatic fixes for eligible issues. +- `checkcode(path)` remains useful for text-oriented or legacy automation. +- `codeCompatibilityReport(path)` finds potential issues after a release + upgrade. +- Project Upgrade can check and apply some release migrations and produce a + report. + +Do not auto-apply fixes across a scientific codebase without tests. Analyzer +silence does not establish numerical correctness, security, toolbox +availability, or Octave compatibility. + +Migration sequence: + +1. Freeze representative outputs and tolerance rationale in the old release. +2. Record release, products, compilers, BLAS/threading context, RNG algorithm + and seed, and external data schema. +3. Run static compatibility and dependency analysis. +4. Read every relevant product's release notes and bug reports. +5. Migrate shared libraries before applications. +6. Run unit, integration, numerical-equivalence, graphics, and performance + checks. +7. Investigate differences rather than automatically widening tolerances. + +R2026a-specific checks include changed/removed APIs in release notes, new JSON +table/timetable I/O, Python 3.13 support, string-array Python conversion, +interactive HTML graphics export, and platform/compiler support. R2026a no +longer ships new MATLAB releases for Intel Macs. + +## Unit testing + +Base MATLAB includes script-, function-, and class-based `matlab.unittest` +testing. Keep tests deterministic and independent of order. + +```matlab +classdef TestSummarizeSignal < matlab.unittest.TestCase + methods (Test) + function centersFiniteColumn(testCase) + actual = summarizeSignal([1; 2; 3]); + testCase.verifyEqual(actual.Count, 3); + testCase.verifyEqual(actual.Mean, 0, AbsTol=1e-14); end - % Override method - function result = compute(obj, x) - % Call superclass method - baseResult = compute@BaseClass(obj, x); - result = baseResult + obj.ExtraProp; + function rejectsNonfiniteInput(testCase) + testCase.verifyError( ... + @() summarizeSignal([1; NaN]), ... + "MATLAB:validators:mustBeFinite"); end end end ``` -### Handle vs Value Classes - -```matlab -% Value class (default) - copy semantics -classdef ValueClass - properties - Data - end -end - -a = ValueClass(); -a.Data = 1; -b = a; % b is a copy -b.Data = 2; % a.Data is still 1 - -% Handle class - reference semantics -classdef HandleClass < handle - properties - Data - end -end - -a = HandleClass(); -a.Data = 1; -b = a; % b references same object -b.Data = 2; % a.Data is now 2 -``` - -### Events and Listeners - -```matlab -classdef EventClass < handle - events - DataChanged - end - - properties - Data - end - - methods - function set.Data(obj, value) - obj.Data = value; - notify(obj, 'DataChanged'); - end - end -end - -% Usage -obj = EventClass(); -listener = addlistener(obj, 'DataChanged', @(src, evt) disp('Data changed!')); -obj.Data = 42; % Triggers event -``` +Use domain-derived `AbsTol` and `RelTol`; exact checks remain appropriate for +integers, strings, dimensions, and invariants. Isolate file output in temporary +folders and refuse network, interactive dialogs, or real credentials in unit +tests. + +Product boundaries: + +- `runtests`, `testsuite`, `matlab.unittest.TestCase`, and ordinary framework + plugins are base MATLAB. +- Parallel test execution requires Parallel Computing Toolbox. +- Dependency-based test selection, MATLAB Test Manager, Code Quality Dashboard, + advanced coverage, generated tests, and equivalence workflows can require + MATLAB Test. +- Requirements traceability can require Requirements Toolbox; generated-code + workflows can require MATLAB Coder, MATLAB Compiler SDK, Embedded Coder, or + other named products. + +In R2026a, `runtests` automatically opens a project for tests belonging to a +project that is not already open and closes it afterward. This may execute +reviewed project startup and shutdown actions; it is not safe for untrusted +projects. + +## Review checklist + +- [ ] Main function and filename agree. +- [ ] Inputs, shapes, classes, units, missingness, and outputs are documented. +- [ ] No hidden base-workspace dependency. +- [ ] Dynamic/external execution surfaces are absent or explicitly approved. +- [ ] Paths are project-local and narrow. +- [ ] Resources close on both success and failure. +- [ ] Errors have stable identifiers where tests rely on them. +- [ ] Tests cover edge shapes, empty values, missing values, nonfinite values, + numerical tolerances, and failure behavior. +- [ ] Required products are declared separately from confirmed license status. +- [ ] Migration evidence includes release notes and representative baselines. + +## Sources (verified 2026-07-23) + +- [Scripts vs. Functions](https://www.mathworks.com/help/matlab/matlab_prog/scripts-and-functions.html) +- [Create Scripts](https://www.mathworks.com/help/matlab/matlab_prog/create-scripts.html) +- [`arguments`](https://www.mathworks.com/help/matlab/ref/arguments.html) +- [Local Functions](https://www.mathworks.com/help/matlab/matlab_prog/local-functions.html) +- [MATLAB Projects](https://www.mathworks.com/help/matlab/projects.html) +- [Analyze Project Dependencies](https://www.mathworks.com/help/matlab/matlab_prog/analyze-project-dependencies.html) +- [`requiredFilesAndProducts`](https://www.mathworks.com/help/matlab/ref/matlab.codetools.requiredfilesandproducts.html) +- [MATLAB Code Analyzer Report](https://www.mathworks.com/help/matlab/matlab_prog/matlab-code-analyzer-report.html) +- [`codeCompatibilityReport`](https://www.mathworks.com/help/matlab/ref/codecompatibilityreport.html) +- [Project Upgrade](https://www.mathworks.com/help/matlab/matlab_prog/upgrade-projects.html) +- [Run Unit Tests](https://www.mathworks.com/help/matlab/run-unit-tests.html) +- [`runtests` R2026a history](https://www.mathworks.com/help/matlab/ref/runtests.html) +- [MATLAB Test product boundary](https://www.mathworks.com/products/matlab-test.html) +- [MATLAB R2026a release notes](https://www.mathworks.com/help/matlab/release-notes.html) diff --git a/.agents/skills/matlab/references/python-integration.md b/.agents/skills/matlab/references/python-integration.md index 57cd0fd..ffa8891 100644 --- a/.agents/skills/matlab/references/python-integration.md +++ b/.agents/skills/matlab/references/python-integration.md @@ -1,433 +1,248 @@ -# Python Integration Reference +# MATLAB and Python Integration -## Table of Contents -1. [Calling Python from MATLAB](#calling-python-from-matlab) -2. [Data Type Conversion](#data-type-conversion) -3. [Working with Python Objects](#working-with-python-objects) -4. [Calling MATLAB from Python](#calling-matlab-from-python) -5. [Common Workflows](#common-workflows) +This reference is pinned to MATLAB R2026a as reviewed on 2026-07-23. +Calling Python from MATLAB and calling MATLAB from Python are different +interfaces with different process, data, and license behavior. -## Calling Python from MATLAB +## Exact R2026a compatibility -### Setup +MathWorks' support table lists 64-bit CPython **3.9, 3.10, 3.11, 3.12, +and 3.13** for: -```matlab -% Check Python configuration -pyenv +- MATLAB Interface to Python; +- MATLAB Engine for Python; +- MATLAB Compiler SDK for Python; +- MATLAB Production Server Client Library. -% Set Python version (before calling any Python) -pyenv('Version', '/usr/bin/python3'); -pyenv('Version', '3.10'); +The current MathWorks-maintained PyPI release reviewed here is +`matlabengine==26.1.12`, released 2026-05-08. It requires MATLAB R2026a. +Do not install a floating latest package for a reproducible environment. -% Check if Python is available -pe = pyenv; -disp(pe.Version); -disp(pe.Executable); +```bash +uv pip install "matlabengine==26.1.12" ``` -### Basic Python Calls - -```matlab -% Call built-in functions with py. prefix -result = py.len([1, 2, 3, 4]); % 4 -result = py.sum([1, 2, 3, 4]); % 10 -result = py.max([1, 2, 3, 4]); % 4 -result = py.abs(-5); % 5 - -% Create Python objects -pyList = py.list({1, 2, 3}); -pyDict = py.dict(pyargs('a', 1, 'b', 2)); -pySet = py.set({1, 2, 3}); -pyTuple = py.tuple({1, 2, 3}); - -% Call module functions -result = py.math.sqrt(16); -result = py.os.getcwd(); -wrapped = py.textwrap.wrap('This is a long string'); -``` +Installation does not include MATLAB or grant a license. Engine requires an +installed R2026a on the same machine; MATLAB Runtime alone is insufficient. +The Python architecture must match MATLAB. -### Import and Use Modules +R2026a also ships a preinstalled Engine distribution at the single named path: -```matlab -% Import module -np = py.importlib.import_module('numpy'); -pd = py.importlib.import_module('pandas'); - -% Use module -arr = np.array({1, 2, 3, 4, 5}); -result = np.mean(arr); - -% Alternative: direct py. syntax -arr = py.numpy.array({1, 2, 3, 4, 5}); -result = py.numpy.mean(arr); +```text +/extern/engines/python/dist ``` -### Run Python Code - -```matlab -% Run Python statements -pyrun("x = 5") -pyrun("y = x * 2") -result = pyrun("z = y + 1", "z"); - -% Run Python file -pyrunfile("script.py"); -result = pyrunfile("script.py", "output_variable"); - -% Run with input variables -x = 10; -result = pyrun("y = x * 2", "y", x=x); -``` +Add only that confirmed path to the selected environment when using this +method. Do not print or upload the full `PATH`, `PYTHONPATH`, environment, +license configuration, home directory, or credentials. -### Keyword Arguments +## Plan compatibility without execution -```matlab -% Use pyargs for keyword arguments -result = py.sorted({3, 1, 4, 1, 5}, pyargs('reverse', true)); - -% Multiple keyword arguments -df = py.pandas.DataFrame(pyargs( ... - 'data', py.dict(pyargs('A', {1, 2, 3}, 'B', {4, 5, 6})), ... - 'index', {'x', 'y', 'z'})); -``` - -## Data Type Conversion - -### MATLAB to Python - -| MATLAB Type | Python Type | -|-------------|-------------| -| double, single | float | -| int8, int16, int32, int64 | int | -| uint8, uint16, uint32, uint64 | int | -| logical | bool | -| char, string | str | -| cell array | list | -| struct | dict | -| numeric array | numpy.ndarray (if numpy available) | - -```matlab -% Automatic conversion examples -py.print(3.14); % float -py.print(int32(42)); % int -py.print(true); % bool (True) -py.print("hello"); % str -py.print({'a', 'b'}); % list - -% Explicit conversion to Python types -pyInt = py.int(42); -pyFloat = py.float(3.14); -pyStr = py.str('hello'); -pyList = py.list({1, 2, 3}); -pyDict = py.dict(pyargs('key', 'value')); +```bash +python scripts/plan_python_compatibility.py \ + --matlab-release R2026a \ + --python-version 3.13 \ + --engine-version 26.1.12 ``` -### Python to MATLAB +The planner uses a bundled dated support table. It does not import +`matlab.engine`, inspect the environment, locate installations, start MATLAB, +or check out a license. `--include-launch-snippet` adds an explicitly labeled +launch example to the JSON plan but still does not execute it. -```matlab -% Convert Python types to MATLAB -matlabDouble = double(py.float(3.14)); -matlabInt = int64(py.int(42)); -matlabChar = char(py.str('hello')); -matlabString = string(py.str('hello')); -matlabCell = cell(py.list({1, 2, 3})); - -% Convert numpy arrays -pyArr = py.numpy.array({1, 2, 3, 4, 5}); -matlabArr = double(pyArr); - -% Convert pandas DataFrame to MATLAB table -pyDf = py.pandas.read_csv('data.csv'); -matlabTable = table(pyDf); % Requires pandas2table or similar - -% Manual DataFrame conversion -colNames = cell(pyDf.columns.tolist()); -data = cell(pyDf.values.tolist()); -T = cell2table(data, 'VariableNames', colNames); -``` +## Calling Python from MATLAB -### Array Conversion +Review and pin one interpreter before any `py.*` access: ```matlab -% MATLAB array to numpy -matlabArr = [1 2 3; 4 5 6]; -pyArr = py.numpy.array(matlabArr); - -% numpy to MATLAB -pyArr = py.numpy.random.rand(int64(3), int64(4)); -matlabArr = double(pyArr); - -% Note: numpy uses row-major (C) order, MATLAB uses column-major (Fortran) -% Transposition may be needed for correct layout +environment = pyenv( ... + Version="/reviewed/venv/bin/python", ... + ExecutionMode="OutOfProcess"); ``` -## Working with Python Objects - -### Object Methods and Properties - -```matlab -% Call methods -pyList = py.list({3, 1, 4, 1, 5}); -pyList.append(9); -pyList.sort(); - -% Access properties/attributes -pyStr = py.str('hello world'); -upper = pyStr.upper(); -words = pyStr.split(); - -% Check attributes -methods(pyStr) % List methods -fieldnames(pyDict) % List keys -``` +On Windows, a registered version can be selected by version, but a full named +executable is clearer. On macOS/Linux, use the full executable. The Python +build architecture must match MATLAB. MATLAB does not support CPython from the +Microsoft Store. -### Iterating Python Objects +Interpreter switching: -```matlab -% Iterate over Python list -pyList = py.list({1, 2, 3, 4, 5}); -for item = py.list(pyList) - disp(item{1}); -end - -% Convert to cell and iterate -items = cell(pyList); -for i = 1:length(items) - disp(items{i}); -end - -% Iterate dict keys -pyDict = py.dict(pyargs('a', 1, 'b', 2, 'c', 3)); -keys = cell(pyDict.keys()); -for i = 1:length(keys) - key = keys{i}; - value = pyDict{key}; - fprintf('%s: %d\n', char(key), int64(value)); -end -``` +- in-process: restart MATLAB before changing the loaded interpreter; +- out-of-process: `terminate(pyenv)` can stop the external interpreter, after + which `pyenv` can be reconfigured. -### Error Handling +Out-of-process isolates interpreter crashes and allows reload, but it is not a +security sandbox. Data plus transfer metadata are limited to 2 GiB per +out-of-process transfer. Python modules can perform arbitrary process, file, +network, native, and credential operations. -```matlab -try - result = py.some_module.function_that_might_fail(); -catch ME - if isa(ME, 'matlab.exception.PyException') - disp('Python error occurred:'); - disp(ME.message); - else - rethrow(ME); - end -end -``` +Never run untrusted `py.*`, `pyrun`, `pyrunfile`, Python modules, wheels, or +requirements files. `pyrun` and `pyrunfile` are dynamic code execution +surfaces. ## Calling MATLAB from Python -### Setup MATLAB Engine +Starting Engine is explicit execution: ```python -# Install MATLAB Engine API for Python -# From MATLAB: cd(fullfile(matlabroot,'extern','engines','python')) -# Then: python setup.py install - import matlab.engine -# Start MATLAB engine -eng = matlab.engine.start_matlab() - -# Or connect to shared session (MATLAB: matlab.engine.shareEngine) -eng = matlab.engine.connect_matlab() - -# List available sessions -matlab.engine.find_matlab() +engine = matlab.engine.start_matlab() +try: + result = engine.sqrt(16.0) +finally: + engine.quit() ``` -### Call MATLAB Functions - -```python -import matlab.engine - -eng = matlab.engine.start_matlab() - -# Call built-in functions -result = eng.sqrt(16.0) -result = eng.sin(3.14159 / 2) - -# Multiple outputs -mean_val, std_val = eng.std([1, 2, 3, 4, 5], nargout=2) +`start_matlab()` creates a MATLAB process, can execute startup code, and can +check out a MATLAB license. Never call it merely to probe availability. +Review startup paths/actions and confirm entitlement first. -# Matrix operations -A = matlab.double([[1, 2], [3, 4]]) -B = eng.inv(A) -C = eng.mtimes(A, B) # Matrix multiplication +`connect_matlab()` connects to a deliberately shared local MATLAB session; +`find_matlab()` lists shared sessions. Sharing changes the trust boundary and +must be explicitly approved. Do not connect to an unknown session. -# Call custom function (must be on MATLAB path) -result = eng.myfunction(arg1, arg2) - -# Cleanup -eng.quit() -``` - -### Data Conversion (Python to MATLAB) +Only add narrow reviewed paths: ```python -import matlab.engine -import numpy as np - -eng = matlab.engine.start_matlab() - -# Python to MATLAB types -matlab_double = matlab.double([1.0, 2.0, 3.0]) -matlab_int = matlab.int32([1, 2, 3]) -matlab_complex = matlab.double([1+2j, 3+4j], is_complex=True) - -# 2D array -matlab_matrix = matlab.double([[1, 2, 3], [4, 5, 6]]) - -# numpy to MATLAB -np_array = np.array([[1, 2], [3, 4]], dtype=np.float64) -matlab_array = matlab.double(np_array.tolist()) - -# Call MATLAB with numpy data -result = eng.sum(matlab.double(np_array.flatten().tolist())) -``` - -### Async Calls +engine.addpath("/reviewed/project/src", nargout=0) +value = engine.analyzeSignal( + matlab.double([[1.0], [2.0], [3.0]]), + nargout=1, +) +``` + +Do not use recursive path additions. Pass fixed function names, not text to +`eval`, and set `nargout` deliberately. Redirect output to bounded +`io.StringIO` only when needed; logs can contain paths or data. + +## MATLAB-to-Python conversion in R2026a + +Scalar automatic mappings include: + +| MATLAB | Python | +|---|---| +| real `double`/`single` | `float` by default; type hints can select `int` | +| complex float | `complex` | +| integer scalar | `int` | +| logical scalar | `bool` | +| string scalar/character vector | `str` | +| missing string | `None`-like string conversion documented by MathWorks | +| `dictionary`/scalar `struct` | `dict` | +| `table`/`timetable` | pandas `DataFrame` | +| `datetime` | `datetime.datetime` | +| `duration` | `datetime.timedelta` | + +With NumPy available, numeric/logical MATLAB arrays convert to NumPy arrays +with corresponding precision/sign/complex dtype. Without NumPy, numeric arrays +use Python buffer/memoryview behavior. Since R2025a, array conversion behavior +changed; test code migrated from older vector `array.array` assumptions. + +R2026a additions: + +- MATLAB string vectors automatically convert to Python lists; +- `pystringarray` converts MATLAB string arrays to NumPy `StringDType` + arrays; +- missing string entries need explicit round-trip tests. + +No automatic conversion is documented for multidimensional character/cell +arrays or M-by-N string arrays where both dimensions exceed one. Sparse +arrays, nonscalar structure arrays, categorical arrays, `containers.Map`, +MATLAB objects, and metadata classes are unsupported in the MATLAB-to-Python +interface. + +## Python-to-MATLAB conversion + +MATLAB automatically converts selected scalar Python returns. Other values use +explicit conversion: + +| Python value | MATLAB conversion | +|---|---| +| `py.str` | `string` or `char` | +| Python numeric scalar | `double`, `single`, or integer constructors | +| `py.bytes` | `uint8` | +| `py.numpy.ndarray` | matching MATLAB numeric class or `string` where supported | +| `py.list`/`py.tuple` | numeric/logical/string/cell conversion when homogeneous/compatible | +| mapping protocol / `py.dict` | `dictionary` or `struct` | +| pandas `DataFrame` | `table`/`timetable` with documented conversion | +| Python datetime/timedelta/NumPy time | MATLAB `datetime`/`duration` conversions | + +Always test: + +- rank and row/column orientation; +- C-order versus column-major interpretation; +- dtype width/sign and complex values; +- `NaN`, `Inf`, `None`, `NaT`, missing strings, and categorical values; +- table index versus timetable row times; +- time zones and units; +- dictionary key restrictions and column-name normalization; +- copies versus shared/buffer-backed memory. + +Do not flatten an array to "fix" a shape mismatch without recording the +ordering contract. + +## MATLAB Engine array classes + +The `matlab` Python package provides MATLAB array classes such as +`matlab.double`, `matlab.single`, signed/unsigned integer classes, and +`matlab.logical`. These are for Engine calls, not general NumPy replacements. ```python -import matlab.engine - -eng = matlab.engine.start_matlab() - -# Asynchronous call -future = eng.sqrt(16.0, background=True) - -# Do other work... - -# Get result when ready -result = future.result() - -# Check if done -if future.done(): - result = future.result() - -# Cancel if needed -future.cancel() -``` - -## Common Workflows - -### Using Python Libraries in MATLAB - -```matlab -% Use scikit-learn from MATLAB -sklearn = py.importlib.import_module('sklearn.linear_model'); - -% Prepare data -X = rand(100, 5); -y = X * [1; 2; 3; 4; 5] + randn(100, 1) * 0.1; - -% Convert to Python/numpy -X_py = py.numpy.array(X); -y_py = py.numpy.array(y); - -% Train model -model = sklearn.LinearRegression(); -model.fit(X_py, y_py); - -% Get coefficients -coefs = double(model.coef_); -intercept = double(model.intercept_); - -% Predict -y_pred = double(model.predict(X_py)); +column = matlab.double([[1.0], [2.0], [3.0]]) +matrix = matlab.double([[1.0, 2.0], [3.0, 4.0]]) ``` -### Using MATLAB in Python Scripts - -```python -import matlab.engine -import numpy as np - -# Start MATLAB -eng = matlab.engine.start_matlab() - -# Use MATLAB's optimization -def matlab_fmincon(objective, x0, A, b, Aeq, beq, lb, ub): - """Wrapper for MATLAB's fmincon.""" - # Convert to MATLAB types - x0_m = matlab.double(x0.tolist()) - A_m = matlab.double(A.tolist()) if A is not None else matlab.double([]) - b_m = matlab.double(b.tolist()) if b is not None else matlab.double([]) - - # Call MATLAB (assuming objective is a MATLAB function) - x, fval = eng.fmincon(objective, x0_m, A_m, b_m, nargout=2) - - return np.array(x).flatten(), fval +Construct the intended dimensions explicitly. Function outputs can be Engine +proxy/object types; convert only through documented APIs. -# Use MATLAB's plotting -def matlab_plot(x, y, title_str): - """Create plot using MATLAB.""" - eng.figure(nargout=0) - eng.plot(matlab.double(x.tolist()), matlab.double(y.tolist()), nargout=0) - eng.title(title_str, nargout=0) - eng.saveas(eng.gcf(), 'plot.png', nargout=0) +## Serialization and exchange -eng.quit() -``` +- MATLAB does not support saving Python objects into MAT files. +- Never use Python pickle for MATLAB exchange. Pickle deserialization executes + attacker-controlled behavior. +- For simple data, prefer schema-documented CSV/JSON/Parquet/HDF5. +- For trusted MATLAB arrays, use MAT with explicit version and inventory it + before loading in another process. +- SciPy `loadmat` is not used by this skill's inventory. It can deserialize + complex structures and is not a safety scanner. -### Sharing Data Between MATLAB and Python +## Compiler SDK distinction -```matlab -% Save data for Python -data = rand(100, 10); -labels = randi([0 1], 100, 1); -save('data_for_python.mat', 'data', 'labels'); - -% In Python: -% import scipy.io -% mat = scipy.io.loadmat('data_for_python.mat') -% data = mat['data'] -% labels = mat['labels'] - -% Load data from Python (saved with scipy.io.savemat) -loaded = load('data_from_python.mat'); -data = loaded.data; -labels = loaded.labels; - -% Alternative: use CSV for simple data exchange -writematrix(data, 'data.csv'); -% Python: pd.read_csv('data.csv') - -% Python writes: df.to_csv('results.csv') -results = readmatrix('results.csv'); -``` +MATLAB Compiler SDK for Python packages are not MATLAB Engine: -### Using Python Packages Not Available in MATLAB +- building requires MATLAB, MATLAB Compiler SDK, and source dependencies; +- deployed components use a compatible MATLAB Runtime under applicable terms; +- generated Python packages are not supported as Python modules called back + from MATLAB's Python interface; +- Engine itself requires full installed MATLAB, not Runtime. -```matlab -% Example: Use Python's requests library -requests = py.importlib.import_module('requests'); +Confirm product, target release, platform, package, runtime, and license terms. -% Make HTTP request -response = requests.get('https://api.example.com/data'); -status = int64(response.status_code); +## Troubleshooting without broad disclosure -if status == 200 - data = response.json(); - % Convert to MATLAB structure - dataStruct = struct(data); -end +Collect only named facts: -% Example: Use Python's PIL/Pillow for advanced image processing -PIL = py.importlib.import_module('PIL.Image'); +- MATLAB release/update and architecture; +- one Python executable path and `major.minor`; +- Engine package version; +- selected `pyenv` status/mode (not all environment values); +- one failing function, input classes/shapes, and redacted traceback; +- whether NumPy/pandas are required and their pinned versions. -% Open image -img = PIL.open('image.png'); +Do not ask for `env`, `set`, complete `PATH`, complete `sys.path`, license +files, tokens, home-directory listings, or credentials. -% Resize -img_resized = img.resize(py.tuple({int64(256), int64(256)})); +## Sources (verified 2026-07-23) -% Save -img_resized.save('image_resized.png'); -``` +- [Python Compatibility by MATLAB Release](https://www.mathworks.com/support/requirements/python-compatibility.html) +- [Install MATLAB Engine API for Python](https://www.mathworks.com/help/matlab/matlab_external/install-the-matlab-engine-for-python.html) +- [MathWorks `matlabengine` 26.1.12 package](https://pypi.org/project/matlabengine/26.1.12/) +- [Call MATLAB from Python](https://www.mathworks.com/help/matlab/matlab-engine-for-python.html) +- [Get Started with MATLAB Engine](https://www.mathworks.com/help/matlab/matlab_external/get-started-with-matlab-engine-for-python.html) +- [Configure Python for MATLAB](https://www.mathworks.com/help/matlab/matlab_external/install-supported-python-implementation.html) +- [Call Python from MATLAB](https://www.mathworks.com/help/matlab/call-python-libraries.html) +- [Pass Data from MATLAB to Python](https://www.mathworks.com/help/matlab/matlab_external/passing-data-to-python.html) +- [Pass Data from Python to MATLAB](https://www.mathworks.com/help/matlab/matlab_external/pass-data-between-matlab-and-python-from-python.html) +- [Python Interface Limitations](https://www.mathworks.com/help/matlab/matlab_external/limitations-to-python-support.html) +- [MATLAB Engine Limitations](https://www.mathworks.com/help/matlab/matlab_external/limitations-to-the-matlab-engine-for-python.html) +- [R2026a Release Highlights](https://www.mathworks.com/products/new_products/latest_features.html) diff --git a/.agents/skills/matlab/scripts/_common.py b/.agents/skills/matlab/scripts/_common.py new file mode 100644 index 0000000..ebdef51 --- /dev/null +++ b/.agents/skills/matlab/scripts/_common.py @@ -0,0 +1,263 @@ +#!/usr/bin/env python3 +"""Shared bounded local-only helpers for the MATLAB skill CLIs.""" + +from __future__ import annotations + +import hashlib +import json +import re +import sys +from pathlib import Path +from typing import Any, Iterable + +MAX_INPUT_BYTES = 64 * 1024 * 1024 +MAX_TEXT_BYTES = 4 * 1024 * 1024 +MAX_JSON_BYTES = 2 * 1024 * 1024 +MAX_FILES = 500 +MAX_JSON_ITEMS = 20_000 +MAX_JSON_DEPTH = 24 +MAX_PATH_CHARS = 4096 +MATLAB_IDENTIFIER = re.compile(r"^[A-Za-z][A-Za-z0-9_]{0,62}$") +MATLAB_RELEASE = re.compile(r"^R(20[0-9]{2})([ab])$") +URL_LIKE = re.compile(r"^[A-Za-z][A-Za-z0-9+.-]*://") + + +class CliError(ValueError): + """Expected, user-actionable CLI failure.""" + + +def _raw_path(value: str) -> Path: + if not value or len(value) > MAX_PATH_CHARS: + raise CliError("path is empty or exceeds the length limit") + if "\x00" in value or URL_LIKE.match(value): + raise CliError("only local filesystem paths are accepted") + return Path(value) + + +def _reject_symlink_chain(path: Path) -> None: + """Reject every existing symlink component without following it.""" + absolute = path.absolute() + current = Path(absolute.anchor) + for part in absolute.parts[1:]: + current = current / part + if current.exists() and current.is_symlink(): + raise CliError(f"symlink paths are not accepted: {current}") + + +def checked_root(value: str | Path) -> Path: + raw = _raw_path(str(value)) + _reject_symlink_chain(raw) + try: + root = raw.resolve(strict=True) + except OSError as exc: + raise CliError(f"root does not exist or is inaccessible: {raw}") from exc + if not root.is_dir(): + raise CliError(f"root is not a directory: {root}") + return root + + +def checked_input( + value: str | Path, + *, + root: Path, + kind: str = "file", + suffixes: Iterable[str] | None = None, + max_bytes: int = MAX_INPUT_BYTES, +) -> Path: + raw = _raw_path(str(value)) + candidate = raw if raw.is_absolute() else root / raw + _reject_symlink_chain(candidate) + try: + path = candidate.resolve(strict=True) + path.relative_to(root) + except (OSError, ValueError) as exc: + raise CliError(f"input must exist within root {root}") from exc + if kind == "file" and not path.is_file(): + raise CliError(f"input is not a regular file: {path}") + if kind == "directory" and not path.is_dir(): + raise CliError(f"input is not a directory: {path}") + if kind == "any" and not (path.is_file() or path.is_dir()): + raise CliError(f"input is not a regular file or directory: {path}") + if kind not in {"file", "directory", "any"}: + raise CliError(f"unsupported input kind: {kind}") + if suffixes is not None and path.is_file(): + allowed = {suffix.casefold() for suffix in suffixes} + if path.suffix.casefold() not in allowed: + raise CliError( + f"unsupported suffix {path.suffix!r}; expected one of {sorted(allowed)}" + ) + if path.is_file(): + try: + size = path.stat().st_size + except OSError as exc: + raise CliError(f"cannot stat input: {path}") from exc + if size > max_bytes: + raise CliError(f"input is {size} bytes; limit is {max_bytes}") + return path + + +def checked_output( + value: str | Path, + *, + root: Path, + suffixes: Iterable[str] | None = None, +) -> Path: + raw = _raw_path(str(value)) + candidate = raw if raw.is_absolute() else root / raw + _reject_symlink_chain(candidate) + try: + parent = candidate.parent.resolve(strict=True) + parent.relative_to(root) + except (OSError, ValueError) as exc: + raise CliError(f"output parent must exist within root {root}") from exc + path = parent / candidate.name + if path.exists() or path.is_symlink(): + raise CliError(f"refusing to overwrite existing output: {path}") + if suffixes is not None: + allowed = {suffix.casefold() for suffix in suffixes} + if path.suffix.casefold() not in allowed: + raise CliError( + f"unsupported output suffix {path.suffix!r}; " + f"expected one of {sorted(allowed)}" + ) + return path + + +def read_bytes(path: Path, *, max_bytes: int = MAX_INPUT_BYTES) -> bytes: + try: + size = path.stat().st_size + if size > max_bytes: + raise CliError(f"input is {size} bytes; limit is {max_bytes}") + return path.read_bytes() + except CliError: + raise + except OSError as exc: + raise CliError(f"cannot read input: {path}") from exc + + +def read_text(path: Path, *, max_bytes: int = MAX_TEXT_BYTES) -> str: + payload = read_bytes(path, max_bytes=max_bytes) + try: + return payload.decode("utf-8") + except UnicodeDecodeError as exc: + raise CliError(f"input is not UTF-8 text: {path}") from exc + + +def _reject_duplicate_keys(pairs: list[tuple[str, Any]]) -> dict[str, Any]: + output: dict[str, Any] = {} + for key, value in pairs: + if key in output: + raise CliError(f"duplicate JSON key: {key!r}") + output[key] = value + return output + + +def _validate_json_shape(value: Any, *, depth: int = 0) -> int: + if depth > MAX_JSON_DEPTH: + raise CliError(f"JSON nesting exceeds {MAX_JSON_DEPTH}") + if value is None or isinstance(value, (bool, int, float, str)): + if isinstance(value, str) and len(value) > 100_000: + raise CliError("JSON string exceeds 100000 characters") + return 1 + if isinstance(value, list): + return 1 + sum(_validate_json_shape(item, depth=depth + 1) for item in value) + if isinstance(value, dict): + if not all(isinstance(key, str) for key in value): + raise CliError("JSON object keys must be strings") + return 1 + sum( + _validate_json_shape(item, depth=depth + 1) for item in value.values() + ) + raise CliError(f"unsupported JSON value type: {type(value).__name__}") + + +def parse_json_text(text: str) -> Any: + try: + value = json.loads(text, object_pairs_hook=_reject_duplicate_keys) + except CliError: + raise + except (json.JSONDecodeError, ValueError) as exc: + raise CliError(f"invalid JSON: {exc}") from exc + item_count = _validate_json_shape(value) + if item_count > MAX_JSON_ITEMS: + raise CliError(f"JSON has {item_count} values; limit is {MAX_JSON_ITEMS}") + return value + + +def load_json(path: Path) -> Any: + return parse_json_text(read_text(path, max_bytes=MAX_JSON_BYTES)) + + +def bounded_int( + value: str | int, + *, + name: str, + minimum: int, + maximum: int, +) -> int: + try: + number = int(value) + except (TypeError, ValueError) as exc: + raise CliError(f"{name} must be an integer") from exc + if number < minimum or number > maximum: + raise CliError(f"{name} must be between {minimum} and {maximum}") + return number + + +def validate_identifier(value: str, *, name: str = "identifier") -> str: + if not MATLAB_IDENTIFIER.fullmatch(value): + raise CliError( + f"{name} must be a MATLAB identifier of at most 63 characters" + ) + return value + + +def validate_release(value: str) -> str: + if not MATLAB_RELEASE.fullmatch(value): + raise CliError("MATLAB release must look like R2026a") + return value + + +def sha256_file(path: Path, *, max_bytes: int = MAX_INPUT_BYTES) -> str: + size = path.stat().st_size + if size > max_bytes: + raise CliError(f"input is {size} bytes; hash limit is {max_bytes}") + digest = hashlib.sha256() + try: + with path.open("rb") as handle: + while chunk := handle.read(64 * 1024): + digest.update(chunk) + except OSError as exc: + raise CliError(f"cannot hash input: {path}") from exc + return digest.hexdigest() + + +def relative_id(path: Path, root: Path) -> str: + return path.relative_to(root).as_posix() + + +def write_new_text(path: Path, text: str) -> None: + try: + with path.open("x", encoding="utf-8", newline="\n") as handle: + handle.write(text) + except FileExistsError as exc: + raise CliError(f"refusing to overwrite existing output: {path}") from exc + except OSError as exc: + raise CliError(f"cannot write output: {path}") from exc + + +def emit_json(value: Any) -> None: + json.dump(value, sys.stdout, indent=2, sort_keys=True, ensure_ascii=False) + sys.stdout.write("\n") + + +def fail_json(tool: str, exc: Exception) -> int: + emit_json( + { + "error": str(exc), + "executes_external_code": False, + "network_accessed": False, + "ok": False, + "tool": tool, + } + ) + return 2 diff --git a/.agents/skills/matlab/scripts/generate_function_scaffold.py b/.agents/skills/matlab/scripts/generate_function_scaffold.py new file mode 100644 index 0000000..42fc9ad --- /dev/null +++ b/.agents/skills/matlab/scripts/generate_function_scaffold.py @@ -0,0 +1,165 @@ +#!/usr/bin/env python3 +"""Dry-run or write deterministic MATLAB function and unit-test scaffolds.""" + +from __future__ import annotations + +import argparse +import re +import sys +from pathlib import Path +from typing import Any + +from _common import ( + CliError, + checked_input, + checked_root, + emit_json, + fail_json, + relative_id, + validate_identifier, + write_new_text, +) + +TOOL = "generate_function_scaffold" +SAFE_DIRECTORY = re.compile(r"^[A-Za-z][A-Za-z0-9_.-]{0,63}$") + + +def _directory(root: Path, name: str, *, write: bool) -> Path: + if not SAFE_DIRECTORY.fullmatch(name): + raise CliError("source/test directory must be one simple local directory name") + path = root / name + if path.exists(): + return checked_input(path, root=root, kind="directory") + if not write: + return path + try: + path.mkdir() + except FileExistsError: + return checked_input(path, root=root, kind="directory") + except OSError as exc: + raise CliError(f"cannot create directory: {path}") from exc + return path + + +def _title_name(name: str) -> str: + return name[0].upper() + name[1:] + + +def function_source(name: str) -> str: + upper = name.upper() + return f"""function result = {name}(data, options) +%{upper} Scale a finite numeric matrix. +% RESULT = {upper}(DATA, Scale=VALUE) multiplies DATA by VALUE. + +arguments + data (:,:) double {{mustBeFinite}} + options.Scale (1,1) double {{mustBeFinite}} = 1 +end + +result = data .* options.Scale; +end +""" + + +def test_source(name: str, class_name: str) -> str: + return f"""classdef {class_name} < matlab.unittest.TestCase + methods (Test) + function scalesFiniteMatrix(testCase) + actual = {name}([1 2; 3 4], Scale=2); + expected = [2 4; 6 8]; + testCase.verifyEqual(actual, expected, AbsTol=1e-14); + end + + function preservesShape(testCase) + actual = {name}(zeros(2, 3)); + testCase.verifySize(actual, [2 3]); + end + end +end +""" + + +def generate(args: argparse.Namespace) -> dict[str, Any]: + root = checked_root(args.root) + name = validate_identifier(args.name, name="function name") + class_name = validate_identifier( + "Test" + _title_name(name), name="test class name" + ) + if args.source_dir == args.test_dir: + raise CliError("source and test directories must be distinct") + if not SAFE_DIRECTORY.fullmatch(args.source_dir) or not SAFE_DIRECTORY.fullmatch( + args.test_dir + ): + raise CliError( + "source/test directory must be one simple local directory name" + ) + source_dir = _directory(root, args.source_dir, write=args.write) + test_dir = _directory(root, args.test_dir, write=args.write) + function_path = source_dir / f"{name}.m" + test_path = test_dir / f"{class_name}.m" + for path in (function_path, test_path): + if path.exists() or path.is_symlink(): + raise CliError(f"refusing to overwrite existing scaffold: {path}") + function_text = function_source(name) + test_text = test_source(name, class_name) + if args.write: + write_new_text(function_path, function_text) + try: + write_new_text(test_path, test_text) + except Exception: + try: + function_path.unlink(missing_ok=True) + except OSError: + pass + raise + return { + "contents": { + "function": function_text, + "test": test_text, + }, + "dry_run": not args.write, + "executes": False, + "files": { + "function": relative_id(function_path, root), + "test": relative_id(test_path, root), + }, + "network_accessed": False, + "ok": True, + "requires_matlab_to_run_tests": True, + "target_release": "R2026a", + "tool": TOOL, + "wrote_files": bool(args.write), + "warning": ( + "Generated text is a starting point. Review domain semantics, " + "tolerances, products, paths, and safety before execution." + ), + } + + +def parser() -> argparse.ArgumentParser: + result = argparse.ArgumentParser( + description=( + "Generate a MATLAB R2026a function and matlab.unittest class. " + "Default is dry-run; --write refuses collisions." + ) + ) + result.add_argument("name", help="MATLAB function identifier") + result.add_argument("--root", default=".", help="allowed project root") + result.add_argument("--source-dir", default="src") + result.add_argument("--test-dir", default="tests") + result.add_argument( + "--write", action="store_true", help="create new files and directories" + ) + return result + + +def main() -> int: + try: + emit_json(generate(parser().parse_args())) + return 0 + except CliError as exc: + return fail_json(TOOL, exc) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.agents/skills/matlab/scripts/inventory_mat_file.py b/.agents/skills/matlab/scripts/inventory_mat_file.py new file mode 100644 index 0000000..857f0e4 --- /dev/null +++ b/.agents/skills/matlab/scripts/inventory_mat_file.py @@ -0,0 +1,351 @@ +#!/usr/bin/env python3 +"""Inventory bounded MAT/HDF5 metadata without deserializing object values.""" + +from __future__ import annotations + +import argparse +import hashlib +import sys +from collections import Counter +from pathlib import Path +from typing import Any + +from _common import ( + CliError, + bounded_int, + checked_input, + checked_root, + emit_json, + fail_json, + relative_id, + sha256_file, +) + +TOOL = "inventory_mat_file" +HDF5_SIGNATURE = b"\x89HDF\r\n\x1a\n" +OBJECT_LIKE_CLASSES = { + "cell", + "function", + "java", + "object", + "opaque", + "struct", + "table", + "timetable", +} + + +def identify_header(path: Path) -> dict[str, Any]: + try: + with path.open("rb") as handle: + prefix = handle.read(8192) + except OSError as exc: + raise CliError(f"cannot read MAT header: {path}") from exc + signature_offset = next( + ( + offset + for offset in (0, 512, 1024, 2048, 4096) + if prefix[offset : offset + 8] == HDF5_SIGNATURE + ), + None, + ) + header = prefix[:128] + if header.startswith(b"MATLAB 7.3 MAT-file"): + kind = "matlab_v7_3_hdf5" + elif header.startswith(b"MATLAB 5.0 MAT-file"): + kind = "matlab_level5_v6_or_v7" + elif signature_offset is not None: + kind = "hdf5_not_confirmed_matlab_v7_3" + elif prefix.startswith(b"\x80"): + kind = "python_pickle_signature_refused" + else: + kind = "unknown_or_mat_v4" + return { + "detected_kind": kind, + "hdf5_signature_offset": signature_offset, + "matlab_text_header_present": header.startswith(b"MATLAB "), + } + + +def _redacted_name(name: str, index: int, kind: str) -> dict[str, Any]: + return { + "id": f"{kind}-{index:06d}", + "name_emitted": False, + "name_sha256": hashlib.sha256(name.encode("utf-8", "surrogatepass")).hexdigest(), + } + + +def scipy_inventory( + path: Path, *, max_nodes: int +) -> tuple[list[dict[str, Any]], list[str]]: + try: + from scipy import io as scipy_io + except ImportError as exc: + raise CliError( + "SciPy is optional and not installed; use --backend header or " + "install a pinned scipy in an approved environment" + ) from exc + try: + entries = scipy_io.whosmat(str(path), appendmat=False) + except Exception as exc: + raise CliError(f"SciPy could not inventory MAT metadata: {exc}") from exc + if len(entries) > max_nodes: + raise CliError(f"MAT file has more than {max_nodes} variables") + output: list[dict[str, Any]] = [] + warnings: list[str] = [] + for index, (name, shape, class_name) in enumerate(entries, start=1): + class_text = str(class_name) + record = { + **_redacted_name(str(name), index, "variable"), + "class": class_text, + "object_like": class_text.casefold() in OBJECT_LIKE_CLASSES, + "shape": [int(value) for value in shape], + "values_loaded": False, + } + if record["object_like"]: + warnings.append( + f"{record['id']} has object-like class {class_text!r}; " + "do not load without expert review" + ) + output.append(record) + return output, warnings + + +def hdf5_inventory( + path: Path, *, max_nodes: int, max_depth: int +) -> tuple[list[dict[str, Any]], list[str], dict[str, int]]: + try: + import h5py + except ImportError as exc: + raise CliError( + "h5py is optional and not installed; use --backend header or " + "install a pinned h5py in an approved environment" + ) from exc + + records: list[dict[str, Any]] = [] + warnings: list[str] = [] + link_counts: Counter[str] = Counter() + seen_objects: set[int] = set() + + def append_record(path_name: str, kind: str, details: dict[str, Any]) -> None: + if len(records) >= max_nodes: + raise CliError(f"HDF5 object/link count exceeds {max_nodes}") + records.append( + { + **_redacted_name(path_name, len(records) + 1, "hdf5-node"), + "kind": kind, + **details, + } + ) + + def walk(group: Any, prefix: str, depth: int) -> None: + if depth > max_depth: + raise CliError(f"HDF5 group depth exceeds {max_depth}") + try: + names = sorted(group.keys()) + except Exception as exc: + raise CliError(f"cannot enumerate HDF5 group metadata: {exc}") from exc + for name in names: + full_name = f"{prefix}/{name}" if prefix else f"/{name}" + try: + link = group.get(name, getlink=True) + except Exception as exc: + raise CliError(f"cannot inspect HDF5 link metadata: {exc}") from exc + if isinstance(link, h5py.SoftLink): + link_counts["soft"] += 1 + append_record( + full_name, + "soft_link", + {"followed": False, "target_emitted": False}, + ) + warnings.append("soft HDF5 link found and not followed") + continue + if isinstance(link, h5py.ExternalLink): + link_counts["external"] += 1 + append_record( + full_name, + "external_link", + {"followed": False, "target_emitted": False}, + ) + warnings.append("external HDF5 link found and not followed") + continue + link_counts["hard"] += 1 + try: + obj = group[name] + object_key = hash(obj.id) + except Exception as exc: + raise CliError(f"cannot inspect HDF5 object metadata: {exc}") from exc + attribute_names = sorted(str(key) for key in obj.attrs.keys()) + if len(attribute_names) > 1000: + raise CliError("HDF5 object has more than 1000 attributes") + base = { + "attribute_count": len(attribute_names), + "attribute_names_emitted": False, + "attribute_name_hashes": [ + hashlib.sha256(name.encode("utf-8")).hexdigest() + for name in attribute_names + ], + "hard_link_revisited": object_key in seen_objects, + "values_loaded": False, + } + if isinstance(obj, h5py.Dataset): + dtype_text = str(obj.dtype) + if len(dtype_text) > 500: + dtype_text = dtype_text[:500] + "..." + reference_dtype = h5py.check_dtype(ref=obj.dtype) is not None + object_like = bool(reference_dtype or obj.dtype.kind == "O") + append_record( + full_name, + "dataset", + { + **base, + "chunks": list(obj.chunks) if obj.chunks is not None else None, + "compression": obj.compression, + "dtype": dtype_text, + "object_like": object_like, + "shape": [int(value) for value in obj.shape], + }, + ) + if object_like: + warnings.append( + "HDF5 object/reference dtype found; values were not read" + ) + seen_objects.add(object_key) + elif isinstance(obj, h5py.Group): + append_record(full_name, "group", base) + if object_key not in seen_objects: + seen_objects.add(object_key) + walk(obj, full_name, depth + 1) + else: + append_record(full_name, "unknown_hdf5_object", base) + warnings.append("unknown HDF5 object type found") + + try: + with h5py.File(path, "r") as handle: + walk(handle, "", 0) + except CliError: + raise + except Exception as exc: + raise CliError(f"h5py could not inventory HDF5 metadata: {exc}") from exc + return records, warnings, dict(sorted(link_counts.items())) + + +def inventory(args: argparse.Namespace) -> dict[str, Any]: + root = checked_root(args.root) + max_file_bytes = bounded_int( + args.max_file_bytes, + name="max_file_bytes", + minimum=1, + maximum=512 * 1024 * 1024, + ) + max_nodes = bounded_int( + args.max_nodes, name="max_nodes", minimum=1, maximum=100_000 + ) + max_depth = bounded_int( + args.max_depth, name="max_depth", minimum=1, maximum=128 + ) + path = checked_input( + args.input, + root=root, + suffixes={".mat"}, + max_bytes=max_file_bytes, + ) + header = identify_header(path) + warnings = [ + "Inventory is metadata triage, not a safety certificate; do not load " + "untrusted MAT/HDF5 content." + ] + backend_used = "header" + records: list[dict[str, Any]] = [] + link_counts: dict[str, int] = {} + backend = args.backend + if backend == "auto": + backend = ( + "hdf5" + if header["detected_kind"] + in {"matlab_v7_3_hdf5", "hdf5_not_confirmed_matlab_v7_3"} + else "scipy" + ) + if backend == "scipy": + if header["detected_kind"] in { + "matlab_v7_3_hdf5", + "hdf5_not_confirmed_matlab_v7_3", + }: + raise CliError("SciPy metadata backend is not used for HDF5/v7.3") + records, extra = scipy_inventory(path, max_nodes=max_nodes) + warnings.extend(extra) + backend_used = "scipy.io.whosmat" + elif backend == "hdf5": + if header["hdf5_signature_offset"] is None: + raise CliError("HDF5 signature was not found at a recognized userblock offset") + records, extra, link_counts = hdf5_inventory( + path, max_nodes=max_nodes, max_depth=max_depth + ) + warnings.extend(extra) + backend_used = "h5py-metadata" + elif backend != "header": + raise CliError(f"unsupported backend: {backend}") + if header["detected_kind"] == "python_pickle_signature_refused": + warnings.append( + "Python pickle signature found under .mat suffix; never deserialize it" + ) + object_like_count = sum(bool(record.get("object_like")) for record in records) + return { + "backend": backend_used, + "deserializes_objects": False, + "executes": False, + "file": relative_id(path, root), + "file_name_emitted": True, + "file_sha256": sha256_file(path, max_bytes=max_file_bytes), + "file_size_bytes": path.stat().st_size, + "header": header, + "hdf5_links": link_counts, + "name_values_emitted": False, + "network_accessed": False, + "object_like_count": object_like_count, + "ok": ( + header["detected_kind"] != "python_pickle_signature_refused" + and object_like_count == 0 + and link_counts.get("external", 0) == 0 + ), + "records": records, + "records_count": len(records), + "safe_to_load": False, + "tool": TOOL, + "values_loaded": False, + "warnings": sorted(set(warnings)), + } + + +def parser() -> argparse.ArgumentParser: + result = argparse.ArgumentParser( + description=( + "Inventory local MAT technical metadata without MATLAB, loadmat, " + "pickle, dataset reads, or object deserialization. Header-only is default." + ) + ) + result.add_argument("input", help="local .mat file") + result.add_argument("--root", default=".", help="allowed local root") + result.add_argument( + "--backend", + choices=("header", "auto", "scipy", "hdf5"), + default="header", + help="optional metadata backend; header is dependency-free and safest", + ) + result.add_argument("--max-file-bytes", default=64 * 1024 * 1024) + result.add_argument("--max-nodes", default=5000) + result.add_argument("--max-depth", default=24) + return result + + +def main() -> int: + try: + report = inventory(parser().parse_args()) + emit_json(report) + return 0 if report["ok"] else 1 + except CliError as exc: + return fail_json(TOOL, exc) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.agents/skills/matlab/scripts/plan_batch_command.py b/.agents/skills/matlab/scripts/plan_batch_command.py new file mode 100644 index 0000000..32facb4 --- /dev/null +++ b/.agents/skills/matlab/scripts/plan_batch_command.py @@ -0,0 +1,257 @@ +#!/usr/bin/env python3 +"""Create a bounded MATLAB or Octave command plan without executing it.""" + +from __future__ import annotations + +import argparse +import math +import re +import sys +from pathlib import Path +from typing import Any + +from _common import ( + CliError, + checked_input, + checked_root, + emit_json, + fail_json, + parse_json_text, + relative_id, + validate_identifier, +) + +TOOL = "plan_batch_command" +SAFE_COMMAND = re.compile(r"^[A-Za-z0-9_.+-]{1,64}$") + + +def matlab_string(value: str) -> str: + if len(value) > 100_000: + raise CliError("MATLAB string literal exceeds 100000 characters") + if any(ord(character) < 32 for character in value): + raise CliError("control characters are not accepted in MATLAB strings") + return '"' + value.replace('"', '""') + '"' + + +def _numeric_row(values: list[Any]) -> str | None: + if not values or not all( + isinstance(item, (bool, int, float)) and not isinstance(item, str) + for item in values + ): + return None + return "[" + " ".join(matlab_literal(item) for item in values) + "]" + + +def matlab_literal(value: Any, *, depth: int = 0) -> str: + if depth > 12: + raise CliError("argument nesting exceeds 12") + if value is None: + return "[]" + if isinstance(value, bool): + return "true" if value else "false" + if isinstance(value, int): + if abs(value) > 2**53: + raise CliError("JSON integers outside exact MATLAB double range are refused") + return str(value) + if isinstance(value, float): + if not math.isfinite(value): + raise CliError("nonfinite JSON numbers are refused") + return format(value, ".17g") + if isinstance(value, str): + return matlab_string(value) + if isinstance(value, list): + row = _numeric_row(value) + if row is not None: + return row + if value and all(isinstance(item, list) for item in value): + rows = [_numeric_row(item) for item in value] + if all(row_value is not None for row_value in rows): + widths = {len(item) for item in value} + if len(widths) == 1: + return "[" + "; ".join( + row_value[1:-1] for row_value in rows if row_value + ) + "]" + if value and all(isinstance(item, str) for item in value): + return "[" + " ".join(matlab_string(item) for item in value) + "]" + return "{" + ", ".join( + matlab_literal(item, depth=depth + 1) for item in value + ) + "}" + if isinstance(value, dict): + fields: list[str] = [] + for key in sorted(value): + validate_identifier(key, name="JSON object field") + fields.extend( + [ + matlab_string(key), + matlab_literal(value[key], depth=depth + 1), + ] + ) + return "struct(" + ", ".join(fields) + ")" + raise CliError(f"unsupported argument type: {type(value).__name__}") + + +def _executable(value: str, engine: str) -> str: + default = "matlab" if engine == "matlab" else "octave" + command = value or default + if not SAFE_COMMAND.fullmatch(command): + raise CliError( + "executable must be a bare command name; edit the reviewed argv " + "manually for an absolute installation path" + ) + return command + + +def build_plan(args: argparse.Namespace) -> dict[str, Any]: + root = checked_root(args.root) + target = checked_input( + args.target, + root=root, + suffixes={".m"}, + max_bytes=args.max_input_bytes, + ) + executable = _executable(args.executable, args.engine) + function_name = args.function_name or target.stem + validate_identifier(function_name, name="function name") + if args.mode == "function" and function_name != target.stem: + raise CliError("function name must match the target .m filename") + if args.arg_json and args.mode != "function": + raise CliError("--arg-json is accepted only in function mode") + literals = [ + matlab_literal(parse_json_text(argument)) for argument in args.arg_json + ] + target_literal = matlab_string(str(target)) + warnings = [ + "This plan does not execute or prove the target safe.", + "Confirm the exact runtime, products, licenses, startup behavior, " + "inputs, outputs, and external effects before execution.", + ] + argv: list[str] + statement: str | None + if args.engine == "matlab": + argv = [executable] + if args.disable_graphics: + argv.append("-noFigureWindows") + startup = target.parent if args.mode == "function" else root + argv.extend(["-sd", str(startup)]) + if args.mode == "script": + statement = f"run({target_literal})" + elif args.mode == "function": + statement = f"{function_name}({', '.join(literals)})" + else: + statement = ( + f"results = runtests({target_literal}); assertSuccess(results)" + ) + warnings.append( + "R2026a runtests can automatically open and close a containing " + "project, including reviewed startup/shutdown actions." + ) + argv.extend(["-batch", statement]) + warnings.append( + "MATLAB -batch still processes launcher/startup configuration; " + "-sd is not isolation." + ) + prerequisites = [ + "Installed MATLAB compatible with the reviewed source", + "Confirmed MATLAB and required-product license availability", + ] + else: + argv = [ + executable, + "--no-init-all", + "--no-history", + "--quiet", + "--no-gui", + ] + if args.disable_graphics: + argv.append("--no-window-system") + statement = None + if args.mode == "script": + argv.append(str(target)) + elif args.mode == "function": + statement = f"{function_name}({', '.join(literals)})" + argv.extend(["--path", str(target.parent), "--eval", statement]) + else: + statement = ( + f"success = test({target_literal}, " + f'{matlab_string("quiet")}); assert(success)' + ) + argv.extend(["--eval", statement]) + warnings.append( + "Octave BIST test is not MATLAB matlab.unittest compatibility." + ) + prerequisites = [ + "Installed GNU Octave compatible with the reviewed source", + "Confirmed required Octave package availability", + ] + return { + "arguments": { + "count": len(literals), + "json_arrays": "numeric rectangular arrays map to MATLAB arrays; " + "other arrays map to cells", + }, + "command_argv": argv, + "disable_graphics": bool(args.disable_graphics), + "engine": args.engine, + "executes": False, + "mode": args.mode, + "network_accessed": False, + "ok": True, + "prerequisites": prerequisites, + "root": str(root), + "statement": statement, + "target": relative_id(target, root), + "tool": TOOL, + "warnings": warnings, + } + + +def parser() -> argparse.ArgumentParser: + result = argparse.ArgumentParser( + description=( + "Plan a MATLAB -batch or GNU Octave command. The tool validates " + "local paths and JSON literals but never launches either runtime." + ) + ) + result.add_argument("engine", choices=("matlab", "octave")) + result.add_argument("mode", choices=("script", "function", "tests")) + result.add_argument("target", help="reviewed local .m file") + result.add_argument("--root", default=".", help="allowed local root") + result.add_argument( + "--arg-json", + action="append", + default=[], + help="one bounded JSON function argument; repeat as needed", + ) + result.add_argument("--function-name", help="must match target stem") + result.add_argument( + "--executable", + default="", + help="bare command name only (default: matlab or octave)", + ) + result.add_argument( + "--disable-graphics", + action="store_true", + help="plan no figure windows/window system", + ) + result.add_argument( + "--max-input-bytes", + type=int, + default=4 * 1024 * 1024, + help="maximum target .m bytes (default: 4194304)", + ) + return result + + +def main() -> int: + try: + args = parser().parse_args() + if args.max_input_bytes < 1 or args.max_input_bytes > 64 * 1024 * 1024: + raise CliError("--max-input-bytes must be between 1 and 67108864") + emit_json(build_plan(args)) + return 0 + except CliError as exc: + return fail_json(TOOL, exc) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.agents/skills/matlab/scripts/plan_python_compatibility.py b/.agents/skills/matlab/scripts/plan_python_compatibility.py new file mode 100644 index 0000000..248cd1b --- /dev/null +++ b/.agents/skills/matlab/scripts/plan_python_compatibility.py @@ -0,0 +1,176 @@ +#!/usr/bin/env python3 +"""Plan MATLAB R2026a Python compatibility without importing or starting Engine.""" + +from __future__ import annotations + +import argparse +import re +import sys +from pathlib import Path +from typing import Any + +from _common import ( + CliError, + checked_input, + checked_root, + emit_json, + fail_json, + load_json, + validate_release, +) + +TOOL = "plan_python_compatibility" +PYTHON_VERSION = re.compile(r"^([0-9]{1,2})\.([0-9]{1,2})(?:\.[0-9]{1,3})?$") +PACKAGE_VERSION = re.compile(r"^[0-9]+\.[0-9]+\.[0-9]+$") + + +def compatibility_data() -> dict[str, Any]: + skill_root = checked_root(Path(__file__).resolve().parents[1]) + path = checked_input( + skill_root / "assets" / "python_compatibility_r2026a.json", + root=skill_root, + suffixes={".json"}, + ) + value = load_json(path) + if not isinstance(value, dict) or value.get("schema_version") != "1.0": + raise CliError("bundled compatibility data is invalid") + return value + + +def normalize_python(value: str) -> tuple[str, str]: + match = PYTHON_VERSION.fullmatch(value) + if not match: + raise CliError("Python version must look like 3.13 or 3.13.5") + major_minor = f"{int(match.group(1))}.{int(match.group(2))}" + return value, major_minor + + +def plan(args: argparse.Namespace) -> tuple[dict[str, Any], int]: + release = validate_release(args.matlab_release) + data = compatibility_data() + if release != data["matlab_release"]: + raise CliError( + "bundled compatibility data is pinned only to R2026a; consult the " + "official support table for another release" + ) + requested_python, major_minor = normalize_python(args.python_version) + engine_version = args.engine_version + if not PACKAGE_VERSION.fullmatch(engine_version): + raise CliError("Engine version must be a three-part numeric version") + supported = major_minor in data["supported_python_versions"] + exact_engine = engine_version == data["matlab_engine_package"]["version"] + bits_ok = args.python_bits == data["python_architecture_bits"] + implementation_ok = args.implementation == data["python_implementation"] + software_compatible = supported and exact_engine and bits_ok and implementation_ok + installed_ok = args.matlab_installed == "yes" + license_ok = args.license_status == "confirmed" + ready_to_launch = software_compatible and installed_ok and license_ok + warnings = [ + "This static plan does not inspect PATH, PYTHONPATH, sys.path, the " + "environment, installations, credentials, or licenses.", + "Package installation does not install MATLAB or grant a license.", + "MATLAB Runtime cannot host MATLAB Engine for Python.", + ] + if args.matlab_installed == "unknown": + warnings.append("Installed MATLAB R2026a has not been confirmed.") + if args.license_status == "unknown": + warnings.append("MATLAB and required-product license availability is unknown.") + if not supported: + warnings.append( + f"CPython {major_minor} is outside the R2026a supported set." + ) + if not exact_engine: + warnings.append( + "Engine package version does not match the reviewed R2026a package." + ) + if not bits_ok: + warnings.append("R2026a Engine requires matching 64-bit Python.") + if not implementation_ok: + warnings.append("The reviewed interface supports CPython.") + launch_snippet = None + if args.include_launch_snippet: + launch_snippet = [ + "import matlab.engine", + "engine = matlab.engine.start_matlab() # explicit MATLAB launch/license action", + "try:", + " pass # call one reviewed function with explicit nargout", + "finally:", + " engine.quit()", + ] + warnings.append( + "Launch snippet is informational and was not executed; review MATLAB " + "startup code and obtain explicit approval before using it." + ) + report = { + "as_of": data["as_of"], + "engine_launch_explicit": bool(args.include_launch_snippet), + "engine_launch_snippet": launch_snippet, + "executes": False, + "implementation": args.implementation, + "install_plan_argv": [ + "uv", + "pip", + "install", + f"matlabengine=={data['matlab_engine_package']['version']}", + ], + "license_status": args.license_status, + "matlab_installed": args.matlab_installed, + "matlab_release": release, + "network_accessed": False, + "ok": software_compatible, + "preinstalled_engine_path": ( + "/" + data["preinstalled_engine_relative_path"] + ), + "python_architecture_bits": args.python_bits, + "python_version_requested": requested_python, + "ready_to_launch": ready_to_launch, + "reviewed_engine_package": data["matlab_engine_package"], + "software_compatible": software_compatible, + "supported_python_versions": data["supported_python_versions"], + "tool": TOOL, + "warnings": warnings, + } + return report, 0 if software_compatible else 1 + + +def parser() -> argparse.ArgumentParser: + result = argparse.ArgumentParser( + description=( + "Check a requested CPython and MATLAB Engine version against the " + "bundled R2026a support record. No environment probing or launch occurs." + ) + ) + result.add_argument("--matlab-release", default="R2026a") + result.add_argument("--python-version", required=True) + result.add_argument("--engine-version", default="26.1.12") + result.add_argument("--python-bits", type=int, choices=(32, 64), default=64) + result.add_argument("--implementation", choices=("CPython",), default="CPython") + result.add_argument( + "--matlab-installed", + choices=("yes", "no", "unknown"), + default="unknown", + ) + result.add_argument( + "--license-status", + choices=("confirmed", "unavailable", "unknown"), + default="unknown", + ) + result.add_argument( + "--include-launch-snippet", + action="store_true", + help="include, but do not execute, an explicit Engine lifecycle snippet", + ) + return result + + +def main() -> int: + try: + report, status = plan(parser().parse_args()) + emit_json(report) + return status + except CliError as exc: + return fail_json(TOOL, exc) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.agents/skills/matlab/scripts/reproducibility_report.py b/.agents/skills/matlab/scripts/reproducibility_report.py new file mode 100644 index 0000000..3ac083f --- /dev/null +++ b/.agents/skills/matlab/scripts/reproducibility_report.py @@ -0,0 +1,233 @@ +#!/usr/bin/env python3 +"""Create a deterministic named-file reproducibility report.""" + +from __future__ import annotations + +import argparse +import json +import math +import re +import sys +from pathlib import Path +from typing import Any + +from _common import ( + CliError, + checked_input, + checked_output, + checked_root, + emit_json, + fail_json, + load_json, + relative_id, + sha256_file, + validate_release, + write_new_text, +) + +TOOL = "reproducibility_report" +OCTAVE_VERSION = re.compile(r"^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$") +ALLOWED_SUFFIXES = { + ".csv", + ".fig", + ".h5", + ".hdf5", + ".json", + ".m", + ".mat", + ".mlx", + ".pdf", + ".png", + ".svg", + ".tsv", + ".txt", +} + + +def _finite_nonnegative(value: float | None, name: str) -> float | None: + if value is None: + return None + if not math.isfinite(value) or value < 0: + raise CliError(f"{name} must be finite and nonnegative") + return value + + +def _runtime_version(runtime: str, version: str) -> str: + if runtime == "matlab": + return validate_release(version) + if not OCTAVE_VERSION.fullmatch(version): + raise CliError("Octave runtime version must be a three-part numeric version") + return version + + +def _named_fact(value: str | None, name: str, *, maximum: int = 500) -> str | None: + if value is None: + return None + if not value or len(value) > maximum or any( + ord(character) < 32 for character in value + ): + raise CliError(f"{name} must be bounded printable text") + return value + + +def build(args: argparse.Namespace) -> dict[str, Any]: + root = checked_root(args.root) + if len(args.file) > 200: + raise CliError("at most 200 named files are accepted") + if not args.file: + raise CliError("provide at least one --file") + files = [ + checked_input( + value, + root=root, + suffixes=ALLOWED_SUFFIXES, + max_bytes=args.max_file_bytes, + ) + for value in args.file + ] + ids = [relative_id(path, root) for path in files] + if len(set(ids)) != len(ids): + raise CliError("--file paths must be unique") + file_records = [ + { + "path": relative_id(path, root), + "sha256": sha256_file(path, max_bytes=args.max_file_bytes), + "size_bytes": path.stat().st_size, + } + for path in sorted(files, key=lambda item: relative_id(item, root)) + ] + product_manifest: dict[str, Any] | None = None + if args.product_manifest: + path = checked_input( + args.product_manifest, + root=root, + suffixes={".json"}, + max_bytes=2 * 1024 * 1024, + ) + data = load_json(path) + if not isinstance(data, dict) or data.get("schema_version") != "1.0": + raise CliError("product manifest must be a schema_version 1.0 object") + product_manifest = { + "path": relative_id(path, root), + "sha256": sha256_file(path, max_bytes=2 * 1024 * 1024), + "license_status_verified": False, + } + command_plan: dict[str, Any] | None = None + if args.command_plan: + path = checked_input( + args.command_plan, + root=root, + suffixes={".json"}, + max_bytes=2 * 1024 * 1024, + ) + data = load_json(path) + if not isinstance(data, dict) or data.get("executes") is not False: + raise CliError("command plan must be a nonexecuting JSON plan") + command_plan = { + "path": relative_id(path, root), + "sha256": sha256_file(path, max_bytes=2 * 1024 * 1024), + } + runtime_version = _runtime_version(args.runtime, args.runtime_version) + abs_tol = _finite_nonnegative(args.absolute_tolerance, "absolute_tolerance") + rel_tol = _finite_nonnegative(args.relative_tolerance, "relative_tolerance") + if (abs_tol is not None or rel_tol is not None) and not args.tolerance_rationale: + raise CliError( + "--tolerance-rationale is required when a numeric tolerance is recorded" + ) + if args.rng_seed is not None and not 0 <= args.rng_seed <= 2**32 - 1: + raise CliError("--rng-seed must be between 0 and 4294967295") + if args.rng_substream is not None and not 1 <= args.rng_substream <= 2**31 - 1: + raise CliError("--rng-substream must be between 1 and 2147483647") + rng_algorithm = _named_fact(args.rng_algorithm, "rng_algorithm", maximum=100) + tolerance_rationale = _named_fact( + args.tolerance_rationale, "tolerance_rationale", maximum=2000 + ) + platform_os = _named_fact(args.platform_os, "platform_os", maximum=200) + platform_arch = _named_fact(args.platform_arch, "platform_arch", maximum=200) + return { + "command_plan": command_plan, + "environment_dumped": False, + "executes": False, + "named_files": file_records, + "network_accessed": False, + "numeric_policy": { + "absolute_tolerance": abs_tol, + "relative_tolerance": rel_tol, + "rationale": tolerance_rationale, + }, + "ok": True, + "platform": { + "architecture": platform_arch, + "operating_system": platform_os, + "source": "caller-supplied; not probed", + }, + "products_manifest": product_manifest, + "randomness": { + "algorithm": rng_algorithm, + "seed": args.rng_seed, + "substream": args.rng_substream, + }, + "root_emitted": False, + "runtime": args.runtime, + "runtime_version": runtime_version, + "schema_version": "1.0", + "tool": TOOL, + "warning": ( + "Hashes and named facts support provenance but do not establish " + "safety, scientific validity, or license availability." + ), + } + + +def parser() -> argparse.ArgumentParser: + result = argparse.ArgumentParser( + description=( + "Hash only named local artifacts and emit a deterministic MATLAB/" + "Octave reproducibility report. No environment or runtime probe occurs." + ) + ) + result.add_argument("--root", default=".", help="allowed local root") + result.add_argument( + "--file", action="append", default=[], help="named local artifact; repeat" + ) + result.add_argument("--runtime", choices=("matlab", "octave"), default="matlab") + result.add_argument("--runtime-version", default="R2026a") + result.add_argument("--product-manifest") + result.add_argument("--command-plan") + result.add_argument("--rng-algorithm") + result.add_argument("--rng-seed", type=int) + result.add_argument("--rng-substream", type=int) + result.add_argument("--absolute-tolerance", type=float) + result.add_argument("--relative-tolerance", type=float) + result.add_argument("--tolerance-rationale") + result.add_argument("--platform-os", default="unknown") + result.add_argument("--platform-arch", default="unknown") + result.add_argument("--max-file-bytes", type=int, default=64 * 1024 * 1024) + result.add_argument("--output", help="optional new .json output under root") + return result + + +def main() -> int: + try: + args = parser().parse_args() + if args.max_file_bytes < 1 or args.max_file_bytes > 512 * 1024 * 1024: + raise CliError("--max-file-bytes must be between 1 and 536870912") + report = build(args) + if args.output: + root = checked_root(args.root) + output = checked_output(args.output, root=root, suffixes={".json"}) + write_new_text( + output, + json.dumps( + report, indent=2, sort_keys=True, ensure_ascii=False + ) + + "\n", + ) + emit_json(report) + return 0 + except CliError as exc: + return fail_json(TOOL, exc) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.agents/skills/matlab/scripts/scan_m_code.py b/.agents/skills/matlab/scripts/scan_m_code.py new file mode 100644 index 0000000..b0d3145 --- /dev/null +++ b/.agents/skills/matlab/scripts/scan_m_code.py @@ -0,0 +1,433 @@ +#!/usr/bin/env python3 +"""Bounded static risk triage for MATLAB source and opaque artifacts.""" + +from __future__ import annotations + +import argparse +import re +import sys +from collections import Counter +from pathlib import Path +from typing import Any + +from _common import ( + CliError, + bounded_int, + checked_input, + checked_root, + emit_json, + fail_json, + read_text, + relative_id, +) + +TOOL = "scan_m_code" +SEVERITY = {"low": 1, "medium": 2, "high": 3, "critical": 4} +SOURCE_SUFFIX = ".m" +OPAQUE_SUFFIXES = { + ".fig": ("opaque_figure", "high", "MATLAB figure object file is opaque."), + ".mat": ("mat_file", "high", "MAT file can contain objects and callbacks."), + ".mlapp": ("opaque_app", "high", "MATLAB app archive is opaque and executable."), + ".mlx": ("opaque_live_script", "high", "Live script archive is opaque."), + ".p": ("protected_code", "high", "Protected MATLAB code is not reviewable."), + ".prj": ("project_file", "medium", "Project metadata can configure actions."), + ".slx": ("simulink_model", "high", "Simulink model archive can execute callbacks."), + ".mdl": ("simulink_model", "high", "Simulink model can execute callbacks."), +} +RULES: tuple[tuple[str, str, str, re.Pattern[str]], ...] = ( + ( + "dynamic_eval", + "critical", + "Dynamic evaluation can execute text as MATLAB code.", + re.compile(r"\b(?:eval|evalin)\s*\(", re.IGNORECASE), + ), + ( + "workspace_injection", + "high", + "assignin mutates another workspace and can hide data flow.", + re.compile(r"\bassignin\s*\(", re.IGNORECASE), + ), + ( + "text_dispatch", + "high", + "Review feval/str2func inputs; text-derived dispatch executes code.", + re.compile(r"\b(?:feval|str2func)\s*\(", re.IGNORECASE), + ), + ( + "shell_execution", + "critical", + "Shell entry point can execute external commands.", + re.compile(r"(?:^\s*!|\b(?:system|unix|dos)\s*\()", re.IGNORECASE), + ), + ( + "python_execution", + "high", + "Python integration can execute Python/native code.", + re.compile(r"\b(?:pyrun|pyrunfile|pyenv)\s*\(|\bpy\.", re.IGNORECASE), + ), + ( + "java_execution", + "high", + "Java integration or class-path mutation crosses the runtime boundary.", + re.compile( + r"\b(?:javaObject|javaMethod|javaaddpath|javarmpath|javaclasspath)" + r"\s*\(|\bjava\.", + re.IGNORECASE, + ), + ), + ( + "dotnet_execution", + "high", + ".NET assembly/type access can execute external code.", + re.compile(r"\bNET\.addAssembly\s*\(|\bSystem\.", re.IGNORECASE), + ), + ( + "native_execution", + "critical", + "Native library or MEX entry point can execute process-native code.", + re.compile( + r"\b(?:mex|loadlibrary|calllib|libpointer|javaaddpath)\s*\(", + re.IGNORECASE, + ), + ), + ( + "code_generation", + "high", + "Code generation/build invokes separate products and toolchains.", + re.compile(r"\b(?:codegen|mcc|compiler\.build)\s*\(", re.IGNORECASE), + ), + ( + "mat_load", + "high", + "Loading MAT/object data can invoke installed class deserialization code.", + re.compile(r"(?:^\s*load(?:\s|\()|\bload\s*\()", re.IGNORECASE), + ), + ( + "deserialization_callback", + "critical", + "Object load callback/custom serialization executes during restoration.", + re.compile( + r"\bloadobj\s*\(|\bloadObjectImpl\s*\(|" + r"matlab\.mixin\.CustomElementSerialization", + re.IGNORECASE, + ), + ), + ( + "network_io", + "high", + "Network API can transmit data or retrieve executable/untrusted content.", + re.compile( + r"\b(?:webread|webwrite|websave|urlread|urlwrite|tcpclient|udpport)" + r"\s*\(", + re.IGNORECASE, + ), + ), + ( + "callback", + "medium", + "Review callback provenance and lifecycle.", + re.compile( + r"\b(?:addlistener|timer)\s*\(|" + r"\b(?:Callback|TimerFcn|StartFcn|StopFcn)\b", + re.IGNORECASE, + ), + ), + ( + "path_mutation", + "medium", + "Path mutation can shadow trusted functions.", + re.compile( + r"\b(?:addpath|rmpath|path|userpath|savepath|genpath)\s*\(", + re.IGNORECASE, + ), + ), + ( + "file_mutation", + "medium", + "File mutation requires reviewed local paths and collision policy.", + re.compile( + r"\b(?:delete|movefile|copyfile|mkdir|rmdir|fopen|diary)\s*\(", + re.IGNORECASE, + ), + ), + ( + "serialization_write", + "medium", + "Save/export can overwrite files or serialize executable objects.", + re.compile(r"\b(?:save|savefig|exportgraphics|writetable)\s*\(", re.IGNORECASE), + ), + ( + "interactive_input", + "medium", + "Interactive input/dialog can hang or fail in batch mode.", + re.compile( + r"\b(?:input|uigetfile|uiputfile|questdlg|inputdlg)\s*\(", + re.IGNORECASE, + ), + ), + ( + "broad_clear", + "low", + "Broad clearing hides workspace/function-state dependencies.", + re.compile(r"\bclear\s+all\b", re.IGNORECASE), + ), +) + + +def _without_comments(line: str, *, in_block: bool) -> tuple[str, bool]: + stripped = line.lstrip() + if in_block: + if stripped.startswith("%}"): + return "", False + return "", True + if stripped.startswith("%{"): + return "", True + single = False + double = False + index = 0 + while index < len(line): + char = line[index] + if char == "'" and not double: + if single and index + 1 < len(line) and line[index + 1] == "'": + index += 2 + continue + single = not single + elif char == '"' and not single: + if double and index + 1 < len(line) and line[index + 1] == '"': + index += 2 + continue + double = not double + elif char == "%" and not single and not double: + return line[:index], False + index += 1 + return line, False + + +def scan_source( + path: Path, root: Path, *, max_file_bytes: int +) -> list[dict[str, Any]]: + findings: list[dict[str, Any]] = [] + text = read_text(path, max_bytes=max_file_bytes) + in_block = False + for line_number, line in enumerate(text.splitlines(), start=1): + code, in_block = _without_comments(line, in_block=in_block) + for rule, severity, message, pattern in RULES: + for match in pattern.finditer(code): + findings.append( + { + "column": match.start() + 1, + "file": relative_id(path, root), + "line": line_number, + "message": message, + "rule": rule, + "severity": severity, + } + ) + if path.name.casefold() in {"startup.m", "finish.m", "pathdef.m"}: + findings.append( + { + "column": 1, + "file": relative_id(path, root), + "line": 1, + "message": "Lifecycle/path file can execute implicitly.", + "rule": "implicit_lifecycle_file", + "severity": "high", + } + ) + return findings + + +def _opaque_finding(path: Path, root: Path) -> dict[str, Any] | None: + suffix = path.suffix.casefold() + details = OPAQUE_SUFFIXES.get(suffix) + if suffix.startswith(".mex"): + details = ( + "mex_binary", + "critical", + "MEX file is unreviewed native executable code.", + ) + if details is None: + return None + rule, severity, message = details + return { + "column": None, + "file": relative_id(path, root), + "line": None, + "message": message, + "rule": rule, + "severity": severity, + } + + +def collect( + source: Path, + *, + root: Path, + recursive: bool, + max_files: int, + max_file_bytes: int, + max_total_bytes: int, +) -> list[Path]: + if source.is_file(): + suffix = source.suffix.casefold() + supported = ( + suffix == SOURCE_SUFFIX + or suffix in OPAQUE_SUFFIXES + or suffix.startswith(".mex") + ) + if not supported: + raise CliError("input file is not .m or a recognized opaque artifact") + size = source.stat().st_size + if size > max_file_bytes or size > max_total_bytes: + raise CliError( + f"file is {size} bytes; per-file/total limits are " + f"{max_file_bytes}/{max_total_bytes}" + ) + return [source] + output: list[Path] = [] + stack = [source] + total_bytes = 0 + while stack: + directory = stack.pop() + try: + entries = sorted(directory.iterdir(), key=lambda item: item.name.casefold()) + except OSError as exc: + raise CliError(f"cannot read directory: {directory}") from exc + for entry in entries: + if entry.is_symlink(): + raise CliError(f"symlink encountered during scan: {entry}") + if entry.is_dir(): + if recursive: + stack.append(entry) + continue + if not entry.is_file(): + continue + suffix = entry.suffix.casefold() + if suffix != SOURCE_SUFFIX and suffix not in OPAQUE_SUFFIXES and not ( + suffix.startswith(".mex") + ): + continue + size = entry.stat().st_size + if size > max_file_bytes: + raise CliError(f"file is {size} bytes; limit is {max_file_bytes}") + total_bytes += size + if total_bytes > max_total_bytes: + raise CliError( + f"selected files total {total_bytes} bytes; " + f"limit is {max_total_bytes}" + ) + output.append(entry) + if len(output) > max_files: + raise CliError(f"more than {max_files} relevant files found") + return sorted(output) + + +def scan(args: argparse.Namespace) -> tuple[dict[str, Any], int]: + root = checked_root(args.root) + source = checked_input(args.input, root=root, kind="any") + max_files = bounded_int( + args.max_files, name="max_files", minimum=1, maximum=5000 + ) + max_file_bytes = bounded_int( + args.max_file_bytes, + name="max_file_bytes", + minimum=1, + maximum=64 * 1024 * 1024, + ) + max_total_bytes = bounded_int( + args.max_total_bytes, + name="max_total_bytes", + minimum=1, + maximum=256 * 1024 * 1024, + ) + files = collect( + source, + root=root, + recursive=args.recursive, + max_files=max_files, + max_file_bytes=max_file_bytes, + max_total_bytes=max_total_bytes, + ) + findings: list[dict[str, Any]] = [] + scanned_text = 0 + opaque = 0 + for path in files: + if path.suffix.casefold() == ".m": + scanned_text += 1 + findings.extend( + scan_source(path, root, max_file_bytes=max_file_bytes) + ) + else: + finding = _opaque_finding(path, root) + if finding: + opaque += 1 + findings.append(finding) + findings.sort( + key=lambda item: ( + item["file"], + item["line"] if item["line"] is not None else 0, + item["column"] if item["column"] is not None else 0, + item["rule"], + ) + ) + counts = Counter(item["severity"] for item in findings) + threshold = 5 if args.fail_on == "none" else SEVERITY[args.fail_on] + failing = sum(SEVERITY[item["severity"]] >= threshold for item in findings) + report = { + "content_emitted": False, + "executes": False, + "fail_on": args.fail_on, + "files_considered": len(files), + "findings": findings, + "network_accessed": False, + "ok": failing == 0, + "opaque_files": opaque, + "static_only": True, + "summary": { + severity: counts.get(severity, 0) + for severity in ("critical", "high", "medium", "low") + }, + "text_files_scanned": scanned_text, + "tool": TOOL, + "warning": ( + "Static pattern triage can miss dynamic behavior and can produce " + "false positives; never treat a clean report as permission to execute." + ), + } + return report, 1 if failing else 0 + + +def parser() -> argparse.ArgumentParser: + result = argparse.ArgumentParser( + description=( + "Statically triage reviewed MATLAB text and flag opaque artifacts. " + "No MATLAB/Octave runtime is launched." + ) + ) + result.add_argument("input", help="local .m file or directory") + result.add_argument("--root", default=".", help="allowed local root") + result.add_argument( + "--recursive", action="store_true", help="scan nested directories" + ) + result.add_argument( + "--fail-on", + choices=("critical", "high", "medium", "low", "none"), + default="high", + ) + result.add_argument("--max-files", default=500) + result.add_argument("--max-file-bytes", default=4 * 1024 * 1024) + result.add_argument("--max-total-bytes", default=32 * 1024 * 1024) + return result + + +def main() -> int: + try: + report, status = scan(parser().parse_args()) + emit_json(report) + return status + except CliError as exc: + return fail_json(TOOL, exc) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.agents/skills/matlab/scripts/validate_project_manifest.py b/.agents/skills/matlab/scripts/validate_project_manifest.py new file mode 100644 index 0000000..0f4523d --- /dev/null +++ b/.agents/skills/matlab/scripts/validate_project_manifest.py @@ -0,0 +1,348 @@ +#!/usr/bin/env python3 +"""Validate a bounded MATLAB/Octave project and product manifest.""" + +from __future__ import annotations + +import argparse +import re +import sys +from pathlib import Path +from typing import Any + +from _common import ( + CliError, + checked_input, + checked_root, + emit_json, + fail_json, + load_json, + relative_id, + validate_release, +) + +TOOL = "validate_project_manifest" +TOP_LEVEL = { + "schema_version", + "project_name", + "runtime", + "matlab_release", + "octave_version", + "entry_points", + "test_paths", + "required_products", + "optional_products", + "octave_packages", + "startup_actions", + "shutdown_actions", + "external_interfaces", + "generated_artifacts", + "notes", +} +REQUIRED = { + "schema_version", + "project_name", + "runtime", + "entry_points", + "test_paths", + "required_products", + "optional_products", + "octave_packages", + "startup_actions", + "shutdown_actions", + "external_interfaces", + "generated_artifacts", + "notes", +} +PRODUCT_KEYS = {"name", "purpose", "minimum_release", "license_status"} +ENTRY_KEYS = {"path", "kind"} +OCTAVE_VERSION = re.compile(r"^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$") +SHA256 = re.compile(r"^[0-9a-f]{64}$") +SAFE_NAME = re.compile(r"^[A-Za-z0-9][A-Za-z0-9 ._+()/-]{0,126}$") + + +def _string(value: Any, name: str, *, maximum: int = 500) -> str: + if not isinstance(value, str) or not value or len(value) > maximum: + raise CliError(f"{name} must be a nonempty string up to {maximum} characters") + if any(ord(character) < 32 for character in value): + raise CliError(f"{name} contains control characters") + return value + + +def _list(value: Any, name: str, *, maximum: int = 500) -> list[Any]: + if not isinstance(value, list): + raise CliError(f"{name} must be an array") + if len(value) > maximum: + raise CliError(f"{name} has {len(value)} entries; limit is {maximum}") + return value + + +def _validate_product( + value: Any, *, field: str, index: int, warnings: list[str] +) -> str: + if not isinstance(value, dict) or set(value) != PRODUCT_KEYS: + raise CliError( + f"{field}[{index}] must contain exactly {sorted(PRODUCT_KEYS)}" + ) + name = _string(value["name"], f"{field}[{index}].name", maximum=127) + if not SAFE_NAME.fullmatch(name): + raise CliError(f"{field}[{index}].name contains unsupported characters") + _string(value["purpose"], f"{field}[{index}].purpose", maximum=1000) + release = value["minimum_release"] + if release is not None: + validate_release(_string(release, f"{field}[{index}].minimum_release")) + status = value["license_status"] + if status not in {"unknown", "confirmed", "unavailable"}: + raise CliError( + f"{field}[{index}].license_status must be unknown, confirmed, " + "or unavailable" + ) + if status == "confirmed": + warnings.append( + f"{name}: manifest says license confirmed; validator cannot verify " + "installation, entitlement, or checkout availability" + ) + return name + + +def _validate_path( + raw: Any, + *, + root: Path, + name: str, + allow_missing: bool, + kind: str = "any", + suffixes: set[str] | None = None, +) -> str: + text = _string(raw, name) + if allow_missing: + candidate = Path(text) + if ( + candidate.is_absolute() + or ".." in candidate.parts + or "://" in text + or "\x00" in text + ): + raise CliError(f"{name} must be a relative path without traversal") + if suffixes and candidate.suffix.casefold() not in suffixes: + raise CliError(f"{name} has unsupported suffix") + return candidate.as_posix() + path = checked_input(text, root=root, kind=kind, suffixes=suffixes) + return relative_id(path, root) + + +def validate(args: argparse.Namespace) -> tuple[dict[str, Any], int]: + root = checked_root(args.root) + manifest_path = checked_input( + args.manifest, root=root, suffixes={".json"}, max_bytes=2 * 1024 * 1024 + ) + data = load_json(manifest_path) + if not isinstance(data, dict): + raise CliError("manifest root must be a JSON object") + unknown = set(data) - TOP_LEVEL + missing = REQUIRED - set(data) + if unknown: + raise CliError(f"unknown manifest fields: {sorted(unknown)}") + if missing: + raise CliError(f"missing manifest fields: {sorted(missing)}") + if data["schema_version"] != "1.0": + raise CliError("schema_version must be 1.0") + project_name = _string(data["project_name"], "project_name", maximum=127) + if not SAFE_NAME.fullmatch(project_name): + raise CliError("project_name contains unsupported characters") + runtime = data["runtime"] + if runtime not in {"matlab", "octave", "both"}: + raise CliError("runtime must be matlab, octave, or both") + matlab_release = data.get("matlab_release") + octave_version = data.get("octave_version") + if runtime in {"matlab", "both"}: + validate_release(_string(matlab_release, "matlab_release")) + elif matlab_release is not None: + validate_release(_string(matlab_release, "matlab_release")) + if runtime in {"octave", "both"}: + version = _string(octave_version, "octave_version") + if not OCTAVE_VERSION.fullmatch(version): + raise CliError("octave_version must be a three-part numeric version") + elif octave_version is not None: + version = _string(octave_version, "octave_version") + if not OCTAVE_VERSION.fullmatch(version): + raise CliError("octave_version must be a three-part numeric version") + + warnings: list[str] = [] + entry_ids: list[str] = [] + for index, entry in enumerate(_list(data["entry_points"], "entry_points", maximum=100)): + if not isinstance(entry, dict) or set(entry) != ENTRY_KEYS: + raise CliError( + f"entry_points[{index}] must contain exactly {sorted(ENTRY_KEYS)}" + ) + kind = entry["kind"] + if kind not in {"function", "script", "live_script"}: + raise CliError( + f"entry_points[{index}].kind must be function, script, or live_script" + ) + suffixes = {".mlx"} if kind == "live_script" else {".m"} + entry_id = _validate_path( + entry["path"], + root=root, + name=f"entry_points[{index}].path", + allow_missing=args.allow_missing_paths, + kind="file", + suffixes=suffixes, + ) + entry_ids.append(entry_id) + if kind == "live_script": + warnings.append( + f"{entry_id}: live script is opaque; export reviewed code to .m " + "before execution" + ) + if len(set(entry_ids)) != len(entry_ids): + raise CliError("entry point paths must be unique") + + test_ids = [ + _validate_path( + value, + root=root, + name=f"test_paths[{index}]", + allow_missing=args.allow_missing_paths, + kind="any", + ) + for index, value in enumerate( + _list(data["test_paths"], "test_paths", maximum=100) + ) + ] + if len(set(test_ids)) != len(test_ids): + raise CliError("test paths must be unique") + + product_names: list[str] = [] + for field in ("required_products", "optional_products"): + for index, value in enumerate(_list(data[field], field, maximum=200)): + product_names.append( + _validate_product( + value, field=field, index=index, warnings=warnings + ).casefold() + ) + if len(set(product_names)) != len(product_names): + raise CliError("product names must be unique across required and optional lists") + required_names = { + value["name"].casefold() for value in data["required_products"] + } + if runtime in {"matlab", "both"} and "matlab" not in required_names: + raise CliError("MATLAB runtime manifests must declare MATLAB as required") + + package_names: set[str] = set() + for index, value in enumerate( + _list(data["octave_packages"], "octave_packages", maximum=200) + ): + if not isinstance(value, dict) or set(value) != { + "name", + "version", + "source", + "sha256", + }: + raise CliError( + "each octave_packages entry must contain name, version, source, " + "and sha256" + ) + name = _string(value["name"], f"octave_packages[{index}].name", maximum=127) + _string(value["version"], f"octave_packages[{index}].version", maximum=64) + _string(value["source"], f"octave_packages[{index}].source", maximum=500) + checksum = _string( + value["sha256"], f"octave_packages[{index}].sha256", maximum=64 + ) + if not SHA256.fullmatch(checksum): + raise CliError(f"octave_packages[{index}].sha256 must be lowercase SHA-256") + if name.casefold() in package_names: + raise CliError("Octave package names must be unique") + package_names.add(name.casefold()) + if runtime == "matlab" and data["octave_packages"]: + warnings.append("Octave packages are declared for a MATLAB-only runtime") + + action_ids: list[str] = [] + for field in ("startup_actions", "shutdown_actions"): + for index, value in enumerate(_list(data[field], field, maximum=100)): + action_ids.append( + _validate_path( + value, + root=root, + name=f"{field}[{index}]", + allow_missing=args.allow_missing_paths, + kind="file", + suffixes={".m"}, + ) + ) + if action_ids: + warnings.append( + "Startup/shutdown actions require explicit code review before a project opens" + ) + + generated_ids = [ + _validate_path( + value, + root=root, + name=f"generated_artifacts[{index}]", + allow_missing=True, + ) + for index, value in enumerate( + _list(data["generated_artifacts"], "generated_artifacts", maximum=200) + ) + ] + external = [ + _string(value, f"external_interfaces[{index}]", maximum=500) + for index, value in enumerate( + _list(data["external_interfaces"], "external_interfaces", maximum=100) + ) + ] + notes = [ + _string(value, f"notes[{index}]", maximum=1000) + for index, value in enumerate(_list(data["notes"], "notes", maximum=100)) + ] + report = { + "allow_missing_paths": bool(args.allow_missing_paths), + "entry_points": entry_ids, + "executes": False, + "external_interface_count": len(external), + "generated_artifacts": generated_ids, + "license_verified": False, + "manifest": relative_id(manifest_path, root), + "network_accessed": False, + "notes_count": len(notes), + "octave_package_count": len(package_names), + "ok": True, + "product_count": len(product_names), + "project_name": project_name, + "runtime": runtime, + "schema_version": "1.0", + "test_paths": test_ids, + "tool": TOOL, + "warnings": warnings, + } + return report, 0 + + +def parser() -> argparse.ArgumentParser: + result = argparse.ArgumentParser( + description=( + "Validate a strict local MATLAB/Octave project, product, and license-" + "status manifest. The validator never launches either runtime." + ) + ) + result.add_argument("manifest", help="local JSON manifest") + result.add_argument("--root", default=".", help="allowed project root") + result.add_argument( + "--allow-missing-paths", + action="store_true", + help="schema-check paths without requiring them to exist", + ) + return result + + +def main() -> int: + try: + report, status = validate(parser().parse_args()) + emit_json(report) + return status + except CliError as exc: + return fail_json(TOOL, exc) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.agents/skills/matstudylab-bootstrap/SKILL.md b/.agents/skills/matstudylab-bootstrap/SKILL.md index 488f6c9..7d6bc61 100644 --- a/.agents/skills/matstudylab-bootstrap/SKILL.md +++ b/.agents/skills/matstudylab-bootstrap/SKILL.md @@ -1,10 +1,12 @@ --- name: matstudylab-bootstrap -description: "Check-on-use skills sync before workflow commands — not user-facing." +description: "Check-on-use skills setup/update before workflow commands — not user-facing." disable-model-invocation: true --- -Runs automatically as **Step 0** of `/accept`, `/explain`, `/build`, `/new`, and `/modify`. End users never invoke this skill directly. +Runs automatically as **Step 0** of `/accept`, `/explain`, `/build`, `/new`, and `/modify`. End users never invoke this skill directly. Not a user-facing slash command; harness-agnostic (no Cursor-only rule). + +English is the default for all agent UX strings here. Localize only when `LORE.md` or a `/build` language choice says so. ## Step 1 — Run bootstrap script @@ -14,32 +16,92 @@ From the repository root, run: ./scripts/bootstrap-skills.sh ``` -**Completion criterion:** the script prints `bootstrap: OK` and exits with status 0. +The script chooses the branch (`setup` vs `update` vs skip). Do not pick the branch yourself. + +**Completion criterion:** the script finished and you captured its stdout/stderr (including any `bootstrap:` lines and exit status). ## Step 2 — Interpret outcome -| Output line | Meaning | Action | -|-------------|---------|--------| -| `bootstrap: SKIPPED_FRESH` | `skills-lock.json` checked within the last 24 hours | Proceed to the workflow command | -| `bootstrap: SYNC_OK` | Lockfile was stale; `npx skills@latest update -p -y` succeeded | Proceed to the workflow command | -| `bootstrap: WOULD_SYNC` | Dry-run only (`BOOTSTRAP_DRY_RUN=1`) | Tests only — do not use in production | -| `bootstrap: SYNC_FAILED_CONTINUE` | Sync failed (network, auth, or CLI missing) | **Warn the user**; proceed with vendored skills already in `.agents/skills/` | +Match every `bootstrap:` line the script printed. Outcomes (in addition to a final `bootstrap: OK` on success paths): + +| Output line | Meaning | Parent-command action | +|-------------|---------|------------------------| +| `bootstrap: SKIPPED_LOCAL_ONLY` | Preference `mode` is `local-only` — no Node/network | Proceed (vendored skills). `/build` may still offer HITL to re-enable sync — see Step 3. | +| `bootstrap: SKIPPED_FRESH` | `last_synced_at` within 24h — no sync needed | Proceed | +| `bootstrap: NODE_MISSING` | Node/`npx` not available | See Step 3 (warn+continue vs `/build` HITL) | +| `bootstrap: WOULD_SETUP` | Dry-run only (`BOOTSTRAP_DRY_RUN=1`); would run setup | Tests only — do not use in production | +| `bootstrap: WOULD_UPDATE` | Dry-run only; would run update | Tests only — do not use in production | +| `bootstrap: SETUP_OK` | First-time / incomplete catalog install succeeded | Proceed | +| `bootstrap: UPDATE_OK` | Stale catalog update succeeded | Proceed | +| `bootstrap: SYNC_FAILED_CONTINUE` | Setup or update failed (network, auth, or CLI) | See Step 3 (warn+continue vs `/build` HITL) | +| `bootstrap: NAME_GUARD_FAIL` | Owned skill name collision; script exits **non-zero** | **Stop.** Do not continue the parent command until the collision is fixed | +| `bootstrap: OK` | Terminal success marker (printed after most branches) | Required on exit 0; alone does not identify the branch | + +On `SYNC_FAILED_CONTINUE` / `NODE_MISSING`, the script still exits 0 after `bootstrap: OK` — treat those as warnings for non-`/build` commands, not hard stops. `NAME_GUARD_FAIL` is the hard stop (exit ≠ 0). + +**Completion criterion:** you named exactly which outcome branch ran (including `NAME_GUARD_FAIL` if exit ≠ 0) and know whether Step 3 requires a user warning or `/build` HITL. + +## Step 3 — Parent-command policy + +### `/new`, `/modify`, `/accept`, `/explain` + +On `NODE_MISSING` or `SYNC_FAILED_CONTINUE`, **warn the user in English** (default) and continue with vendored skills in `.agents/skills/`: + +- **Node missing:** + ```text + Warning: Node/`npx` missing — could not check for skill updates. + Using vendored `.agents/skills/`. (Use /build to open setup or set local-only.) + Continuing with the command… + ``` +- **Sync failed:** + ```text + Warning: skills sync failed; using vendored `.agents/skills/`. + Continuing with the command… + ``` + +Do not open a HITL menu on these commands. -When sync fails, the script still exits 0 after printing `bootstrap: OK` — treat `SYNC_FAILED_CONTINUE` as a warning, not a hard stop. +### `/build` -**Completion criterion:** you have identified which branch ran and told the user when sync failed. +Do **not** use the warn+continue strings above. After this skill finishes, `/build` **must** run its own **Step 1 — Skills HITL** in `.agents/skills/build/SKILL.md`: -## Step 3 — Continue workflow +- `SKIPPED_LOCAL_ONLY` → short notice + optional re-enable (`auto`) +- `NODE_MISSING`, `SYNC_FAILED_CONTINUE`, or incomplete setup → three-option menu (continue session / save `local-only` / open setup) +- Quiet success (`SETUP_OK` / `UPDATE_OK`) → one-line notice, no menu; script sets `mode: auto` and stamps `last_synced_at` +- Quiet skip (`SKIPPED_FRESH`) → one-line notice, no menu; **does not** refresh `last_synced_at` (real sync only) -Proceed with the parent command only after Step 2. +**Completion criterion:** for non-`/build` parents, any required English warning was shown (or no warn was needed); for `/build`, bootstrap outcome is captured so Step 1 HITL can run (or quiet success continues); `NAME_GUARD_FAIL` never continues. -**Completion criterion:** bootstrap is complete; no further sync attempts in this session unless the user explicitly asks. +## Step 4 — Continue workflow + +Proceed with the parent command only after Steps 1–3. Do not re-run bootstrap in this session unless the user explicitly asks or `/build` HITL triggers another setup/update. + +**Completion criterion:** bootstrap is complete; parent command may start its next step. ## Reference -- Lockfile: `skills-lock.json` (`last_checked` ISO timestamp, updated every run) -- Staleness threshold: **24 hours** (see `scripts/lib/skills_bootstrap.py`) -- Upstream catalog skills sync via `npx skills@latest update`; project command skills (`matstudylab-bootstrap`, `accept`, `build`, `new`, `modify`, `explain`) are **not** updated by that command -- `matlab` and `matlab-performance-optimizer` are listed in the lockfile for inventory but **bootstrap does not sync them** — install manually (see `docs/agents/matlab-skills-gate.md`) +### Setup vs update (script decides) + +- **Setup** — catalog missing / incomplete: adds full Pocock catalog plus `matlab` and `matlab-performance-optimizer` (copy into `.agents/skills/`). +- **Update** — lock present and previously synced OK but stale: project-scoped `npx skills@latest update -p -y` (never pass a repo id as update positional). +- Owned project skills (`matstudylab-bootstrap`, `accept`, `build`, `new`, `modify`, `explain`) are name-guarded and not updated by the catalog CLI. + +### Preference sidecar + +Path: `.matstudylab/skills-pref.json` (gitignored; absence ⇒ `auto` and never synced). + +```json +{ "mode": "auto" | "local-only", "last_synced_at": "" } +``` + +- Freshness uses **`last_synced_at`** (24h), **not** a lockfile `last_checked` field. +- `mode: local-only` → `SKIPPED_LOCAL_ONLY` (no Node/network probe). +- Successful setup/update stamps `last_synced_at` and forces `mode: auto`. `SKIPPED_FRESH` does **not** slide the timestamp. + +### Pointers + - Step 0 contract for new command skills: `docs/agents/command-skill-step-0.md` -- Spec: `docs/spec.md` — Skills layout, T2 +- Spec: `docs/spec.md` — Skills layout +- Canonical English HITL copy: `.scratch/skills-auto-provisioning-v2/assets/prototype-node-gate-messages.md` +- OS install checklists: `docs/agents/skills-node-os-checklist.md` +- Manual HITL verification: `docs/agents/build-skills-hitl-checklist.md` diff --git a/.agents/skills/migrate-to-shoehorn/SKILL.md b/.agents/skills/migrate-to-shoehorn/SKILL.md new file mode 100644 index 0000000..ae4f965 --- /dev/null +++ b/.agents/skills/migrate-to-shoehorn/SKILL.md @@ -0,0 +1,118 @@ +--- +name: migrate-to-shoehorn +description: Migrate test files from `as` type assertions to @total-typescript/shoehorn. Use when user mentions shoehorn, wants to replace `as` in tests, or needs partial test data. +--- + +# Migrate to Shoehorn + +## Why shoehorn? + +`shoehorn` lets you pass partial data in tests while keeping TypeScript happy. It replaces `as` assertions with type-safe alternatives. + +**Test code only.** Never use shoehorn in production code. + +Problems with `as` in tests: + +- Trained not to use it +- Must manually specify target type +- Double-as (`as unknown as Type`) for intentionally wrong data + +## Install + +```bash +npm i @total-typescript/shoehorn +``` + +## Migration patterns + +### Large objects with few needed properties + +Before: + +```ts +type Request = { + body: { id: string }; + headers: Record; + cookies: Record; + // ...20 more properties +}; + +it("gets user by id", () => { + // Only care about body.id but must fake entire Request + getUser({ + body: { id: "123" }, + headers: {}, + cookies: {}, + // ...fake all 20 properties + }); +}); +``` + +After: + +```ts +import { fromPartial } from "@total-typescript/shoehorn"; + +it("gets user by id", () => { + getUser( + fromPartial({ + body: { id: "123" }, + }), + ); +}); +``` + +### `as Type` → `fromPartial()` + +Before: + +```ts +getUser({ body: { id: "123" } } as Request); +``` + +After: + +```ts +import { fromPartial } from "@total-typescript/shoehorn"; + +getUser(fromPartial({ body: { id: "123" } })); +``` + +### `as unknown as Type` → `fromAny()` + +Before: + +```ts +getUser({ body: { id: 123 } } as unknown as Request); // wrong type on purpose +``` + +After: + +```ts +import { fromAny } from "@total-typescript/shoehorn"; + +getUser(fromAny({ body: { id: 123 } })); +``` + +## When to use each + +| Function | Use case | +| --------------- | -------------------------------------------------- | +| `fromPartial()` | Pass partial data that still type-checks | +| `fromAny()` | Pass intentionally wrong data (keeps autocomplete) | +| `fromExact()` | Force full object (swap with fromPartial later) | + +## Workflow + +1. **Gather requirements** - ask user: + - What test files have `as` assertions causing problems? + - Are they dealing with large objects where only some properties matter? + - Do they need to pass intentionally wrong data for error testing? + +2. **Install and migrate**: + - [ ] Install: `npm i @total-typescript/shoehorn` + - [ ] Find test files with `as` assertions: `grep -r " as [A-Z]" --include="*.test.ts" --include="*.spec.ts"` + - [ ] Replace `as Type` with `fromPartial()` + - [ ] Replace `as unknown as Type` with `fromAny()` + - [ ] Add imports from `@total-typescript/shoehorn` + - [ ] Run type check to verify diff --git a/.agents/skills/migrate-to-shoehorn/agents/openai.yaml b/.agents/skills/migrate-to-shoehorn/agents/openai.yaml new file mode 100644 index 0000000..3bd79ee --- /dev/null +++ b/.agents/skills/migrate-to-shoehorn/agents/openai.yaml @@ -0,0 +1,3 @@ +interface: + display_name: "Migrate to Shoehorn" + short_description: "Replace test assertions with shoehorn" diff --git a/.agents/skills/modify/SKILL.md b/.agents/skills/modify/SKILL.md index ab81d14..e64fc6e 100644 --- a/.agents/skills/modify/SKILL.md +++ b/.agents/skills/modify/SKILL.md @@ -11,9 +11,9 @@ Read `LORE.md`, `AGENTS.md`, `docs/spec.md`, `docs/matlab-guidelines.md`, and `C ## Step 0 — Bootstrap (mandatory) Read and execute `.agents/skills/matstudylab-bootstrap/SKILL.md` before any other step. -Do not proceed until bootstrap completes (sync, skip, or failed-with-continue). +Do not proceed until bootstrap completes (skip, setup/update, or warn-and-continue). -**Completion criterion:** bootstrap outcome identified (fresh, synced, or failed-with-continue). +**Completion criterion:** bootstrap outcome identified (skip, setup/update OK, or warn-and-continue). ## Step 1 — Select bundle and script diff --git a/.agents/skills/new/SKILL.md b/.agents/skills/new/SKILL.md index 0cc1947..7babc4c 100644 --- a/.agents/skills/new/SKILL.md +++ b/.agents/skills/new/SKILL.md @@ -11,9 +11,10 @@ Read `LORE.md`, `AGENTS.md`, `docs/spec.md`, `docs/matlab-guidelines.md`, and `C ## Step 0 — Bootstrap (mandatory) Read and execute `.agents/skills/matstudylab-bootstrap/SKILL.md` before any other step. -Do not proceed until bootstrap completes (sync, skip, or failed-with-continue). +Do not proceed until bootstrap completes (skip, setup/update, or warn-and-continue). +On `NODE_MISSING` or `SYNC_FAILED_CONTINUE`: **warn + continue** with vendored skills only — no three-option HITL menu (that menu is `/build` only). -**Completion criterion:** bootstrap outcome identified (fresh, synced, or failed-with-continue). +**Completion criterion:** bootstrap outcome identified (skip, setup/update OK, or warn-and-continue). ## Step 1 — Read context diff --git a/.agents/skills/prototype/LOGIC.md b/.agents/skills/prototype/LOGIC.md index 526ecb1..5f5a3fd 100644 --- a/.agents/skills/prototype/LOGIC.md +++ b/.agents/skills/prototype/LOGIC.md @@ -1,13 +1,15 @@ # Logic Prototype -A tiny interactive terminal app that lets the user drive a state model by hand. Use this when the question is about **business logic, state transitions, or data shape** — the kind of thing that looks reasonable on paper but only feels wrong once you push it through real cases. +A single, self-contained HTML file — a **shareable demo** — that lets anyone drive a state model by clicking buttons. Use this when the question is about **business logic, state transitions, or data shape** — the kind of thing that looks reasonable on paper but only feels wrong once you push it through real cases. + +Because it's one file with nothing to install, you can hand it to a non-developer — a designer, a PM, a domain expert — and let them feel the model for themselves. So it speaks their language, not the code's. ## When this is the right shape - "I'm not sure if this state machine handles the edge case where X then Y." - "Does this data model actually let me represent the case where..." - "I want to feel out what the API should look like before writing it." -- Anything where the user wants to **press buttons and watch state change**. +- Anything where someone wants to **press buttons and watch state change**. If the question is "what should this look like" — wrong branch. Use [UI.md](UI.md). @@ -15,17 +17,11 @@ If the question is "what should this look like" — wrong branch. Use [UI.md](UI ### 1. State the question -Before writing code, write down what state model and what question you're prototyping. One paragraph, in the prototype's README or a comment at the top of the file. A logic prototype that answers the wrong question is pure waste — make the question explicit so it can be checked later, whether the user is watching now or returning to it AFK. - -### 2. Pick the language - -Use whatever the host project uses. If the project has no obvious runtime (e.g. a docs repo), ask. - -Match the project's existing conventions for tooling — don't add a new package manager or runtime just for the prototype. +Before writing code, write down what state model and what question you're prototyping. One paragraph, at the top of the demo (in a visible intro, not just a comment). A logic prototype that answers the wrong question is pure waste — make the question explicit so it can be checked later, whether the user is watching now or returning to it AFK. -### 3. Isolate the logic in a portable module +### 2. Isolate the logic in a portable module -Put the actual logic — the bit that's answering the question — behind a small, pure interface that could be lifted out and dropped into the real codebase later. The TUI around it is throwaway; the logic module shouldn't be. +Put the actual logic — the bit that's answering the question — in a single `