fix: template chat and tasks, honest prompt optimization, readable reports, working Studio

.github/workflows/sync-studio.yml

@@ -165,8 +165,10 @@ jobs:
 
           ---
           *This PR was automatically created by the [sync-studio workflow](https://github.com/UnicoLab/agentomatic/actions/workflows/sync-studio.yml).*" \
-              --label "automated,studio-ui" \
               --base main
 
             echo "✅ Pull request created successfully"
+            # Labels are best-effort: a missing label must not fail the sync.
+            gh pr edit "${BRANCH}" --add-label "automated,studio-ui" \
+              || echo "::warning::Could not add labels (create 'automated' and 'studio-ui')"
           fi

.gitignore

@@ -48,3 +48,6 @@ site/
 # Poetry (legacy)
 poetry.lock
 inspiration/
+examples/prompt_optimization/out/
+# ModelCheckpoint default output (runtime artefacts, never committed)
+optimization_results/checkpoints/

.pre-commit-config.yaml

@@ -1,3 +1,8 @@
+# The Studio UI bundle is a build artifact synced from agentomatic-studio
+# releases (sync-studio.yml). Whitespace fixers would rewrite its minified
+# chunks, so the shipped files would no longer match the release tarball.
+exclude: ^src/agentomatic/studio/static/
+
 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v5.0.0

CLAUDE.md

@@ -145,6 +145,7 @@ into the image/compose that drive the same `main.py`.
 | `AGENTOMATIC_ENABLE_CONTROL_PLANE` / `AGENTOMATIC_CONTROL_TOKEN` | Control plane |
 | `AGENTOMATIC_ENABLE_RATE_LIMIT` | Rate limiting |
 | `AGENTOMATIC_LOGS_HISTORY` | Persist per-agent invoke/chat/stream history (default off) |
+| `AGENTOMATIC_EPHEMERAL_THREADS` | With no store/DB configured, keep chat threads in a bounded in-memory store (default on; `0` disables — thread routes then answer 400) |
 | `AGENTOMATIC_ALLOW_LOGSLLM_ANALYSIS` | Enable LLM analysis over those logs (default off) |
 | `AGENTOMATIC_INGESTION_ROOT` | Confine ingestion source/output paths to this dir (default: cwd) |
 | `AGENTOMATIC_TASK_EVENT_LOG` | Task-event replay backend: `memory` (default), `none` to disable, or a registered provider |
@@ -235,6 +236,12 @@ register_vector_store_adapter("my_db", MyDbStore)  # upsert/query/delete
   `"skipped: <reason>"`).
 - Model defaults resolve provider-agnostically from `AGENTOMATIC_TASK_MODEL` /
   `LLM__MODEL` before any local fallback.
+- Data roles: **train** is what the optimizer learns from, **validation** selects
+  candidates, **holdout** only vetoes (no transfer / regression / widened gap),
+  **test** is never passed to the fitter — `agent.evaluate(data.test)` before and
+  after `fit()`. `generate_fit_report(history, baseline_eval=..., final_eval=...)`
+  shows the prompt diff, every candidate's decision and reason, and per-example
+  answers. Guide: `docs/optimization/`; runnable: `examples/prompt_optimization/`.
 
 ## Project conventions (match these when editing the framework)
 

README.md

@@ -764,7 +764,7 @@ Failure cluster 2:
 agentomatic run
 
 # 2. Prepare a labelled JSONL evaluation dataset (query + expected_answer).
-# See docs/guide/optimization.md for the format and metric guidance.
+# See docs/optimization/ for the format, metrics and anti-overfitting guidance.
 
 # 3. Optimize a running agent with the current fitter CLI.
 agentomatic optimize scope_agent \

docs/getting-started/quickstart.md

@@ -671,7 +671,7 @@ Now that your first agent is running, explore these resources:
 | **[Deep Agent Integration](../guide/deep-agents.md)** | Register and debug Deep Agent workflows with full Studio support |
 | **[Chat Interface](../guide/debug-ui.md)** | Chainlit-based conversational testing |
 | **[Prompt Management](../guide/prompts.md)** | Template versioning, hot-reload, and A/B testing |
-| **[Prompt Optimization](../guide/optimization.md)** | Auto-tune prompts with DSPy-inspired optimization |
+| **[Prompt Optimization](../optimization/index.md)** | Auto-tune prompts with DSPy-inspired optimization |
 | **[Storage Backends](../guide/storage.md)** | Configure PostgreSQL, SQLite, or custom adapters |
 | **[Middleware](../guide/middleware.md)** | Auth, rate limiting, metrics, and custom middleware |
 | **[Configuration](../guide/configuration.md)** | Platform settings, CORS, environment variables |

docs/guide/class-agents.md

@@ -1163,7 +1163,7 @@ agents/my_agent/
 
     Class scaffolds emit flat scripts (full abstraction):
     `TrainCliSettings` / `EvalCliSettings` →
-    [`train_and_report`](optimization.md) / [`evaluate_and_report`](optimization.md).
+    [`train_and_report`](../optimization/class-agents.md#one-call-train_and_report) / [`evaluate_and_report`](../optimization/reports.md#evaluation-only-reports).
     `train.py` also includes a **commented staged** path
     (`compile_agent` → `fit_agent` → `evaluate_agent`) for full control —
     same primitives under the hood.
@@ -1205,7 +1205,7 @@ agents/my_agent/
     ```
 
     For the current in-process fitter API and explicit search spaces, see
-    [Prompt Optimization](optimization.md#modern-python-api-promptfitter).
+    [Low level: PromptFitter](../optimization/prompt-fitter.md).
 
 ??? example "Generated `dataset.jsonl`"
 

docs/guide/configuration.md

@@ -189,6 +189,7 @@ directory, host, port, and stack values passed to `agentomatic run`.
 | `AGENTOMATIC_ENABLE_CONTROL_PLANE` / `AGENTOMATIC_CONTROL_TOKEN` | control plane | `true` / empty in generated `main.py`; `false` in CLI fallback | Mount control-plane routes and protect mutations |
 | `AGENTOMATIC_ENABLE_RATE_LIMIT` / `AGENTOMATIC_RATE_LIMIT_TRUST_PROXY_HEADERS` | rate limiting | `false` / `false` | Enable the limiter; trust forwarded client IPs only behind a trusted proxy |
 | `AGENTOMATIC_LOGS_HISTORY` / `DATABASE_URL` | invocation history / storage | `false` / empty | Persist logs and threads using an async SQLAlchemy URL |
+| `AGENTOMATIC_EPHEMERAL_THREADS` | thread-store fallback | `true` | With no `store=`, MEMORY connection or database URL, keep conversation threads in a bounded in-memory store (1000 threads, lost on restart) so Studio chat and `/chat` history work out of the box. `0` disables it — thread routes then answer `400 Thread storage not configured` |
 | `AGENTOMATIC_ARTIFACT_ROOT` | `artifact_root` | `.local/artifacts` | Versioned plugin/model artifact bundles |
 | `AGENTOMATIC_PLUGIN_AUTORELOAD` | `plugin_autoreload` | `false` | Watch the promoted artifact version and reload plugins safely |
 | `AGENTOMATIC_PLUGIN_AUTORELOAD_INTERVAL` | `plugin_autoreload_interval` | `5` | Seconds between artifact-pointer checks |