Skip to content

docs: document the Task API - #1094

Open
adithya-s-k wants to merge 2 commits into
huggingface:mainfrom
adithya-s-k:docs/task-api
Open

docs: document the Task API#1094
adithya-s-k wants to merge 2 commits into
huggingface:mainfrom
adithya-s-k:docs/task-api

Conversation

@adithya-s-k

@adithya-s-k adithya-s-k commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

What

The Task API — the optional discovery layer that lets a dataset-backed environment publish its tasks and splits — has been in the codebase since #726 but has never been documented. Before this PR:

  • docs/ had zero mentions of the Task API or any of its routes
  • TaskProvider was missing from reference/core.md, as were ListTasksRequest, NumTasksRequest, GetTaskRequest, and GetTaskRangeRequest

It shipped in #726 as one bullet of the external-env importer work (976c4121, "Adds ORS-compatible task/split APIs to the OpenEnv server") and hasn't been touched since. The only implementations in-repo are the generated ors_environment.py.tpl wrapper and #1003 (LaTeX OCR), which is what surfaced the gap.

Changes

New: docs/source/guides/task-api.md — follows the structure of guides/mcp-environment-lifecycle.md:

  • the two halves — the TaskProvider structural protocol, and the routes HTTPEnvServer derives from it
  • when to implement it, and when not to
  • route table and error-semantics table: unknown env_name → 404, missing method / NotImplementedError → 501, IndexError → 400
  • the two easy-to-miss contract rules: task methods must be side-effect-free, and must work on a freshly constructed instance (each route builds a short-lived env and closes it)
  • env_name resolution — defaults to the factory's class name and is matched case-insensitively, so implementers should pass it explicitly
  • split normalization: bare strings become {"name", "type"}, with type preserved only for train/validation/test
  • task selection is not part of the Task API — it rides on ResetRequest's extra="allow", filtered against the env's reset() signature, so reset(split=..., index=...) is what actually starts an episode
  • streaming / large-split caveats: bounded previews from list_tasks() while num_tasks() keeps reporting the honest total
  • the client-side pattern (core clients ship no task methods; env clients add thin HTTP helpers), plus a training-loop example and curl equivalents
  • a pre-ship checklist

docs/source/reference/core.md — adds the missing [[autodoc]] entries for TaskProvider and the four request models, plus a cross-link from the intro.

docs/source/_toctree.yml — "The Task API" under Advanced Guides.

src/openenv/core/env_server/interfaces.py — reformats the TaskProvider docstrings into the HF doc-builder style used by the reference docs (Args/Returns/Raises, *optional*, Examples:), now that they're rendered by autodoc. No behavior change.

Notes for reviewers

  • Everything asserted in the guide is read off the implementation (http_server.py:1062-1151, _get_valid_kwargs, _normalize_split, _call_task_method). [feature] Import external envs (ors, verifiers) #726 shipped no design doc, so the guide states the docstring's own side-effect-free / fresh-instance rules as the contract — happy to strengthen or soften that wording if there's a stronger intended contract.
  • Placement is under Advanced Guides. Given the Task API is the only route to dataset-backed RL, top-level Guides next to Rewards may be the better home — one-line change.
  • doc-builder isn't in the dev env here, so the rendered output is unverified; _toctree.yml parses and all autodoc targets import.

Verification

  • usort check / ruff format --check / ruff check clean on the touched file
  • pytest tests/core/test_task_api.py — 8 passed
  • all autodoc targets import

Note

Low Risk
Documentation and docstring-only updates with no changes to server or protocol behavior.

Overview
Adds first-class documentation for the Task API — the optional TaskProvider discovery layer and the HTTP routes HTTPEnvServer exposes under /{env_name}/….

A new Advanced Guides page (guides/task-api.md) explains when to implement the five task methods, that discovery is metadata-only (episodes still start via reset(split=..., index=...)), route tables and error semantics (404/501/400), split normalization, client patterns, and a pre-ship checklist. The docs site nav and Core API reference now link to it and autodoc TaskProvider plus ListTasksRequest, NumTasksRequest, GetTaskRequest, and GetTaskRangeRequest.

TaskProvider docstrings in interfaces.py are expanded into Hugging Face doc-builder style (Args/Returns/Raises, examples) so they render in the reference; no runtime behavior changes.

Reviewed by Cursor Bugbot for commit 3ba7e28. Bugbot is set up for automated code reviews on this repo. Configure here.

The Task API landed in huggingface#726 as part of the external-env importer work and
has never been documented: `TaskProvider` and the four task request models
were absent from the API reference, and `docs/` had no mention of the
`/{env_name}/splits`, `/tasks`, `/num_tasks`, `/task`, `/task_range` routes.

Adds a `guides/task-api.md` guide covering the protocol, the HTTP routes and
their error semantics, split normalization, `env_name` resolution, the
fresh-instance-per-request lifecycle, task selection via `reset(split=,
index=)`, streaming caveats, and the client-side helper pattern.

Also adds the missing autodoc entries and reformats the `TaskProvider`
docstrings into the HF doc-builder style used by the reference docs.
@bot-ci-comment

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@sergiopaniego sergiopaniego left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants