Peel: three-layer progressive-disclosure output system

docs/peel.md

@@ -0,0 +1,110 @@
+---
+title: Peel — the output system
+description: >-
+  Peel is freshdata's progressive-disclosure output system: a glance-level
+  answer, an inspect layer of evidence, and a full audit layer with zero
+  information loss, shared across the core API, semantic models, the AI
+  copilot, and parsing.
+keywords: freshdata output, progressive disclosure, clean report, terminal rendering, notebook html, machine readable
+---
+
+# Peel — the output system
+
+Peel is how freshdata report objects present themselves. A result is fresh
+produce: the **skin** tells you at a glance whether it is good, you can **peel**
+to the flesh to inspect it, and the **core** is always intact.
+
+Every report — `CleanReport`, `ParseResult`, the experimental `CopilotReport` —
+renders through the same three layers:
+
+| Layer | Name | What it answers |
+|---|---|---|
+| 1 | **Skin** (glance) | Did it succeed? What changed? Does anything need my attention? What's the one next step? |
+| 2 | **Flesh** (inspect) | The evidence — per-column changes, ranked findings, semantic proposals, frame inventory. |
+| 3 | **Core** (audit) | Every field, machine-readable, nothing dropped. |
+
+Peel never removes information; it only decides what you see first. Layer 3 is
+the existing structured objects and their `to_dict()`/`to_json()` — display can
+be turned off entirely and no data is lost.
+
+## Status and severity language
+
+Every result carries a **text status label** (colour and icons only reinforce
+it, so output stays readable when piped, in CI, or for screen readers):
+
+`CLEAN` · `CHANGED` · `REVIEW` · `BLOCKED` · `PARTIAL` · `SKIPPED` · `FAILED`
+
+Findings in the attention list are ranked by one shared order:
+
+1. privacy & safety → 2. data-corruption risk → 3. policy/contract → 4. analysis
+reliability → 5. cosmetic consistency
+
+so a high trust score can never bury a privacy or policy finding.
+
+## Seeing Peel output
+
+Peel is **opt-in** while it stabilizes; existing output is unchanged by default.
+
+```python
+import freshdata as fd
+
+cleaned, report = fd.clean(df, return_report=True)
+
+report.show()                    # legacy behavior unchanged (inline HTML / temp file)
+report.show(mode="standard")     # Peel text: glance + attention + next step
+report.show(mode="compact")      # two lines, for pipelines
+report.show(mode="verbose")      # + per-column, semantic, and action detail
+report.show(renderer="terminal") # styled panel when `rich` is installed
+```
+
+### Display modes
+
+| Mode | Use |
+|---|---|
+| `auto` | environment-aware: `standard` in a terminal, `compact` when piped |
+| `compact` | one-screen, two lines |
+| `standard` | glance + ranked attention + next step |
+| `verbose` | full human-readable diagnostics |
+| `debug` | + internal audit metadata |
+| `json` | stable `to_dict()` on stdout |
+| `plain` | no ANSI, ASCII icons |
+| `silent` | render nothing |
+
+### Notebook
+
+`fd.set_display("peel")` (or `FRESHDATA_DISPLAY=peel`) switches the notebook
+`_repr_html_` to the Peel card; `FRESHDATA_LEGACY_DISPLAY=1` forces the legacy
+layout. Objects with only a Peel view (like `ParseResult`) always render as Peel.
+Disclosure uses native `<details>`, so it works in static notebook exports with
+no JavaScript.
+
+### Command line
+
+`freshdata clean` gains additive display flags; default output is unchanged:
+
+```bash
+freshdata clean input.csv                    # unchanged (legacy summary)
+freshdata clean input.csv --verbose          # Peel verbose
+freshdata clean input.csv -vv                # Peel debug
+freshdata clean input.csv --output-format json  # report JSON to stdout
+freshdata clean input.csv --no-color         # no ANSI
+freshdata clean input.csv --display peel     # Peel standard
+```
+
+Display flags never change cleaning behavior. `fd.set_display(...)` sets
+process-wide preferences; `NO_COLOR` and `FRESHDATA_NO_PREVIEWS` are honored.
+
+## Machine-readable access is unchanged
+
+`report.to_dict()` / `report.to_json()` schemas are frozen — additive only.
+`report.actions`, `report.warnings`, `report.domain_findings`, `report.attention`
+(the ranked queue) and every other attribute keep working. Peel is a rendering
+layer over these objects, never a replacement for them.
+
+## Privacy
+
+Values shown on evidence cards and previews are escaped before HTML rendering
+and never placed in HTML attributes. `fd.set_display(previews=False)` (or
+`FRESHDATA_NO_PREVIEWS=1`) switches to schema-only display. `undo_log` is never
+serialized, and provider credentials are never captured — only failure states
+appear in the audit layer.

mkdocs.yml

@@ -106,6 +106,7 @@ nav:
       - Cleaning engine: cleaning-engine.md
       - Validation (suites & contracts): validation.md
       - Interactive output: interactive.md
+      - Peel output system: peel.md
       - Decision-preserving workflow: decision-workflow.md
       - Scalable backends: backends.md
       - Backend fallback matrix: fallback-matrix.md

pyproject.toml

@@ -91,6 +91,10 @@ outofcore = [
 cli = [
     "pyyaml>=5.1",
 ]
+# Styled terminal output (Peel). Everything renders as plain text without it.
+rich = [
+    "rich>=13",
+]
 # Interactive output layer. The renderers produce self-contained HTML with zero
 # of these installed; these libraries *upgrade* tables/charts when present and
 # are never imported by `import freshdata`.

src/freshdata/__init__.py

@@ -77,6 +77,7 @@
 )
 from .profile import ColumnProfile, Profile
 from .quality import QualityDebtGate, evaluate_quality_debt
+from .render.options import get_display, reset_display, set_display
 from .repairplan import (
     FrameSignature,
     PlanDriftError,
@@ -163,6 +164,9 @@
     "apply_field_policy",
     "clean_text",
     "clean_text_value",
+    "get_display",
+    "reset_display",
+    "set_display",
     "validate_fields",
     "cdc_profile",
     "apply",

src/freshdata/enterprise/cli.py

@@ -30,6 +30,62 @@
 from .metrics import compute_trust_score
 
 
+def _add_display_flags(parser: argparse.ArgumentParser) -> None:
+    """Additive Peel display flags (spec §11.3). Default output is unchanged."""
+    group = parser.add_argument_group("display")
+    group.add_argument(
+        "-v",
+        "--verbose",
+        action="count",
+        default=0,
+        help="Peel diagnostics: -v verbose, -vv debug",
+    )
+    group.add_argument(
+        "--output-format",
+        choices=("text", "json"),
+        default="text",
+        help="'json' prints the report as JSON to stdout",
+    )
+    group.add_argument("--no-color", action="store_true", help="disable ANSI styling")
+    group.add_argument(
+        "--display",
+        choices=("legacy", "peel"),
+        default="legacy",
+        help="'peel' opts into the Peel layout for this run",
+    )
+
+
+def _emit_report(report: Any, args: argparse.Namespace, legacy_text: str) -> None:
+    """Print a clean report honoring the display flags.
+
+    With no display flags the legacy text is printed verbatim (backward
+    compatible); ``--output-format json``/``--verbose``/``--display peel``
+    opt into Peel output.
+    """
+    fmt = getattr(args, "output_format", "text")
+    verbose = getattr(args, "verbose", 0)
+    display = getattr(args, "display", "legacy")
+
+    if fmt == "json":
+        print(json.dumps(report.to_dict(), default=str, indent=2))
+        return
+    if verbose == 0 and display == "legacy":
+        print(legacy_text)
+        return
+
+    from ..render.normalize import normalize
+    from ..render.options import get_display
+    from ..render.terminal import render_terminal_text
+
+    mode = "debug" if verbose >= 2 else "verbose" if verbose == 1 else "standard"
+    color = "never" if getattr(args, "no_color", False) else "auto"
+    try:
+        options = get_display(mode=mode, color=color)
+        print(render_terminal_text(normalize(report), options))
+    except Exception:
+        print(legacy_text)  # display must never break the command
+
+
 def _infer_format(path: str) -> str:
     low = path.lower()
     if low.endswith((".parquet", ".pq")):
@@ -183,7 +239,7 @@ def cmd_clean(args: argparse.Namespace) -> int:
     if args.lineage:
         result.lineage.emit(args.lineage)
     if not args.quiet:
-        print(result.summary())
+        _emit_report(result.clean_report, args, result.summary())
         for event in result.clean_report.fallback_events:
             if event.get("fallback_step") == "semantic":
                 print(
@@ -237,7 +293,7 @@ def _cmd_clean_engine(args: argparse.Namespace) -> int:
         with open(args.report, "w", encoding="utf-8") as fh:
             json.dump(report.to_dict(), fh, indent=2, default=str)
     if not args.quiet:
-        print(report.summary())
+        _emit_report(report, args, report.summary())
         if report.backend_differences:
             print(
                 f"\n{len(report.backend_differences)} backend difference(s) recorded "
@@ -627,6 +683,7 @@ def build_parser() -> argparse.ArgumentParser:
     )
     clean.add_argument("--actor", help="who ran this (recorded in lineage)")
     clean.add_argument("--quiet", action="store_true")
+    _add_display_flags(clean)
     clean.add_argument(
         "--context-file",
         metavar="PATH",

src/freshdata/experimental/ai_copilot.py

@@ -66,6 +66,7 @@
 from ..enterprise.config import ClusterConfig, MaskingRule
 from ..enterprise.metrics import TrustScore, compute_trust_score
 from ..enterprise.privacy import PIIDetectionConfig, anonymize, detect_pii
+from ..render.mixins import HtmlReprMixin
 
 __all__ = [
     "CleaningPlan",
@@ -172,7 +173,7 @@ def __str__(self) -> str:
 
 
 @dataclass(frozen=True)
-class CopilotReport:
+class CopilotReport(HtmlReprMixin):
     """Everything one :func:`analyze_dataset` run produced.
 
     ``summary``, ``cleaning_plan``, and ``recommended_code`` are all directly
@@ -194,6 +195,18 @@ class CopilotReport:
     #: default deterministic path or when the provider call failed).
     narrative: str | None = None
 
+    _render_kind = "copilot"
+
+    @property
+    def attention(self) -> tuple:
+        """The ranked attention queue (privacy/policy first), as ``AttentionItem``\\s.
+
+        Terminal, notebook, and programmatic callers all read the same order.
+        """
+        from ..render import normalize  # noqa: PLC0415
+
+        return normalize.normalize(self).attention
+
     def to_dict(self) -> dict[str, Any]:
         return {
             "goal": self.goal,

src/freshdata/parsers/base.py

@@ -17,11 +17,13 @@
 
 import pandas as pd
 
+from ..render.mixins import HtmlReprMixin
+
 _MAX_XML_BYTES = 10 * 1024 * 1024
 
 
 @dataclass
-class ParseResult:
+class ParseResult(HtmlReprMixin):
     """The structural output of a :class:`Parser`.
 
     Attributes
@@ -46,6 +48,21 @@ class ParseResult:
     metadata: dict[str, Any] = field(default_factory=dict)
     warnings: list[str] = field(default_factory=list)
 
+    _render_kind = "parse"
+
+    def summary(self) -> str:
+        """Plain-text Peel summary — parsing is structural only, so this makes
+        the parsed/validated/cleaned distinction explicit (spec §9)."""
+        # Deferred: render.normalize imports back into freshdata internals; a
+        # module-level import here would be circular at package-init time.
+        from ..render import normalize, plain  # noqa: PLC0415
+        from ..render.options import get_display  # noqa: PLC0415
+
+        return plain.render_plain(normalize.normalize(self), get_display(mode="standard"))
+
+    def __str__(self) -> str:
+        return self.summary()
+
     @property
     def frame(self) -> pd.DataFrame:
         """The single frame, for formats that yield exactly one."""