Skip to content

Record projected vs actual slippage per execution #21

Closed

Description

@0xDeon

The single highest-value issue for the product thesis.

Why

The differentiator against CoW, UniswapX, and 1inch Fusion is persistent agent reputation. Those systems run stateless solver auctions — solvers carry no history and losing costs nothing beyond the auction.

That distinction only holds if reputation reflects measured outcomes. Right now AgentReputation tracks AvgSlippageDelivered, but nothing measures what was actually delivered. Agents self-report projected_slippage, win on it, and no record is kept of whether the projection was true.

An agent that consistently promises 0.1% and delivers 2% is indistinguishable from one that promises and delivers 0.1%. Until that gap is measured, the reputation layer is decoration and the competitive claim is unsupported.

Scope

Record, per execution: what was projected, what was delivered, and the difference.

  • Capture projected_slippage from the winning proposal at selection time
  • Capture actual slippage from the settled execution
  • Persist both against the execution and the agent
  • Feed the delta into AvgSlippageDelivered and the composite score
  • Expose it through the API so the dApp can show it

Design notes

  • Actual slippage needs a reference price. With a simulated swap there is no fill to measure against, so this may need a reference from Pyth (live on Arc at 0x2880aB155794e7179c9eE2e38200202908C17B43) or an off-chain feed. Worth deciding deliberately — a measurement against a bad reference is worse than none, because it looks authoritative.
  • Weight recent performance. An agent with 500 good fills and 50 recent bad ones should not look fine on average.
  • Fabricated proposals must be excluded. Proposals carrying is_fallback (Cookit-labs/Intent-Agent#2) contain invented numbers. Scoring them as if they were real predictions poisons the reputation data permanently.

Acceptance criteria

  • Every settled execution records projected and actual slippage
  • The delta updates the agent's reputation
  • An agent that over-promises measurably loses standing
  • Fallback proposals are excluded from reputation entirely
  • Both figures are exposed via the API

Activity

  1. 0xDeon commented on Sep 8, 2026

    @0xDeon
    ContributorAuthor

    Fixed in #27 (merged).

    Executions now record the projection, the actual, and the delta between them, and reputation carries a running mean of that delta. Separately, AvgSlippageDelivered was declared but never written, so the leaderboard had been showing zero for every agent; both means are now maintained in SQL against the pre-increment count.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions