Skip to content

Track whether a bug is a regression from the most recent release #5925

Description

@andygrove

Problem

Triage records how bad a bug is but not whether it is new. A defect that shipped in the last release and one introduced since it read identically on the issue tracker, even though the second one hits users who upgrade without changing anything on their side. There is no label for that distinction and no documented procedure for establishing it.

A pass over the 19 open priority:critical issues, each checked against the 1.0.0 tag, found 18 pre-existing and one regression (#5783). That one was only identifiable by building the tag and running the reproducer: 1.0.0 failed loudly on a struct with byte-identical duplicate Parquet field names, and main silently returns 6 rows where Spark returns 3, because #5602 removed the assert_eq! guard that used to catch it.

Proposal

  • Add a regression label for a bug that did not affect the most recent release.
  • Document in docs/source/contributor-guide/bug_triage.md how to establish regression status, resolving the comparison point from the release tags at triage time rather than hard-coding a version, and comparing against the tag's commit date rather than the release's publication date.
  • Order the evidence steps cheapest-first: issue creation date, then the implicated code at the tag, then whether the path was reachable at all, then building the tag and running the reproducer.
  • Treat regression as orthogonal to priority and as an escalation trigger.
  • Add the matching step to the bug-triage skill so automated passes apply it and report it.

Two cases worth writing down, both of which came up in the pass:

  • Issues found during PR review routinely say "this is pre-existing, not caused by this PR". That is a claim about the pull request under review, not about the last release.
  • A query that failed loudly on the release and now returns silently wrong data is a regression, even though it never produced the right answer on either version.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions