Skip to content

feat: re-analyse an item and replace the AI's earlier tags - #301

Open
Mallard-Marsh wants to merge 5 commits into
Anyesh:mainfrom
Mallard-Marsh:pr/replace-reanalysis
Open

Mallard-Marsh wants to merge 5 commits into
Anyesh:mainfrom
Mallard-Marsh:pr/replace-reanalysis

Conversation

@Mallard-Marsh

@Mallard-Marsh Mallard-Marsh commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Description

Adds re-analyse and replace. Re-analysis only ever filled empty fields, so a better analysis (a larger image, a better model, a newer prompt) never reached items that already had tags. In replace mode the AI redoes its own earlier answers, and anything a person edited by hand is kept.

Builds on #300: it uses the tags.user_edited_keys list that #300 records. The first three commits here are #300's; please review and merge #300 first and I'll rebase this so only this PR's own two commits remain.

Backend

  • POST /items/{id}/analyze takes ?replace=true, passed to tag_item_image as a replace argument. It is only sent when replacing, so a plain re-analysis enqueues exactly what it always did. It also works when retrying a failed item.
  • In replace mode the job overwrites type, subtype, primary_color, colors, pattern, material, formality, style, season and fit (the tags JSON keys and the columns), except the keys in tags.user_edited_keys. It never replaces a value with an empty or "unknown" answer, leaves the user-owned keys (size, care_instructions, source_url) and the edit list alone, and keeps a primary colour the person chose in the new colour list.
  • One function, _ai_may_set, now makes the fill-or-replace decision for both the tags JSON and the columns. Behaviour without replace is unchanged.

Frontend: a "Re-analyze and replace AI tags" button next to the existing re-analyse button opens a confirm dialog that says what is replaced and what is kept (including that edits made before edit-tracking existed can be replaced). Strings in all 8 locales.

Related Issue

Part of #270 (re-analysing after correcting tags). Depends on #300.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)
  • CI/CD or build changes

Checklist

  • I have read the CONTRIBUTING guide
  • My code follows the project's coding style
  • I have added tests that prove my fix/feature works
  • New and existing tests pass locally
  • I have updated documentation as needed
  • My changes don't introduce new warnings or errors

Testing

Test Environment

  • Docker Compose
  • Kubernetes
  • Local development

Tests Performed

  • backend: ruff, full pytest on current main + feat: edit an item's AI tags and secondary and tertiary colours #300 (same 4 macOS-only failures as main). Tests cover the decision function, replacing everything, keeping edited keys, user-owned keys and the edit list, never erasing with nothing, the primary colour rule, columns and tags agreeing, normal analysis unchanged, an item with no edit list, and the endpoint with and without replace and on a failed item.
  • frontend: tsc, vitest (749 passing), i18n:check, vocab:check; a test for the confirm dialog flow.
  • In daily use for about a week on a self-hosted instance.

Additional Notes

A bulk version (replace for a ticked selection or the whole wardrobe from the bulk toolbar) exists in my fork too; happy to send it as a follow-up if you want it.

Summary by Sourcery

Enable users to re-analyse wardrobe items by replacing AI-generated tags while preserving manual edits and improving structured tag management.

New Features:

  • Add an option to re-analyse an item and replace prior AI-generated tags while preserving fields recorded as manually edited.
  • Add item tag editing for structured attributes, colors, and vocabulary-backed values with localized UI labels.
  • Add localized confirmation flow for replacing AI analysis and support the option when retrying failed analyses.

Bug Fixes:

  • Prevent partial or empty tag updates from erasing existing tags and keep duplicated color data synchronized across item columns and tag JSON.
  • Ensure manually edited tag values are tracked and protected from later AI replacement.
  • Make item color preferences account for secondary colors as well as the primary color.

Enhancements:

  • Centralize garment vocabularies for backend validation, AI prompts, and generated frontend options.
  • Improve optimistic item updates and preserve concurrent AI changes while an edit form is open.

Documentation:

  • Update contributor guidance for maintaining pattern, style, season, and fit vocabularies and localized labels.

Tests:

  • Add backend and frontend coverage for tag validation, edit tracking, replacement behavior, endpoint options, localized confirmation, and tag editing.

@sourcery-ai

sourcery-ai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Reviewer's Guide

This PR adds an end-to-end “re-analyze and replace” flow: the API and worker selectively overwrite prior AI tags while protecting tracked human edits and user-owned metadata, and the UI adds localized confirmation plus tag editing and edit tracking. It also centralizes garment vocabularies, validates manual tag changes, updates generated frontend data, and broadens color-based recommendation scoring to include secondary colors.

Sequence diagram for re-analyze and replace

sequenceDiagram
    actor User
    participant UI as ItemDetailDialog
    participant API as POST /items/{id}/analyze
    participant Queue as TaggingQueue
    participant Worker as tag_item_image
    participant DB as ClothingItem

    User->>UI: handleReanalyze(true)
    UI->>API: POST /items/{id}/analyze?replace=true
    API->>Queue: enqueue_job(replace=true)
    Queue->>Worker: tag_item_image(replace=true)
    Worker->>DB: load item and user_edited_keys
    Worker->>Worker: _ai_may_set(field, value, replace=true, kept)
    alt AI field is not user-edited and value is non-empty
        Worker->>DB: replace tag and column
    else Field is user-edited or value is empty or unknown
        Worker->>DB: keep existing value
    end
    Worker->>DB: preserve user-owned keys and edit list
Loading

Flow diagram for tracked manual tag edits

flowchart TD
    A[User edits item tags] --> B[ItemService.update]
    B --> C[Compare incoming values with existing item]
    C --> D[Append changed AI-owned keys to user_edited_keys]
    D --> E[Merge partial tags without replacing user-owned keys]
    E --> F[Later replace analysis reads user_edited_keys]
    F --> G[_ai_may_set skips tracked keys]
Loading

File-Level Changes

Change Details Files
Adds replace-mode AI re-analysis while preserving fields tracked as manually edited.
  • Accepts an optional replace query parameter and forwards it through both normal and failed-item job enqueue paths.
  • Centralizes fill-versus-replace behavior in _ai_may_set for tags JSON and denormalized columns.
  • Replaces AI-owned values only when the new answer is meaningful, preserves user-owned metadata and the edit list, and retains a user-selected primary color in the color list.
  • Tracks changed tag and item fields server-side while preventing clients from modifying the tracking list.
backend/app/api/items.py
backend/app/schemas/item.py
backend/app/services/item_service.py
backend/app/workers/tagging.py
backend/tests/test_item_tag_edits.py
backend/tests/test_replace_reanalysis.py
Introduces vocabulary-driven validation and editing for clothing tags.
  • Moves pattern, style, season, and fit vocabularies into shared garment data and derives backend validators, prompts, and frontend constants from it.
  • Normalizes and validates manually edited closed-vocabulary values while allowing legacy off-list values to remain untouched.
  • Adds an item tag editor with partial diffs, null-based clearing, color ordering, subtype cleanup, and optimistic update support.
backend/app/data/garment_vocabulary.json
backend/app/prompts/clothing_analysis.txt
backend/app/services/ai_service.py
backend/app/utils/garment_vocabulary.py
backend/app/utils/tag_validation.py
frontend/components/item-detail-dialog.tsx
frontend/components/item-tags-editor.tsx
frontend/lib/tag-edits.ts
frontend/lib/generated/garment-vocabulary.ts
frontend/scripts/gen-shared-data.mjs
frontend/lib/types.ts
frontend/lib/hooks/use-items.ts
backend/tests/test_garment_vocabulary.py
frontend/tests/item-detail-tag-edit.test.tsx
frontend/tests/item-tags-editor.test.tsx
frontend/tests/tag-edits.test.ts
Adds the frontend replace-analysis workflow and localized confirmation messaging.
  • Adds a dedicated replace button and confirmation dialog explaining that AI answers are replaced while tracked manual edits and user-owned fields are retained.
  • Sends replace=true only after confirmation and leaves the existing re-analysis request unchanged.
  • Provides translations across all supported locales and covers confirmation, cancellation, cooldown, and disabled states with tests.
frontend/components/replace-analysis-dialog.tsx
frontend/components/item-detail-dialog.tsx
frontend/lib/hooks/use-items.ts
frontend/messages/de/wardrobe.json
frontend/messages/en/wardrobe.json
frontend/messages/fr/wardrobe.json
frontend/messages/it/wardrobe.json
frontend/messages/ja/wardrobe.json
frontend/messages/ko/wardrobe.json
frontend/messages/zh-CN/wardrobe.json
frontend/messages/zh-TW/wardrobe.json
frontend/tests/item-detail-replace-analysis.test.tsx
Updates recommendation color scoring to consider all stored colors rather than only the primary color.
  • Uses case-insensitive color sets for explicit and learned favorite/avoid preferences.
  • Applies each preference bonus or penalty once when any item color matches.
  • Adds coverage for secondary colors, learned preferences, duplicate matches, and case differences.
backend/app/services/item_scorer.py
backend/tests/test_item_scorer.py
Documents the expanded vocabulary generation workflow and keeps generated data synchronized.
  • Documents patterns, styles, seasons, and fits as vocabulary-backed fields and lists the required locale labels.
  • Updates generated shared vocabulary output and validation tests to ensure prompt, backend, and frontend lists agree.
CONTRIBUTING.md
backend/app/data/garment_vocabulary.json
frontend/lib/generated/garment-vocabulary.ts
backend/tests/test_garment_vocabulary.py
frontend/tests/item-tag-labels.test.tsx

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@github-actions github-actions Bot added javascript Pull requests that update javascript code python Pull requests that update python code labels Oct 27, 2026

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 4 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="frontend/components/item-tags-editor.tsx" line_range="95" />
<code_context>
+
+function ChipGroup({ label, values, selected, labelFor, onChange }: ChipGroupProps) {
+  // Values already on the item that the list no longer offers can still be switched off.
+  const all = [...values, ...selected.filter((value) => !values.includes(value))];
+  return (
+    <div className="space-y-2" role="group" aria-label={label}>
</code_context>
<issue_to_address>
**Legacy tags block style edits**

When a style or season contains an off-vocabulary value that the user leaves selected while changing another value, `ChipGroup` includes the retained value in the submitted selection, and `normalize_tag_edits` rejects it, so the PATCH returns 422 and the edit cannot be saved.

Exclude retained off-vocabulary values from submitted style and season selections while still allowing users to remove them.

Also at `backend/app/utils/tag_validation.py:54-57`, `frontend/components/item-tags-editor.tsx:109`.
</issue_to_address>

### Comment 2
<location path="backend/app/workers/tagging.py" line_range="336-345" />
<code_context>
+                    setattr(item, field, value)
+
+            # A colour the person chose as primary must stay in the list when the AI replaced
+            # the rest of it.
+            if (
+                replace
+                and "primary_color" in kept
+                and "colors" not in kept
+                and item.primary_color
+                and item.colors
+                and item.primary_color not in item.colors
+            ):
+                item.colors = [item.primary_color, *item.colors]

             # The guards above may have kept a user's column value while the
</code_context>
<issue_to_address>
**Primary color is missing from list**

When a person chose a primary color, the color list is empty, and replacement analysis returns no colors, the worker's `and item.colors` guard skips adding `item.primary_color`, leaving the primary color absent from the color list and affecting color displays and preference scoring.

Add the primary color to the list whenever it is missing, including when the list is empty.
</issue_to_address>

### Comment 3
<location path="backend/app/services/item_service.py" line_range="266" />
<code_context>
+        edited = [
+            field
+            for field in _EDITABLE_ITEM_FIELDS
+            if field in update_data and update_data[field] != getattr(item, field)
+        ]
+
</code_context>
<issue_to_address>
**Stale form values become protected edits**

When tagging finishes after the user opens the edit form but before they save an unrelated change, `update_data` can contain values from an edit form opened before tagging finished, so comparing them with the now-updated database row marks untouched `type`, `subtype`, or color fields as user-edited and can overwrite the fresh AI values; later replace analyses then preserve those stale values.

Track edits using explicit changed-field intent or a versioned comparison against the form’s original values, rather than treating every submitted difference from the current row as a hand edit.
</issue_to_address>

### Comment 4
<location path="backend/app/services/item_service.py" line_range="267" />
<code_context>
+            field
+            for field in _EDITABLE_ITEM_FIELDS
+            if field in update_data and update_data[field] != getattr(item, field)
+        ]
+
+        if incoming or edited:
</code_context>
<issue_to_address>
**Concurrent edit loses protection**

When a PATCH's item read precedes a replacement commit, but its write follows that commit, `ItemService.update` compares an item-level edit against the earlier ORM snapshot, so a concurrent replacement can change the database value after that comparison but before the PATCH writes. The PATCH then overwrites the replacement without recording the key in `user_edited_keys`, and a later replacement overwrites the person's submitted value.

Serialize item-field comparison and update under a row lock, and determine edit tracking against the locked current row.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 4 findings to address first, and with replace enabled, new AI guesses are saved over existing tags that were not recorded as manually edited. Incorrect tags affect only the item record and can be corrected or regenerated, though reverting the code will not restore the previous values.

Blocking findings: frontend/components/item-tags-editor.tsx:95, backend/app/workers/tagging.py:345, backend/app/services/item_service.py:266, backend/app/services/item_service.py:267


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread backend/app/workers/tagging.py Resolved
Comment thread backend/app/services/item_service.py Resolved
Comment thread backend/app/services/item_service.py Resolved
Mallard-Marsh and others added 5 commits October 28, 2026 08:04
The item editor only covered name, type, subtype, brand, primary colour,
notes and wash interval. Pattern, material, formality, fit, style, season and
extra colours were read-only chips, so a wrong tag (a herringbone blazer
tagged "solid") could not be fixed on screen.

Editor (components/item-tags-editor.tsx, mounted in the edit form):
- selects for pattern, material, formality and fit, each with a None option
  that clears the tag
- toggle chips for style and season
- secondary and tertiary colour selects after the existing primary colour
  field (which keeps its eyedropper); colours past the third are preserved
- only the tags that differ are sent, and none at all when nothing changed

Backend:
- The pattern, style, season and fit lists move into
  garment_vocabulary.json next to the other vocabulary, are generated for the
  frontend, and fill new <<PATTERNS>> <<STYLES>> <<SEASONS>> <<FITS>> prompt
  tokens. The rendered tagging prompt is byte-identical to before.
- PATCH validates the pattern, formality, fit, style and season values it is
  given (422 for unknown ones) and stores them stripped and lower-cased. Only
  keys in the request are checked, so an item that still holds an older
  off-list value can be saved. Material stays free text and colours stay
  lenient, as before.
- A null or empty `tags` object is now a no-op. It used to set the whole tags
  JSON to NULL, dropping the AI confidence and the user-owned keys; clearing
  one tag is a null inside the object.
- The server records the AI-produced keys a person changed in
  tags.user_edited_keys (from the tags object and from type, subtype,
  primary_color and colors), ignoring a value sent by the client. Nothing
  reads it yet; it is for a later "re-analyse and replace".
- The colour preference score looks at every colour on the item, not only
  the primary.

Edit form: changing the type drops a subtype that belongs to a different
type's list ("button-down" after a T-shirt becomes a polo) and keeps one that
fits the new type or that the app has no list for. The optimistic update now
merges edited tags instead of replacing the whole tags object.

No new database column and no migration: fit stays in the tags JSON.

Tests cover validation and cleaning, the null and empty no-ops, clearing a
tag and its column, edit tracking from both paths, the scorer, the prompt
tokens, the diffing helpers, the editor controls and the full edit dialog
(including that saving a plain field sends no tags).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
CI type-checks the test files with tsc. The check read a property the Item
type does not declare.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…rs in step

From review:
- The edit dialog compares the form with the item as it was when editing began and sends only
  the fields and tags that changed. Before, tags the AI added while the form was open were sent
  back as clears, and untouched fields were resent (and recorded as edits). Nothing changed
  means no request.
- ItemService.update locks the row first and compares edits against it, not against the item
  loaded before the lock, so a re-analysis committing in between cannot slip past edit tracking.
- A primary colour or colour list sent outside `tags` now reaches the tags JSON, and a new
  primary colour replaces the old one at the front of the colour list (Anyesh#294 for any client).
- A style or season list may keep an off-list value the item already has, so an old value left
  switched on no longer makes the save fail with 422; a new off-list value is still rejected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-analysis only ever filled empty fields, so a better analysis (a larger
photo, a better model) never reached items that already had tags. This adds
"re-analyse and replace": the AI redoes its own earlier answers, and anything
a person edited by hand is kept.

Backend:
- POST /items/{id}/analyze takes ?replace=true, passed to tag_item_image as a
  `replace` argument. It is only sent when replacing, so a plain re-analysis
  enqueues exactly what it always did. It works when retrying a failed item
  too.
- In replace mode the job overwrites type, subtype, primary_color, colors,
  pattern, material, formality, style, season and fit (the tags JSON keys as
  well as the columns), except the keys in tags.user_edited_keys. It never
  replaces a value with an empty or "unknown" answer, leaves the user-owned
  keys (size, care_instructions, source_url) and the edit list alone, and
  keeps a primary colour the person chose in the new colour list.
- One function, _ai_may_set, now makes the fill-or-replace decision for the
  tags JSON and the columns. Behaviour without replace is unchanged.

Frontend: a "Re-analyze and replace AI tags" button next to the existing
re-analyse one opens a confirm dialog that says what is replaced and kept,
including that edits made before they were recorded can be replaced. The hook
accepts an id or { id, replace }. Strings are in all eight locales.

Depends on the tag editor branch, which records tags.user_edited_keys.

Tests: the decision function, replacing everything, keeping edited keys,
user-owned keys and the edit list, never erasing with nothing, the primary
colour rule, columns and tags agreeing, normal analysis unchanged, an item
with no edit list, the endpoint with and without replace and on a failed
item, and the dialog flow.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…empty

From review: the rule that keeps a hand-picked primary colour in the colour list after
"re-analyse and replace" skipped an empty list, so the primary could end up missing from it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

javascript Pull requests that update javascript code python Pull requests that update python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants