Skip to content

fix: preserve row_number columns in CSVToDocument - #12788

Merged
anakin87 merged 3 commits into
deepset-ai:mainfrom
LimbC-C:fix/csv-row-number-metadata
Oct 2, 2026
Merged

anakin87 merged 3 commits into
deepset-ai:mainfrom
LimbC-C:fix/csv-row-number-metadata

Conversation

@LimbC-C

@LimbC-C LimbC-C commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Related Issues

Fixes #12787.

Proposed Changes:

Preserve a CSV column named row_number in CSVToDocument row mode. Previously the generated row index overwrote that column after metadata collision handling, silently losing source record identifiers.

Reserve row_number before merging CSV columns so the existing csv_ prefix and numeric suffix handling preserves the original value. Add regressions for ordinary and occupied-prefix metadata, plus a control selecting row_number as the content column. Update the component docstring, current/versioned docs, and release note.

How did you test it?

  • Before the fix: both preservation regressions fail with the expected missing-metadata assertions; the content-column control passes.
  • After the fix: all 20 tests in test/components/converters/test_csv_to_document.py pass; CSV converter statement coverage is 95%.
  • Focused mypy: no issues found in the source and test files.
  • All applicable pre-commit hooks and git diff --cached --check pass.

Validation ran via Hatch on Windows / Python 3.12.14 using local ByteStreams. Pytest emitted two NumPy-reload warnings from Haystack imports. No live model, external service, or full integration suite was run.

Notes for the reviewer

Generated, reviewed by the coding agent, and tested with Codex at the account owner's request. No human review is claimed.

The change is independent of #12772, which fixes textual NA markers in CSVDocumentCleaner. The converter, its tests, and contribution guidelines were unchanged between the local base and current upstream at verification.

Checklist

@LimbC-C
LimbC-C requested a review from a team as a code owner September 17, 2026 02:01
@LimbC-C
LimbC-C requested review from anakin87 and removed request for a team September 17, 2026 02:01
@vercel

vercel Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

@LimbC-C is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/components/converters
  csv.py
Project Total  

This report was generated by python-coverage-comment-action

@anakin87 anakin87 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed a few simplifications.

Looks good!

@anakin87
anakin87 enabled auto-merge (squash) October 2, 2026 13:20
@anakin87
anakin87 merged commit 0e4b7cd into deepset-ai:main Oct 2, 2026
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CSVToDocument row mode overwrites a CSV column named row_number

2 participants