Skip to content

Added the Batch Processing Assistant - #901

Open
j-erler wants to merge 1 commit into
MindWorkAI:mainfrom
j-erler:feature/batch-processing-assistant
Open

Added the Batch Processing Assistant#901
j-erler wants to merge 1 commit into
MindWorkAI:mainfrom
j-erler:feature/batch-processing-assistant

Conversation

@j-erler

@j-erler j-erler commented Aug 9, 2026

Copy link
Copy Markdown

Adds an assistant that processes all documents of a folder in one batch run. Each document is extracted to Markdown by the Rust runtime and sent to the selected provider together with the user's instructions.

Instructions come from one of three sources: a free prompt, one of the existing document analysis policies (including its minimum provider confidence), or a file the user imports.

Output is either one Markdown file per document, or one CSV results table in which each answer becomes a row. The user can name the results table; its columns are the document and the answer.

Every run writes a log named log.csv with the document, time, model, status, and the reason for any error. When a later run finds a log in the output folder, the assistant asks whether to continue it. Continuing processes only the documents that failed or whose results no longer exist — this recovers runs interrupted by a crash or by documents exceeding the model's context window. A single failing document never stops the run, and the run can be canceled at any time.

Screenshots

The assistant within the overview:
Assistant_Menu_German

The empty form:
Assistant_UI_German

A running batch:
Assistant_Running_German

The log of a run, opened in a spreadsheet application:
Output_log

The assistant ships with German and English localization:
Assistant_UI_English

Design decisions worth flagging
Columns are separated by a vertical bar and quoted per RFC 4180, so the files open in spreadsheet applications regardless of the user's list separator. The files are written with a UTF-8 BOM for the same reason.
The status column holds the raw enum values (DONE, FAILED, CANCELED) so that filters stay stable across languages, while the headers are localized.
The output files are rewritten after every document. That is quadratic, but it is what makes a crashed run recoverable — the trade-off is deliberate.
Documents are identified by their path relative to the input folder, because two subfolders may contain a document of the same name.

Known limitation
The run is driven by the assistant component, so navigating away from the page while a batch is running leaves the loop without its UI. I saw that AIJobService exists for running chats independently of the assistant UI, but did not want to pull that into this PR unasked — happy to follow up if that is the direction you'd prefer.

Tests
The repository has no .NET test project, so this PR adds none. BatchProcessingCsv (writer and parser) and the file-matching helpers are pure logic and would be straightforward to cover if a test project is planned — glad to add that.

Localization for English and German is included via the I18N process.

The assistant processes all documents of a folder in one batch run. Each
document is extracted to Markdown by the Rust runtime and sent to the
selected provider together with the user's instructions.

The instructions come from one of three sources: a free prompt, one of
the existing document analysis policies including its minimum provider
confidence, or a file the user imports.

The output is either one Markdown file per document, or one CSV results
table in which each answer becomes a row. The user can name the results
table; its columns are the document and the answer.

Every run writes a log named log.csv with the document, time, model,
status, and the reason for any error. When a later run finds a log in
the output folder, the assistant asks whether to continue it. Continuing
processes only the documents that failed or whose results no longer
exist, which recovers runs interrupted by a crash or by documents
exceeding the context window of the model. A single failing document
never stops the run, and the run can be canceled at any time.

Columns are separated by a vertical bar and quoted per RFC 4180, so that
the files open in spreadsheet applications regardless of the list
separator of the user. Documents are identified by their path relative
to the input folder, because two subfolders may contain a document of
the same name.

Includes the English and German localization.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@j-erler
j-erler force-pushed the feature/batch-processing-assistant branch from cdbc336 to 05790e8 Compare August 9, 2026 16:05
@j-erler j-erler changed the title Feature/batch processing assistant Added the Batch Processing Assistant Aug 9, 2026
@SommerEngineering

Copy link
Copy Markdown
Member

Thank you so much for this cool assistant. How may we thank you in the changelog? Under your GitHub name j-erler, or may we mention your name (which we don't know yet)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants