Skip to content

fix: flush pending filename index tasks on service exit to avoid full rescan - #265

Merged
Johnson-zs merged 1 commit into
linuxdeepin:develop/snipe-20260923from
Johnson-zs:verify-patch
Sep 30, 2026
Merged

Johnson-zs merged 1 commit into
linuxdeepin:develop/snipe-20260923from
Johnson-zs:verify-patch

Conversation

@Johnson-zs

@Johnson-zs Johnson-zs commented Sep 30, 2026 •

Copy link
Copy Markdown

fix: flush pending filename index tasks on service exit to avoid full rescan

Log: FileNameIndexDBus::cleanup() immediately marked Dirty and stopped
all tasks on exit, forcing a full rescan on next startup even when only
a few incremental tasks remained. Add TaskManager::flushPendingTasks()
which drains the task queue within a bounded 5 s timeout before cleanup
gives up. A warning is logged whenever a real flush actually occurs, so
the event is visible in logs.

Influence: When the service exits with a small backlog, pending tasks are
completed and the index stays Clean — no full rescan on restart. Large
backlogs (> 5000 queued incremental events) skip flushing and fall back
to the existing Dirty + StopCurrentTask() path, so shutdown is never
blocked for too long.

PMS: BUG-505079

Summary by Sourcery

Flush manageable filename index work during shutdown while retaining a bounded dirty-state fallback for large or stalled backlogs.

Bug Fixes:

  • Flush pending filename index tasks during service shutdown when the backlog is small enough to preserve a clean index and avoid an unnecessary full rescan on restart.
  • Fall back to marking the index dirty when pending work cannot be completed within the bounded shutdown window or exceeds the backlog threshold.

Enhancements:

  • Add bounded task-draining support that completes queued incremental work without re-entrant event-loop processing.

@deepin-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Johnson-zs

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sourcery-ai

sourcery-ai Bot commented Sep 30, 2026

Copy link
Copy Markdown

Reviewer's Guide

Shutdown now attempts to drain small pending filename-index backlogs within a bounded timeout, preserving a clean index when successful while retaining the existing Dirty fallback for large, timed-out, or unstartable work.

Sequence diagram for bounded filename-index flush during shutdown

sequenceDiagram
    participant Service as FileNameIndexDBus
    participant Manager as TaskManager
    participant Worker as WorkerThread
    participant State as StateStore

    Service->>Manager: flushPendingTasks(5000)
    alt no pending work
        Manager-->>Service: true
    else backlog exceeds 5000 or task cannot start
        Manager-->>Service: false
        Service->>State: setIndexState(Dirty)
    else pending work within threshold
        loop until queue empty or timeout
            Manager->>Manager: schedule()
            Manager->>Worker: quit()
            Manager->>Worker: wait(remaining)
            Worker-->>Manager: task completed
            Manager->>Manager: finalizeIndexState()
        end
        alt all tasks completed
            Manager-->>Service: true
        else timeout
            Manager-->>Service: false
            Service->>State: setIndexState(Dirty)
        end
    end
Loading

File-Level Changes

Change Details Files
Flushes pending indexing work during service cleanup before falling back to a dirty index state.
  • Disables filesystem events, then attempts a bounded task drain during shutdown.
  • Marks the index Dirty only when flushing fails and unfinished work remains.
  • Preserves the existing stop/dirty fallback for large or unflushable backlogs.
src/index/dbus/filenameindexdbus.cpp
Adds a bounded, synchronous task-draining workflow that safely advances queued tasks without relying on the main event loop.
  • Returns immediately for empty work and rejects queues above the 5,000 incremental-task threshold.
  • Waits for each worker task within the remaining timeout, manually finalizes completed tasks, and schedules subsequent work.
  • Logs actual flushes and timeout failures, returning success only when the queue is drained.
src/index/task/taskmanager.cpp
src/index/task/taskmanager.h

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="src/index/task/taskmanager.cpp" line_range="1172-1185" />
<code_context>
+        // processed because the main thread was blocked in wait().  Finalize
+        // the index state and clean up manually to prevent a stale
+        // onTaskFinished from firing later.
+        if (currentTask) {
+            const auto type = currentTask->taskType();
+            const bool succeeded = (currentTask->status() == IndexTask::Status::Finished);
+
+            if (succeeded) {
+                HandlerResult result;
+                result.success = true;
+                finalizeIndexState(type, result);
+            }
+
+            currentTask->disconnect();
+            currentTask->deleteLater();
+            currentTask = nullptr;
+            m_currentIncrementalPending = 0;
+        }
+    }
</code_context>
<issue_to_address>
**issue (broader_impact):** The manual completion path bypasses `onTaskFinished`, so successful flushed tasks never execute `handleRootPathFailure`, `updateIndexStatusOnSuccess`, failure bookkeeping, completion signals, or the normal `cleanupTask` bookkeeping. A flushed Create/Update task therefore can leave index-status metadata or recovery state stale even though `finalizeIndexState` marks the index clean.

**Triggers:** When a task completes during shutdown and its queued `onTaskFinished` callback is replaced by this manual cleanup.

**Suggested fix:** Refactor the common task-completion handling into a synchronous helper and invoke it from both `onTaskFinished` and the flush path, rather than calling only `finalizeIndexState`.
</issue_to_address>

### Comment 2
<location path="src/index/task/taskmanager.cpp" line_range="1172-1185" />
<code_context>
+        // processed because the main thread was blocked in wait().  Finalize
+        // the index state and clean up manually to prevent a stale
+        // onTaskFinished from firing later.
+        if (currentTask) {
+            const auto type = currentTask->taskType();
+            const bool succeeded = (currentTask->status() == IndexTask::Status::Finished);
+
+            if (succeeded) {
+                HandlerResult result;
+                result.success = true;
+                finalizeIndexState(type, result);
+            }
+
+            currentTask->disconnect();
+            currentTask->deleteLater();
+            currentTask = nullptr;
+            m_currentIncrementalPending = 0;
+        }
+    }
</code_context>
<issue_to_address>
**issue (bug_risk):** The flush path calls `currentTask->deleteLater()` after stopping and waiting for `workerThread`, but the deferred-delete event belongs to that worker thread and is not processed while the thread is stopped. The task object consequently remains allocated until the thread is restarted or the process exits, leaking each task flushed during shutdown.

**Triggers:** When cleanup is the final service shutdown and the worker thread is not restarted before TaskManager destruction.

**Suggested fix:** Destroy the completed task safely after the worker thread has stopped, or ensure its deferred deletion is delivered before abandoning the worker thread.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread src/index/task/taskmanager.cpp Outdated
Comment on lines +1172 to +1185
if (currentTask) {
const auto type = currentTask->taskType();
const bool succeeded = (currentTask->status() == IndexTask::Status::Finished);

if (succeeded) {
HandlerResult result;
result.success = true;
finalizeIndexState(type, result);
}

currentTask->disconnect();
currentTask->deleteLater();
currentTask = nullptr;
m_currentIncrementalPending = 0;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (broader_impact): The manual completion path bypasses onTaskFinished, so successful flushed tasks never execute handleRootPathFailure, updateIndexStatusOnSuccess, failure bookkeeping, completion signals, or the normal cleanupTask bookkeeping. A flushed Create/Update task therefore can leave index-status metadata or recovery state stale even though finalizeIndexState marks the index clean.

Triggers: When a task completes during shutdown and its queued onTaskFinished callback is replaced by this manual cleanup.

Suggested fix: Refactor the common task-completion handling into a synchronous helper and invoke it from both onTaskFinished and the flush path, rather than calling only finalizeIndexState.

Comment thread src/index/task/taskmanager.cpp Outdated
Comment on lines +1172 to +1185
if (currentTask) {
const auto type = currentTask->taskType();
const bool succeeded = (currentTask->status() == IndexTask::Status::Finished);

if (succeeded) {
HandlerResult result;
result.success = true;
finalizeIndexState(type, result);
}

currentTask->disconnect();
currentTask->deleteLater();
currentTask = nullptr;
m_currentIncrementalPending = 0;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): The flush path calls currentTask->deleteLater() after stopping and waiting for workerThread, but the deferred-delete event belongs to that worker thread and is not processed while the thread is stopped. The task object consequently remains allocated until the thread is restarted or the process exits, leaking each task flushed during shutdown.

Triggers: When cleanup is the final service shutdown and the worker thread is not restarted before TaskManager destruction.

Suggested fix: Destroy the completed task safely after the worker thread has stopped, or ensure its deferred deletion is delivered before abandoning the worker thread.

… rescan

Log: FileNameIndexDBus::cleanup() immediately marked Dirty and stopped
all tasks on exit, forcing a full rescan on next startup even when only
a few incremental tasks remained.  Add TaskManager::flushPendingTasks()
which drains the task queue within a bounded 5 s timeout before cleanup
gives up.  A warning is logged whenever a real flush actually occurs, so
the event is visible in logs.

Influence: When the service exits with a small backlog, pending tasks are
completed and the index stays Clean — no full rescan on restart.  Large
backlogs (> 5000 queued incremental events) skip flushing and fall back
to the existing Dirty + StopCurrentTask() path, so shutdown is never
blocked for too long.

PMS: BUG-505079
@Johnson-zs
Johnson-zs merged commit faf25c1 into linuxdeepin:develop/snipe-20260923 Sep 30, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants