Repository navigation
Workspace persistence is fragile on localStorage; move graph/session storage to IndexedDB with migration + recovery #377
Description
We currently autosave full graph payloads in localStorage. This is brittle for larger workflows and can silently drop history when writes fail. Session restore is also partial (open tabs, active tab, and file metadata are not reliably recovered).
Based on discussion, this will follow a hybrid storage model:
keep filesystem storage for file/binary artifacts (imports/exports)
move graph/session metadata persistence to IndexedDB
This keeps binaries out of DB, while fixing reliability for autosave + restore.
Scope
- add a versioned IndexedDB storage layer for graph/session metadata
- run one-time migration of existing localStorage metadata on first successful init
- wire autosave/load/session-restore to the new metadata store
- handle corrupted entries by skipping bad records and continuing load
- keep a sane fallback path when IndexedDB is unavailable
- remove silent history-loss behavior during normal saves
- add automated tests for migration, restore, corrupted-record skip, and IndexedDB-unavailable fallback
- add docs for schema, migration flow, fallback behavior, and manual verification steps
Acceptance criteria
- reload restores previously open graphs and active tab
- large workflows save/load reliably without quota-related loss
- existing users are migrated automatically
- one corrupted record does not block full workspace restore
Activity
avinxshKD commentedon Mar 29, 2026
Hey @pradeeban what do you think about adding this? It should make saves more reliable for bigger workflows and stop the random session/history loss we still see with localStorage.
pradeeban commentedon Mar 31, 2026
Fair points about the local storage. But then the counter point is, often filesystem is better than database to store binary data. If you are thinking of indexing and retrieval, perhaps a hybrid approach of storage of the files in the local file system and the metadata in a database could be the winner approach.
avinxshKD commentedon Apr 1, 2026
@pradeeban Thanks, that makes sense. I agree with the hybrid direction. We can keep filesystem storage for files/binary exports, and use IndexedDB for autosave/session metadata + recovery (graph state, history, open tabs, active tab). That should fix the localStorage reliability issues without moving binaries into DB. If you’re aligned with this scope, I’ll update the issue and start implementation :p
pradeeban commentedon Apr 1, 2026
I had a similar discussion when I started implementing Niffler back in 2020. I went with local file-system based storage for the binary data (DICOM images) with metadata extracted into csv formats which itself then stored into a MongoDB instance as a metadata archive. I am seeing parallels.
avinxshKD commentedon Apr 2, 2026
Thanks for the context. Got it, that lines up well. I’ll update this to make the hybrid split explicit: filesystem for binaries/exports, IndexedDB for graph/session metadata with migration + fallback. I’ll open a PR for the storage layer and migration path.
avinxshKD commentedon Apr 2, 2026
@pradeeban Updated the issue with the hybrid scope.
pradeeban commentedon Apr 3, 2026
@avinxshKD perfect.