feat(db): add dateCreated and lastModified to Repo#1656
Conversation
Populate dates on create and bump lastModified on user-role mutations. Backfill missing fields on NeDB/Mongo startup so Date Created/Modified sorting in the repo list works for existing records. Fixes finos#1486.
✅ Deploy Preview for endearing-brigadeiros-63f9d0 canceled.
|
andypols
left a comment
There was a problem hiding this comment.
Thanks for the contribution — it will be really useful to see recently added repositories in the proxy.
I do have reservations about the migration approach. Running one-off backfills during application startup is difficult to track, retry, test, and roll back, and it will not scale well as more migrations are introduced.
I think the migration work should move to a separate PR that introduces a small, explicit migration framework with versioned migrations, tracking of completed migrations, proper error handling, and documented recovery or rollback behaviour. We can then add the backfill for these new fields as the first migration.
We should also use a more meaningful creation date for existing records rather than defaulting everything to 1970. For MongoDB records with a standard ObjectId, the creation timestamp can be derived from _id, for example:
await collection.updateMany(
{
$or: [
{ dateCreated: { $exists: false } },
{ lastModified: { $exists: false } },
],
},
[
{
$set: {
dateCreated: {
$ifNull: ["$dateCreated", { $toDate: "$_id" }],
},
lastModified: {
$ifNull: [
"$lastModified",
{ $ifNull: ["$dateCreated", { $toDate: "$_id" }] },
],
},
},
},
],
);Remove backfillRepoDates and startup hooks; existing-repo migration will follow in a separate PR with a versioned migration framework. Co-authored-by: Cursor <cursoragent@cursor.com>
Remove backfillRepoDates and startup hooks; existing-repo migration will follow in a separate PR with a versioned migration framework. Co-authored-by: Cursor <cursoragent@cursor.com>
e41612a to
1d4da08
Compare
Remove backfillRepoDates and startup hooks; existing-repo migration will follow in a separate PR with a versioned migration framework. Co-authored-by: Cursor <cursoragent@cursor.com>
1d4da08 to
f74be7d
Compare
Remove backfillRepoDates and startup hooks; existing-repo migration will follow in a separate PR with a versioned migration framework.
f74be7d to
065fab5
Compare
|
Thanks @andypols — agreed on deferring migration to a dedicated PR. Changes in 065fab5:
This PR now only adds |
NeDB adaptor already bumps lastModified on all user-role mutations; extend finos#1486 date-field tests to assert removeUserCanAuthorise preserves dateCreated while advancing lastModified.
fe8f6e3 to
10f21a6
Compare
|
The migration/backfill code remains removed as requested. I have now completed the two available behavior checks and updated the PR checklist:
Sorting remains explicitly deferred to #1464. Could you please re-review the narrowed persistence-only PR when convenient? |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1656 +/- ##
==========================================
+ Coverage 85.97% 86.23% +0.26%
==========================================
Files 87 87
Lines 8621 8646 +25
Branches 1492 1494 +2
==========================================
+ Hits 7412 7456 +44
+ Misses 1178 1160 -18
+ Partials 31 30 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
@andypols Migration/backfill is out of this PR as requested — persistence-only now. Mind taking another look? |
Summary
dateCreated/lastModified(ISO-8601) on the Repo modellastModifiedon all user-role mutations (NeDB + Mongo)Scope / issue linkage
This PR lands the persistence slice of #1486 (model fields + create/mutation timestamps). Remaining #1486 work is intentionally split out:
dateCreatedfrom$toDate("$_id")rather than defaulting to epoch (as suggested in review)Unblocks Date Created / Date Modified sorting for newly created and recently mutated repos once #1464 lands.
Related: #1420 (broader admin sorting/pagination — #1464 covers the server half)
Test plan
npm test -- test/db(163 passed; 45 integration tests skipped because no external MongoDB was configured)lastModifiedadvances,dateCreatedunchanged