Skip to content

feat(db): add dateCreated and lastModified to Repo#1656

Open
sankalpsthakur wants to merge 3 commits into
finos:mainfrom
sankalpsthakur:prep/1486-repo-date-fields
Open

feat(db): add dateCreated and lastModified to Repo#1656
sankalpsthakur wants to merge 3 commits into
finos:mainfrom
sankalpsthakur:prep/1486-repo-date-fields

Conversation

@sankalpsthakur

@sankalpsthakur sankalpsthakur commented Jul 22, 2026

Copy link
Copy Markdown

Summary

  • Add optional dateCreated / lastModified (ISO-8601) on the Repo model
  • Stamp both on create; bump lastModified on all user-role mutations (NeDB + Mongo)
  • No startup backfill — existing records without these fields stay as-is until a follow-up migration PR (per @andypols review in feat(db): add dateCreated and lastModified to Repo #1656)

Scope / issue linkage

This PR lands the persistence slice of #1486 (model fields + create/mutation timestamps). Remaining #1486 work is intentionally split out:

  • sortBy allowlist + UI sort options — blocked on / coordinated with feat: add server-side pagination to API endpoints #1464 server-side pagination
  • Existing-record backfill — follow-up PR introducing a versioned migration framework; first migration will derive Mongo dateCreated from $toDate("$_id") rather than defaulting to epoch (as suggested in review)

Unblocks Date Created / Date Modified sorting for newly created and recently mutated repos once #1464 lands.

Related: #1420 (broader admin sorting/pagination — #1464 covers the server half)

Test plan

  • npm test -- test/db (163 passed; 45 integration tests skipped because no external MongoDB was configured)
  • Create a repo and confirm both dates are set
  • Mutate canPush/canAuthorise → lastModified advances, dateCreated unchanged
  • Sort repo list by Date Created / Date Modified (once feat: add server-side pagination to API endpoints #1464 lands)

Populate dates on create and bump lastModified on user-role mutations.
Backfill missing fields on NeDB/Mongo startup so Date Created/Modified
sorting in the repo list works for existing records. Fixes finos#1486.
@sankalpsthakur
sankalpsthakur requested a review from a team as a code owner July 22, 2026 15:04
@netlify

netlify Bot commented Jul 22, 2026

Copy link
Copy Markdown

Deploy Preview for endearing-brigadeiros-63f9d0 canceled.

Name Link
🔨 Latest commit 10f21a6
🔍 Latest deploy log https://app.netlify.com/projects/endearing-brigadeiros-63f9d0/deploys/6a611b609f82010008e6fde2

@sankalpsthakur sankalpsthakur changed the title feat(db): add dateCreated and lastModified on repos feat(db): add dateCreated and lastModified to Repo Jul 22, 2026

@andypols andypols left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution — it will be really useful to see recently added repositories in the proxy.

I do have reservations about the migration approach. Running one-off backfills during application startup is difficult to track, retry, test, and roll back, and it will not scale well as more migrations are introduced.

I think the migration work should move to a separate PR that introduces a small, explicit migration framework with versioned migrations, tracking of completed migrations, proper error handling, and documented recovery or rollback behaviour. We can then add the backfill for these new fields as the first migration.

We should also use a more meaningful creation date for existing records rather than defaulting everything to 1970. For MongoDB records with a standard ObjectId, the creation timestamp can be derived from _id, for example:

await collection.updateMany(
  {
    $or: [
      { dateCreated: { $exists: false } },
      { lastModified: { $exists: false } },
    ],
  },
  [
    {
      $set: {
        dateCreated: {
          $ifNull: ["$dateCreated", { $toDate: "$_id" }],
        },
        lastModified: {
          $ifNull: [
            "$lastModified",
            { $ifNull: ["$dateCreated", { $toDate: "$_id" }] },
          ],
        },
      },
    },
  ],
);

sankalpsthakur added a commit to sankalpsthakur/git-proxy that referenced this pull request Jul 22, 2026
Remove backfillRepoDates and startup hooks; existing-repo migration
will follow in a separate PR with a versioned migration framework.

Co-authored-by: Cursor <cursoragent@cursor.com>
@linux-foundation-easycla

linux-foundation-easycla Bot commented Jul 22, 2026

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

sankalpsthakur added a commit to sankalpsthakur/git-proxy that referenced this pull request Jul 22, 2026
Remove backfillRepoDates and startup hooks; existing-repo migration
will follow in a separate PR with a versioned migration framework.

Co-authored-by: Cursor <cursoragent@cursor.com>
@sankalpsthakur
sankalpsthakur force-pushed the prep/1486-repo-date-fields branch from e41612a to 1d4da08 Compare July 22, 2026 19:09
sankalpsthakur added a commit to sankalpsthakur/git-proxy that referenced this pull request Jul 22, 2026
Remove backfillRepoDates and startup hooks; existing-repo migration
will follow in a separate PR with a versioned migration framework.

Co-authored-by: Cursor <cursoragent@cursor.com>
@sankalpsthakur
sankalpsthakur force-pushed the prep/1486-repo-date-fields branch from 1d4da08 to f74be7d Compare July 22, 2026 19:10
Remove backfillRepoDates and startup hooks; existing-repo migration
will follow in a separate PR with a versioned migration framework.
@sankalpsthakur
sankalpsthakur force-pushed the prep/1486-repo-date-fields branch from f74be7d to 065fab5 Compare July 22, 2026 19:10
@sankalpsthakur

Copy link
Copy Markdown
Author

Thanks @andypols — agreed on deferring migration to a dedicated PR.

Changes in 065fab5:

  • Removed backfillRepoDates from both Mongo and NeDB adaptors
  • Removed the startup backfill hooks in src/db/index.ts
  • Dropped the related unit tests

This PR now only adds dateCreated / lastModified on new repos and bumps lastModified on user-role mutations. I'll follow up separately with a versioned migration framework and use $toDate on _id for existing Mongo records as you suggested.

NeDB adaptor already bumps lastModified on all user-role mutations;
extend finos#1486 date-field tests to assert removeUserCanAuthorise preserves
dateCreated while advancing lastModified.
@sankalpsthakur
sankalpsthakur force-pushed the prep/1486-repo-date-fields branch from fe8f6e3 to 10f21a6 Compare July 22, 2026 19:34
@sankalpsthakur

Copy link
Copy Markdown
Author

The migration/backfill code remains removed as requested. I have now completed the two available behavior checks and updated the PR checklist:

  • npm test -- test/db: 163 passed; 45 external-Mongo integration tests skipped because no MongoDB service was configured.
  • A direct in-memory NeDB exercise created a real repo record, confirmed ISO-8601 dateCreated === lastModified, then ran add/remove mutations for both canPush and canAuthorise; each mutation advanced lastModified while preserving dateCreated.

Sorting remains explicitly deferred to #1464. Could you please re-review the narrowed persistence-only PR when convenient?

@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.23%. Comparing base (9bc9c0d) to head (10f21a6).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1656      +/-   ##
==========================================
+ Coverage   85.97%   86.23%   +0.26%     
==========================================
  Files          87       87              
  Lines        8621     8646      +25     
  Branches     1492     1494       +2     
==========================================
+ Hits         7412     7456      +44     
+ Misses       1178     1160      -18     
+ Partials       31       30       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@sankalpsthakur

Copy link
Copy Markdown
Author

@andypols Migration/backfill is out of this PR as requested — persistence-only now. Mind taking another look?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants