Skip to content

CLARIN-DSpace upgrade to v.767 - #201

Open
xlukacm wants to merge 1599 commits into
ufal:clarin-v7from
dataquest-dev:clarin-v7-merge-767
Open

xlukacm wants to merge 1599 commits into
ufal:clarin-v7from
dataquest-dev:clarin-v7-merge-767

Conversation

@xlukacm

@xlukacm xlukacm commented Nov 12, 2026 •

Copy link
Copy Markdown

Description

Upgrade CLARIN DSpace to v.767

tdonohue and others added 30 commits February 18, 2026 16:00
…nts-7x

[Port dspace-7_x] Update DSO creation and edition selectors in admin menu to use more specific authorization endpoints (Includes port of DSpace#4639)
- In 7.6.0, ./cypress/integration/ dir was renamed to ./cypress/e2e/
- Related PR: DSpace#2127

(cherry picked from commit 359d355)
[Port dspace-7_x] Update Cypress section in README.md
Bumps [isbot](https://github.com/omrilotan/isbot) from 5.1.32 to 5.1.35.
- [Changelog](https://github.com/omrilotan/isbot/blob/main/CHANGELOG.md)
- [Commits](https://github.com/omrilotan/isbot/commits)

---
updated-dependencies:
- dependency-name: isbot
  dependency-version: 5.1.35
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [Wandalen/wretry.action](https://github.com/wandalen/wretry.action) from 1.3.0 to 3.8.0.
- [Release notes](https://github.com/wandalen/wretry.action/releases)
- [Commits](Wandalen/wretry.action@v1.3.0...v3.8.0)

---
updated-dependencies:
- dependency-name: Wandalen/wretry.action
  dependency-version: 3.8.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [korthout/backport-action](https://github.com/korthout/backport-action) from 2 to 4.
- [Release notes](https://github.com/korthout/backport-action/releases)
- [Commits](korthout/backport-action@v2...v4)

---
updated-dependencies:
- dependency-name: korthout/backport-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/add-to-project](https://github.com/actions/add-to-project) from 1.0.0 to 1.0.2.
- [Release notes](https://github.com/actions/add-to-project/releases)
- [Commits](actions/add-to-project@v1.0.0...v1.0.2)

---
updated-dependencies:
- dependency-name: actions/add-to-project
  dependency-version: 1.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4 to 6.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](actions/setup-node@v4...v6)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/cache](https://github.com/actions/cache) from 4 to 5.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](actions/cache@v4...v5)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps the testing group with 2 updates in the / directory: [axe-core](https://github.com/dequelabs/axe-core) and [ng-mocks](https://github.com/help-me-mom/ng-mocks).


Updates `axe-core` from 4.11.0 to 4.11.1
- [Release notes](https://github.com/dequelabs/axe-core/releases)
- [Changelog](https://github.com/dequelabs/axe-core/blob/develop/CHANGELOG.md)
- [Commits](dequelabs/axe-core@v4.11.0...v4.11.1)

Updates `ng-mocks` from 14.14.0 to 14.15.0
- [Release notes](https://github.com/help-me-mom/ng-mocks/releases)
- [Changelog](https://github.com/help-me-mom/ng-mocks/blob/master/CHANGELOG.md)
- [Commits](help-me-mom/ng-mocks@v14.14.0...v14.15.0)

---
updated-dependencies:
- dependency-name: axe-core
  dependency-version: 4.11.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: testing
- dependency-name: ng-mocks
  dependency-version: 14.15.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: testing
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps the sass group with 1 update: [sass](https://github.com/sass/dart-sass).


Updates `sass` from 1.94.2 to 1.97.1
- [Release notes](https://github.com/sass/dart-sass/releases)
- [Changelog](https://github.com/sass/dart-sass/blob/main/CHANGELOG.md)
- [Commits](sass/dart-sass@1.94.2...1.97.1)

---
updated-dependencies:
- dependency-name: sass
  dependency-version: 1.97.1
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: sass
...

Signed-off-by: dependabot[bot] <support@github.com>
[Port dspace-7_x] fix: distortion of community/collection logo
…overflow_7_x

[Port dspace-7_x] Fix: Full item view - bitstreams continuous text overflow in full file section
…dspace-7_x/actions/cache-5

Bump actions/cache from 4 to 5
…dspace-7_x/actions/setup-node-6

Bump actions/setup-node from 4 to 6
…dspace-7_x/actions/add-to-project-1.0.2

Bump actions/add-to-project from 1.0.0 to 1.0.2
…pace-7_x/multi-9b612d52e6

Bump lodash and @types/lodash
…pace-7_x/sass-9edc815acf

Bump sass from 1.94.2 to 1.97.1 in the sass group
…pace-7_x/testing-090809f7a5

Bump the testing group across 1 directory with 2 updates
…pace-7_x/isbot-5.1.35

Bump isbot from 5.1.32 to 5.1.35
…pace-7_x/express-4.22.1

Bump express from 4.21.2 to 4.22.1
Bumps the eslint group with 1 update: [eslint-plugin-jsonc](https://github.com/ota-meshi/eslint-plugin-jsonc).


Updates `eslint-plugin-jsonc` from 2.21.0 to 2.21.1
- [Release notes](https://github.com/ota-meshi/eslint-plugin-jsonc/releases)
- [Changelog](https://github.com/ota-meshi/eslint-plugin-jsonc/blob/master/CHANGELOG.md)
- [Commits](ota-meshi/eslint-plugin-jsonc@v2.21.0...v2.21.1)

---
updated-dependencies:
- dependency-name: eslint-plugin-jsonc
  dependency-version: 2.21.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: eslint
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [@types/lodash](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/lodash) from 4.17.23 to 4.17.24.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/lodash)

---
updated-dependencies:
- dependency-name: "@types/lodash"
  dependency-version: 4.17.24
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [sortablejs](https://github.com/SortableJS/Sortable) from 1.15.6 to 1.15.7.
- [Release notes](https://github.com/SortableJS/Sortable/releases)
- [Commits](SortableJS/Sortable@1.15.6...1.15.7)

---
updated-dependencies:
- dependency-name: sortablejs
  dependency-version: 1.15.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [axios](https://github.com/axios/axios) from 1.13.2 to 1.13.5.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md)
- [Commits](axios/axios@v1.13.2...v1.13.5)

---
updated-dependencies:
- dependency-name: axios
  dependency-version: 1.13.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
…dspace-7_x/Wandalen/wretry.action-3.8.0

Bump Wandalen/wretry.action from 1.3.0 to 3.8.0
milanmajchrak and others added 30 commits October 27, 2026 17:16
Since 7.6.7 the locale interceptor reads the logged in user's language through an EPerson request. When that request goes stale (after creating an EPerson, or when the cache expires after 15 minutes), its reload passes the same interceptor and waits for itself. All other REST requests then wait too and the UI stops loading data. EPerson requests now skip the EPerson language, like the root request already does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DSpace/dspace-angular PR 5323 fixed the same freeze in main and was
backported to 8.x and 9.x, not to 7.x. Using the same code and tests
keeps later vanilla merges free of conflicts. The spec also resets the
spy calls, so each test only sees its own request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Node 18 is end of life since April 2025 and gets no security fixes,
including CVE-2026-111497 in the Brotli decoder bundled with Node.
DSpace 7.6.7 also ships on Node 22.

CI now tests on Node 20 and 22.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…origin and language (#1669)

* Load SSR static content from the configured UI origin, not the Host header

During SSR, HtmlContentService built the static-file URL from the request
protocol and Host header. A request with a spoofed Host made the server fetch
/static-files/*.html from an attacker origin and render it unsanitized
(dsSafeHtml); the SSR page caches, keyed by path only, could then serve that
render to other visitors.

Use the origin of the configured ui.baseUrl (the address the Node server
listens on) instead and drop the now unused REQUEST injection.

dspace-customers#1083, finding R22.

* Key SSR page caches by origin and language, not only by path

getCacheKey returned req.url, but an anonymous SSR render also depends on the
request origin (protocol and Host header, e.g. citation_pdf_url via
getCurrentOrigin) and on the dsLanguage cookie. A request with a forged Host or
another language filled the bot and anonymous caches for everyone requesting
that path.

Move the key to getSsrCacheKey so it can be unit tested and include protocol,
Host and the language cookie. The /reload check now reads req.url, because the
key no longer starts with the path.

dspace-customers#1083, finding R22.

* Skip the SSR bot cache for authenticated requests

cacheCheck read the bot cache for any bot user agent before checking for an
authentication cookie, so an authenticated request with a bot user agent got
the anonymous cached page instead of its own render. Read either cache only
for anonymous requests; saveToCache already stores anonymous renders only.

dspace-customers#1083, finding R22.

* Do not load static page names that leave static-files once decoded

The static page file name comes from the URL. Express decodes the path before
serving files, so /static/..%2Findex loaded the app's own index.html into the
page. Return no content for a name that contains a path separator after
decoding (the query part is ignored), which renders the existing 404.

dspace-customers#1083, finding R22.

---------

Co-authored-by: jm <jm@maz>
…vity (#1668)

* Fix dspace-customers#1083 R23: gate Matomo on Klaro consent

main.browser.ts loaded matomo.js and queued tracking on every page load,
before and regardless of cookie consent; Klaro had no Matomo service.

- Klaro gets an opt-in "matomo" service whose callback drives
  MatomoConsentTracker.
- The tracker loads matomo.js only with consent and a configured
  hostUrl and siteId. Without consent nothing is pushed to _paq; only
  the current page is kept and tracked once consent is given.
- Revocation calls forgetConsentGiven, and a request processor empties
  requests built while consent is revoked, so Matomo's consent queue
  cannot replay them after a later accept.
- enableCookieConsentPopup: false keeps tracking without consent.

* R23: keep Matomo consent state consistent and pin the init wiring

- Reset the pending item dimension on every consent change, and let
  each page_view event set or clear it, so an identifier seen in one
  consent period cannot be attached to a page view in another.
- Push setConsentGiven as a function: Matomo runs queued
  'setConsentGiven' names before other commands, so accept, revoke,
  accept before matomo.js loaded left Matomo in the revoked state.
- Treat a blank hostUrl or siteId as not configured.
- Test that BrowserInitService routes Angulartics Matomo tracking
  through the consent gate.
- Define MATOMO_KLARO_KEY next to the other Klaro service keys.

---------

Co-authored-by: jm <jm@maz>
…#1663)

* Harden dsAuthInfo auth cookie with Secure and SameSite attributes

Set secure and sameSite='lax' on the auth JWT cookie written in storeToken(),
as defense-in-depth. The Secure flag is derived from the actual page protocol
(location.protocol === 'https:') so it is applied on any https page, including
behind a TLS-terminating reverse proxy, while local http dev still works.
'lax' is used rather than 'strict' so the cookie is still sent on the top-level
navigation that returns from external authentication.

* docs: shorten auth cookie security comment

---------

Co-authored-by: Matus Kasak <matus.kasak@dataquest.sk>
Pages that skip SSR (search, browse, admin) load the trimmed config.json. Since 7.6.7 the browser reads ui.baseUrl for the statistics referrer, citation URLs and the legacy bitstream redirect, and it was undefined there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add dtq-dev-767-base to the docker.yml push branches so pushes to this
base branch build and push the Docker images, mirroring how
dtq-dev-9-base generates its images.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
build.yml keeps upload-artifact v7 from the 7.6.7 branch and uploads
coverage from Node 22. Dockerfile.dist keeps the 7.6.7 layout, which
already runs on Node 22.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ci: build Docker images on push to dtq-dev-767-base
Port the instance deploy setup from dtq-dev-9-base (#1383 and follow-ups)
to dtq-dev-767-base, retargeted from 8603 to 8633.

- deploy.yml: replace the stale dev-5 jobs (instances 5/8) with probe,
  deploy-8633 and import-8633 on dspace-${DEV_MACHINE}-dep-1. Adds
  DEV_MACHINE, DUMP_PATH, ASSETSTORE_PATH and PROBE_ONLY inputs.
- start.sh: honour INSTANCE_OVERLAY so the per-instance overlay's
  `ports: !override` replaces the fixed host ports; set
  COMPOSE_PROJECT_NAME for the CLI container.
- docker.yml: deploy after a push via this branch's deploy.yml to 8633
  instead of dtq-dev's deploy.yml to instance 5.
- Replace import-weekly.yml (erase + import of instance 8) with the
  non-destructive redeploy-weekly.yml.
- playwright-tests.yml: point HOME_URL at dev-6.pc:8633.

Differences from 9-base: IMPORT defaults to false on manual runs and
DUMP_PATH has no default, since 8633 should start empty. The v7-specific
`chown dspace:dspace` on the assetstore and `-o dataquest` on
create-administrator are kept.
Restore deploy.yml verbatim from dtq-dev-9-base (f6cf600) with only
8603->8633, 6603->6633, dev-6->dev-20 substituted, plus the two v7
specifics: chown of the assetstore and `-o dataquest` on
create-administrator. Restore template wording in start.sh comment.
DUMP_PATH and DEV_MACHINE were interpolated straight into `run:` scripts,
so a crafted workflow_dispatch value could inject shell commands on the
self-hosted runner. Expose them as env vars and reference those instead.
The dev-5 dump is now anonymized (accounts, e-mails, tokens) before it is
placed on dev-20 as /opt/dspace-envs/8633/dump/dspace_dev5_dump_26.07.16_anon.sql.
…rrors

- psql loads the dump with ON_ERROR_STOP=1: the anonymized dump restores
  with zero errors, so any error now means a partial restore that later
  steps would not catch.
- The import replaces the whole DB, dropping the dev user start.sh
  created; recreate it, since the REST tests after import log in as it.
- The probe verdict printed "OK" when `compose config` failed and only
  looked for the dtq-dev-9-base fixed ports. Check `config -q` first and
  also flag the 6+ digit ports this branch builds from INSTANCE.
Co-authored-by: Matus Kasak <matus.kasak@dataquest.sk>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
ci(deploy): deploy dtq-dev-767-base to instance 8633 on dev-20
…wnload (#1720)

* fix(license): ensure IP address is included in anonymous user payload (ufal/dspace-angular PR 163)

* fix(license): ensure IP address is included in anonymous user payload

* test(license): add missing template dependencies to spec

(cherry picked from commit e183ad8)

* issue 162: filter clarinusermetadata/manage payload to contain only required metadata keys (ufal/dspace-angular PR 164)

* issue 162: filter clarinusermetadata/manage payload to contain only fields required by the license

* test(license): cover required-info payload filter and optional ORGANIZATION

Spec cases for ufal/dspace-angular PR 164 on top of the ufal/dspace-angular PR 163 spec setup:
- only metadata keys required by the license (plus the current IP) are sent
- an empty ORGANIZATION does not block "I agree"; an empty
  REQUIRED_ORGANIZATION still does

Both cases fail without ufal/dspace-angular PR 164's component change.

---------

(cherry picked from commit a026caf)

---------

(cherry picked from commit 33013a0)

Co-authored-by: Ondrej Kosarko <kosarko@ufal.mff.cuni.cz>
Co-authored-by: Amad Ul Hassan <hassan@ufal.mff.cuni.cz>
Co-authored-by: Milan Kuchtiak <kuchtiak@ufal.mff.cuni.cz>
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…e page (#1721)

* fix(static-page): interpolate  placeholders from config (ufal/dspace-angular PR 185)

(cherry picked from commit 45ab229)

* fix(static-page): read ${...} placeholder values from backend config (ufal/dspace-angular PR 188)

Fill the Terms of Service placeholders from /api/config/properties instead of
a new staticPage UI config section, as XMLUI did. Rename the placeholders to
the backend keys (description.*), join comma-split values with ", ", and
leave a placeholder as is when the property is missing or fails to load.
Requires ufal/clarin-dspace PR 1424.

(cherry picked from commit 550ec11)

---------

(cherry picked from commit 544e127)

Co-authored-by: Ondřej Košarko <kosarko@ufal.mff.cuni.cz>
Co-authored-by: Amad Ul Hassan <hassan@ufal.mff.cuni.cz>
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…1722)

* CorefUD v1.4 License Agreement

(cherry picked from commit 4e7d72c)

* Update by MN

license hasn't been used yet

(cherry picked from commit c6b80fb)

* New licence MWE-2.0

(cherry picked from commit 146fc6d)

* Add the UMR 2.2 license

(cherry picked from commit 46a9021)

* New license licence-unisegs-1.5.html

(cherry picked from commit 1d4906f)

* Copied previous UD license.

(cherry picked from commit 8aa0ab4)

* Changes for the upcoming UD release.

(cherry picked from commit 958827e)

* add cwl 1.0 license text

(cherry picked from commit fc92e44)

* license for atrium nonpublic record

(cherry picked from commit 07d11b6)

---------

(cherry picked from commit a19cb74)

Co-authored-by: Ondrej Kosarko <kosarko@ufal.mff.cuni.cz>
Co-authored-by: Jan Štěpánek <jan.stepanek@matfyz.cz>
Co-authored-by: Dan Zeman <zeman@ufal.mff.cuni.cz>
Co-authored-by: Ondřej Košarko <ko_ok@centrum.cz>
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…ed formats (#1723)

* docs(static): refresh deposit page for v7 UI (ufal/dspace-angular PR 183)

* docs(static): refresh deposit page for v7 UI

* docs(static): fix grammar and Czech icon label per review

* docs(static): cap deposit images to container width

(cherry picked from commit 60b2d5d)

* Point depositors to LINDAT format recommendations in upload step and FAQ

Add a sentence to the upload step info alert (en, cs) linking the LINDAT
centre page in the CLARIN Standards Information System, restoring the
format hint the DSpace 5 upload step had. Repoint the static FAQ
(en, cs) from the generic SIS recommended-formats list to the same
centre page.

(cherry picked from commit 29dfc5b)

---------

(cherry picked from commit fe1f7c4)

Co-authored-by: Amad Ul Hassan <hassan@ufal.mff.cuni.cz>
Co-authored-by: Ondrej Kosarko <kosarko@ufal.mff.cuni.cz>
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…pty file description (#1724)

* fix(subject-badge): add text-wrap and text-left to prevent overflow

(cherry picked from commit 5f5b9f4)

* fix(file-description): display "N/A" when file description is empty (ufal/dspace-angular PR 167)

* fix(file-description): display "N/A" when file description is empty

* chore: apply review fixes - restore missing i18n keys and use robust test selectors

* chore(i18n): restore blank lines in cs.json5 removed by sync-i18n

Keep the key moves and new translations from the sync, but put back
the blank-line separators so the diff only shows real changes.

* chore(i18n): restore cs.json5 values mangled by sync-i18n on duplicate keys

sync-i18n mixes up keys that appear twice in en.json5/cs.json5 and put
the first Czech translation on the second (effective) entry. Restore
the clarin-v7 values for admin.update-config.file-info.modified,
admin.update-config.success.save.title and
admin.update-config.error.save.message, and drop the bogus TODOs.

---------

(cherry picked from commit 3eaf3f4)

---------

(cherry picked from commit 7f16e81)

Co-authored-by: Ondrej Kosarko <kosarko@ufal.mff.cuni.cz>
Co-authored-by: amadulhaxxani <hassan@ufal.mff.cuni.cz>
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
… path (#1725)

No functional change. Adds the commented-out line
`#- /opt/DATA/assetstore:/dspace/assetstore` that customer/lindat has under the
assetstore volume.

(cherry picked from commit 471d2f6)
(cherry picked from commit f9aa172)

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
Same as dtq-dev does for instances 5 and 8.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The merge dropped the gitlink but kept its .gitmodules entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.