Skip to content

VSB-TUO/Replace the retired Creative Commons API with the local CC legal tools CSV - #1645

Merged
xlukacm merged 13 commits into
customer/vsb-tuofrom
vsb-tuo/cc-license-local-csv
Nov 5, 2026
Merged

xlukacm merged 13 commits into
customer/vsb-tuofrom
vsb-tuo/cc-license-local-csv

Conversation

@xlukacm

@xlukacm xlukacm commented Nov 4, 2026 •

Copy link
Copy Markdown
Phases MP MM MB MR JM Total
ETA 0 0 0 0 0 0
Developing 0 0 0 0 0 0
Review 0 0 0 0 0 0
Total - - - - - 0
ETA est. 0
ETA cust. - - - - - 0

Problem description

Reported issues

The Creative Commons API (api.creativecommons.org) was shut down on 2026-11-02, so the cclicense submission step on VSB-TUO no longer loads licences, and accepting one fails.

Not-reported issues

None.

Analysis

Backport of the upstream fix DSpace/DSpace#12424 (issue DSpace/DSpace#11808), merged into main on 2026-11-04 at the head cherry-picked here (b12ab4f22d). The licences come from the CC legal tools CSV shipped in dspace/config/cc-license/cc-legal-tools.csv (dspace.cc-license.csv), so nothing calls Creative Commons at runtime. The 8 upstream commits are cherry-picked with -x; conflicts were only in unrelated dspace.cfg context and in mocks/ITs the PR deletes.

VSB-TUO commits on top:

  • keep the cclicense step enabled (reverts the upstream commit that comments it out, VSB enables it on purpose in VSB-TUO/Creative Commons license #1053)
  • Collectors.toList() instead of Stream.toList() for Java 11
  • cc.license.classfilter = CC3.0, recombo, mark, certification: with the old API and the jurisdiction commented out (VSB-TUO/Use international license 4.0. instead of czech 3.0. #1059), VSB submitters could only pick CC 4.0 International or CC0; the CSV lists 3.0 separately, so it is hidden to keep the same choice
  • existing items: the old API stored http:// URIs and the dc.rights.uri form field often has no trailing slash, and the CSV lookup was an exact match on https://…/, so re-accepting such a licence returned HTTP 500. The lookup now normalises scheme and trailing slash; the stored URI is not rewritten
  • the test local.cfg pins the upstream classfilter/jurisdiction the CC ITs assert

Behaviour changes for VSB:

  • cc.license.locale and cc.submit.addbitstream are gone: question labels come from the UI translations, and no license_rdf bitstream is added to new items (existing ones are still exported and removed when the licence changes)
  • new licences are stored as https://creativecommons.org/…, existing items keep http://
  • licence names (dc.rights) stay English, as the old API returned them

Tested: the CC ITs (CCLicenseConnectorServiceImplIT, SubmissionCCLicenseRestRepositoryIT, SubmissionCCLicenseUrlRepositoryIT, 28 tests), plus a local stack built from this branch and the matching frontend branch: new item, reopen, Czech UI, and an existing item with an http:// URI.

Requires the frontend PR: dataquest-dev/dspace-angular#1714

Problems

  • A non-CC URI typed into dc.rights.uri (e.g. a publisher licence) still gives HTTP 500 when the CC section is re-accepted, as before.
  • Upstream deleted CCLicenseAddPatchOperationIT / CCLicenseRemovePatchOperationIT, so the PATCH operations have no IT.
  • Upstream ported the fix to 8.x, 9.x and 10.x but not to 7.x, so this backport stays until VSB-TUO moves to a newer DSpace version.

🤖 Generated with Claude Code

Nathan Buckingham and others added 13 commits November 4, 2026 09:58
…ense name when document is null

(cherry picked from commit 4024db7)
…ues inside of cc.license.classfilter

(cherry picked from commit 075cb89)
…e classfilter list, allow excluding licenses based on UNIT column, check CC version, fix issues with array property

(cherry picked from commit 49b3155)
…ampling licenses to CSV. Fix CC license remove check to use URI instead of name

(cherry picked from commit d5d6d21)
Reverts the upstream commit 3945722 from DSpace#12424, which
comments the step out because vanilla DSpace ships it disabled. VSB-TUO
enables it on purpose (#1053).

This reverts commit 9e854be.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ava 11

Stream.toList() is Java 16+; customer/vsb-tuo builds with release 11.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the old API, cc.license.jurisdiction commented out (#1059) meant the
'standard' class always resolved to the international 4.0 licence, so VSB
submitters could only pick CC 4.0 or CC0. The CSV now lists 3.0 as its own
entry, so exclude it to keep the same choice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Items licensed through the retired CC API carry http URIs, and URIs typed
into the dc.rights.uri form field often lack the trailing slash. The CSV
lookup was an exact match on the https form, so re-accepting such a licence
in the submission failed with a 500 (CCLicenseAddPatchOperation). Normalise
the scheme and the trailing slash on both sides of the lookup; the stored
URI itself is left as it is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The CC licence ITs assert the upstream defaults (CC 3.0 offered, jurisdiction
us), which the VSB dspace.cfg changes on purpose.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An active finding remains because the URI-based license-removal guard lacks integration coverage for existing-license removal, the no-license error case, and the cc.submit.setname=false configuration.

1 open issue
What changed in this PR

This pull request replaces the retired Creative Commons API with a locally bundled CC Legal Tools CSV, preserving license selection while removing runtime API and RDF-license dependencies.

Changes:

  • Adds CSV-backed license loading, filtering, jurisdiction handling, URI resolution, and legacy URI normalization.
  • Updates Creative Commons services, REST repositories, Spring wiring, and configuration for local license data.
  • Stores metadata-only licenses for new submissions while cleaning up legacy license_rdf bundles.
  • Removes obsolete API mocks and RDF transformations and updates the remaining integration coverage.

Review findings remain unresolved: localized CSV titles for some 3.0 jurisdiction rows can overwrite dc.rights instead of preserving an English name (moderate, 1 vote). The add/update and remove PATCH flows also lack integration coverage after their tests were deleted (nits, 1 and 3 votes).

File Description
dspace/​config/​spring/​api/​core-services.xml Registers the CSV license repository bean.
dspace/​config/​dspace.cfg Configures the local CSV and VSB-specific license filtering.
dspace/​config/​cc-license/​cc-legal-tools.csv Bundles Creative Commons license metadata and canonical URLs.
dspace-server-webapp/​src/​test/​java/​org/​dspace/​license/​MockCCLicenseConnectorServiceImpl.java Removes the obsolete API mock.
dspace-server-webapp/​src/​test/​java/​org/​dspace/​app/​rest/​SubmissionCCLicenseUrlRepositoryIT.java Updates license URL resolution tests for CSV-backed licenses.
dspace-server-webapp/​src/​test/​java/​org/​dspace/​app/​rest/​SubmissionCCLicenseRestRepositoryIT.java Updates REST tests for local license definitions and filtering.
dspace-server-webapp/​src/​test/​java/​org/​dspace/​app/​rest/​matcher/​SubmissionCCLicenseMatcher.java Updates REST matchers for the new license representation.
dspace-server-webapp/​src/​test/​java/​org/​dspace/​app/​rest/​CCLicenseRemovePatchOperationIT.java Removes the obsolete remove-PATCH integration tests; replacement coverage is recommended.
dspace-server-webapp/​src/​test/​java/​org/​dspace/​app/​rest/​CCLicenseAddPatchOperationIT.java Removes the obsolete add-PATCH integration tests; replacement coverage is recommended.
dspace-server-webapp/​src/​test/​data/​dspaceFolder/​config/​spring/​api/​core-services-mock.xml Removes obsolete mocked connector wiring.
dspace-server-webapp/​src/​main/​java/​org/​dspace/​app/​rest/​submit/​factory/​impl/​CCLicenseRemovePatchOperation.java Detects licenses by URI and removes legacy license bundles; the removal path lacks integration coverage.
dspace-server-webapp/​src/​main/​java/​org/​dspace/​app/​rest/​repository/​SubmissionCCLicenseRestRepository.java Uses the default CSV-backed license definitions for REST responses.
dspace-api/​src/​test/​java/​org/​dspace/​license/​MockCCLicenseConnectorServiceImpl.java Removes the obsolete API mock.
dspace-api/​src/​test/​java/​org/​dspace/​license/​CCLicenseConnectorServiceImplIT.java Adds integration coverage for CSV loading, filtering, jurisdiction, URI resolution, and legacy URI handling.
dspace-api/​src/​test/​data/​dspaceFolder/​config/​local.cfg Sets test-specific Creative Commons filtering and jurisdiction settings.
dspace-api/​src/​main/​resources/​org/​dspace/​license/​LicenseCleanup.xsl Removes the obsolete license cleanup transformation.
dspace-api/​src/​main/​resources/​org/​dspace/​license/​CreativeCommons.xsl Removes the obsolete Creative Commons RDF transformation.
dspace-api/​src/​main/​java/​org/​dspace/​license/​service/​CreativeCommonsService.java Removes RDF-related service APIs and updates license addition signatures.
dspace-api/​src/​main/​java/​org/​dspace/​license/​LicenseCleanup.java Removes the obsolete RDF cleanup utility.
dspace-api/​src/​main/​java/​org/​dspace/​license/​CreativeCommonsServiceImpl.java Stores metadata-only licenses and removes legacy RDF bundles; PATCH add/update coverage is missing.
dspace-api/​src/​main/​java/​org/​dspace/​license/​CCLicenseCSVRow.java Defines the in-memory CSV license row model.
dspace-api/​src/​main/​java/​org/​dspace/​license/​CCLicenseCSVRepository.java Loads and indexes local CSV license data.
dspace-api/​src/​main/​java/​org/​dspace/​license/​CCLicenseConnectorServiceImpl.java Implements local license filtering, URI resolution, and legacy lookup; some 3.0 rows can provide localized names instead of English dc.rights values.
dspace-api/​src/​main/​java/​org/​dspace/​license/​CCLicenseConnectorService.java Updates the connector contract for local CSV license data.

Review effort: Lite

@xlukacm
xlukacm marked this pull request as ready for review November 5, 2026 06:44
@xlukacm
xlukacm removed the request for review from milanmajchrak November 5, 2026 06:46
@xlukacm
xlukacm merged commit def3014 into customer/vsb-tuo Nov 5, 2026
15 checks passed
@jr-rk
jr-rk deleted the vsb-tuo/cc-license-local-csv branch November 5, 2026 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants