Skip to content

Add isolde rules - #572

Merged
zzacharo merged 4 commits into
masterfrom
add-isolde-rules
Aug 26, 2026
Merged

Add isolde rules#572
zzacharo merged 4 commits into
masterfrom
add-isolde-rules

Conversation

@zzacharo

@zzacharo zzacharo commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Comment thread cds_migrator_kit/rdm/streams.yaml

Returns the model instance if found, otherwise None.
"""
models = _load_models(self.dojson_model.entry_point_models)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you don't have to load it again, it is loaded somewhere in the app already, if you pass the loaded models in the exception then we could just select by the key

Comment thread scripts/snapshot.sh
@zzacharo
zzacharo force-pushed the add-isolde-rules branch 5 times, most recently from 3a16f03 to 9e07d5b Compare August 7, 2026 13:49

__query__ = '693__.a:"CERN ISOLDE" AND (980__:ARTICLE OR 980__:PREPRINT OR 980__:conferencepaper OR 980__:NOTE OR 980__:REPORT) -980__:DELETED -980__.c:MIGRATED -980__:DUMMY'

__ignore_keys__ = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we import and concatenate with the keys specified in _config.py?

Comment thread cds_migrator_kit/rdm/migration_config.py
cli_logger.info(
f"[apprn] Found approval report number {identifier!r} — storing as apprn"
)
scheme = "apprn"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about adding them as apprn in the rule, if we' do this we need to change ep-approval workflow, see: restricted entry, public entry. We need to separate the report numbers somehow, for example faser records both have cern-ep-draft- and cern-ep- rep numbers, will this going to keep cern-ep-draft- as cdsrn?

I think we should raise if there's a ep report number and no ep information to check if there is any other record exists in the system. If we say we'll fix it when we have the error (when we found the internal record or etc) it might be more manual work to fix, wdyt?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DIscussed IRL: we should add a step to check for all the identified EP numbers and collect the duplicates

@zzacharo
zzacharo force-pushed the add-isolde-rules branch 2 times, most recently from 36860d6 to 4dbc74b Compare August 10, 2026 14:59
Comment thread cds_migrator_kit/rdm/records/transform/models/isolde.py Outdated
raise IgnoreKey("title")


_FUNDING_MODEL_MAP = {

@kpsherva kpsherva Aug 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should it be moved to config.py in transform folder? also the mapping is a bit strange...

"intnoteatspubl": {"id": "publication-note"},
"intnotehie-isoldepubl": {"id": "publication-note"},
"hie-isolde-project-notes": {"id": "publication-note"},
"intnotebepubl": {"id": "publication-note"},

@kpsherva kpsherva Aug 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

records from BE shouldn't be in this data set, unless ISOLDE owns them, same with ATS above (to be checked)

@zzacharo
zzacharo force-pushed the add-isolde-rules branch 3 times, most recently from 88a42c2 to acbb1dd Compare August 26, 2026 08:46
@zzacharo
zzacharo merged commit bc12e35 into master Aug 26, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

migration: ISOLDE

3 participants