Repository navigation
Copy-edit the Data Contracts spec text and test its examples against the shapes (#275) - #282
matthiasautrata wants to merge 20 commits into
Conversation
|
@matthiasautrata is attempting to deploy a commit to the EKGF Team on Vercel. A member of the Team first needs to authorize it. |
…EKGF#271) Resolves EKGF#271. §7.2 Step 1 collected a duty only if `matches(n, request)`, i.e. if the duty's own action and asset matched the request. A duty states what the bearer must do, not what is requested, so a request to display ex:marketData never collected the duty to report on ex:usageStats, and §12.3's example returned no duties. Step 2 then partitioned by `d.subject = grantor` / `d.subject = grantee`, so a duty without dprod:subjectOfDuty belonged to neither party, though specification §3.2 and term-mapping describe consumer duties without a subject. The ontology comment on dprod:subjectOfDuty contradicted those documents: it defaulted the bearer to the data product owner, which is the provider. - §3.2: before evaluation, a duty without a subject takes the policy's grantee, or the requesting agent in a policy without one. `d.subject` is then the bearer everywhere, including `performed`. A duty borne by the requesting agent stands for one duty per agent, tracked separately in Σ, so one agent's missed duty is not held against another. - §7.2: duties are collected from every policy with a permission in force for the request; grantor duties are those borne by the grantor, and all others are grantee duties. §12.2 matches. - Collecting every duty of a granting policy makes a provider's missed delivery reachable from every consumer request. Violated is terminal, so denying on it would lock the consumer out permanently. Step 6 now denies only on violations by the grantee's side; the grantor's are reported in `violations` on the Permit. - dprod:subjectOfDuty comment, specification §4.10, and term-mapping state the default. - Add tests/test_duty_bearer.py, which fails on the previous text. Rebased onto EKGF#268 (order-independent errors): a policy grants only if a permission is active (= true), and a permission in error in a policy that does not otherwise grant (E4) gives Failure on the Permit path, since resolving it could engage that policy's duties. Section 9.7 now also keeps the grantor violations reported with a Permit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[Carried forward onto develop for EKGF#275 by Matthias Autrata: conflicts resolved against EKGF#244/EKGF#245/EKGF#262/EKGF#268/EKGF#270/EKGF#272, the namespace changed from https://www.omg.org/spec/DPROD/dprod/ to https://ekgf.org/dprod/spec/develop/, and EKGF#244's duty-only shape and test migrated from dprod:recurrence to dprod:schedule, which is not duty-only.]
Shapes: accept tab-separated POSIX crontab fields and reject smuggled tab/CR/LF content; allow EXDATE/RDATE lines in RFC 5545 expressions while rejecting non-standard or duplicate RRULE parts; reject a dprod:scheduleTimeZone that conflicts with the embedded TZID; move the IRI-identifier message onto its own shape so it no longer annotates unrelated violations. Ontology: every dprod:ScheduleFormat now declares dprod:carriesTimeZone, and the timezone-required rule keys on it so extension formats get the same enforcement as POSIX crontab. Semantics: schedule errors freeze the affected duty (Rule D-FREEZE), surface in the result's errors field, and the processor registry is part of Σ, so the totality and determinism theorems hold for unprocessable schedules too. Tests and CI: every schedule case is validated both with the ontology graph and in the bare documented CLI configuration, making the sh:class constraints falsifiable; the obsolete-recurrence sweep covers specification.md; validate.py runs in the build workflow. Docs: define the previously missing example schedules, rewrite the RRULE components table as complete expressions, and sync the guide checklist and specification with the new rules. [Carried forward onto develop for EKGF#275 by Matthias Autrata: the build-spec.yml change is dropped (EKGF#262 already runs validate.py in CI), and the namespace in the new shapes is the develop one. Schedule errors are fitted to the error model of EKGF#267: instead of an `errors` field on a Permit result, an unprocessable schedule of a collected duty is gathered in E₅ and gives Failure on the Permit path, like E₃ and E₄. Deny and NotApplicable are unaffected, since resolving the schedule can only add duties or violations. EvaluationError admits ScheduleError, and §9.7's completion covers schedule errors.] Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…EKGF#184) The core spec's contract section, its worked contract example, the contracts examples under examples/contracts/ and the overview diagram were added after EKGF#234 and still used the removed dprod:recurrence. Each recurring duty now references an identified dprod:Schedule (an RFC 5545 DTSTART with TZID plus RRULE), embedded as a node object in the JSON-LD; all five documents expand without dropping a term and conform to the contracts shapes. The schedule identifier shape is renamed RejectAnonymousScheduleShape, following the Reject* convention, so the spec generator no longer renders it as a second Schedule class; it is listed with the other rejection shapes in the specification. Schedule and ScheduleFormat are now contract classes in the generated spec. dprod:schedule has no domain, so it is no longer expected among the properties extending ODRL classes. Refs: EKGF#184, EKGF#234, EKGF#275 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…KGF#273) Resolves: EKGF#273 Rewrites EKGF#274 on top of the reusable schedule model (EKGF#280), following the EKGF#184 anchor rule: the schedule's DTSTART/TZ sets the occurrence times, and activation only limits the range. §5.5 left scheduled duties unbounded and partly undefined: - expand(schedule, processors, clock) had no lower bound, so a rule without COUNT or UNTIL generated infinitely many instances. - The condition guard evaluated ⟦duty.condition⟧(Σ.env): Σ has no env, the guard did not depend on the occurrence time, and under EKGF#267 the condition is three-valued. - instantiate gave equal values for every t, and nothing in §5.2 or §7.2 generated or evaluated instances: the template itself was fulfilled or violated like any duty. - performed had no time bounds, so one delivery fulfilled every daily instance, past and future. §5.2 also had a stuck state: D-FULFILL required deadline ≥ clock at evaluation time, and D-VIOLATE required no performance, so an action taken before the deadline but evaluated after it left the duty Active. Changes: - expand takes a range (from, to) and returns the schedule's own times within it. The expression alone fixes the times (DTSTART/TZID for RFC 5545, COUNT and UNTIL counted from DTSTART); the range never moves them, and whether expand errs does not depend on it. occurrences uses the range [activatedAt, clock], so a template never activated has no instances. The truncate(anchor, FREQ) of the previous EKGF#274 is no longer needed, since DTSTART fixes the time of day. - The template's condition is evaluated once, by D-ACTIVATE (= true). An error there leaves it Pending and is collected in E₃. - An instance is identified by (template, t), and is Active with activatedAt = t, so relative deadlines run from t. updateDutyStates updates instances after templates; templates are neither fulfilled nor violated, and Steps 6 and 8 of §7.2 report instances through tracked(D). - performed takes a window (since(d), effectiveDeadline(d, Σ)). An instance counts only actions at or after t; other duties count actions from their policy's effectiveDate (EKGF#269). D-FULFILL no longer compares the deadline with the evaluation time. A duty without an asset matches any performed asset. - Specification §4.3 and the dprod:schedule comment say the same. - Add tests/test_schedule_bounds.py, which fails on the previous text. Refs: EKGF#184, EKGF#274, EKGF#275, EKGF#280 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Under the instance generation in this PR, an occurrence becomes an instance only once its scheduled time has passed, and it is Active from then on. The example's 6 February instance, recorded as Pending and "not yet due", therefore cannot exist. It is removed, and the README says when an occurrence becomes an instance. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The overview gave each instance a "fulfilment window (dprod:deadline)". Under this PR the window runs from the occurrence to the deadline, and an unscheduled duty can have a deadline as well. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A readability pass over the text added in EKGF#259 and EKGF#261: - Prohibition-wins is the profile's default conflict strategy, not a fixed rule (EKGF#197, EKGF#279). The overview bullet and the data use policy example now say "by default". - The deterministic-evaluation bullet leads with its claim and is split into sentences. - The overview says that lifecycle status and duty state share the same four optional concepts but use different properties, which the "kept apart on purpose" sentence left unexplained. - A DPROD-aware processor "evaluates" rather than "enforces" (EKGF#263). - The figure sentence had an "it" with no antecedent. - Two descriptions went stale when EKGF#261 simplified the examples: the data offer example has no domain vocabulary, parties or duty instances, and only some examples declare terms inline. - The data offer README listed four points after "Three things", and repeated the spec's note on plain ODRL names almost verbatim. - Typos: "soem", "[[dprod]]]". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…KGF#275) The spec says every Data Contract worked example validates against dprod-contracts-shapes.ttl, but nothing checked it: validate.py and the CI job from EKGF#262 cover only the Turtle examples in dprod-contracts/examples/. tests/test_examples.py now validates the contracts examples, their README blocks and the spec's eg-contract with the same settings as validate.py (RDFS inference, the vocabulary as ont_graph, advanced SHACL). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…KGF#275) A DataContract is an odrl:Agreement and, like any agreement, complete in itself: dprod:acceptsOffer records which offer it came from but contributes no rules. The example contracts now restate the offer's targets, permissions, prohibitions and duties they take over, and the vocabulary, specification, guide, overview and formal semantics say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A policy's operands say which facts it needs, and the source follows from who knows each fact. Facts the platform already holds about assets and parties (classification, retention, completeness, timeliness, volume, role) now come from the state snapshot; only what the requester alone can say, such as purpose, environment and processing mode, stays on the request. The policy writers' guide states the rule in its operand reference, and the data use policy and ODCS READMEs say which facts the caller must assemble. The example operand descriptions now describe the measured fact, not a requirement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A contract records the covenant, not what happened under it. The two evaluated delivery records sat in the contract's odrl:obligation, where an evaluator would read them as two more clauses of the agreement. The example now states only the daily delivery duty, and the README explains that instance state is kept outside the contract and supplied with each evaluation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A dprod:deadline runs forward from when a duty becomes active, so a notify duty with a P14D deadline meant "notify within 14 days", and was violated after 14 days even with no change planned. DPROD cannot yet express a notice period before a later action. The notify duties keep their bearer and target but lose the deadlines, and the provider duties README, the contracts guide and the term mapping say what cannot be expressed yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…EKGF#275) The notify duties in the provider duties example, baseline.ttl and data-contract.ttl record the agreed notice period with ex:noticePeriod. A footnote in the README, the contracts guide and the term mapping says DPROD does not evaluate it yet. EKGF#284 proposes a notice-period duty. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A DataContract is an odrl:Agreement, complete in itself, and dprod:acceptsOffer records the offer it accepts. "Activated offer" suggested the contract is the offer in another state. The class description, the specification's definition, the overview and the policy writers' guide now say the contract accepts an offer and binds both parties to the rules it states. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A constraint on an ODRL rule is a condition on the rule (ODRL 2.2 §2.5.3), and DPROD reads a duty's constraint as its activation condition (Rule D-ACTIVATE). The quality duties put the required level there, so "conform to, completeness = 100" read as "conform once completeness is 100" and could never be violated by incomplete data. ODRL says what a duty requires through its action: a duty is fulfilled when its action, with all refinements satisfied, has been exercised (§2.6.3). - Examples: the quality levels (timeliness, completeness, row count, latency) and the support channels and service window move from the duty's odrl:constraint to an odrl:refinement of its action, written odrl:action [ rdf:value ... ; odrl:refinement ... ]. Triggers stay constraints: retention, classification, purpose. The ODCS delivery duty loses its purpose = regulatory constraint, since ODCS driver says why the SLA exists, not when it applies. - Semantics: Duty gains a refinement field, carried to schedule instances. Refinements are recorded as agreed terms; Rules D-FULFILL and D-VIOLATE still match the action alone. - Shapes: ActionShape accepts an IRI or a refined action with one rdf:value; permissions and prohibitions reject refinements, since an unevaluated one would change what they grant or forbid. New tests. - Prose: the provider duties README, the guide, the term mapping and the ODCS comparison say what a refinement states and what a constraint states. odrl:refinement is coerced as IRI-valued in the contexts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The formal semantics called an Offer a proposal, but Eval treated it like a signed policy: an open offer's permissions granted to any requesting agent and its duties activated. An Offer states the rules its grantor proposes and binds no one until an Agreement restates them, so PolicyApplicable is now false for an Offer, and §3.4 and §7.4 no longer describe offers as granting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README said DPROD-aware processors enforce lifecycle; DPROD declares what a policy grants, forbids and obliges, and evaluators compute duty state. It also counted 8 offers in baseline.ttl (there are 9), and left odcs.ttl, odcs.md, validate.py and tests/ out of the repository tree and the quick start. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dbff08a to
c55b2df
Compare
Resolves the conflicts with develop. EKGF#280 and EKGF#272 landed as squash merges, so this branch's copies of their commits conflicted with them. The resolved tree is this branch's own commits and EKGF#274's replayed on develop: - Shapes use the dprod-contracts-shapes: prefix that develop adopted (EKGF#295), including the new ActionShape and the specification's shape tables. - ActionShape gains the dct:description every contracts shape carries since EKGF#302. Core 93 OK, contracts 121 OK, validate.py clean (Python 3.13, as CI). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jgeluk
left a comment
There was a problem hiding this comment.
Conflicts resolved in e7fa44c (merge of develop, no force-push). #280 and #272 landed as squash merges, so this branch's copies of their commits conflicted with them. The resolved tree is this branch's own commits plus #274's, replayed on develop. Two changes beyond picking sides:
developrenamed the contracts shapes prefix todprod-contracts-shapes:(#295). The newActionShapeand the specification's shape tables now use it.ActionShapegains thedct:descriptionthat every contracts shape carries since #302.
Core 93 OK, contracts 121 OK, validate.py clean, generator clean (Python 3.13, as CI; 3.15 fails 19 JSON-LD tests on plain develop too). Note that #274's commits are in this branch, so merging this lands #274.
Review
- A refined duty is reported Fulfilled when its required level is missed. Moving quality levels from constraint to refinement fixes "incomplete data can never violate" (the duty never activating). But D-FULFILL and D-VIOLATE match on the action alone, so a nightly delivery at 80% completeness against a
completeness eq 100refinement now evaluates to Fulfilled. That reads as compliance. Either say in §5.2 / the guide that Fulfilled on a refined duty means only that the action was exercised, or have the evaluator return the duty's unevaluated refinements with its state, so callers can't mistake one for the other. - §7.3 has overlapping
PolicyApplicableclauses. The genericPolicyApplicable(p, Env)(validity window) still matches an Offer, alongside the newPolicyApplicable(Offer(...), Env) = false. The Agreement clause uses... ∧, so it extends the generic one, but the Offer clause replaces it. Please write it as an explicit case split on the policy type, or say that the Offer clause overrides. As it stands, two readings are possible. - Offers keep
effectiveDate/expirationDatein the §3.4 grammar but never reach the validity check now. It's worth one sentence on what an offer's dates mean, presumably lifecycle and catalogue only. - Interop.
odrl:actionobjects can now be blank nodes ([ rdf:value …; odrl:refinement … ]). That is valid ODRL 2.2, but consumers that read the action as an IRI (SPARQL over?rule odrl:action ?a) will now get a blank node for refined duties. Consider a note in the term mapping. - PR description. It cites commit hashes (
95cc248,4b6e87f,df96f04,904a512,f9d2f0c,dbff08a) that aren't on the branch. Also, "Merge after #264, #266, #270 and #274": only #274 is still open.
The rest looks good: the contract-is-authoritative decision, the state-snapshot sourcing rule, dropping the bogus P14D lead time, and the new JSON-LD shape-validation test (which would have caught the prefix issue above).
Part of #275. This is a readability pass over the Data Contracts text added in #259 and #261. It also adds the test that the spec's validation claim was missing, it settles what
dprod:acceptsOffermeans, it says where each fact a policy reads comes from, it keeps evaluator output out of contracts, it records change-notice periods as agreed terms rather than claiming a lead time, it states quality levels as refinements of a duty's action rather than as its activation condition, and it stops evaluating an offer as if it had been signed.Merge after #264, #266, #270 and #274. This branch merges all four, so until they land the diff includes their commits.
Copy edits (
b78d8c7)eg-contractpromised a full example with a domain vocabulary, parties and duty instances. The data offer example has none of these.Test (
4d0d41f)The spec says every Data Contract worked example validates against
dprod-contracts-shapes.ttl, but nothing checked this.validate.pyand the CI job from #262 cover only the Turtle examples indprod-contracts/examples/.tests/test_examples.pynow validates every contracts example, every README block and the spec'seg-contract. It uses the same settings asvalidate.py: RDFS inference, the vocabulary asont_graph, and advanced SHACL. I checked that it fails when an example is broken (a missing assignee, a string deadline).A contract is authoritative (
4ea9f1c)The examples left it open whether a contract inherits its offer's rules. Most contracts stated only some of them, and the collections contract stated none. Nothing in the formal semantics gives
dprod:acceptsOfferan evaluation meaning, so the contracts bound fewer rules than their prose claimed.Decision: a
dprod:DataContractis anodrl:Agreement, and like any ODRL agreement it is complete in itself.dprod:acceptsOfferrecords lineage only and contributes no rules. A contract restates the offer's rules it takes over, and it may narrow them, override them or add duties on the consumer.eg-contractrestates the offer's three rules.dprod:acceptsOfferdescription indprod-contracts.ttland §4.4 of the specification now say the link records lineage. The formal semantics' Agreement note and the guide follow, and the guide no longer calls a contract an "activated offer".Where facts come from (
2e9831b)Since #268 a fact the evaluator cannot find is an error, not
false. The examples read some facts from the request that a requester cannot vouch for, such as the data's classification, retention, completeness and timeliness. Nothing said what the calling system has to send.The rule: a policy's operands say which facts it needs, and the source follows from who knows each fact. Facts the platform already holds about assets and parties go in the state snapshot. The agent and the clock come from the context. Only what the requester alone can say goes on the request: purpose, environment, processing mode and the like. The calling system assembles all three before it calls the evaluator.
ex:recipientTypestays on the request, since the recipient need not be the requesting agent.ex:classification,ex:retentionPeriod,ex:completeness,ex:volumeCountandex:timelinessnow usedprod:stateSourcein the data use policy, ODCS and provider duties JSON-LD examples, inbaseline.ttl,data-use-policy.ttlandodcs.ttl, and in the profile README's operand sample. The data use policy and ODCS READMEs say which facts come from where, and that a missing fact gives an evaluation failure, not a permit.ex:retentionPeriodandex:timelinesswere described as a maximum and a requirement. The requirement is the constraint's right operand, so they now describe the measured fact.A contract is not a log (
aff4025)The lifecycle contract listed two evaluated deliveries, Fulfilled and Violated, in its
odrl:obligation. A contract records the covenant, not what happened under it, and an evaluator would read the two records as two more clauses. The example now states only the daily delivery duty. The README explains that instance state is kept outside the contract and supplied with each evaluation. #283 covers how such state should be recorded, and the Turtle examples' "Duty Instances" sections.No lead time for change notification (
95cc248)The examples modelled "14 days' notice of a change" as a notify duty with a
P14Ddeadline. A deadline runs forward from when the duty becomes active, so the duty meant "notify within 14 days", and the provider was in violation after 14 days even with no change planned. DPROD cannot yet express a notice period before a later action. The notify duties in the provider duties example,baseline.ttlanddata-contract.ttlkeep their bearer and target but lose the deadline. Instead (4b6e87f), each notify duty records the agreed notice period with the example's ownex:noticePeriod, as an agreed term. A footnote in the provider duties README, the contracts guide and the term mapping says DPROD does not evaluate it yet. #284 proposes a notice-period duty tied to the action.What a duty requires (
df96f04)A constraint on an ODRL rule is a condition on the rule (ODRL 2.2 §2.5.3), and DPROD reads a duty's constraint as its activation condition (Rule D-ACTIVATE). The quality duties put the required level there. "Conform, with completeness = 100" therefore meant "conform once completeness is 100", so incomplete data could never violate it. ODRL states what a duty requires through its action: a duty is fulfilled when its action, with all refinements satisfied, has been exercised (§2.6.3).
odrl:constraintto anodrl:refinementof its action, writtenodrl:action [ rdf:value ex:conformTo ; odrl:refinement [...] ]. Conditions on when a duty applies stay constraints: retention, classification and purpose. The ODCS delivery duty loses itsodrl:purpose eq ex:regulatoryconstraint, because ODCSdriversays why the SLA exists, not when it applies.Dutygains arefinementfield, which schedule instances carry over. The profile records refinements as agreed terms. Rules D-FULFILL and D-VIOLATE still match on the action alone. We are declaring what the parties agreed, not adding an enforcement point.ActionShapeaccepts an IRI, or a refined action with exactly onerdf:value. Permissions and prohibitions reject refinements, since an unevaluated refinement would change what they grant or forbid.test_action_refinement.pycovers both.odrl:refinementis coerced as IRI-valued in the JSON-LD contexts.904a512drops the leftover "activated offer" wording fordprod:DataContract. The class description, the specification's definition, the overview and the policy writers' guide now say that a contract accepts an offer and binds both parties to the rules it states.An offer binds no one (
f9d2f0c)The formal semantics called an Offer a proposal, but
Evaltreated it like a signed policy. An open offer's permissions granted to any requesting agent (§7.4: no assignee means any agent), and its duties activated. §7.3 now makesPolicyApplicablefalse for an Offer, so an offer passed toEvalcontributes no permission, prohibition or duty. Its rules take effect only when an Agreement restates them, which follows from the "contract is authoritative" change above. §3.4 and §7.4 no longer describe offers as granting. Offers are still validated by SHACL. The prose already treated offers as proposals, so only the formal semantics changes.Contracts README (
dbff08a)baseline.ttlhas 9 offers, not 8.odcs.ttl,odcs.md,validate.pyandtests/, and the quick start lists the ODCS and baseline examples.The
dprod:recurrencewording in this file is replaced by #280 and #274.Tests
validate.pyclean.🤖 Generated with Claude Code