Repository navigation
fix(scoring): detect primaryPurpose/suppliedBy so completeness profiles can reach Standard and Advanced (#124) - #126
Open
drk-ram wants to merge 1 commit into
Conversation
…Standard and Advanced (GenAI-Security-Project#124) determine_completeness_profile() checked each required field with the legacy check_field_in_aibom() helper, which had no mapping for primaryPurpose (emitted as modelCard.modelParameters.task) or suppliedBy (emitted as supplier.name). The registry jsonpaths for these fields, and for datasets, ethicalConsiderations, technicalLimitations and intendedUse, pointed at a root-level `$.component` object that does not exist in a CycloneDX document, so the registry-driven path could not find them either. A fully populated BOM therefore never left Basic. - field_registry.json: point the six `$.component.` jsonpaths at `$.components[0]`; ethicalConsiderations now reads the emitted `name`. - scoring.py: add primaryPurpose and suppliedBy aliases to check_field_in_aibom(). - scoring.py: determine_completeness_profile() accepts extraction_results and uses check_field_with_enhanced_results(), so the profile and the per-field checklist share one detection path. - tests: Standard and Advanced profiles are reachable, absent fields keep the profile at Basic, and the checklist marks both fields present. Fixes GenAI-Security-Project#124 Signed-off-by: Raghuvar Karthik Durga <177662415+drk-ram@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #124.
determine_completeness_profile()could never classify a BOM as Standard or Advanced.primaryPurposeandsuppliedBywere not detectable by the legacy helper (the generator emits them asmodelCard.modelParameters.taskandsupplier.name), and several registryjsonpathentries pointed at a root-level$.componentobject that does not exist in a CycloneDX document.Changes
field_registry.json: the six$.component.*jsonpaths (datasets,primaryPurpose,suppliedBy,ethicalConsiderations,technicalLimitations,intendedUse) now use$.components[0].*.ethicalConsiderationsreads[0].name, which is what_create_model_card_section()emits. TheexternalReferenceslocations are left to fix(scoring): align externalReferences registry paths with emitted BOM (#76) #120.scoring.py:check_field_in_aibom()resolvesprimaryPurpose(modelParameters.task) andsuppliedBy(supplier.name).scoring.py:determine_completeness_profile()takes an optionalextraction_resultsand goes throughcheck_field_with_enhanced_results(), so the profile and the per-field checklist use one detection path.calculate_completeness_score()passes it through.Tests (
tests/test_completeness_profile.py)task/supplier.nameare detected by both the legacy and the enhanced check, and removing them flips the resultStandard; a score below 70, or a missing supplier, staysBasicAdvancedcalculate_completeness_score()checks both fields off and does not list them as missingFour of the seven tests fail on
origin/main(the profile staysBasic).Validation
pytest -q --no-cov: all passedpytest tests/test_completeness_profile.py --no-covonorigin/mainsources: 4 failed, 3 passed; with this change: 7 passed