Repository navigation
Add cited knowledge consultation skill for Business Central - #222
Javier Armesto Gonzalez (javiarmesto) wants to merge 2 commits into
Conversation
|
👋 Heads up Javier Armesto Gonzalez (@javiarmesto) — and cc maintainers — this PR introduces new top-level entries that aren't part of BCQuality's known repository structure:
This isn't a block — just a flag. 🚩 New top-level folders and files are usually unintended (a stray export, a tool's scratch dir, or content that meant to land inside an existing layer). BCQuality keeps a deliberately small root: plugin metadata, If this was intentional and the new entry genuinely belongs at the repo root, a maintainer can review and merge as normal — no action needed beyond a quick sanity check. If it wasn't, please move the content into the right existing layer (or drop it) and push an update. 🙏 A maintainer will take a look before merging. |
There was a problem hiding this comment.
Reviewed draft head eb0db6358e860508773cd389ab49886a80c9be48. The consultation action is cleanly separated from code review, preserves the exact question, uses bounded complete-body reads with explicit applicability/layer handling, and introduces knowledge-response without repurposing findings-report. The schema, semantic tests (12 schema cases, 18 semantic/CLI tests), frontmatter, knowledge index/retrieval, and skill-index validation all pass, and the regenerated skill index is unchanged for all 21 existing action skills.
However, one merge-critical dispatch defect remains (details in the follow-up review): al-knowledge.md declares no filter dimensions, so under READ's matching rules Entry rejects it with filter-mismatch whenever the caller supplies technologies: [al], which the new SKILL.md directs for AL questions. My earlier statement that citation acceptance requires independently collected read evidence also holds only when the optional validator is used; in inline hosts the check is self-attested.
There was a problem hiding this comment.
Requesting changes on eb0db6358e860508773cd389ab49886a80c9be48.
Merge-critical
al-knowledgeis not dispatched on the normal AL path.microsoft/skills/knowledge/al-knowledge.mdis the only action skill withoutbc-version/technologies/countries/application-areafrontmatter, so the generated index records empty arrays. DO says skill filters follow READ semantics, and READ'stechnologiesrule requires a non-empty intersection with no sentinel.skills/al-knowledge/SKILL.mdstep 2 tells the agent to pass target context when reliably determined, using the same rules asal-code-review, which setstechnologies: [al]for AL input. Entry therefore drops the onlyknowledge-querycandidate asfilter-mismatchand returnsno-match. Please declarebc-version: [all],technologies: [al],countries: [w1],application-area: [all](asal-code-reviewdoes), and add aTest-SkillIndex.ps1assertion thatal-knowledgeis admitted whentechnologies: [al]is supplied.
Please also address
plugin.jsonand.claude-plugin/marketplace.jsonset version0.3.0-knowledge-preview.3. A fork preview label should not land onmain; please leave the version for the maintainers' release process.- The contract text and PR description imply citation integrity is independently verified. That is only true when a consumer runs
tools/validate_knowledge_response.pywith its own evidence. In inline hosts (the VS Code caseSKILL.mdtargets) the answering agent attests its own reads. Please state that limitation in DO /docs/knowledge-response-validation.mdrather than implying independent verification. - Minor: the validator does not check that cited or
suppressed[]paths belong to enabled layers.
For maintainers: the flag-new-top-level notice about schemas/ is a false positive; the folder already exists on main. The PR remains draft, and the stable Entry/DO/schema changes still need both maintainers' approval.
|
Thanks for the review, Jesper Schulz-Wedde (@JesperSchulz). I have addressed the four points:
I also tightened question relevance after an inline test returned general localization advice for an unsupported customs question. The action now distinguishes applicability from relevance and returns Validation passes. I tested inline consultation in Claude Code and VS Code Insiders; the final Insiders candidate returned both a supported answer and I have kept the PR in draft for maintainer review and the required approvals. |
Jesper Schulz-Wedde (JesperSchulz)
left a comment
There was a problem hiding this comment.
Reviewed 64095627e9885ae23807a889d188ef1d8f0e9c7e. All four points from my changes-requested review are resolved:
- Dispatch:
al-knowledgenow declaresbc-version: [all],technologies: [al],countries: [w1],application-area: [all]. The newTest-SkillIndex.ps1assertions are effective: removing the filters makes the test fail with "Entry must admit knowledge-query with technologies: [al]". ALfile-pathinput still routes toal-code-reviewand excludesal-knowledge. - Version: both manifests are back to
0.2.0. - Verification mode: DO,
SKILL.md, and the validation doc now state plainly that inline checks are self-attestation, and independent checks need a trusted consumer read collector. - Enabled layers: citations and
layer-precedencesuppressions must belong to enabled layers.configurationsuppressions may name disabled-layer articles, but paths must be canonical and exist. Covered by 7 new regression tests.
The relevance tightening (adjacent-topic material leads to no-knowledge with no references) is a sound addition.
Exact-head validation passes: frontmatter (0 errors), 12 schema cases, 25 semantic/CLI tests, knowledge index/retrieval, and the skill index (all 19 review leaves preserved). No merge-critical issue remains.
Non-blocking nits
- DO says "Report the mode used", but
knowledge-responsehasadditionalProperties: falseand no mode field. Please say where the mode is reported (the consumer's record, not the response). validate_responsechecks reference paths in both the new layer loop and the citation loop, so a bad path is reported twice.skills/do.mdanddocs/knowledge-response-validation.mdgained stray double blank lines.
Before merge, the substantive validation workflows (frontmatter/knowledge response, skill index, knowledge index, review fixtures) still need to run on GitHub; only flag/guard/CLA have reported so far. The PR is still draft, and the stable Entry/DO/schema changes need both maintainers' approval.
Purpose
The goal is to help AL coding agents design and specify Business Central
solutions using cited guidance from BCQuality.
Add
al-knowledgeso a user or agent can ask a Business Central developmentquestion without supplying source code or starting a code review. The skill
reads relevant BCQuality articles and returns guidance with exact citations and
explicit applicability conditions.
The skill exposes a public skill entry point:
skills/al-knowledge/SKILL.md.This is the file the host agent reads to start the skill. It explains how to
prepare the question and follow BCQuality's routing instructions. Entry owns
the routing decision, and the internal action owns knowledge consultation.
The skill follows BCQuality's Entry, READ and DO contracts. It has
no dependency on a particular consumer framework or agent role. The
host supplies the agent that follows the instructions, either through its skill
mechanism or inline in the caller's context.
Behavior
The public skill entry point tells the agent to preserve the caller's exact
question as
knowledge-queryand pass it to Entry with only the target contextactually known. Entry selects
the compatible knowledge action. A question-only request cannot dispatch the
existing review actions because their input types do not intersect.
The action discovers articles in enabled layers, applies READ's applicability
and precedence rules, and reads each selected article in full before citing it.
The index is discovery metadata, not evidence of a complete read. Unknown
applicability dimensions remain explicit on conditional references.
The action returns one
knowledge-responseJSON object containingskill,outcome,question,answer,referencesandsuppressed:completedrequires an answer to the exact question and at least one checked supporting reference. Adjacent-topic guidance alone is not an answer.no-knowledgeidentifies a gap in applicable knowledge and returns no citations.partialexplains why the work is incomplete and cites only articles read.failedexplains the failure and returns no citations.Entry's
no-matchandfaileddispatch records remain separate from actionresponses and pass back unchanged.
al-code-reviewkeeps its review role andfindings-report output. Knowledge guidance does not constitute a review verdict
or approval gate.
Implementation
flowchart TD Q["User or agent<br/>Exact question and known context"] S["Public skill entry point<br/>Prepare knowledge-query<br/>Resolve the BCQuality root"] E["Entry<br/>Prepare index when possible<br/>Select the knowledge action"] A["Internal al-knowledge action<br/>Apply READ rules<br/>Read complete articles"] R["knowledge-response<br/>Answer, outcome and citations"] V{"Consumer validation<br/>Schema, exact question<br/>and complete citation reads"} OK["Accept the unchanged response"] BAD["Preserve the raw response<br/>Record validation errors separately"] STOP["Return Entry dispatch record<br/>no-match or failed"] Q --> S --> E E -->|routed| A --> R --> V E -->|no-match or failed| STOP V -->|valid| OK V -->|invalid| BADThe same host agent can follow the skill instructions inline. Article discovery
does not replace complete reads. In inline hosts, the answering agent checks
its own reads and response; this is self-attestation, not independent verification.
Independent evidence checks require a trusted consumer collector outside the
answering agent.
no-knowledgeandfailedaction responses contain no citations.skills/al-knowledge/SKILL.md.microsoft/skills/knowledge/al-knowledge.md.knowledge-queryandknowledge-response.complete bodies and fallback through EOF when helpers are unavailable.
tools/validate_knowledge_response.py, semantic regressiontests, and documentation of independently collected read evidence.
When a consumer runs the optional validator with its own trusted read evidence,
the validator checks the schema, exact question, citation paths within
the live corpus, and same-run complete-read evidence with matching file hashes
and byte counts. Consumers collect that evidence from successful tool results,
not from the answering agent's assertions. Invalid output remains unchanged and
the consumer records a separate validation failure.
The validator also enforces enabled layers for citations and layer-precedence
suppressions. A configuration suppression may identify an excluded article in a
disabled layer; it is an exclusion record, never supporting evidence. All paths
must be canonical existing corpus files.
The validator does not independently prove comprehension, applicability or
source support for every claim. READ and actual execution evidence still govern
those checks. Documentation links to the canonical contracts instead of
redefining them. Consumer-specific integration material is outside this PR's
intended scope.
Prerequisites
DO, action skills, response schema and enabled knowledge articles.
index is needed only when running Entry's generator; path discovery remains
available when index preparation cannot run.
No AL app, compiler or Business Central tenant is required
for a question-only consultation. Available guidance remains limited to the
installed corpus. Authentication to the agent provider follows the chosen host.
Optional tooling and alignment with existing BCQuality contracts
PowerShell 7.2 or later supports the existing bounded retrieval helpers. READ
already permits path discovery and complete native file reads when PowerShell,
a helper or a prepared index is unavailable. This proposal reuses that approach
for knowledge consultation.
Existing DO rules require complete article reads, citation integrity and
deterministic validation of review reports before acceptance. The proposal
extends those principles to
knowledge-response.Python with
jsonschemais required only to run the proposed consumer validator.It is not a prerequisite for using the skill and is not an existing BCQuality
runtime requirement. The original repository uses Python with PyYAML for its
frontmatter CI checks; this validator is an additional optional tool.
The validator's read-evidence manifest is also proposed tooling, not an existing
BCQuality evidence standard. It provides one way to check complete citation reads
in the current run. Hosts can use their native reading tools and another
validation implementation that enforces the same response contract. The
obligation is to validate the response and its citations, not to use this Python
script or its manifest format.
Validation
The changes pass validation.
Maintainer review
This proposal changes stable Entry and DO contracts and adds an output schema.
It requires review and approval from both maintainers. No upstream merge or
release is assumed by the proposal.