Skip to content

Add cited knowledge consultation skill for Business Central - #222

Draft
Javier Armesto Gonzalez (javiarmesto) wants to merge 2 commits into
microsoft:mainfrom
javiarmesto:feat/al-knowledge-consultation
Draft

Javier Armesto Gonzalez (javiarmesto) wants to merge 2 commits into
microsoft:mainfrom
javiarmesto:feat/al-knowledge-consultation

Conversation

@javiarmesto

@javiarmesto Javier Armesto Gonzalez (javiarmesto) commented Oct 7, 2026 •

Copy link
Copy Markdown

Purpose

The goal is to help AL coding agents design and specify Business Central
solutions using cited guidance from BCQuality.

Add al-knowledge so a user or agent can ask a Business Central development
question without supplying source code or starting a code review. The skill
reads relevant BCQuality articles and returns guidance with exact citations and
explicit applicability conditions.

The skill exposes a public skill entry point: skills/al-knowledge/SKILL.md.
This is the file the host agent reads to start the skill. It explains how to
prepare the question and follow BCQuality's routing instructions. Entry owns
the routing decision, and the internal action owns knowledge consultation.

The skill follows BCQuality's Entry, READ and DO contracts. It has
no dependency on a particular consumer framework or agent role. The
host supplies the agent that follows the instructions, either through its skill
mechanism or inline in the caller's context.

Behavior

The public skill entry point tells the agent to preserve the caller's exact
question as knowledge-query and pass it to Entry with only the target context
actually known. Entry selects
the compatible knowledge action. A question-only request cannot dispatch the
existing review actions because their input types do not intersect.

The action discovers articles in enabled layers, applies READ's applicability
and precedence rules, and reads each selected article in full before citing it.
The index is discovery metadata, not evidence of a complete read. Unknown
applicability dimensions remain explicit on conditional references.

The action returns one knowledge-response JSON object containing skill,
outcome, question, answer, references and suppressed:

  • completed requires an answer to the exact question and at least one checked supporting reference. Adjacent-topic guidance alone is not an answer.
  • no-knowledge identifies a gap in applicable knowledge and returns no citations.
  • partial explains why the work is incomplete and cites only articles read.
  • failed explains the failure and returns no citations.

Entry's no-match and failed dispatch records remain separate from action
responses and pass back unchanged. al-code-review keeps its review role and
findings-report output. Knowledge guidance does not constitute a review verdict
or approval gate.

Implementation

flowchart TD
    Q["User or agent<br/>Exact question and known context"]
    S["Public skill entry point<br/>Prepare knowledge-query<br/>Resolve the BCQuality root"]
    E["Entry<br/>Prepare index when possible<br/>Select the knowledge action"]
    A["Internal al-knowledge action<br/>Apply READ rules<br/>Read complete articles"]
    R["knowledge-response<br/>Answer, outcome and citations"]
    V{"Consumer validation<br/>Schema, exact question<br/>and complete citation reads"}
    OK["Accept the unchanged response"]
    BAD["Preserve the raw response<br/>Record validation errors separately"]
    STOP["Return Entry dispatch record<br/>no-match or failed"]
    Q --> S --> E
    E -->|routed| A --> R --> V
    E -->|no-match or failed| STOP
    V -->|valid| OK
    V -->|invalid| BAD
Loading

The same host agent can follow the skill instructions inline. Article discovery
does not replace complete reads. In inline hosts, the answering agent checks
its own reads and response; this is self-attestation, not independent verification.
Independent evidence checks require a trusted consumer collector outside the
answering agent. no-knowledge and failed action responses contain no citations.

  • Public skill entry point: skills/al-knowledge/SKILL.md.
  • Internal action: microsoft/skills/knowledge/al-knowledge.md.
  • New input/output contracts: knowledge-query and knowledge-response.
  • Response schema and support in Entry, DO, the skill index and frontmatter validator.
  • Explicit READ coverage for bounded knowledge retrieval, continuation pages,
    complete bodies and fallback through EOF when helpers are unavailable.
  • Consumer validator: tools/validate_knowledge_response.py, semantic regression
    tests, and documentation of independently collected read evidence.
  • English explanatory and technical diagrams for a user or any agent.

When a consumer runs the optional validator with its own trusted read evidence,
the validator checks the schema, exact question, citation paths within
the live corpus, and same-run complete-read evidence with matching file hashes
and byte counts. Consumers collect that evidence from successful tool results,
not from the answering agent's assertions. Invalid output remains unchanged and
the consumer records a separate validation failure.

The validator also enforces enabled layers for citations and layer-precedence
suppressions. A configuration suppression may identify an excluded article in a
disabled layer; it is an exclusion record, never supporting evidence. All paths
must be canonical existing corpus files.

The validator does not independently prove comprehension, applicability or
source support for every claim. READ and actual execution evidence still govern
those checks. Documentation links to the canonical contracts instead of
redefining them. Consumer-specific integration material is outside this PR's
intended scope.

Prerequisites

  • A host agent that can read the skill entry point and follow its instructions.
  • A readable BCQuality plugin or checkout containing the skill entry point, Entry, READ,
    DO, action skills, response schema and enabled knowledge articles.
  • Permissions to read the corpus and retain complete tool results. Writing an
    index is needed only when running Entry's generator; path discovery remains
    available when index preparation cannot run.

No AL app, compiler or Business Central tenant is required
for a question-only consultation. Available guidance remains limited to the
installed corpus. Authentication to the agent provider follows the chosen host.

Optional tooling and alignment with existing BCQuality contracts

PowerShell 7.2 or later supports the existing bounded retrieval helpers. READ
already permits path discovery and complete native file reads when PowerShell,
a helper or a prepared index is unavailable. This proposal reuses that approach
for knowledge consultation.

Existing DO rules require complete article reads, citation integrity and
deterministic validation of review reports before acceptance. The proposal
extends those principles to knowledge-response.

Python with jsonschema is required only to run the proposed consumer validator.
It is not a prerequisite for using the skill and is not an existing BCQuality
runtime requirement. The original repository uses Python with PyYAML for its
frontmatter CI checks; this validator is an additional optional tool.

The validator's read-evidence manifest is also proposed tooling, not an existing
BCQuality evidence standard. It provides one way to check complete citation reads
in the current run. Hosts can use their native reading tools and another
validation implementation that enforces the same response contract. The
obligation is to validate the response and its citations, not to use this Python
script or its manifest format.

Validation

The changes pass validation.

Maintainer review

This proposal changes stable Entry and DO contracts and adds an output schema.
It requires review and approval from both maintainers. No upstream merge or
release is assumed by the proposal.

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

👋 Heads up Javier Armesto Gonzalez (@javiarmesto) — and cc maintainers — this PR introduces new top-level entries that aren't part of BCQuality's known repository structure:

  • 📁 schemas/ (new top-level folder)

This isn't a block — just a flag. 🚩 New top-level folders and files are usually unintended (a stray export, a tool's scratch dir, or content that meant to land inside an existing layer). BCQuality keeps a deliberately small root: plugin metadata, community/, custom/, microsoft/, skills/, tools/, docs/, evaluation/, .github/, and a handful of root docs. Partner guides belong under docs/; shared knowledge belongs beside the skill that owns its domain.

If this was intentional and the new entry genuinely belongs at the repo root, a maintainer can review and merge as normal — no action needed beyond a quick sanity check. If it wasn't, please move the content into the right existing layer (or drop it) and push an update. 🙏

A maintainer will take a look before merging.

@JesperSchulz Jesper Schulz-Wedde (JesperSchulz) left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed draft head eb0db6358e860508773cd389ab49886a80c9be48. The consultation action is cleanly separated from code review, preserves the exact question, uses bounded complete-body reads with explicit applicability/layer handling, and introduces knowledge-response without repurposing findings-report. The schema, semantic tests (12 schema cases, 18 semantic/CLI tests), frontmatter, knowledge index/retrieval, and skill-index validation all pass, and the regenerated skill index is unchanged for all 21 existing action skills.

However, one merge-critical dispatch defect remains (details in the follow-up review): al-knowledge.md declares no filter dimensions, so under READ's matching rules Entry rejects it with filter-mismatch whenever the caller supplies technologies: [al], which the new SKILL.md directs for AL questions. My earlier statement that citation acceptance requires independently collected read evidence also holds only when the optional validator is used; in inline hosts the check is self-attested.

@JesperSchulz Jesper Schulz-Wedde (JesperSchulz) left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes on eb0db6358e860508773cd389ab49886a80c9be48.
Merge-critical

  1. al-knowledge is not dispatched on the normal AL path. microsoft/skills/knowledge/al-knowledge.md is the only action skill without bc-version/technologies/countries/application-area frontmatter, so the generated index records empty arrays. DO says skill filters follow READ semantics, and READ's technologies rule requires a non-empty intersection with no sentinel. skills/al-knowledge/SKILL.md step 2 tells the agent to pass target context when reliably determined, using the same rules as al-code-review, which sets technologies: [al] for AL input. Entry therefore drops the only knowledge-query candidate as filter-mismatch and returns no-match. Please declare bc-version: [all], technologies: [al], countries: [w1], application-area: [all] (as al-code-review does), and add a Test-SkillIndex.ps1 assertion that al-knowledge is admitted when technologies: [al] is supplied.

Please also address

  1. plugin.json and .claude-plugin/marketplace.json set version 0.3.0-knowledge-preview.3. A fork preview label should not land on main; please leave the version for the maintainers' release process.
  2. The contract text and PR description imply citation integrity is independently verified. That is only true when a consumer runs tools/validate_knowledge_response.py with its own evidence. In inline hosts (the VS Code case SKILL.md targets) the answering agent attests its own reads. Please state that limitation in DO / docs/knowledge-response-validation.md rather than implying independent verification.
  3. Minor: the validator does not check that cited or suppressed[] paths belong to enabled layers.

For maintainers: the flag-new-top-level notice about schemas/ is a false positive; the folder already exists on main. The PR remains draft, and the stable Entry/DO/schema changes still need both maintainers' approval.

@javiarmesto

Copy link
Copy Markdown
Author

Thanks for the review, Jesper Schulz-Wedde (@JesperSchulz). I have addressed the four points:

  1. I added the requested applicability filters to the internal al-knowledge action and an AL technology admission assertion in Test-SkillIndex.ps1, while preserving knowledge-only input separation from review actions.
  2. I restored the original 0.2.0 version in both plugin manifests, leaving version changes to the maintainers' release process.
  3. I clarified DO, the skill entry point, the validation documentation and this PR description: inline checks are self-attestation by the answering agent. Independent evidence checks require a consumer's own trusted read collector. Neither mode proves comprehension or source support for every claim.
  4. I added enabled-layer checks to the validator. Citations and layer-precedence suppressions cannot reference disabled layers. One explicit exception is a suppressed entry with reason: "configuration": it may identify an article excluded because its layer is disabled, but never counts as supporting evidence. All paths must still be canonical, exist and remain within the corpus. This distinction is documented and covered by regression tests.

I also tightened question relevance after an inline test returned general localization advice for an unsupported customs question. The action now distinguishes applicability from relevance and returns no-knowledge with no references when only adjacent-topic material is available.

Validation passes. I tested inline consultation in Claude Code and VS Code Insiders; the final Insiders candidate returned both a supported answer and no-knowledge for the unsupported question. Exported logs truncate some tool results, so I am not presenting those runs as independent complete-read verification.

I have kept the PR in draft for maintainer review and the required approvals.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 64095627e9885ae23807a889d188ef1d8f0e9c7e. All four points from my changes-requested review are resolved:

  1. Dispatch: al-knowledge now declares bc-version: [all], technologies: [al], countries: [w1], application-area: [all]. The new Test-SkillIndex.ps1 assertions are effective: removing the filters makes the test fail with "Entry must admit knowledge-query with technologies: [al]". AL file-path input still routes to al-code-review and excludes al-knowledge.
  2. Version: both manifests are back to 0.2.0.
  3. Verification mode: DO, SKILL.md, and the validation doc now state plainly that inline checks are self-attestation, and independent checks need a trusted consumer read collector.
  4. Enabled layers: citations and layer-precedence suppressions must belong to enabled layers. configuration suppressions may name disabled-layer articles, but paths must be canonical and exist. Covered by 7 new regression tests.

The relevance tightening (adjacent-topic material leads to no-knowledge with no references) is a sound addition.

Exact-head validation passes: frontmatter (0 errors), 12 schema cases, 25 semantic/CLI tests, knowledge index/retrieval, and the skill index (all 19 review leaves preserved). No merge-critical issue remains.

Non-blocking nits

  • DO says "Report the mode used", but knowledge-response has additionalProperties: false and no mode field. Please say where the mode is reported (the consumer's record, not the response).
  • validate_response checks reference paths in both the new layer loop and the citation loop, so a bad path is reported twice.
  • skills/do.md and docs/knowledge-response-validation.md gained stray double blank lines.

Before merge, the substantive validation workflows (frontmatter/knowledge response, skill index, knowledge index, review fixtures) still need to run on GitHub; only flag/guard/CLA have reported so far. The PR is still draft, and the stable Entry/DO/schema changes need both maintainers' approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants