Skip to content

fix(entity-store): write maintainer relationships as { raw_identifiers } objects - #494

Merged
gurevichdmitry merged 1 commit into
mainfrom
fix/entity-maintainers-relationships-shape
Oct 20, 2026
Merged

gurevichdmitry merged 1 commit into
mainfrom
fix/entity-maintainers-relationships-shape

Conversation

@gurevichdmitry

Copy link
Copy Markdown
Contributor

Summary

generate-entity-maintainers-data (the relationships maintainer) writes entity.relationships.<type> as bare string arrays, for example owns: ['laptop-ab12']. Since elastic/kibana#262242 (merged 2026-04-13), Entity Store v2 maps every relationship except resolution as an object of { raw_identifiers, ids }, so Elasticsearch rejects these documents.

The command still exits 0, so the failure is easy to miss. On a 9.6.0-SNAPSHOT stack with --space default --quick it logged about 27,000 failed writes, on both the history indices (.entities.v2.history.security_default.*) and the latest index (.entities.v2.latest.default-00001):

document_parsing_exception: object mapping for [entity.relationships.owns] tried to parse field [null] as object, but found a concrete value
document_parsing_exception: failed to parse field [entity.relationships.supervises] of type [text] ...

The relationship code was written in #326 (2026-03-13), when Kibana's schema still used string arrays, and hasn't changed since.

Change

generateRelationshipsFields now emits { raw_identifiers: { user | host | service: { name: [...] } } }:

  • Targets are classified by kind: synthetic hosts as host, synthetic services as service, other entity names as user. owns targets are devices, so they default to host.
  • Nested rather than dotted keys, so the result deep-merges cleanly with the entity _source, which Elasticsearch returns in nested form (snapshot merges it into the history documents).
  • The identifier kinds used (user.name, host.name, service.name) are all in Kibana's EntityRelationship.raw_identifiers schema.

ids is not populated: it holds target entity EUIDs, and this generator only has names.

Test plan

  • tsc, eslint and prettier --check pass.
  • Run generate-entity-maintainers-data --space default --quick against a 9.6 stack after org-data and confirm there are no document_parsing_exception errors and that entity.relationships.* is populated on entities and in the history snapshots.

Opened as a draft until the runtime check is done.

🤖 Generated with Claude Code

…s } objects

Entity Store v2 maps each entity.relationships.<type> (except resolution) as
an object of { raw_identifiers, ids} since elastic/kibana#262242. The
generate-entity-maintainers-data relationships maintainer still wrote bare
string arrays, so Elasticsearch rejected the documents with
document_parsing_exception on both the latest and history indices.

Emit { raw_identifiers: { user|host|service: { name: [...] } } } instead, in
the nested form Elasticsearch returns in _source so it deep-merges cleanly
with the existing entity.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@gurevichdmitry
gurevichdmitry marked this pull request as ready for review October 20, 2026 11:31
@gurevichdmitry
gurevichdmitry requested a review from a team as a code owner October 20, 2026 11:31
@gurevichdmitry
gurevichdmitry requested review from chennn1990 and a balanced review from Copilot October 20, 2026 11:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Validation of latest-index and history writes against a 9.6 stack remains pending.

No open issues 🎉

What changed in this PR

Updates entity maintainer relationship data to match Entity Store v2’s object schema, addressing Elasticsearch parsing failures.

Changes:

  • Wraps relationship targets in nested raw_identifiers objects.
  • Classifies targets as users, hosts, or services, with owned devices defaulting to hosts.
File Description
src/​commands/​entity_store/​entity_maintainers.ts Converts relationship arrays into typed identifier objects compatible with snapshot merging.

Review effort: Balanced

// Entity Store v2 maps `entity.relationships.<type>` as an object
// ({ ids, raw_identifiers }), so bare string arrays are rejected with a
// document_parsing_exception on both the latest and history indices.
type RelationshipIdentifierKind = 'user' | 'host' | 'service';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non blocking, but we do have a 4th type called 'generic', although there's been discussion on moving away from it

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tiansivive, thanks for the review. Checked with @uri-weisman , and it’s okay to omit generic.

@tiansivive tiansivive left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gurevichdmitry
gurevichdmitry merged commit ca078eb into main Oct 20, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants