Skip to content

[Security][CPS] Document detection rule behavior and correct app availability - #8064

Open
nastasha-solomon wants to merge 20 commits into
mainfrom
issue-8050
Open

[Security][CPS] Document detection rule behavior and correct app availability#8064
nastasha-solomon wants to merge 20 commits into
mainfrom
issue-8050

Conversation

@nastasha-solomon

@nastasha-solomon nastasha-solomon commented Aug 24, 2026

Copy link
Copy Markdown
Member

Summary

Contributes to #8050.

Corrects the cross-project search (CPS) availability information for Elastic Security apps and documents rule-level CPS behavior ahead of GA.

Previews

Deliberately not included

The "Using cross-project search with Elastic Security" guidance page (item 2 of #8050) is not included as it's not a blocker to CPS GA'ing. It will be added in a separate PR.

Lifecycle note

CPS pages remain at serverless: preview. Flipping them to GA is intentionally left to a separate PR. The one inline {applies_to} tag added here, on Max alerts per run, will need to flip with them.

Generative AI disclosure

  1. Did you use a generative AI (GenAI) tool to assist in creating this contribution?
  • Yes - Cursor + Claude Opus
  • No

@nastasha-solomon nastasha-solomon self-assigned this Aug 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Elastic Docs AI PR menu

Check the box to run an AI review for this pull request.

  • Review docs changes (docs-review). Status: not started.

Powered by GitHub Agentic Workflows and docs-actions. For more information, reach out to the docs team.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!


The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

Comment thread deploy-manage/_snippets/cps-limitations-core.md
Co-authored-by: Cursor <cursoragent@cursor.com>

@shainaraskas shainaraskas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hello thank you!

overall thoughts:

  • I think it would be good if we could be clearer about when things use the space scope
  • We can probably cut back/unify some of the info about API keys on rules - it reads as pretty cohesive between the two surfaces now and we should probably just document what is different about detection rules in this spot

Comment thread deploy-manage/_snippets/cps-limitations-core.md
Comment thread explore-analyze/cross-project-search/_snippets/cps-availability-security-apps.md Outdated
Comment thread explore-analyze/cross-project-search/_snippets/cps-availability-security-apps.md Outdated
:::

- **{{ml-cap}}:** {{anomaly-detect-cap}} job {{dfeeds}} and transforms can read data from linked projects. Jobs and results live on the origin project. {{ml-cap}} rules alert on those origin results, including anomalies produced from linked-project data.
- **Timeline:** Tables display documents from linked projects. Actions that don't apply to remote documents are disabled.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this using the space scope?

@nastasha-solomon nastasha-solomon Aug 27, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PhilippeOberti @michaelolo24 does Timeline always use the space’s default CPS scope, or can it follow the CPS scope selector selection in the project's header? For reference, here's the latest draft.

Comment on lines 14 to 15
- **Dashboards:** The Detection & Response and Data Quality dashboards support {{cps-init}}.
- **Intelligence:** Threat intelligence indicator searches support {{cps-init}}.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

space scope?

@nastasha-solomon nastasha-solomon Aug 27, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chuddy-elastic who can clarify whether the threat intel features always use the space’s default CPS scope, or can follow the CPS scope selector selection in the project's header? For reference, here's the latest draft.

Comment thread solutions/_snippets/cps-sec-obs-rules.md Outdated
Comment thread explore-analyze/cross-project-search/_snippets/cps-availability-security-apps.md Outdated
Comment thread solutions/security/detect-and-alert/cross-project-search-detection-rules.md Outdated
Comment thread solutions/security/detect-and-alert/cross-project-search-detection-rules.md Outdated
Comment thread solutions/security/detect-and-alert/cross-project-search-detection-rules.md Outdated

@shainaraskas shainaraskas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also avoid remote

Comment thread explore-analyze/cross-project-search/_snippets/cps-availability-security-apps.md Outdated
Comment thread explore-analyze/cross-project-search/_snippets/cps-availability-security-apps.md Outdated
nastasha-solomon and others added 6 commits August 26, 2026 16:17
Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com>
…y-security-apps.md

Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com>
…y-security-apps.md

Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com>
…y-security-apps.md

Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com>

@shainaraskas shainaraskas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking nice! a couple of considerations for you about the table columns and some questions about rule scope.

the cutback on the API keys stuff for rules is looking especially nice 🎉

Comment thread solutions/security/detect-and-alert/advanced-data-source-configuration.md Outdated
Comment thread solutions/security/detect-and-alert/advanced-data-source-configuration.md Outdated
Comment thread explore-analyze/cross-project-search/_snippets/cps-availability-security-apps.md Outdated
Comment thread solutions/security/detect-and-alert/cross-project-search-detection-rules.md Outdated
Comment thread solutions/security/detect-and-alert/cross-project-search-detection-rules.md Outdated
**Max alerts per run** (optional)
: Specify the maximum number of alerts the rule can create each time it executes. Default is 100.

{applies_to}`serverless: preview` When {{cps}} is enabled, this limit covers the combined results from the origin project and all linked projects in the space-level scope rather than each project separately. Refer to [{{cps-cap}} and detection rules](/solutions/security/detect-and-alert/cross-project-search-detection-rules.md#cps-rules-max-alerts).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 if we can limit the scope with project routing, is this exactly true? or is it the scope of the query, which is by default the same as the space-level scope unless there's a query-level override?

@nastasha-solomon nastasha-solomon Sep 2, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, good callout. I'm unsure what the relationship is and if the setting supersedes the space-level scope and or query-level overrides. @chuddy-elastic can you answer this?

Claude edited this description as follows:

{applies_to}serverless: preview When {{cps}} is enabled, this limit is shared across every project the rule searched, not applied separately to each project. A query-level override (SET project_routing or a qualified index expression) changes which projects that execution searches; the limit still applies once. Refer to {{cps-cap}} and detection rules.

Comment thread solutions/_snippets/cps-sec-obs-rules.md Outdated
Comment thread solutions/_snippets/cps-sec-obs-rules.md Outdated
Comment thread solutions/_snippets/cps-sec-obs-rules.md Outdated
Comment thread solutions/_snippets/cps-sec-obs-rules.md

If your data spans ECH, ECE, ECK, or self-managed clusters rather than linked {{serverless-short}} projects, refer to [{{ccs-cap}} and detection rules](/solutions/security/detect-and-alert/cross-cluster-search-detection-rules.md) instead.

## Which projects a rule searches and where alerts are stored [cps-rules-scope-and-alerts]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this stuff covered in the core alerting/rules docs?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the moment, it's not. @chuddy-elastic @tiamliu who would be able to provide this information for Kibana alerting v1 and Observability incident management?

@shainaraskas shainaraskas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for these edits! just a couple more comments, mostly around precision re: rule scope, and some stuff about making the snippet fit for purpose across its two homes :) ✅ anyway


- **{{esql}} rules:** Add [`SET project_routing`](/explore-analyze/cross-project-search/cross-project-search-project-routing.md) at the start of the rule query to override the space-level scope.
- **Rules that use index patterns:** Use [qualified index expressions](/explore-analyze/cross-project-search/cross-project-search-search.md#search-expressions) in the index pattern to target specific projects.
- **{{ml-cap}} rules:** These rules alert on {{anomaly-detect}} results stored on the origin project. {{anomaly-jobs-cap}} can read linked-project data; jobs and results stay on the origin.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this doesn't get to the heart of the takeaway, but is more like implementation detail. consider

Suggested change
- **{{ml-cap}} rules:** These rules alert on {{anomaly-detect}} results stored on the origin project. {{anomaly-jobs-cap}} can read linked-project data; jobs and results stay on the origin.
- **{{ml-cap}} rules:** These rules alert on {{anomaly-detect}} job results, so the job's scope determines what the rule can detect. To control which projects the job searches, set [`project_routing`](/explore-analyze/machine-learning/anomaly-detection/ml-ad-run-jobs.md) on the job's {{dfeed}}.

**Max alerts per run** (optional)
: Specify the maximum number of alerts the rule can create each time it executes. Default is 100.

{applies_to}`serverless: preview` When {{cps}} is enabled, this limit is shared across every project the rule searched, not applied separately to each project. A query-level override (`SET project_routing` or a qualified index expression) changes which projects that execution searches. The limit still applies once. Refer to [{{cps-cap}} and detection rules](/solutions/security/detect-and-alert/cross-project-search-detection-rules.md#cps-rules-max-alerts).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not exactly true because not all rules allow query-level overrides. we prob want to lean on https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8064/explore-analyze/alerting/alerts/create-manage-rules#cps-scope-for-rules


The **Max alerts per run** [advanced setting](/solutions/security/detect-and-alert/common-rule-settings.md#rule-ui-advanced-params) limits the number of alerts a rule creates in a single execution. Under {{cps}}, that limit covers the combined results from every project the rule queries in that run, rather than each project separately. By default, the rule queries the origin project and all linked projects in the space-level scope. The default limit is 100.

A rule that stayed under the limit on a single project can reach the limit after you link projects, which leaves matching events without alerts. Review the limit for rules that run across a broad scope. To search fewer projects, add [`SET project_routing`](/explore-analyze/cross-project-search/cross-project-search-project-routing.md) at the start of the **{{esql}} query**, or enter a [qualified index expression](/explore-analyze/cross-project-search/cross-project-search-search.md#search-expressions) in **Index patterns**, instead of raising the limit.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are not the only ways

a) space scope
b) these two
c) ml job

consider linking up/out to wherever solutions/_snippets/cps-sec-obs-rules.md goes

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry I keep coming back to this single snippet.

The problem we have now is that it's being used to do two things:

  1. explain availability/behavior here
  2. act as a limitations list here (where it is pretty overwhelming)

... but it's not doing double duty as well because more things work now

I suggest that we do the following:

  1. kill the snippet
  2. put the content directly on Managing cross-project search scope in your project apps and rebrand as "support" rather than "limitations"
  3. Create a mini list reduced to limitations, and put it on Configure cross-project search, and link across

shortlist of limitations might look like this

- **Alerts:** The Alerts page doesn't show alerts that a linked project generated independently. Only alerts created by origin project rules appear.
- **Attack Discovery:** Discoveries are based on origin project alerts only. Alerts from linked projects aren't included.
- **Cases:** You can't attach an alert or event from a linked project to a case.
- **Entity store:** A host that appears in more than one project isn't combined into a single entity at the origin. Risk scoring runs on the origin project only.
- **Flyouts:** Session View isn't available for documents from linked projects. Some actions are hidden or disabled.
- **{{elastic-defend}} and Osquery:** Policies, artifacts, response actions, and Osquery saved queries and packs can't be shared or managed across linked projects.
- **{{ml-cap}} rules:** {{ml-cap}} rules don't use the space-level {{cps}} scope. They use the scope of the underlying {{anomaly-detect}} job's {{dfeed}}, which might differ from the space default.
- **SIEM Readiness and Value report:** These features don't include data from linked projects.
- **Timeline:** Some actions are disabled for documents from linked projects.

- **Detection rules:** {{esql}} rules support `SET project_routing`. For non-{{esql}} rules that use index patterns, you can use [qualified index expressions](/explore-analyze/cross-project-search/cross-project-search-search.md#search-expressions). Origin rules write alerts to the origin project. The **Max alerts per run** limit applies across the projects the rule queries. A rule searches only the linked projects the user who last saved it can access. For details, refer to [{{cps-cap}} and detection rules](/solutions/security/detect-and-alert/cross-project-search-detection-rules.md).
- **{{elastic-defend}} and Osquery:** The **Endpoints** page, host details, **Response actions history**, and Osquery query results include data from linked projects. Policies, artifacts, response action dispatch, and Osquery saved queries and packs stay per project because they're managed through Fleet.
- **Entity store:** Origin profiles include entities from every linked project. Each project still builds its own store, and risk scoring stays on the origin project. A host that appears in more than one project isn't combined into a single entity at the origin.
- **{{ml-cap}}:** {{anomaly-detect-cap}} job {{dfeeds}} can read data from linked projects. Jobs and results are stored on the origin project. {{ml-cap}} rules alert on those stored results, including anomalies produced from linked-project data.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

siiide note, I was doing a little poking around for another ml thing and I suspect that when i try to start a prebuilt job using this flow, it searches every linked project (doesn't use the space scope): https://www.elastic.co/docs/solutions/security/advanced-entity-analytics/anomaly-detection#manage-jobs

might be worth adding as a limitation and/or tweaking that page directly (or maybe it's a bug? :/)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants