Improve clarity on index management recommendations - #8082
Conversation
Clarified recommendations regarding index count and size, emphasizing the importance of avoiding large numbers of small indices for resource efficiency and cost-effectiveness.
Elastic Docs AI PR menuCheck the box to run an AI review for this pull request.
Powered by GitHub Agentic Workflows and docs-actions. For more information, reach out to the docs team. |
🔍 Preview links for changed docs |
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
| * **Index count and size**: Consolidate small indices for better efficiency. | ||
| In general, avoid a design where your project contains hundreds of very small indices, specifically those under 1GB each. | ||
| Avoiding small indices is important because every index in {{es}} has a certain amount of resource overhead. | ||
| Avoiding large numbers of small indices is important because every index in {{es}} has a certain amount of resource overhead. |
There was a problem hiding this comment.
What does large number mean? (Is there a definition that we can push out as large can mean something different for us vs customer A vs customer B).
Is there an explicit need to mention large number? As we wanted to avoid that to indicate that small indices adds overhead and depending on customer + their costs, they can decide how many small indices they want to keep vs not.
| Avoiding small indices is important because every index in {{es}} has a certain amount of resource overhead. | ||
| Avoiding large numbers of small indices is important because every index in {{es}} has a certain amount of resource overhead. | ||
| {{es}} needs to maintain metadata for each index to keep it running smoothly. | ||
| When you have a very large number of small indices, the combined overhead from all of them can consume more CPU resources than if the same data were stored in fewer, larger indices. |
There was a problem hiding this comment.
Why omitting this line as that seems important for customers to understand what overhead leads to additional resources. I will prefer we keep it.
| Higher resource consumption associated with large numbers of small indices leads to higher costs and can ultimately impact the overall performance of your project. | ||
|
|
||
| If your use case naturally generates many small, separate streams of data, the recommended approach is to implement a process to consolidate them into fewer, larger indices. This practice leads to more efficient resource utilization. By grouping your data into larger indices, you can ensure a more performant and cost-efficient experience with {{es-serverless}}. | ||
| If your use case naturally generates many small, separate streams of data, the recommended approach is to implement a data model which consolidates them into fewer, larger indices. This practice leads to more efficient resource utilization. By grouping your data into larger indices, you can ensure a more performant and cost-efficient experience with {{es-serverless}}. |
There was a problem hiding this comment.
Do customers know what is a data model?
There was a problem hiding this comment.
Maybe mechanism is a better generic word to use instead of process/datamodel?
2 participants
Clarified recommendations regarding index count and size, emphasizing the importance of avoiding large numbers of small indices for resource efficiency and cost-effectiveness.
Summary
Generative AI disclosure