Skip to content

Conversation

MichelLosier
Copy link
Contributor

@MichelLosier MichelLosier commented Sep 11, 2025

Proposed commit message

Resolves: https://github.com/elastic/ingest-dev/issues/5692

This adds a 90 day data retention limit for the elastic_agent.status_change data stream.

Prior to this change, this data stream did not have a retention limit set. Kibana has a task that writes to this elastic_agent.status_change every minute only if there is a change in status.

The primary use case for this data is for setting up alert rule monitoring of elastic agents. 90 days would allow users to analyze trends with status changes that may occur with an agent over the span of weeks or months. This also aligns with similar retention of our per 1m metric default policies for APM: https://www.elastic.co/docs/solutions/observability/apm/index-lifecycle-management.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@MichelLosier MichelLosier requested a review from a team as a code owner September 11, 2025 21:31
@MichelLosier MichelLosier added the enhancement New feature or request label Sep 11, 2025
@MichelLosier MichelLosier self-assigned this Sep 11, 2025
@MichelLosier MichelLosier requested a review from a team September 11, 2025 21:43
@MichelLosier MichelLosier marked this pull request as draft September 11, 2025 22:48
@MichelLosier MichelLosier marked this pull request as ready for review September 11, 2025 23:05
Copy link
Member

@nchaulet nchaulet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@andrewkroh andrewkroh added Integration:elastic_agent Elastic Agent Team:Elastic-Agent Platform - Ingest - Agent [elastic/elastic-agent] labels Sep 12, 2025
@elasticmachine
Copy link

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

@jlind23
Copy link
Contributor

jlind23 commented Sep 15, 2025

@MichelLosier if this datastream only exist from 9.2 onwards, shouldn't we update the Kibana version constraint of this package?

@nchaulet
Copy link
Member

nchaulet commented Sep 15, 2025

@MichelLosier if this datastream only exist from 9.2 onwards, shouldn't we update the Kibana version constraint of this package?

The datastream exist as part of the elastic-agent package, so both the datastream definition and the template are contained in that version, and could be installed in kibana < 9.2

@MichelLosier
Copy link
Contributor Author

The datastream was already introduced in version 2.3.0 of this package, if we were to kibana version constraint this maybe a breaking change for the package.

@MichelLosier MichelLosier force-pushed the add-data-retention-ea-status-change-logs branch from 18e1427 to b9f9b95 Compare September 16, 2025 15:56
@elasticmachine
Copy link

💚 Build Succeeded

History

cc @MichelLosier

Copy link

@MichelLosier MichelLosier merged commit 30f1321 into elastic:main Sep 16, 2025
9 checks passed
@elastic-vault-github-plugin-prod

Package elastic_agent - 2.6.2 containing this change is available at https://epr.elastic.co/package/elastic_agent/2.6.2/

robester0403 pushed a commit to robester0403/integrations that referenced this pull request Sep 17, 2025
…lastic#15296)

* Add data retention and ILM policy for elastic agent status_change_logs data stream
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Integration:elastic_agent Elastic Agent Team:Elastic-Agent Platform - Ingest - Agent [elastic/elastic-agent]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants