[ti_cyware_intel_exchange] Update Readme and add ioc_expiration_duration config parameter #15286

muskan-agarwal26 · 2025-09-11T11:05:40Z

Type of change

Enhancement

Proposed commit message

This release includes addition of ioc_expiration_duration configuration parameter to define the time of deletion of a log from transform.
It also includes the updated Readme as per the new template.
Field type of custom_scores is changed from long to flattened.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

Clone integrations repo.
Install elastic package locally.
Start elastic stack using elastic-package.
Move to integrations/packages/ti_cyware_intel_exchange directory.
Run the following command to run tests.

elastic-package test

2025/09/11 16:35:32  INFO New version is available - v0.114.0. Download from: https://github.com/elastic/elastic-package/releases/tag/v0.114.0
Run asset tests for the package
2025/09/11 16:35:33  INFO License text found in "/root/GITHUB/integrations/LICENSE.txt" will be included in package
--- Test results for package: ti_cyware_intel_exchange - START ---
╭──────────────────────────┬─────────────┬───────────┬───────────────────────────────────────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE                  │ DATA STREAM │ TEST TYPE │ TEST NAME                                                                         │ RESULT │ TIME ELAPSED │
├──────────────────────────┼─────────────┼───────────┼───────────────────────────────────────────────────────────────────────────────────┼────────┼──────────────┤
│ ti_cyware_intel_exchange │             │ asset     │ dashboard ti_cyware_intel_exchange-56ee88b2-39b0-44f1-a122-46ff83bdbcb0 is loaded │ PASS   │       2.55µs │
│ ti_cyware_intel_exchange │             │ asset     │ search ti_cyware_intel_exchange-d3c12e4c-1d77-4c81-8223-5f909ffb433f is loaded    │ PASS   │        226ns │
│ ti_cyware_intel_exchange │ indicator   │ asset     │ index_template logs-ti_cyware_intel_exchange.indicator is loaded                  │ PASS   │        219ns │
│ ti_cyware_intel_exchange │ indicator   │ asset     │ ingest_pipeline logs-ti_cyware_intel_exchange.indicator-0.1.1 is loaded           │ PASS   │        300ns │
╰──────────────────────────┴─────────────┴───────────┴───────────────────────────────────────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: ti_cyware_intel_exchange - END   ---
Done
Run pipeline tests for the package
--- Test results for package: ti_cyware_intel_exchange - START ---
╭──────────────────────────┬─────────────┬───────────┬───────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE                  │ DATA STREAM │ TEST TYPE │ TEST NAME                                     │ RESULT │ TIME ELAPSED │
├──────────────────────────┼─────────────┼───────────┼───────────────────────────────────────────────┼────────┼──────────────┤
│ ti_cyware_intel_exchange │ indicator   │ pipeline  │ (ingest pipeline warnings test-indicator.log) │ PASS   │ 727.724563ms │
│ ti_cyware_intel_exchange │ indicator   │ pipeline  │ test-indicator.log                            │ PASS   │ 372.805349ms │
╰──────────────────────────┴─────────────┴───────────┴───────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: ti_cyware_intel_exchange - END   ---
Done
Run policy tests for the package
--- Test results for package: ti_cyware_intel_exchange - START ---
No test results
--- Test results for package: ti_cyware_intel_exchange - END   ---
Done
Run static tests for the package
--- Test results for package: ti_cyware_intel_exchange - START ---
╭──────────────────────────┬─────────────┬───────────┬──────────────────────────┬────────┬──────────────╮
│ PACKAGE                  │ DATA STREAM │ TEST TYPE │ TEST NAME                │ RESULT │ TIME ELAPSED │
├──────────────────────────┼─────────────┼───────────┼──────────────────────────┼────────┼──────────────┤
│ ti_cyware_intel_exchange │ indicator   │ static    │ Verify sample_event.json │ PASS   │ 295.866369ms │
╰──────────────────────────┴─────────────┴───────────┴──────────────────────────┴────────┴──────────────╯
--- Test results for package: ti_cyware_intel_exchange - END   ---
Done
Run system tests for the package
2025/09/11 16:35:41  INFO License text found in "/root/GITHUB/integrations/LICENSE.txt" will be included in package
2025/09/11 16:36:52  INFO Write container logs to file: /root/GITHUB/integrations/build/container-logs/indicator-api-1757588812989465565.log
2025/09/11 16:36:56  INFO Write container logs to file: /root/GITHUB/integrations/build/container-logs/elastic-agent-1757588816205599723.log
--- Test results for package: ti_cyware_intel_exchange - START ---
╭──────────────────────────┬─────────────┬───────────┬───────────┬────────┬───────────────╮
│ PACKAGE                  │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │  TIME ELAPSED │
├──────────────────────────┼─────────────┼───────────┼───────────┼────────┼───────────────┤
│ ti_cyware_intel_exchange │ indicator   │ system    │ default   │ PASS   │ 54.619711399s │
╰──────────────────────────┴─────────────┴───────────┴───────────┴────────┴───────────────╯
--- Test results for package: ti_cyware_intel_exchange - END   ---
Done

1. Updated the readme as per the new template. 2. Added ilm policy inside manifest file.

1. Add ioc_expiration_duration config parameter. 2. Updated dashboard description. 3. Change field type from long to flattened of custom_scores field.

elasticmachine · 2025-09-11T12:01:03Z

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

packages/ti_cyware_intel_exchange/_dev/build/docs/README.md

...ges/ti_cyware_intel_exchange/data_stream/indicator/_dev/test/pipeline/test-common-config.yml

...ges/ti_cyware_intel_exchange/data_stream/indicator/elasticsearch/ingest_pipeline/default.yml

kcreddy · 2025-09-13T04:36:54Z

/test

elastic-vault-github-plugin-prod · 2025-09-13T04:59:22Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

efd6 · 2025-09-15T02:06:23Z

packages/ti_cyware_intel_exchange/_dev/build/docs/README.md


-## Requirements
+The Cyware Intel Exchange integration is compatible with `v3` version.


Suggested change

The Cyware Intel Exchange integration is compatible with `v3` version.

The Cyware Intel Exchange integration is compatible with Cyware version `v3`.

?

Updating with:
The Cyware Intel Exchange integration is compatible with CTIX API version v3.

efd6 · 2025-09-15T02:09:29Z

packages/ti_cyware_intel_exchange/_dev/build/docs/README.md


-Agentless deployments are only supported in Elastic Serverless and Elastic Cloud environments. This functionality is in beta and is subject to change. Beta features are not subject to the support SLA of official GA features.
+This integration periodically queries the CTIX API to retrieve IOC indicators.


Suggested change

This integration periodically queries the CTIX API to retrieve IOC indicators.

This integration periodically queries the [CTIX API](https://ctixapiv3.cyware.com/intel-exchange-api-reference) to retrieve Indicators of Compromise (IOCs).

efd6 · 2025-09-15T02:10:17Z

packages/ti_cyware_intel_exchange/_dev/build/docs/README.md


 1. Go to **Administration** > **Integration Management**.
 2. In **Third Party Developers**, click **CTIX Integrators**.
 3. Click **Add New**. Enter the following details:
-   - **Name**: Enter a unique name for the API credentials in 50 characters.
+   - **Name**: Enter a unique name for the API credentials within 50 characters.


Suggested change

- **Name**: Enter a unique name for the API credentials within 50 characters.

- **Name**: Enter a unique name for the API credentials up to 50 characters long.

efd6 · 2025-09-15T02:10:35Z

packages/ti_cyware_intel_exchange/_dev/build/docs/README.md


 1. Go to **Administration** > **Integration Management**.
 2. In **Third Party Developers**, click **CTIX Integrators**.
 3. Click **Add New**. Enter the following details:
-   - **Name**: Enter a unique name for the API credentials in 50 characters.
+   - **Name**: Enter a unique name for the API credentials within 50 characters.
   - **Description**: Enter a description for the credentials within 1000 characters.


Suggested change

- **Description**: Enter a description for the credentials within 1000 characters.

- **Description**: Enter a description for the credentials up to 1000 characters long.

efd6 · 2025-09-15T02:11:38Z

packages/ti_cyware_intel_exchange/_dev/build/docs/README.md

+#### ILM Policy
+
+To facilitate IoC expiration, source data stream-backed indices `.ds-logs-ti_cyware_intel_exchange.indicator-*` are allowed to contain duplicates from each polling interval. ILM policy `logs-ti_cyware_intel_exchange.indicator-default_policy` is added to these source indices, so it doesn't lead to unbounded growth. This means that in these source indices data will be deleted after `5 days` from ingested date.
+


Delete this empty line.

efd6 · 2025-09-15T02:13:08Z

packages/ti_cyware_intel_exchange/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "0.1.1"
+  changes:
+    - description: Update documentation and add ioc_expiration_duration config parameter.


Suggested change

- description: Update documentation and add ioc_expiration_duration config parameter.

- description: Update documentation and add "IOC Expiration Duration" configuration parameter.

(ioc_expiration_duration is not a user-facing entity, but the changelog is user-facing documentation)

efd6 · 2025-09-15T02:17:14Z

packages/ti_cyware_intel_exchange/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "0.1.1"


Suggested change

- version: "0.1.1"

- version: "0.2.0"

(enhancement bumps minor)

efd6 · 2025-09-15T02:19:46Z

packages/ti_cyware_intel_exchange/changelog.yml

+  changes:
+    - description: Update documentation and add ioc_expiration_duration config parameter.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/15286


Suggested change

link: https://github.com/elastic/integrations/pull/15286

link: https://github.com/elastic/integrations/pull/15286

- description: Fix type of `ti_cyware_intel_exchange.indicator.external_references` field in transform field definitions.

type: bugfix

link: https://github.com/elastic/integrations/pull/15286

- description: Fix handling of `ti_cyware_intel_exchange.indicator.custom_scores`.

type: bugfix

link: https://github.com/elastic/integrations/pull/15286

- description: Change type of `ti_cyware_intel_exchange.indicator.custom_scores` from `long` to `flattened`.

type: breaking-change

link: https://github.com/elastic/integrations/pull/15286

efd6 · 2025-09-15T02:25:23Z

packages/ti_cyware_intel_exchange/data_stream/indicator/fields/fields.yml

        - name: custom_scores
-          type: long
+          type: flattened


This makes it a breaking change. Why is it being changed?

In live logs, we are now receiving values like: {"x_ctix_customscore_2": 2}.
Previously, the value used to be null, so we had defined the type as long, assuming it would be numeric based on the name. However, since actual values are now appearing in live logs, we changed it to flattened.

efd6 · 2025-09-16T06:36:54Z

/test

elasticmachine · 2025-09-16T06:56:29Z

💔 Build Failed

Buildkite Build
Commit: 431f445

Failed CI Steps

Check integrations ti_cyware_intel_exchange

History

💚 Build #31277 succeeded 313d60c

elastic-sonarqube · 2025-09-16T06:58:50Z

Quality Gate failed

Failed conditions
62.9% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

ShourieG

LGTM from my end but please wait for Dan's approval before merging

muskan-crest added 2 commits September 5, 2025 16:25

Enchancements -

fa7c4af

1. Updated the readme as per the new template. 2. Added ilm policy inside manifest file.

Enhancements in Cyware-

5e0bac2

1. Add ioc_expiration_duration config parameter. 2. Updated dashboard description. 3. Change field type from long to flattened of custom_scores field.

muskan-agarwal26 requested a review from a team as a code owner September 11, 2025 11:05

Updated changelog entry

313d60c

ShourieG reviewed Sep 11, 2025

View reviewed changes

efd6 reviewed Sep 15, 2025

View reviewed changes

Resolved comments provided by @efd6 and @ShourieG

431f445

muskan-agarwal26 requested review from efd6 and ShourieG September 15, 2025 10:04

ShourieG reviewed Sep 16, 2025

View reviewed changes

ShourieG approved these changes Sep 16, 2025

View reviewed changes


		## Requirements
		The Cyware Intel Exchange integration is compatible with `v3` version.


		Agentless deployments are only supported in Elastic Serverless and Elastic Cloud environments. This functionality is in beta and is subject to change. Beta features are not subject to the support SLA of official GA features.
		This integration periodically queries the CTIX API to retrieve IOC indicators.

	- Name: Enter a unique name for the API credentials within 50 characters.
	- Name: Enter a unique name for the API credentials up to 50 characters long.

	- Description: Enter a description for the credentials within 1000 characters.
	- Description: Enter a description for the credentials up to 1000 characters long.

		#### ILM Policy

		To facilitate IoC expiration, source data stream-backed indices `.ds-logs-ti_cyware_intel_exchange.indicator-*` are allowed to contain duplicates from each polling interval. ILM policy `logs-ti_cyware_intel_exchange.indicator-default_policy` is added to these source indices, so it doesn't lead to unbounded growth. This means that in these source indices data will be deleted after `5 days` from ingested date.

	- description: Update documentation and add ioc_expiration_duration config parameter.
	- description: Update documentation and add "IOC Expiration Duration" configuration parameter.

-      link: https://github.com/elastic/integrations/pull/15286
+      link: https://github.com/elastic/integrations/pull/15286
+    - description: Fix type of `ti_cyware_intel_exchange.indicator.external_references` field in transform field definitions.
+      type: bugfix
+      link: https://github.com/elastic/integrations/pull/15286
+    - description: Fix handling of `ti_cyware_intel_exchange.indicator.custom_scores`.
+      type: bugfix
+      link: https://github.com/elastic/integrations/pull/15286
+    - description: Change type of `ti_cyware_intel_exchange.indicator.custom_scores` from `long` to `flattened`.
+      type: breaking-change
+      link: https://github.com/elastic/integrations/pull/15286

[ti_cyware_intel_exchange] Update Readme and add ioc_expiration_duration config parameter #15286

Are you sure you want to change the base?

[ti_cyware_intel_exchange] Update Readme and add ioc_expiration_duration config parameter #15286

Conversation

muskan-agarwal26 commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

How to test this PR locally

Uh oh!

elasticmachine commented Sep 11, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kcreddy commented Sep 13, 2025

Uh oh!

elastic-vault-github-plugin-prod bot commented Sep 13, 2025

🚀 Benchmarks report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

efd6 commented Sep 16, 2025

Uh oh!

elasticmachine commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💔 Build Failed

Failed CI Steps

History

Uh oh!

elastic-sonarqube bot commented Sep 16, 2025

Quality Gate failed

Uh oh!

ShourieG left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

muskan-agarwal26 commented Sep 11, 2025 •

edited

Loading

elasticmachine commented Sep 16, 2025 •

edited

Loading