Update module to use the new container insights with CW for EKS

docs/container-insights/eks.md

@@ -1,8 +1,6 @@
 # Setting Up Container Insights for your EKS Cluster
 
-This example deploys AWS Distro of OpenTelemetry on your EKS cluster as a Daemonset which will enable
-Container Insights metrics Dashboard on Amazon CloudWatch.
-
+This example deploys CloudWatch Observability EKS add-on on an exisiting Amazon EKS cluster, which has Container Insights enhanced observability for Amazon EKS and CloudWatch Application Signals enabled by default.
 
 ## Prerequisites
 
@@ -42,12 +40,40 @@ Simply run this command to deploy the example
 terraform apply
 ```
 
-## Visualization
+## Enabling Application Signals (preview) for your services
+CloudWatch Application Signals (preview) is currenlty supported for **Java** applications running on your Amazon EKS cluster.
+
+Next, you have to update your Application to `Configure application metrics and trace sampling`. For this, you must add an annotation to a manifest YAML in your cluster. Adding this annotation auto-instruments the application to send metrics, traces, and logs to Application Signals. You have two options for the annotation:
+
+1. **Annotate Workload** auto-instruments a single workload in the cluster.
+    - Paste the below line into the PodTemplate section of the workload manifest.
+    ```
+    annotations: instrumentation.opentelemetry.io/inject-java: "true"
+    ```
+    - In your terminal, enter `kubectl apply -f your_deployment_yaml` to apply the change.
+
+2. **Annotate Namespace** auto-instruments all workloads deployed in the selected namespace.
+    - Paste the below line into the metadata section of the namespace manifest.
+    ```
+    annotations: instrumentation.opentelemetry.io/inject-java: "true"
+    ```
+    - In your terminal, enter `kubectl apply -f your_namespace_yaml` to apply the change.
+    - In your terminal, enter a command to restart all pods in the namespace. An example command to restart deployment workloads is `kubectl rollout restart deployment -n namespace_name`
+
+## Visualization of Container Insights data
+
+After `terraform apply` is successful, open your Amazon CloudWatch console in the same region as your EKS cluster, then from the left hand side choose `Insights -> Container Insights`, there choose the `EKS` from the drop down and you will see the metrics shown on the dashboard:
+
+<img width="1345" alt="image" src="https://github.com/ruchimo/terraform-aws-observability-accelerator/assets/106240341/31686b29-8ec2-46ff-a266-ebfa1de9768a">
+
+
+## Visualization of CloudWatch Application Signals (preview) data
 
-After apply, open your Amazon CloudWatch console in the same region as your EKS cluster, then from the left hand side choose `Insights -> Container Insights`, there choose the `Performance montoring` from the drop down, choose the `cluster name` and you will see the metrics shown on the dashboard:
+After enabling your Application to pass metrics and traces by following [the steps provided above](#enabling-application-signals-preview-for-your-services), open your Amazon CloudWatch console in the same region as your EKS cluster, then from the left hand side choose `Application Signals -> Services` and you will see the metrics shown on the sample dashboard below:
 
-![image](https://github.com/aws-observability/terraform-aws-observability-accelerator/assets/10175027/c5b9b685-5894-4350-b68a-ca86d1128f6f)
+<img width="1321" alt="image" src="https://github.com/ruchimo/terraform-aws-observability-accelerator/assets/106240341/2fccf784-6560-45a9-8be0-4e843c9653f1">
 
+<img width="1300" alt="image" src="https://github.com/ruchimo/terraform-aws-observability-accelerator/assets/106240341/69749d78-7c9a-4019-a023-f5412ab3b2de">
 
 ## Cleanup
 

examples/eks-container-insights/README.md

@@ -1,10 +1,8 @@
 # Enable Container Insights for EKS cluster
 
-This example deploys ADOT as a daemonset on your EKS cluster which enables Container Insights metrics on CloudWatch.
-
-Step-by-step instructions available on our [docs site](https://aws-observability.github.io/terraform-aws-observability-accelerator/)
-under **Amazon CloudWatch Container Insights**
+This example enables enhanced CloudWatch Container Insights for EKS and CloudWatch Application Signals (preview) through our CloudWatch EKS add-ons, providing comprehensive metrics, logs, and insights for cluster and application monitoring.
 
+Step-by-step instructions available on our [docs site](https://aws-observability.github.io/terraform-aws-observability-accelerator/container-insights/eks/)
 
 <!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
 ## Requirements
@@ -13,9 +11,6 @@ under **Amazon CloudWatch Container Insights**
 |------|---------|
 | <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.1.0 |
 | <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.0.0 |
-| <a name="requirement_helm"></a> [helm](#requirement\_helm) | >= 2.4.1 |
-| <a name="requirement_kubectl"></a> [kubectl](#requirement\_kubectl) | >= 2.0.3 |
-| <a name="requirement_kubernetes"></a> [kubernetes](#requirement\_kubernetes) | >= 2.10 |
 
 ## Providers
 
@@ -42,10 +37,11 @@ under **Amazon CloudWatch Container Insights**
 
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
-| <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region) | EKS cluster region | `string` | n/a | yes |
-| <a name="input_eks_cluster_id"></a> [eks\_cluster\_id](#input\_eks\_cluster\_id) | EKS cluster name | `string` | n/a | yes |
-| <a name="input_irsa_iam_permissions_boundary"></a> [irsa\_iam\_permissions\_boundary](#input\_irsa\_iam\_permissions\_boundary) | IAM permissions boundary for IRSA roles | `string` | `null` | no |
-| <a name="input_irsa_iam_role_path"></a> [irsa\_iam\_role\_path](#input\_irsa\_iam\_role\_path) | IAM role path for IRSA roles | `string` | `"/"` | no |
+| <a name="input_cluster_name"></a> [cluster\_name](#input\_cluster\_name) | EKS cluster name | `string` | n/a | yes |
+| <a name="input_eks_cluster_domain"></a> [eks\_cluster\_domain](#input\_eks\_cluster\_domain) | The domain for the EKS cluster | `string` | `""` | no |
+| <a name="input_eks_cluster_version"></a> [eks\_cluster\_version](#input\_eks\_cluster\_version) | The Kubernetes version for the cluster | `string` | `"1.28"` | no |
+| <a name="input_eks_oidc_provider"></a> [eks\_oidc\_provider](#input\_eks\_oidc\_provider) | The OpenID Connect identity provider (issuer URL without leading `https://`) | `string` | `null` | no |
+| <a name="input_eks_oidc_provider_arn"></a> [eks\_oidc\_provider\_arn](#input\_eks\_oidc\_provider\_arn) | The OpenID Connect identity provider ARN | `string` | `null` | no |
 | <a name="input_tags"></a> [tags](#input\_tags) | Additional tags (e.g. `map('BusinessUnit`,`XYZ`) | `map(string)` | `{}` | no |
 
 ## Outputs

examples/eks-container-insights/data.tf

@@ -0,0 +1,7 @@
+data "aws_partition" "current" {}
+data "aws_caller_identity" "current" {}
+data "aws_region" "current" {}
+
+data "aws_eks_cluster" "eks_cluster" {
+  name = var.cluster_name
+}

examples/eks-container-insights/locals.tf

@@ -1,9 +1,15 @@
-data "aws_partition" "current" {}
+locals {
+  name                = "amazon-cloudwatch-observability"
+  kubernetes_version  = var.eks_cluster_version
+  eks_oidc_issuer_url = replace(data.aws_eks_cluster.eks_cluster.identity[0].oidc[0].issuer, "https://", "")
 
-data "aws_caller_identity" "current" {}
-
-data "aws_region" "current" {}
-
-data "aws_eks_cluster" "eks_cluster" {
-  name = var.eks_cluster_id
+  addon_context = {
+    aws_caller_identity_account_id = data.aws_caller_identity.current.account_id
+    aws_caller_identity_arn        = data.aws_caller_identity.current.arn
+    aws_partition_id               = data.aws_partition.current.partition
+    aws_region_name                = data.aws_region.current.name
+    eks_oidc_provider_arn          = "arn:${data.aws_partition.current.partition}:iam::${data.aws_caller_identity.current.account_id}:oidc-provider/${local.eks_oidc_issuer_url}"
+    eks_cluster_id                 = data.aws_eks_cluster.eks_cluster.id
+    tags                           = var.tags
+  }
 }

examples/eks-container-insights/main.tf

@@ -1,34 +1,7 @@
-provider "aws" {
-  region = var.aws_region
-}
-
-provider "kubernetes" {
-  host                   = local.eks_cluster_endpoint
-  cluster_ca_certificate = base64decode(data.aws_eks_cluster.eks_cluster.certificate_authority[0].data)
-  exec {
-    api_version = "client.authentication.k8s.io/v1beta1"
-    args        = ["eks", "get-token", "--cluster-name", var.eks_cluster_id]
-    command     = "aws"
-  }
-}
-
-provider "helm" {
-  kubernetes {
-    host                   = local.eks_cluster_endpoint
-    cluster_ca_certificate = base64decode(data.aws_eks_cluster.eks_cluster.certificate_authority[0].data)
-    exec {
-      api_version = "client.authentication.k8s.io/v1beta1"
-      args        = ["eks", "get-token", "--cluster-name", var.eks_cluster_id]
-      command     = "aws"
-    }
-  }
-}
-
-
-# Deploy the ADOT Container Insights
-
 module "eks_container_insights" {
-  source = "../../modules/eks-container-insights"
-  # source = "github.com/aws-observability/terraform-aws-observability-accelerator//modules/eks-container-insights?ref=v2.5.4"
-  eks_cluster_id = var.eks_cluster_id
+  source                                    = "../../modules/eks-container-insights"
+  cluster_name                              = var.cluster_name
+  enable_amazon_eks_cw_observability        = true
+  create_cloudwatch_observability_irsa_role = true
+  eks_oidc_provider_arn                     = local.addon_context.eks_oidc_provider_arn
 }

examples/eks-container-insights/variables.tf

@@ -1,25 +1,32 @@
-variable "eks_cluster_id" {
+variable "cluster_name" {
   description = "EKS cluster name"
   type        = string
 }
 
-variable "aws_region" {
-  description = "EKS cluster region"
+variable "eks_cluster_domain" {
+  description = "The domain for the EKS cluster"
   type        = string
+  default     = ""
 }
 
-variable "irsa_iam_role_path" {
-  description = "IAM role path for IRSA roles"
+variable "eks_oidc_provider" {
+  description = "The OpenID Connect identity provider (issuer URL without leading `https://`)"
   type        = string
-  default     = "/"
+  default     = null
 }
 
-variable "irsa_iam_permissions_boundary" {
-  description = "IAM permissions boundary for IRSA roles"
+variable "eks_oidc_provider_arn" {
+  description = "The OpenID Connect identity provider ARN"
   type        = string
   default     = null
 }
 
+variable "eks_cluster_version" {
+  description = "The Kubernetes version for the cluster"
+  type        = string
+  default     = "1.28"
+}
+
 variable "tags" {
   description = "Additional tags (e.g. `map('BusinessUnit`,`XYZ`)"
   type        = map(string)

examples/eks-container-insights/versions.tf

@@ -6,25 +6,5 @@ terraform {
       source  = "hashicorp/aws"
       version = ">= 5.0.0"
     }
-    kubernetes = {
-      source  = "hashicorp/kubernetes"
-      version = ">= 2.10"
-    }
-    kubectl = {
-      source  = "alekc/kubectl"
-      version = ">= 2.0.3"
-    }
-    helm = {
-      source  = "hashicorp/helm"
-      version = ">= 2.4.1"
-    }
   }
-
-  # ##  Used for end-to-end testing on project; update to suit your needs
-  # backend "s3" {
-  #   bucket = "aws-observability-accelerator-terraform-states"
-  #   region = "us-west-2"
-  #   key    = "e2e/eks_container_insights/terraform.tfstate"
-  # }
-
 }

modules/eks-container-insights/README.md

@@ -1,10 +1,10 @@
-# Container Insights ADOT implementation for EKS Cluster Observability
+# Container Insights CloudWatch implementation for EKS Cluster Observability
 
-This module provides an automated experience around enabling Container Insights for your EKS cluster using ADOT (AWS Distro for OpenTelemetry).
-It provides the following resources:
+This module configures AWS CloudWatch Agent used for CloudWatch Application signals and Container Insights.
 
-- ADOT Collector Deployment to your EKS cluster
-- Enabling Container Insights on CloudWatch
+Use CloudWatch Application Signals to automatically instrument your applications on AWS so that you can monitor current application health and track long-term application performance against your business objectives. Application Signals provides you with a unified, application-centric view of your applications, services, and dependencies, and helps you monitor and triage application health.
+
+Use CloudWatch Container Insights to collect, aggregate, and summarize metrics and logs from your containerized applications and microservices. CloudWatch automatically collects metrics for many resources, such as CPU, memory, disk, and network. Container Insights also provides diagnostic information, such as container restart failures, to help you isolate issues and resolve them quickly. You can also set CloudWatch alarms on metrics that Container Insights collects.
 
 
 <!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
@@ -14,9 +14,6 @@ It provides the following resources:
 |------|---------|
 | <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.1.0 |
 | <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.0.0 |
-| <a name="requirement_helm"></a> [helm](#requirement\_helm) | >= 2.4.1 |
-| <a name="requirement_kubectl"></a> [kubectl](#requirement\_kubectl) | >= 2.0.3 |
-| <a name="requirement_kubernetes"></a> [kubernetes](#requirement\_kubernetes) | >= 2.10 |
 
 ## Providers
 
@@ -28,31 +25,32 @@ It provides the following resources:
 
 | Name | Source | Version |
 |------|--------|---------|
-| <a name="module_helm_addon"></a> [helm\_addon](#module\_helm\_addon) | github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon | v4.32.1 |
+| <a name="module_cloudwatch_observability_irsa_role"></a> [cloudwatch\_observability\_irsa\_role](#module\_cloudwatch\_observability\_irsa\_role) | terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks | v5.33.0 |
 
 ## Resources
 
 | Name | Type |
 |------|------|
+| [aws_eks_addon.amazon_cloudwatch_observability](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_addon) | resource |
+| [aws_iam_service_linked_role.application_signals_cw](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_service_linked_role) | resource |
 | [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
+| [aws_eks_addon_version.eks_addon_version](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_addon_version) | data source |
 | [aws_eks_cluster.eks_cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster) | data source |
-| [aws_iam_policy.irsa](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy) | data source |
 | [aws_partition.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) | data source |
 | [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
 
 ## Inputs
 
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
-| <a name="input_adot_otel_helm_chart_verison"></a> [adot\_otel\_helm\_chart\_verison](#input\_adot\_otel\_helm\_chart\_verison) | ADOT collector helm chart version | `string` | `"0.17.0"` | no |
-| <a name="input_eks_cluster_id"></a> [eks\_cluster\_id](#input\_eks\_cluster\_id) | EKS Cluster Id | `string` | n/a | yes |
-| <a name="input_helm_config"></a> [helm\_config](#input\_helm\_config) | Helm provider config for adot-exporter-for-eks-on-ec2 | `any` | `{}` | no |
-| <a name="input_irsa_iam_permissions_boundary"></a> [irsa\_iam\_permissions\_boundary](#input\_irsa\_iam\_permissions\_boundary) | IAM permissions boundary for IRSA roles | `string` | `null` | no |
-| <a name="input_irsa_iam_role_path"></a> [irsa\_iam\_role\_path](#input\_irsa\_iam\_role\_path) | IAM role path for IRSA roles | `string` | `"/"` | no |
-| <a name="input_irsa_policies"></a> [irsa\_policies](#input\_irsa\_policies) | Additional IAM policies for a IAM role for service accounts | `list(string)` | `[]` | no |
-| <a name="input_manage_via_gitops"></a> [manage\_via\_gitops](#input\_manage\_via\_gitops) | Determines if the add-on should be managed via GitOps. | `bool` | `false` | no |
-| <a name="input_service_exporters"></a> [service\_exporters](#input\_service\_exporters) | exporter for adot-ci setup | `string` | `"awsemf"` | no |
-| <a name="input_service_receivers"></a> [service\_receivers](#input\_service\_receivers) | receiver for adot-ci setup | `string` | `"awscontainerinsightreceiver"` | no |
+| <a name="input_addon_config"></a> [addon\_config](#input\_addon\_config) | Amazon EKS Managed CloudWatch Observability Add-on config | `any` | `{}` | no |
+| <a name="input_cluster_name"></a> [cluster\_name](#input\_cluster\_name) | Name of the EKS cluster | `string` | `"eks-cw"` | no |
+| <a name="input_create_cloudwatch_application_signals_role"></a> [create\_cloudwatch\_application\_signals\_role](#input\_create\_cloudwatch\_application\_signals\_role) | Create a Cloudwatch Application Signals service-linked role | `string` | `true` | no |
+| <a name="input_create_cloudwatch_observability_irsa_role"></a> [create\_cloudwatch\_observability\_irsa\_role](#input\_create\_cloudwatch\_observability\_irsa\_role) | Create a Cloudwatch Observability IRSA | `string` | `true` | no |
+| <a name="input_eks_oidc_provider_arn"></a> [eks\_oidc\_provider\_arn](#input\_eks\_oidc\_provider\_arn) | The OIDC Provider ARN of AWS EKS cluster | `string` | `""` | no |
+| <a name="input_enable_amazon_eks_cw_observability"></a> [enable\_amazon\_eks\_cw\_observability](#input\_enable\_amazon\_eks\_cw\_observability) | Enable Amazon EKS CloudWatch Observability add-on | `bool` | `false` | no |
+| <a name="input_kubernetes_version"></a> [kubernetes\_version](#input\_kubernetes\_version) | Kubernetes version | `string` | `"1.28"` | no |
+| <a name="input_most_recent"></a> [most\_recent](#input\_most\_recent) | Most Recent | `string` | `"false"` | no |
 | <a name="input_tags"></a> [tags](#input\_tags) | Additional tags (e.g. `map('BusinessUnit`,`XYZ`) | `map(string)` | `{}` | no |
 
 ## Outputs

modules/eks-container-insights/data.tf

@@ -0,0 +1,7 @@
+data "aws_partition" "current" {}
+data "aws_caller_identity" "current" {}
+data "aws_region" "current" {}
+
+data "aws_eks_cluster" "eks_cluster" {
+  name = var.cluster_name
+}