diff --git a/docs/chaos-engineering/faults/chaos-faults/aws/aws.md b/docs/chaos-engineering/faults/chaos-faults/aws/aws.md index b17afefa118..000ea2cf108 100644 --- a/docs/chaos-engineering/faults/chaos-faults/aws/aws.md +++ b/docs/chaos-engineering/faults/chaos-faults/aws/aws.md @@ -1129,7 +1129,36 @@ Windows EC2 network loss causes network packet loss on Windows VM for the target - Simulates network partitions (split-brain) between peer replicas for a stateful application. - It may stall the EC2 instance or get corrupted waiting endlessly for a packet. - It degrades the network without marking the EC2 instance as unhealthy (or unworthy) of traffic, which is resolved using a middleware that switches traffic based on SLOs (performance parameters). + + + + + +### Windows EC2 Process Kill + +Windows EC2 Process Kill fault kills the target processes running on a Windows EC2 instance. This fault disrupts application-critical processes running on the instance by killing their underlying processes or threads. + + +Use cases + +- Simulate loss of processes on Windows EC2 instances to check the app resiliency +- Evaluate the impact of process loss on application performance. + + + + + + +### Lambda Block TCP Connection + +Lambda Block TCP Connection is an AWS fault that simulates network blocks for TCP connections of a Lambda function. This fault helps you evaluate how your application responds when outbound TCP connections from a Lambda function are blocked. + + +Use cases +- Simulate network blocks to test Lambda function resilience. +- Evaluate the impact of blocked TCP connections on application performance and error handling. +- Test fallback mechanisms and error reporting in serverless architectures. \ No newline at end of file diff --git a/docs/chaos-engineering/faults/chaos-faults/aws/experiments.ts b/docs/chaos-engineering/faults/chaos-faults/aws/experiments.ts index 7fef54ddd43..a8ac6b4034f 100644 --- a/docs/chaos-engineering/faults/chaos-faults/aws/experiments.ts +++ b/docs/chaos-engineering/faults/chaos-faults/aws/experiments.ts @@ -435,4 +435,18 @@ export const experiments: ExperimentDetails[] = [ tags: ['memory', 'stress', 'windows'], category: "aws", }, + { + name: "Windows EC2 Process Kill", + description: + "Windows EC2 Process Kill fault kills the target processes running on a Windows EC2 instance. This fault disrupts application-critical processes such as databases or message queues running on the instance by killing their underlying processes or threads.", + tags: ['process', 'kill', 'windows', 'ec2'], + category: "aws", + }, + { + name: "Lambda Block TCP Connection", + description: + "Lambda Block TCP Connection is an AWS fault that simulates network blocks for TCP connections of a Lambda function. This fault helps you evaluate how your application responds when outbound TCP connections from a Lambda function are blocked.", + tags: ['lambda', 'tcp', 'network', 'block'], + category: "aws", + }, ]; \ No newline at end of file diff --git a/docs/chaos-engineering/faults/chaos-faults/aws/lambda-block-tcp-connection.md b/docs/chaos-engineering/faults/chaos-faults/aws/lambda-block-tcp-connection.md new file mode 100644 index 00000000000..83221011d1c --- /dev/null +++ b/docs/chaos-engineering/faults/chaos-faults/aws/lambda-block-tcp-connection.md @@ -0,0 +1,182 @@ +--- +id: lambda-block-tcp-connection +title: Lambda Block TCP Connection +redirect_from: +- /docs/chaos-engineering/technical-reference/chaos-faults/aws/lambda-block-tcp-connection +- /docs/chaos-engineering/chaos-faults/aws/lambda-block-tcp-connection +--- + +Lambda Block TCP Connection is an AWS fault that simulates network blocks for TCP connections of a Lambda function. This fault helps you evaluate how your application responds when outbound TCP connections from a Lambda function are blocked. + +![Lambda Block TCP Connection](./static/images/lambda-inject-latency.png) + +## Use cases +- Simulate network blocks to test Lambda function resilience. +- Evaluate the impact of blocked TCP connections on application performance and error handling. +- Test fallback mechanisms and error reporting in serverless architectures. + +### Prerequisites +- Kubernetes >= 1.17 +- The Lambda function must be up and running. +- Kubernetes secret must have the AWS access configuration (key) in the `CHAOS_NAMESPACE`. Below is a sample secret file: + ```yaml + apiVersion: v1 + kind: Secret + metadata: + name: cloud-secret + type: Opaque + stringData: + cloud_config.yml: |- + # Add the cloud AWS credentials respectively + [default] + aws_access_key_id = XXXXXXXXXXXXXXXXXXX + aws_secret_access_key = XXXXXXXXXXXXXXX + ``` + +:::tip +HCE recommends that you use the same secret name, that is, `cloud-secret`. Otherwise, you will need to update the `AWS_SHARED_CREDENTIALS_FILE` environment variable in the fault template with the new secret name and you won't be able to use the default health check probes. +::: + +Below is an example AWS policy to execute the fault. + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "lambda:GetFunction", + "lambda:GetFunctionConfiguration", + "lambda:UpdateFunctionConfiguration" + ], + "Resource": "*" + } + ] +} +``` + +:::info note +- Go to [superset permission/policy](/docs/chaos-engineering/faults/chaos-faults/aws/security-configurations/policy-for-all-aws-faults) to execute all AWS faults. +- Go to the [common tunables](/docs/chaos-engineering/faults/chaos-faults/common-tunables-for-all-faults) and [AWS-specific tunables](/docs/chaos-engineering/faults/chaos-faults/aws/aws-fault-tunables) to tune the common tunables for all faults and AWS-specific tunables. +- Go to [AWS named profile for chaos](/docs/chaos-engineering/faults/chaos-faults/aws/security-configurations/aws-switch-profile) to use a different profile for AWS faults. +- Currently, it is supported in Python; support for other languages is coming soon. +::: + +### Mandatory tunables + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Lambda Chaos Fault Tunables
TunableDescriptionNotes
FUNCTION_NAMEName of the target Lambda function.For example, test-function. For more information, go to function name.
REGIONThe AWS region where the Lambda function is deployed.For example, us-east-1. For more information, go to region.
TARGET_HOSTNAMESComma-separated list of hostnames to block TCP connections to.For example, example.com,api.example.com. For more information, go to target hostnames.
+ + +### Optional tunables + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Common Chaos Tunables
TunableDescriptionNotes
TOTAL_CHAOS_DURATIONDuration for which chaos is injected (in seconds).Default: 30 s. For more information, go to duration of the chaos.
CHAOS_INTERVALThe interval (in seconds) between successive network block attempts.Default: 30 s. For more information, go to chaos interval.
AWS_SHARED_CREDENTIALS_FILEPath to the AWS secret credentials.Default: /tmp/cloud_config.yml. For more information, go to AWS shared credentials file.
RAMP_TIMEPeriod to wait before and after injecting chaos (in seconds).Optional. For more information, go to ramp time.
+ + +#### Target Hostnames + +The `TARGET_HOSTNAMES` environment variable defines the list of hostnames to which TCP connections should be blocked from the target Lambda function during chaos execution. + + +Block TCP connections to specific hostnames from a Lambda function. + +[embedmd]:# (./static/manifests/lambda-block-tcp-connection/target-hostnames.yaml yaml) +```yaml +--- +# Block TCP connections to specific hostnames from a Lambda function +apiVersion: litmuschaos.io/v1alpha1 +kind: ChaosEngine +metadata: + name: lambda-block-tcp-connection +spec: + engineState: "active" + chaosServiceAccount: litmus-admin + experiments: + - name: lambda-block-tcp-connection + spec: + components: + env: + - name: TARGET_HOSTNAMES + value: 'example.com,api.example.com' +``` + +#### Function Name + +The `FUNCTION_NAME` environment variable specifies the name of the target AWS Lambda function whose host will be subjected to chaos. + +[embedmd]:# (./static/manifests/lambda-block-tcp-connection/function-name.yaml yaml) +```yaml +--- +# Block TCP connections from a specific Lambda function +apiVersion: litmuschaos.io/v1alpha1 +kind: ChaosEngine +metadata: + name: lambda-block-tcp-connection +spec: + engineState: "active" + chaosServiceAccount: litmus-admin + experiments: + - name: lambda-block-tcp-connection + spec: + components: + env: + - name: FUNCTION_NAME + value: 'test-function' +``` \ No newline at end of file diff --git a/docs/chaos-engineering/faults/chaos-faults/aws/static/manifests/lambda-block-tcp-connection/function-name.yaml b/docs/chaos-engineering/faults/chaos-faults/aws/static/manifests/lambda-block-tcp-connection/function-name.yaml new file mode 100644 index 00000000000..f04092477c0 --- /dev/null +++ b/docs/chaos-engineering/faults/chaos-faults/aws/static/manifests/lambda-block-tcp-connection/function-name.yaml @@ -0,0 +1,15 @@ +# Block TCP connections from a specific Lambda function +apiVersion: litmuschaos.io/v1alpha1 +kind: ChaosEngine +metadata: + name: lambda-block-tcp-connection +spec: + engineState: "active" + chaosServiceAccount: litmus-admin + experiments: + - name: lambda-block-tcp-connection + spec: + components: + env: + - name: FUNCTION_NAME + value: 'test-function' \ No newline at end of file diff --git a/docs/chaos-engineering/faults/chaos-faults/aws/static/manifests/lambda-block-tcp-connection/target-hostnames.yaml b/docs/chaos-engineering/faults/chaos-faults/aws/static/manifests/lambda-block-tcp-connection/target-hostnames.yaml new file mode 100644 index 00000000000..de4e669e8cc --- /dev/null +++ b/docs/chaos-engineering/faults/chaos-faults/aws/static/manifests/lambda-block-tcp-connection/target-hostnames.yaml @@ -0,0 +1,15 @@ +# Block TCP connections to specific hostnames from a Lambda function +apiVersion: litmuschaos.io/v1alpha1 +kind: ChaosEngine +metadata: + name: lambda-block-tcp-connection +spec: + engineState: "active" + chaosServiceAccount: litmus-admin + experiments: + - name: lambda-block-tcp-connection + spec: + components: + env: + - name: TARGET_HOSTNAMES + value: 'example.com,api.example.com' \ No newline at end of file diff --git a/docs/chaos-engineering/faults/chaos-faults/aws/static/manifests/windows-ec2-process-kill/force.yaml b/docs/chaos-engineering/faults/chaos-faults/aws/static/manifests/windows-ec2-process-kill/force.yaml new file mode 100644 index 00000000000..dcbf3f17de8 --- /dev/null +++ b/docs/chaos-engineering/faults/chaos-faults/aws/static/manifests/windows-ec2-process-kill/force.yaml @@ -0,0 +1,17 @@ +# Kills processes by names with FORCE enabled on a Windows EC2 instance +apiVersion: litmuschaos.io/v1alpha1 +kind: ChaosEngine +metadata: + name: windows-ec2-process-kill +spec: + engineState: "active" + chaosServiceAccount: litmus-admin + experiments: + - name: windows-ec2-process-kill + spec: + components: + env: + - name: PROCESS_NAMES + value: 'notepad' + - name: FORCE + value: 'enable' \ No newline at end of file diff --git a/docs/chaos-engineering/faults/chaos-faults/aws/static/manifests/windows-ec2-process-kill/process-ids.yaml b/docs/chaos-engineering/faults/chaos-faults/aws/static/manifests/windows-ec2-process-kill/process-ids.yaml new file mode 100644 index 00000000000..bbb8c675651 --- /dev/null +++ b/docs/chaos-engineering/faults/chaos-faults/aws/static/manifests/windows-ec2-process-kill/process-ids.yaml @@ -0,0 +1,15 @@ +# Kills processes by IDs on a Windows EC2 instance +apiVersion: litmuschaos.io/v1alpha1 +kind: ChaosEngine +metadata: + name: windows-ec2-process-kill +spec: + engineState: "active" + chaosServiceAccount: litmus-admin + experiments: + - name: windows-ec2-process-kill + spec: + components: + env: + - name: PROCESS_IDS + value: '1234,5678' \ No newline at end of file diff --git a/docs/chaos-engineering/faults/chaos-faults/aws/static/manifests/windows-ec2-process-kill/process-names.yaml b/docs/chaos-engineering/faults/chaos-faults/aws/static/manifests/windows-ec2-process-kill/process-names.yaml new file mode 100644 index 00000000000..70a3c1fbb03 --- /dev/null +++ b/docs/chaos-engineering/faults/chaos-faults/aws/static/manifests/windows-ec2-process-kill/process-names.yaml @@ -0,0 +1,15 @@ +# Kills processes by names on a Windows EC2 instance +apiVersion: litmuschaos.io/v1alpha1 +kind: ChaosEngine +metadata: + name: windows-ec2-process-kill +spec: + engineState: "active" + chaosServiceAccount: litmus-admin + experiments: + - name: windows-ec2-process-kill + spec: + components: + env: + - name: PROCESS_NAMES + value: 'notepad,explorer' \ No newline at end of file diff --git a/docs/chaos-engineering/faults/chaos-faults/aws/windows-ec2-process-kill.md b/docs/chaos-engineering/faults/chaos-faults/aws/windows-ec2-process-kill.md new file mode 100644 index 00000000000..8fc6eb40496 --- /dev/null +++ b/docs/chaos-engineering/faults/chaos-faults/aws/windows-ec2-process-kill.md @@ -0,0 +1,239 @@ +--- +id: windows-ec2-process-kill +title: Windows EC2 Process Kill +redirect_from: +- /docs/chaos-engineering/technical-reference/chaos-faults/aws/windows-ec2-process-kill +--- + +Windows EC2 Process Kill fault kills the target processes running on a Windows EC2 instance. This fault disrupts application-critical processes running on the instance by killing their underlying processes or threads. + +![Windows EC2 Process Kill](./static/images/windows-ec2-cpu-hog.png) + +## Use cases + +- Simulate loss of processes on Windows EC2 instances to check the app resiliency +- Evaluate the impact of process loss on application performance. + +### Prerequisites +- Kubernetes >= 1.17 +- The EC2 instance must be in a healthy state. +- SSM agent must be installed and running on the target EC2 instance. +- SSM IAM role must be attached to the target EC2 instance(s). +- Kubernetes secret must have the AWS Access Key ID and Secret Access Key credentials in the `CHAOS_NAMESPACE`. Below is a sample secret file: + ```yaml + apiVersion: v1 + kind: Secret + metadata: + name: cloud-secret + type: Opaque + stringData: + cloud_config.yml: |- + # Add the cloud AWS credentials respectively + [default] + aws_access_key_id = XXXXXXXXXXXXXXXXXXX + aws_secret_access_key = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX + ``` + +:::tip +HCE recommends that you use the same secret name, that is, `cloud-secret`. Otherwise, you will need to update the `AWS_SHARED_CREDENTIALS_FILE` environment variable in the fault template with the new secret name and you won't be able to use the default health check probes. +::: + +Below is an example AWS policy to execute the fault. + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "ssm:GetDocument", + "ssm:DescribeDocument", + "ssm:GetParameter", + "ssm:GetParameters", + "ssm:SendCommand", + "ssm:CancelCommand", + "ssm:CreateDocument", + "ssm:DeleteDocument", + "ssm:GetCommandInvocation", + "ssm:UpdateInstanceInformation", + "ssm:DescribeInstanceInformation" + ], + "Resource": "*" + }, + { + "Effect": "Allow", + "Action": [ + "ec2messages:AcknowledgeMessage", + "ec2messages:DeleteMessage", + "ec2messages:FailMessage", + "ec2messages:GetEndpoint", + "ec2messages:GetMessages", + "ec2messages:SendReply" + ], + "Resource": "*" + }, + { + "Effect": "Allow", + "Action": [ + "ec2:DescribeInstanceStatus", + "ec2:DescribeInstances" + ], + "Resource": [ + "*" + ] + } + ] +} +``` + +:::info note +- Go to the [common tunables](/docs/chaos-engineering/faults/chaos-faults/common-tunables-for-all-faults) to tune the common tunables for all the faults. +- Go to [AWS named profile for chaos](/docs/chaos-engineering/faults/chaos-faults/aws/security-configurations/aws-switch-profile) to use a different profile for AWS faults and the [superset permission/policy](/docs/chaos-engineering/faults/chaos-faults/aws/security-configurations/policy-for-all-aws-faults) to execute all AWS faults. +::: + +### Mandatory tunables + + + + + + + + + + + + + + + + + + + + + + +
Tunable Description Notes
EC2_INSTANCE_ID / EC2_INSTANCE_TAG ID or tag of the target EC2 instance(s).For example, i-044d3cb4b03b8af1f or a tag key-value. For more information, go to EC2 instance ID.
REGION The AWS region ID where the EC2 instance has been created.For example, us-east-1. For more information, go to region.
PROCESS_IDS / PROCESS_NAMES Process IDs or names of the target processes (comma-separated).For example, 183,253,857 or notepad,explorer. For more information, go to process IDs or process names.
+ +### Optional tunables + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Tunable Description Notes
TOTAL_CHAOS_DURATION Duration for which chaos is injected (in seconds).Default: 60 s. For more information, go to duration of the chaos.
INSTANCE_AFFECTED_PERC Percentage of instances to target (if using tags).Optional. For more information, go to instance affected perc.
FORCE Force kill the process.Default: disable. For example: enable. For more information, go to force .
INSTALL_DEPENDENCIES Install dependencies on the target instance.Default: true. For more information, go to install dependencies.
AWS_SHARED_CREDENTIALS_FILE Path to the AWS secret credentials.Default: /tmp/cloud_config.yml. For more information, go to AWS shared credentials file.
SEQUENCE Sequence of chaos execution for multiple instances.Default: parallel. Supports serial and parallel. For more information, go to sequence of chaos execution.
RAMP_TIME Period to wait before and after injecting chaos (in seconds).Optional. For more information, go to ramp time.
+ +#### Process IDs + +The `PROCESS_IDS` environment variable specifies the processes that you want to kill by their IDs on a Windows EC2 instance. + +[embedmd]:# (./static/manifests/windows-ec2-process-kill/process-ids.yaml yaml) +```yaml +--- +# Kills processes by IDs on a Windows EC2 instance +apiVersion: litmuschaos.io/v1alpha1 +kind: ChaosEngine +metadata: + name: windows-ec2-process-kill +spec: + engineState: "active" + chaosServiceAccount: litmus-admin + experiments: + - name: windows-ec2-process-kill + spec: + components: + env: + - name: PROCESS_IDS + value: '1234,5678' +``` + +#### Process Names + +The `PROCESS_NAMES` environment variable specifies the processes that you want to kill by their Names on a Windows EC2 instance. + +[embedmd]:# (./static/manifests/windows-ec2-process-kill/process-names.yaml yaml) +```yaml +--- +# Kills processes by names on a Windows EC2 instance +apiVersion: litmuschaos.io/v1alpha1 +kind: ChaosEngine +metadata: + name: windows-ec2-process-kill +spec: + engineState: "active" + chaosServiceAccount: litmus-admin + experiments: + - name: windows-ec2-process-kill + spec: + components: + env: + - name: PROCESS_NAMES + value: 'notepad,explorer' +``` + +#### Force + +The `FORCE` environment variable specifies the force kill of the processes under chaos. + + +[embedmd]:# (./static/manifests/windows-ec2-process-kill/force.yaml yaml) +```yaml +--- +# Kills processes by names with FORCE enabled on a Windows EC2 instance +apiVersion: litmuschaos.io/v1alpha1 +kind: ChaosEngine +metadata: + name: windows-ec2-process-kill +spec: + engineState: "active" + chaosServiceAccount: litmus-admin + experiments: + - name: windows-ec2-process-kill + spec: + components: + env: + - name: PROCESS_NAMES + value: 'notepad' + - name: FORCE + value: 'enable' +``` \ No newline at end of file diff --git a/docs/chaos-engineering/faults/chaos-faults/kubernetes/experiments.ts b/docs/chaos-engineering/faults/chaos-faults/kubernetes/experiments.ts index 053bfdbdffe..e04d09353f0 100644 --- a/docs/chaos-engineering/faults/chaos-faults/kubernetes/experiments.ts +++ b/docs/chaos-engineering/faults/chaos-faults/kubernetes/experiments.ts @@ -343,4 +343,16 @@ export const experiments: ExperimentDetails[] = [ tags: ["pod"], category: "kubernetes", }, + { + name: "Pod Application Function Error", + description: "Pod Application Function Error injects an error into a specified function of an application running in a Kubernetes pod. This fault is useful for testing the resilience of your application to function-level errors.", + tags: ["pod", "function", "error"], + category: "kubernetes", + }, + { + name: "Pod Application Function Latency", + description: "Pod Application Function Latency injects latency into a specified function of an application running in a Kubernetes pod. This fault is useful for testing the resilience of your application to function-level delays.", + tags: ["pod", "function", "latency"], + category: "kubernetes", + }, ]; diff --git a/docs/chaos-engineering/faults/chaos-faults/kubernetes/kubernetes.md b/docs/chaos-engineering/faults/chaos-faults/kubernetes/kubernetes.md index 2826e2780d5..26898f923c1 100644 --- a/docs/chaos-engineering/faults/chaos-faults/kubernetes/kubernetes.md +++ b/docs/chaos-engineering/faults/chaos-faults/kubernetes/kubernetes.md @@ -925,3 +925,37 @@ It can test the application's resilience for invalid system time. + + + +### Pod Application Function Error + +Pod Application Function Error injects an error into a specified function of an application running in a Kubernetes pod. This fault is useful for testing the resilience of your application to function-level errors. + + +Use cases + +- Simulate third-party API failure by injecting errors into wrapper functions. +- Test application behavior when core business logic functions return unexpected errors. +- Validate retry mechanisms and fallback strategies for critical service methods. + + + + + + + +### Pod Application Function Latency + +Pod Application Function Latency injects latency into a specified function of an application running in a Kubernetes pod. This fault is useful for testing the resilience of your application to function-level delays. + + +Use cases + +- Simulate slow function execution to test application behavior under degraded performance. +- Validate timeout, retry, and circuit breaker mechanisms when core functions are delayed. +- Induce latency that can lead to application-level failures, helping identify resilience gaps and failure thresholds. + + + + diff --git a/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/pod-application-function-error.md b/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/pod-application-function-error.md new file mode 100644 index 00000000000..565bb97f63f --- /dev/null +++ b/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/pod-application-function-error.md @@ -0,0 +1,199 @@ +--- +id: pod-application-function-error +title: Pod Application Function Error +redirect_from: +- /docs/chaos-engineering/technical-reference/chaos-faults/kubernetes/pod/pod-application-function-error +- /docs/chaos-engineering/technical-reference/chaos-faults/kubernetes/pod-application-function-error +- /docs/chaos-engineering/chaos-faults/kubernetes/pod-application-function-error +--- + +The Pod Application Function Error fault injects an error into a specified function of an application running within a Kubernetes pod. This fault helps assess the application's resilience to failures at the function level. + +![Pod Application Function Error](./static/images/pod-cpu-hog.png) + +## Use cases + +- Simulate third-party API failure by injecting errors into wrapper functions. +- Test application behavior when core business logic functions return unexpected errors. +- Validate retry mechanisms and fallback strategies for critical service methods. + +### Prerequisites +- Kubernetes >= 1.12.0 +- The target application pod must be running and accessible. +- The function to inject error into must be identifiable by name. +- The litmus-admin service account must be available in the target namespace. + +### Permissions required + +Below is a sample Kubernetes role that defines the permissions required to execute the fault. + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + namespace: hce + name: pod-application-function-error +spec: + definition: + scope: Cluster # Supports "Namespaced" mode too +permissions: + - apiGroups: [""] + resources: ["pods"] + verbs: ["create", "delete", "get", "list", "patch", "deletecollection", "update"] + - apiGroups: [""] + resources: ["events"] + verbs: ["create", "get", "list", "patch", "update"] + - apiGroups: [""] + resources: ["pods/log"] + verbs: ["get", "list", "watch"] + - apiGroups: [""] + resources: ["deployments, statefulsets"] + verbs: ["get", "list"] + - apiGroups: [""] + resources: ["replicasets, daemonsets"] + verbs: ["get", "list"] + - apiGroups: [""] + resources: ["chaosEngines", "chaosExperiments", "chaosResults"] + verbs: ["create", "delete", "get", "list", "patch", "update"] + - apiGroups: ["batch"] + resources: ["jobs"] + verbs: ["create", "delete", "get", "list", "deletecollection"] +``` + +### Mandatory tunables + + + + + + + + + + + + + + + + + + + + + + +
Target Application Tunables for Function Error Injection
TunableDescriptionNotes
NAMEName of the target application where the function is defined.For example: my-app. For more information, go to name.
FUNCTIONName of the specific function in the application where the error will be injected..For example: myFunction. For more information, go to function.
+ + +### Optional tunables + + + + + + + + + + + + + + + + + + + + + + +
Common Tunables for Function Error Injection
TunableDescriptionNotes
TOTAL_CHAOS_DURATIONTotal duration (in seconds) for which the error will be injected into the function.Default: 60 s. For more information, go to duration of the chaos.
MESSAGECustom error message to be thrown by the function during chaos.For example: Injected error for testing. For more information, go to message.
+ + +### Name + +The `NAME` env specifies the name of the target application that is under chaos. + + +[embedmd]:# (./static/manifests/pod-application-function-error/name.yaml yaml) +```yaml +--- +# Injects error in a given function of the application pod +apiVersion: litmuschaos.io/v1alpha1 +kind: K8sFault +metadata: + name: pod-application-function-error +spec: + definition: + targets: + application: + name: 'my-app' + function: 'myFunction' + chaos: + experiment: pod-application-function-error + serviceAccountName: litmus-admin + image: docker.io/harness/chaos-ddcr-faults:main-latest + imagePullPolicy: Always + env: + - name: MESSAGE + value: 'Injected error for testing' +``` + +### Function + +The `FUNCTION` env specifies the specific function in the application where the error will be injected. + + +[embedmd]:# (./static/manifests/pod-application-function-error/function.yaml yaml) +```yaml +--- +# Injects error in a given function of the application pod +apiVersion: litmuschaos.io/v1alpha1 +kind: K8sFault +metadata: + name: pod-application-function-error +spec: + definition: + targets: + application: + name: 'my-app' + function: 'myFunction' + chaos: + experiment: pod-application-function-error + serviceAccountName: litmus-admin + image: docker.io/harness/chaos-ddcr-faults:main-latest + imagePullPolicy: Always + env: + - name: MESSAGE + value: 'Injected error for testing' +``` + +### Message + +The `MESSAGE` environment variable specifies the exact error message to be thrown as part of the chaos fault injection. + +[embedmd]:# (./static/manifests/pod-application-function-error/message.yaml yaml) +```yaml +--- +# Injects error in a given function of the application pod +apiVersion: litmuschaos.io/v1alpha1 +kind: K8sFault +metadata: + name: pod-application-function-error +spec: + definition: + targets: + application: + name: 'my-app' + function: 'myFunction' + chaos: + experiment: pod-application-function-error + serviceAccountName: litmus-admin + image: docker.io/harness/chaos-ddcr-faults:main-latest + imagePullPolicy: Always + env: + - name: MESSAGE + value: 'Injected error for testing' +``` \ No newline at end of file diff --git a/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/pod-application-function-latency.md b/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/pod-application-function-latency.md new file mode 100644 index 00000000000..6ec54a5419c --- /dev/null +++ b/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/pod-application-function-latency.md @@ -0,0 +1,200 @@ +--- +id: pod-application-function-latency +title: Pod Application Function Latency +redirect_from: +- /docs/chaos-engineering/technical-reference/chaos-faults/kubernetes/pod/pod-application-function-latency +- /docs/chaos-engineering/technical-reference/chaos-faults/kubernetes/pod-application-function-latency +- /docs/chaos-engineering/chaos-faults/kubernetes/pod-application-function-latency +--- + +The Pod Application Function Latency fault introduces artificial delay into a specified function of an application running within a Kubernetes pod. This helps evaluate the application's resilience to function-level latency and performance degradation. + +![Pod Application Function Latency](./static/images/pod-cpu-hog.png) + +## Use cases + +- Simulate slow function execution to test application behavior under degraded performance. +- Validate timeout, retry, and circuit breaker mechanisms when core functions are delayed. +- Induce latency that can lead to application-level failures, helping identify resilience gaps and failure thresholds. + +### Prerequisites +- Kubernetes >= 1.12.0 +- The target application pod must be running and accessible. +- The function to inject latency into must be identifiable by name. +- The litmus-admin service account must be available in the target namespace. + +### Permissions required + +Below is a sample Kubernetes role that defines the permissions required to execute the fault. + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + namespace: hce + name: pod-application-function-latency +spec: + definition: + scope: Cluster # Supports "Namespaced" mode too +permissions: + - apiGroups: [""] + resources: ["pods"] + verbs: ["create", "delete", "get", "list", "patch", "deletecollection", "update"] + - apiGroups: [""] + resources: ["events"] + verbs: ["create", "get", "list", "patch", "update"] + - apiGroups: [""] + resources: ["pods/log"] + verbs: ["get", "list", "watch"] + - apiGroups: [""] + resources: ["deployments, statefulsets"] + verbs: ["get", "list"] + - apiGroups: [""] + resources: ["replicasets, daemonsets"] + verbs: ["get", "list"] + - apiGroups: [""] + resources: ["chaosEngines", "chaosExperiments", "chaosResults"] + verbs: ["create", "delete", "get", "list", "patch", "update"] + - apiGroups: ["batch"] + resources: ["jobs"] + verbs: ["create", "delete", "get", "list", "deletecollection"] +``` + +### Mandatory tunables + + + + + + + + + + + + + + + + + + + + + + +
Target Application Tunables for Function Error Injection
TunableDescriptionNotes
NAMEName of the target application where the function is defined.For example: my-app. For more information, go to name.
FUNCTIONName of the specific function in the application where the error will be injected.For example: myFunction. For more information, go to function.
+ + +### Optional tunables + + + + + + + + + + + + + + + + + + + + + + +
Common Tunables for Function Error Injection
TunableDescriptionNotes
TOTAL_CHAOS_DURATIONTotal duration (in seconds) for which the error will be injected into the function.Default: 60 s. For more information, go to duration of the chaos.
LATENCYProvide the exact latency that you want to cause in your application.For example: Injected error for testing. For more information, go to message.
+ + + +### Name + +The `NAME` env specifies the name of the target application that is under chaos. + + +[embedmd]:# (./static/manifests/pod-application-function-latency/name.yaml yaml) +```yaml +--- +# Injects error in a given function of the application pod +apiVersion: litmuschaos.io/v1alpha1 +kind: K8sFault +metadata: + name: pod-application-function-latency +spec: + definition: + targets: + application: + name: 'my-app' + function: 'myFunction' + chaos: + experiment: pod-application-function-latency + serviceAccountName: litmus-admin + image: docker.io/harness/chaos-ddcr-faults:main-latest + imagePullPolicy: Always + env: + - name: MESSAGE + value: 'Injected error for testing' +``` + +### Function + +The `FUNCTION` env specifies the specific function in the application where the error will be injected. + + +[embedmd]:# (./static/manifests/pod-application-function-latency/function.yaml yaml) +```yaml +--- +# Injects error in a given function of the application pod +apiVersion: litmuschaos.io/v1alpha1 +kind: K8sFault +metadata: + name: pod-application-function-latency +spec: + definition: + targets: + application: + name: 'my-app' + function: 'myFunction' + chaos: + experiment: pod-application-function-latency + serviceAccountName: litmus-admin + image: docker.io/harness/chaos-ddcr-faults:main-latest + imagePullPolicy: Always + env: + - name: MESSAGE + value: 'Injected error for testing' +``` + +### Latency + +The `LATENCY` environment variable specifies the exact latency that you want to cause in your application. + +[embedmd]:# (./static/manifests/pod-application-function-latency/latency.yaml yaml) +```yaml +--- +# Injects error in a given function of the application pod +apiVersion: litmuschaos.io/v1alpha1 +kind: K8sFault +metadata: + name: pod-application-function-latency +spec: + definition: + targets: + application: + name: 'my-app' + function: 'myFunction' + chaos: + experiment: pod-application-function-latency + serviceAccountName: litmus-admin + image: docker.io/harness/chaos-ddcr-faults:main-latest + imagePullPolicy: Always + env: + - name: LATENCY + value: '2000' +``` \ No newline at end of file diff --git a/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-connection-error/function.yaml b/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-connection-error/function.yaml new file mode 100644 index 00000000000..4b04483c55d --- /dev/null +++ b/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-connection-error/function.yaml @@ -0,0 +1,19 @@ +# Injects error in a given function of the application pod +apiVersion: litmuschaos.io/v1alpha1 +kind: K8sFault +metadata: + name: pod-application-function-latency +spec: + definition: + targets: + application: + name: 'my-app' + function: 'myFunction' + chaos: + experiment: pod-application-function-latency + serviceAccountName: litmus-admin + image: docker.io/harness/chaos-ddcr-faults:main-latest + imagePullPolicy: Always + env: + - name: MESSAGE + value: 'Injected error for testing' \ No newline at end of file diff --git a/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-connection-error/message.yaml b/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-connection-error/message.yaml new file mode 100644 index 00000000000..7f639049d3a --- /dev/null +++ b/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-connection-error/message.yaml @@ -0,0 +1,19 @@ +# Injects error in a given function of the application pod +apiVersion: litmuschaos.io/v1alpha1 +kind: K8sFault +metadata: + name: pod-application-function-error +spec: + definition: + targets: + application: + name: 'my-app' + function: 'myFunction' + chaos: + experiment: pod-application-function-error + serviceAccountName: litmus-admin + image: docker.io/harness/chaos-ddcr-faults:main-latest + imagePullPolicy: Always + env: + - name: MESSAGE + value: 'Injected error for testing' \ No newline at end of file diff --git a/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-connection-error/name.yaml b/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-connection-error/name.yaml new file mode 100644 index 00000000000..4b04483c55d --- /dev/null +++ b/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-connection-error/name.yaml @@ -0,0 +1,19 @@ +# Injects error in a given function of the application pod +apiVersion: litmuschaos.io/v1alpha1 +kind: K8sFault +metadata: + name: pod-application-function-latency +spec: + definition: + targets: + application: + name: 'my-app' + function: 'myFunction' + chaos: + experiment: pod-application-function-latency + serviceAccountName: litmus-admin + image: docker.io/harness/chaos-ddcr-faults:main-latest + imagePullPolicy: Always + env: + - name: MESSAGE + value: 'Injected error for testing' \ No newline at end of file diff --git a/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-function-latency/function.yaml b/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-function-latency/function.yaml new file mode 100644 index 00000000000..4b04483c55d --- /dev/null +++ b/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-function-latency/function.yaml @@ -0,0 +1,19 @@ +# Injects error in a given function of the application pod +apiVersion: litmuschaos.io/v1alpha1 +kind: K8sFault +metadata: + name: pod-application-function-latency +spec: + definition: + targets: + application: + name: 'my-app' + function: 'myFunction' + chaos: + experiment: pod-application-function-latency + serviceAccountName: litmus-admin + image: docker.io/harness/chaos-ddcr-faults:main-latest + imagePullPolicy: Always + env: + - name: MESSAGE + value: 'Injected error for testing' \ No newline at end of file diff --git a/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-function-latency/latency.yaml b/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-function-latency/latency.yaml new file mode 100644 index 00000000000..24d4ddb03a5 --- /dev/null +++ b/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-function-latency/latency.yaml @@ -0,0 +1,19 @@ +# Injects error in a given function of the application pod +apiVersion: litmuschaos.io/v1alpha1 +kind: K8sFault +metadata: + name: pod-application-function-latency +spec: + definition: + targets: + application: + name: 'my-app' + function: 'myFunction' + chaos: + experiment: pod-application-function-latency + serviceAccountName: litmus-admin + image: docker.io/harness/chaos-ddcr-faults:main-latest + imagePullPolicy: Always + env: + - name: LATENCY + value: '2000' \ No newline at end of file diff --git a/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-function-latency/name.yaml b/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-function-latency/name.yaml new file mode 100644 index 00000000000..4b04483c55d --- /dev/null +++ b/docs/chaos-engineering/faults/chaos-faults/kubernetes/pod/static/manifests/pod-application-function-latency/name.yaml @@ -0,0 +1,19 @@ +# Injects error in a given function of the application pod +apiVersion: litmuschaos.io/v1alpha1 +kind: K8sFault +metadata: + name: pod-application-function-latency +spec: + definition: + targets: + application: + name: 'my-app' + function: 'myFunction' + chaos: + experiment: pod-application-function-latency + serviceAccountName: litmus-admin + image: docker.io/harness/chaos-ddcr-faults:main-latest + imagePullPolicy: Always + env: + - name: MESSAGE + value: 'Injected error for testing' \ No newline at end of file