Added support for additional validation rules in custom detector (#4413)

* Added support for additional validation rules in custom detector

* refactored the approach to support validations for each regex in config

* resolved comments

Author: kashifkhan0771

examples/generic_with_filters.yml

@@ -11,6 +11,10 @@ detectors:
   regex:
     secret: |-
       (?i)[\w.-]{0,50}?(?:access|auth|(?-i:[Aa]pi|API)|credential|creds|key|passw(?:or)?d|secret|token)(?:[ \t\w.-]{0,20})[\s'"]{0,3}(?:=|>|:{1,3}=|\|\||:|=>|\?=|,)[\x60'"\s=]{0,5}([\w.=-]{10,150}|[a-z0-9][a-z0-9+/]{11,}={0,3})(?:[\x60'"\s;]|\\[nr]|$)
+  validations:
+    secret: # name of the regex to apply these validations to
+      contains_digit: true
+      contains_special_char: true
   entropy: 3.5
   # exclude_regexes_capture:
   #   - |-

pkg/custom_detectors/CUSTOM_DETECTORS.md

@@ -43,6 +43,13 @@ This guide will walk you through setting up a custom detector in TruffleHog to i
    - **`exclude_regexes_match`**: This parameter enables you to define regex patterns to exclude entire matches from being reported as secrets. This applies to the entire matched string, not just the token.
    - **`entropy`**: This parameter is used to assess the randomness of detected strings. High entropy often indicates that a string is a potential secret, such as an API key or password, due to its complexity and unpredictability. It helps in filtering false-positives. While an entropy threshold of `3` can be a starting point, it's essential to adjust this value based on your project's specific requirements and the nature of the data you have.
    - **`exclude_words`**: This parameter allows you to specify a list of words that, if present in a detected string, will cause TruffleHog to ignore that string. This is a substring match and does not enforce word boundaries. It applies only to the token.
+   - **`validations`**: This parameter lets you define extra validation rules for each regex specified in the regex option. These rules address limitations of Go's RE2 engine, such as the lack of lookahead support, and are applied after a regex match to help reduce false positives.
+   Available validation options:
+     - **`contains_digit`**: Ensures the match contains at least one numeric digit (0-9). Useful for API keys or tokens that must include numbers.
+     - **`contains_lowercase`**: Ensures the match contains at least one lowercase letter (a-z). Common requirement for passwords and mixed-case tokens.
+     - **`contains_uppercase`**: Ensures the match contains at least one uppercase letter (A-Z). Helps validate tokens that follow mixed-case conventions.
+     - **`contains_special_char`**: Ensures the match contains at least one special character from the set `!@#$%^&*()_+-=[]{}|;:,.<>?`. Useful for complex passwords or encoded tokens.
+    
 
     [Here](/examples/generic_with_filters.yml) is an example of a custom detector using these parameters. 
 

pkg/custom_detectors/custom_detectors.go

@@ -116,7 +116,7 @@ func (c *CustomRegexWebhook) FromData(ctx context.Context, verify bool, data []b
 
 MatchLoop:
 	for _, match := range matches {
-		for _, values := range match {
+		for key, values := range match {
 			// attempt to use capture group
 			secret := values[0]
 			if len(values) > 1 {
@@ -150,6 +150,26 @@ MatchLoop:
 					continue MatchLoop
 				}
 			}
+
+			if validations := c.GetValidations(); validations != nil {
+				validationRules := []struct {
+					enabled   bool
+					validator func(string) bool
+				}{
+					{validations[key].GetContainsDigit(), ContainsDigit},
+					{validations[key].GetContainsLowercase(), ContainsLowercase},
+					{validations[key].GetContainsUppercase(), ContainsUppercase},
+					{validations[key].GetContainsSpecialChar(), ContainsSpecialChar},
+				}
+
+				for _, rule := range validationRules {
+					if rule.enabled && !rule.validator(secret) {
+						// skip this match if a validation rule is enabled but missing from the secret
+						continue MatchLoop
+					}
+				}
+			}
+
 		}
 
 		g.Go(func() error {