# Detector Notes Working notes about TruffleHog detector behavior and local post-processing ideas. ## GitHub / GitLab Noise Current TruffleHog source contains both modern and legacy detectors. GitHub v2 detects modern prefixed PATs: ```text (ghp|gho|ghu|ghs|ghr|github_pat)_[a-zA-Z0-9_]{36,255} ``` GitHub v1 detects legacy 40-character hex tokens near words such as `github`, `gh`, `pat`, or `token`: ```text (?:github|gh|pat|token).{0,40}([a-f0-9]{40}) ``` GitHubOauth2 detects a 20-character client id and a 40-character client secret near `github`: ```text client_id: [a-zA-Z0-9]{20} client_secret: [a-f0-9]{40} Raw = client_id RawV2 = client_id + client_secret ``` GitLab v2 detects modern PATs: ```text glpat-[a-zA-Z0-9\-=_]{20,22} ``` GitLab v1 detects any 20-22 character token-like value near `gitlab` and skips `glpat-` so v2 can handle it: ```text gitlab ... ([a-zA-Z0-9\-=_]{20,22}) ``` Observed local results show high false-positive volume for unverified GitHub v1, GitHubOauth2, and GitLab v1 detections. The current scanner post-filter drops unverified GitHub/GitLab findings that do not match known modern token prefixes. This reduces noise but can hide real legacy/OAuth credentials if verification cannot run. TODO: Prefer a confidence model over hard dropping: ```text verified -> high confidence modern prefix shape -> high/medium confidence legacy GitHub v1 / GitHubOauth2 / GitLab v1 -> low confidence unless verified ``` Dashboard should hide low-confidence findings by default, but allow explicit review. ## GCP ### GCP service account JSON Detector: `GCP` TruffleHog detects JSON blobs containing `auth_provider_x509_cert_url` and parses service-account style credentials. Useful fields already present in the JSON: ```text type project_id private_key_id private_key client_email client_id auth_uri token_uri auth_provider_x509_cert_url client_x509_cert_url ``` TruffleHog output behavior: ```text Raw = client_email, or full key JSON if client_email is missing RawV2 = full cleaned credential JSON Redacted = client_email ExtraData.project = project_id AnalysisInfo.principal = client_email AnalysisInfo.type = type ``` Practical enrichment fields: ```text gcp_project_id gcp_client_email gcp_client_id gcp_private_key_id gcp_credential_type ``` This detector has enough context to verify/function without extra source-code lookup if `RawV2` is preserved. ### GCP Application Default Credentials Detector: `GCPApplicationDefaultCredentials` TruffleHog detects ADC JSON containing `client_secret` and `.apps.googleusercontent.com` client IDs. Useful fields: ```text client_id client_secret refresh_token type ``` TruffleHog output behavior: ```text Raw = client_id without .apps.googleusercontent.com suffix RawV2 = client_id_without_suffix + refresh_token Redacted = shortened refresh_token ExtraData may contain verification details when verified ``` Risk: `RawV2` is concatenated and does not retain `client_secret` cleanly. The raw finding JSON may not be enough to reconstruct the original ADC JSON unless the source line/file is available. Practical enrichment fields: ```text gcp_client_id gcp_refresh_token_redacted gcp_credential_type ``` TODO: For ADC findings, use source context around the finding to parse the whole JSON and preserve `client_secret`/`refresh_token` as structured fields. ### Google AQ authentication keys `AQ.` credentials are currently classified through the Gemini Developer API at `generativelanguage.googleapis.com`. That result does not establish Vertex AI access. TODO: Add a separate Vertex AI Express probe for `AQ.` credentials against the supported `aiplatform.googleapis.com` key-authenticated methods. Keep Gemini Developer API and Vertex results independent, and do not infer access to the full project/location-scoped Vertex API from the key prefix or from a successful Gemini Developer API check. ## Azure ### Azure Container Registry Detector: `AzureContainerRegistry` Detector finds registry hosts and ACR password-like values. Patterns: ```text registry: .azurecr.io password: [a-zA-Z0-9+/]{42}+ACR[a-zA-Z0-9]{6} ``` TruffleHog output behavior: ```text Raw = password RawV2 = {"username":"","password":""} Redacted = registry name ``` Verification uses: ```text https://.azurecr.io/v2/ BasicAuth(username=, password=) ``` Practical enrichment fields: ```text azure_acr_registry azure_acr_login_server = .azurecr.io ``` This detector has enough context in `RawV2` to be useful. ### Azure OpenAI Detector: `AzureOpenAI` Detector finds API keys and Azure OpenAI endpoints. Patterns: ```text endpoint: .openai.azure.com key: 32 lowercase hex chars near api_key/openai_key keywords ``` TruffleHog output behavior: ```text Raw = api key RawV2 = key:endpoint when endpoint is paired during verification or when only one endpoint exists Redacted = shortened key ``` Verification calls: ```text https:///openai/deployments?api-version=2023-03-15-preview Header: Api-Key: ``` Practical enrichment fields: ```text azure_openai_endpoint azure_openai_resource_name ``` TODO: If RawV2 is empty, scan nearby source context for `.openai.azure.com` to pair keys with endpoints. ### Azure DevOps PAT Detector: `AzureDevopsPersonalAccessToken` Detector finds a 52-character token and an organization-like string near `azure`. TruffleHog output behavior: ```text Raw = PAT RawV2 = PAT + organization ``` Verification calls: ```text https://dev.azure.com//_apis/projects BasicAuth(username="", password=) ``` Risk: `RawV2` is concatenated without delimiter, so organization extraction from RawV2 is ambiguous unless the token length is known. Practical enrichment fields: ```text azure_devops_org ``` TODO: Parse organization from raw finding JSON/source context rather than relying on concatenated RawV2 alone. ## DockerHub Detector: `Dockerhub` DockerHub v2 detects modern PATs: ```text dckr_pat_[a-zA-Z0-9_-]{27} ``` DockerHub v1 detects UUID-like legacy tokens near `docker`. Both versions try to pair the token with nearby usernames or emails: ```text username: user/usr/-u/id nearby value or email address Raw = token RawV2 = username:token when username/email is found ``` Verification calls: ```text POST https://hub.docker.com/v2/users/login {"username":"","password":""} ``` If verified, ExtraData can include: ```text hub_username hub_email hub_scope 2fa_required ``` Practical enrichment fields: ```text dockerhub_username dockerhub_email dockerhub_scope dockerhub_2fa_required ``` Risk: A token without nearby username cannot be verified by this detector, but may still be useful if a username can be found elsewhere in the same package/repo. TODO: For unverified DockerHub PATs with empty RawV2, scan nearby context and package/repo metadata for plausible DockerHub usernames. ## Proposed Enrichment Layer Add a post-processing enrichment layer after TruffleHog result parsing and before DB insert. Input: ```text finding JSON target metadata source file path/line if available optional nearby source context ``` Output fields stored in DB/dashboard: ```text provider credential_kind credential_confidence required_context_missing principal project_id tenant_id organization registry endpoint username email scope resource ``` Suggested confidence levels: ```text verified structured_complete prefix_shape_complete token_only_missing_context legacy_unverified noisy_unverified ``` Priority implementation: 1. Parse GCP service-account JSON from `RawV2`. 2. Parse Azure ACR `RawV2` JSON. 3. Parse Azure OpenAI `RawV2` as `key:endpoint` when available. 4. Parse DockerHub `RawV2` as `username:token` and ExtraData when verified. 5. Add low-confidence classification for GitHub/GitLab legacy detectors instead of hard-dropping them. 6. Optionally read nearby file context for detectors where RawV2 lacks required context.