Initial server source import

This commit is contained in:
sashatrask
2026-09-30 20:30:56 +03:00
commit 170dd941b9
498 changed files with 261563 additions and 0 deletions
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-09-17
@@ -0,0 +1,51 @@
## Context
TruffleHog custom detector entries combine every regex in one detector through match permutation, so multiple regex fields are conjunctive rather than alternative. The Xai and ZaiGLM policies each place context-before and context-after patterns in one detector, making ordinary one-direction matches disappear. Existing tests compile each expression with Python and use `any(...)`, which does not exercise TruffleHog's actual configuration semantics.
The scanner already canonicalizes `CustomRegex` findings from `ExtraData.name`, and downstream candidate routing and keycheckers depend on the canonical names `Xai` and `ZaiGLM`.
## Goals / Non-Goals
**Goals:**
- Make both context directions independently executable for Xai and ZAI/GLM.
- Preserve canonical persisted detector names and existing keycheck routing.
- Exercise the complete policy through the real TruffleHog CLI without network verification.
- Keep the policy compatible with the pinned fork on Windows and Linux.
**Non-Goals:**
- Change provider verification endpoints or keycheck behavior.
- Add new key formats or broaden the existing regex bounds.
- Require TruffleHog to be installed for pure unit-test environments.
- Perform live provider requests or use real credentials.
## Decisions
### Split alternatives into uniquely named detectors
Keep the context-before expression under the existing canonical name and move the context-after expression into `XaiContextAfter` or `ZaiGLMContextAfter`. Duplicate names are not used because the custom detector registry can collapse same-name entries; a synthetic CLI probe confirmed unique names execute both alternatives.
Alternative considered: combine both alternatives into one regex. This was rejected because TruffleHog takes the first capture group as the secret, and Go regex does not support branch-reset groups needed to keep one capture position across both directions.
### Canonicalize compatibility aliases centrally
Extend custom detector normalization with a small alias map from the context-after names to `Xai` and `ZaiGLM`. This keeps candidate routing, keychecker gates, dashboards, deduplication, and persisted detector values unchanged.
Alternative considered: teach every downstream consumer the new names. This would widen the change and create divergent persisted identities for one provider.
### Add an optional real-CLI regression test
The test resolves TruffleHog from an explicit environment override, the existing Windows path, or `PATH`. When available, it scans deterministic high-entropy synthetic fixtures with the complete production YAML, `--no-verification`, and `--no-update`, then checks canonical finding names and candidate services. It skips only when no binary is available, while pure tests continue to validate policy structure and alias normalization everywhere.
## Risks / Trade-offs
- [Risk] A CI environment without TruffleHog can skip the integration gate. -> Keep structural unit coverage and run the real-CLI test in Windows and Linux release jobs.
- [Risk] Native Xai can duplicate the custom result for its exact 80-character format. -> Existing candidate identity deduplication remains authoritative; the regression fixture uses a shorter supported overlay form to isolate custom behavior.
- [Risk] A future TruffleHog release can change custom detector output fields or flags. -> The real-CLI test asserts `CustomRegex`, `ExtraData.name`, normalization, and candidate routing as one contract.
## Migration Plan
Deploy the YAML, scanner alias map, and tests together, rebuild the runtime image, then run the offline compatibility test against both worker binaries before enabling scans. Rollback restores the prior YAML and alias map; no persisted data migration is required.
## Open Questions
None.
@@ -0,0 +1,23 @@
## Why
The Xai and ZaiGLM custom detector policies model alternative context directions as separate regex entries, but TruffleHog combines entries within one detector as an AND condition. This silently prevents the broader Xai overlay and ordinary ZAI/GLM source detection, while the current unit tests incorrectly model the entries as OR alternatives.
## What Changes
- Express each alternative Xai and ZAI/GLM context direction as an independently executable custom detector while preserving the normalized provider names consumed by routing and keychecks.
- Add an offline CLI compatibility test that runs the configured TruffleHog binary with the complete custom detector policy and synthetic high-entropy fixtures.
- Verify that custom findings normalize and route to the expected Xai and ZAI keycheck services without performing provider verification requests.
- Keep the existing native detector, result bundle, candidate, and keycheck contracts unchanged.
## Capabilities
### New Capabilities
- `custom-provider-detection-compatibility`: Defines executable compatibility requirements for external custom detector policies and their normalized keycheck routing.
### Modified Capabilities
None.
## Impact
The change affects `app/trufflehog-custom-detectors.yaml`, scanner finding normalization/routing tests, and the provider detector compatibility test surface. It introduces no production API, schema, dependency, or persisted-data changes.
@@ -0,0 +1,30 @@
## ADDED Requirements
### Requirement: Alternative provider contexts execute independently
The custom detector policy SHALL detect supported Xai and ZAI/GLM credentials when provider context appears either before or after the credential, without requiring both context directions in one input chunk.
#### Scenario: Provider context appears before the credential
- **WHEN** an offline scan processes a bounded synthetic credential preceded by its supported provider context
- **THEN** the policy emits one corresponding custom provider finding
#### Scenario: Provider context appears after the credential
- **WHEN** an offline scan processes a bounded synthetic credential followed by its supported provider context
- **THEN** the policy emits one corresponding custom provider finding
### Requirement: Alternative detector names normalize canonically
The scanner MUST normalize all compatibility-only custom detector aliases to the existing canonical `Xai` or `ZaiGLM` detector identity before persistence and candidate extraction.
#### Scenario: Context-after alias is emitted
- **WHEN** TruffleHog emits `CustomRegex` with a context-after compatibility name in `ExtraData.name`
- **THEN** the scanner retains `CustomRegex` as the original detector and exposes the canonical provider detector name downstream
### Requirement: Compatibility is tested through the real CLI
The compatibility suite SHALL run the complete configured custom detector policy through an available TruffleHog executable using deterministic synthetic credentials, disabled verification, and disabled update checks.
#### Scenario: Compatible executable is available
- **WHEN** a configured Windows or Linux TruffleHog executable scans the compatibility fixtures
- **THEN** both context directions produce canonical findings and route to the expected keycheck candidate services without network verification
#### Scenario: Executable is unavailable
- **WHEN** no TruffleHog executable is available in a general unit-test environment
- **THEN** the real-CLI test is explicitly skipped while policy-structure and normalization unit tests still execute
@@ -0,0 +1,16 @@
## 1. Detector Policy
- [x] 1.1 Split Xai context-before and context-after alternatives into uniquely named single-regex detector entries.
- [x] 1.2 Split ZaiGLM context-before and context-after alternatives into uniquely named single-regex detector entries.
- [x] 1.3 Canonicalize the compatibility-only detector names before persistence and candidate extraction.
## 2. Compatibility Coverage
- [x] 2.1 Add pure unit coverage for detector policy structure and canonical alias normalization.
- [x] 2.2 Add an optional real-TruffleHog CLI regression test for both context directions and candidate routing.
## 3. Verification
- [x] 3.1 Run the focused provider and scanner unit tests.
- [x] 3.2 Run the real CLI compatibility test with the current Windows binary and a Linux binary where available.
- [x] 3.3 Validate the OpenSpec change and confirm the patch is formatting-clean.