truf-server/QUARANTINE_AUDIT.md

Pipeline Quarantine Audit

Initial snapshot and remediation: 2026-08-15

Current Impact

• PostgreSQL quarantine rows: 177
• Accounted capacity: 4354 items / 1,817,503,389 bytes
• Configured admission limit: 10000 items / 1,073,741,824 bytes
• New scan admission is closed because the byte limit is exceeded.
• 49,215 admission intents have already ended with quarantine_admission_closed.

pipeline: ready means that PostgreSQL and workers are healthy. It does not mean that new scan admission is open.

Result Bundles

Two rows account for 4004 items / 1,212,153,856 bytes.

Quarantine ID	Source	Original error	Physical size	Read-only validation now
29	Hugging Face spaces	Transient Windows Permission denied	2,870 B	Valid; 3 frames, no findings/errors/candidates
91	Docker Hub query tokenizer	Transient Windows Permission denied	5,354 B	Valid; 8 frames, 1 finding, 4 errors, no candidates

The bundle contents are valid. Their large capacity cost comes from worst-case reservations transferred into quarantine, not their physical file sizes.

Current code now reports a temporarily unavailable private bundle as an availability error. Result ingester defers it instead of classifying it as invalid content.

Recommendation: recover both bundles through an audited offline path rather than discard them. ID 91 contains one finding and must not be deleted without an explicit decision.

Keycheck Quarantine

There are 175 pending keycheck quarantine rows.

Class	Rows	Distinct credentials	Assessment
Repeated DeepSeek unconsumed rechecks	104	1	Duplicate hourly retries; current state is now NO_CONTEXT
DeepSeek unconsumed findings	15	6	Legitimate historical candidates filtered by routing
Azure Foundry unconsumed	15	12	Historical provider-consumption issue
Hugging Face unconsumed	7	5	Historical provider-consumption issue
Replicate unconsumed	6	3	Historical provider-consumption issue
xAI unconsumed	3	3	Historical provider-consumption issue
DeepSeek route mismatch	24	14	Misrouted non-DeepSeek detectors; source findings remain in PostgreSQL
Azure route mismatch	1	1	Historical Azure Foundry route mismatch; current state is UNKNOWN

The active growth came from one DeepSeek credential whose current state was NETWORK. Hourly network retry created a fresh candidate, provider routing silently rejected it, and the candidate entered quarantine after three unconsumed attempts.

Preventive Changes

• Non-DeepSeek routing decisions now complete as NO_CONTEXT instead of leaving a leased candidate unconsumed.
• DeepSeek has a canonical deepseekNoContext.txt status projection.
• Recheck generation now refuses to enqueue a credential while it has an unresolved provider_candidate_unconsumed or candidate_provider_route_mismatch quarantine row.
• Temporarily unavailable result bundle files are deferred instead of quarantined as validation failures.

Verification:

• Targeted tests: 60 passed.
• Manual recheck deepseek network: code 0, no candidates processed.
• DeepSeek unconsumed quarantine remained exactly 119; no new row was created.

Approved Remediation

1. Stop sources and acquire the offline migration guard.
2. Recover bundle IDs 29 and 91 through deterministic re-ingestion.
3. Discard the 104 duplicate DeepSeek retry rows after exact manifest review.
4. Discard the 24 confirmed DeepSeek route-mismatch rows after exact manifest review.
5. Requeue one current candidate per distinct credential from the remaining legitimate unconsumed groups; keep source attribution.
6. Review the single Azure route mismatch separately.
7. Resolve superseded duplicate candidate rows with an audited discard manifest.
8. Verify physical artifacts, capacity accounting, projections and reopened scan admission before restarting sources.

All destructive decisions must use an exact private truf-pipeline-quarantine-review-v1 manifest containing quarantine ID, reason code, payload hash and action. The offline review command requires both --apply and --sources-stopped.

Applied Result

The user approved recovery plus selective cleanup.

• Manifest: runtime/control/quarantine-remediation-20260815.json
• Manifest SHA-256: c3143e6b0e15e07b6afacffd50b449444b9932e75001f633ac82b5b79e21fe3f
• Reviewed: 177
• Approved retry: 32
• Audited discard: 145
• Duplicate/conflicting reviews: 0
• Final quarantine capacity: 0 items / 0 bytes

Bundle outcomes:

Quarantine ID	Final reservation	Final bundle	Target disposition	Preserved contents
29	acknowledged	acknowledged	done	Clean empty result
91	acknowledged	acknowledged	deferred	1 finding, 4 errors

Keycheck outcomes from the retried unique credentials:

Service	Final current statuses
Azure	12 FOUNDRY_UNRESOLVED, 1 UNKNOWN
DeepSeek	6 NO_CONTEXT
Hugging Face	5 NO_CONTEXT
Replicate	3 NO_CONTEXT
xAI	3 NO_CONTEXT

The malformed legacy provider candidates are now terminal current-state records instead of repeatedly deferred/quarantined candidates. Scan admission reopened and scanner workers returned to 3/3 active operation.