truf-server/openspec/changes/add-web-operations-control-plane/specs/distributed-core-source-processing/spec.md

ADDED Requirements

Requirement: Exact distributed core source set

The server core profile SHALL contain exactly gitlab, dockerhub, and huggingface, and SHALL NOT start GitHub or any other discovery source as part of that profile.

Scenario: Core profile starts

• WHEN Supervisor starts the distributed core profile
• THEN it starts one discovery producer for GitLab, DockerHub, and HuggingFace and no GitHub producer

Requirement: Discovery-only producer isolation

Each core discovery producer SHALL search its provider, normalize targets, and admit them to PostgreSQL without claiming targets, acquiring scan slots, invoking a scanner, or staging result bundles locally.

Scenario: GitLab discovery finds targets

• WHEN the GitLab producer completes a provider search
• THEN it enqueues the normalized targets and records zero local scan requests

Scenario: DockerHub discovery resolves targets

• WHEN the DockerHub producer processes pages, retries, tags, or digests
• THEN it may persist discovery progress and immutable targets but never invokes TruffleHog or reserves those targets locally

Scenario: HuggingFace discovery finds Spaces

• WHEN the HuggingFace producer returns Space identifiers
• THEN it drops records explicitly marked private, protected, gated, or disabled and enqueues the remaining identifiers without entering a local scan path or making a second per-Space verification request

Scenario: Pending backlog exists

• WHEN a scheduled discovery interval arrives while pending targets already exist
• THEN the producer still performs the configured discovery cycle unless discovery is paused

Requirement: Durable operations control state

The system SHALL persist discovery pause, dispatch pause, drain state, revision, actor, operation identity, and update timestamps in PostgreSQL so that control state survives process and host restarts.

Scenario: Runtime restarts while paused

• WHEN discovery and dispatch are paused and the runtime restarts
• THEN both effective gates remain paused after startup

Scenario: Stale control form is submitted

• WHEN a mutation supplies a revision older than the current control revision
• THEN the system rejects it without changing control state or writing a success audit event

Scenario: Explicit pause coexists with drain

• WHEN an operator explicitly pauses discovery, enters drain, and later cancels drain
• THEN the explicit discovery pause remains set

Requirement: Transactional discovery admission gate

The system SHALL enforce the effective discovery gate in the same transaction that admits discovered targets or claims discovery-specific retry work.

Scenario: Pause races target admission

• WHEN discovery pause commits before a producer admission transaction commits
• THEN no newly discovered target is admitted by that transaction

Scenario: Discovery is paused before provider request

• WHEN a producer begins a cycle while discovery is effectively paused
• THEN it performs no provider request and records a paused cycle outcome

Scenario: Upload-derived work arrives during pause

• WHEN result ingestion creates projection or keycheck work while discovery is paused
• THEN that work remains admissible because it is not provider discovery

Requirement: Transactional dispatch gate

The system SHALL enforce the effective dispatch gate within the reservation transaction so that no new remote assignment can be issued after dispatch pause or drain commits.

Scenario: Dispatch pause races claim

• WHEN dispatch pause commits before a worker claim transaction commits
• THEN the claim returns a paused or no-work response and creates no reservation

Scenario: Existing worker uploads while paused

• WHEN dispatch is paused and a worker with an existing assignment reports status or uploads its result
• THEN the server accepts the valid request under the existing assignment authority

Scenario: Assignment expires while paused

• WHEN an existing assignment expires during dispatch pause
• THEN the reaper processes it normally without issuing replacement work

Requirement: Drain lifecycle

Entering drain SHALL effectively pause discovery and dispatch while preserving status, terminal report, upload, receipt replay, ingestion, projection, and maintenance paths needed to finish accepted work.

Scenario: Drain begins with active assignments

• WHEN drain is requested while remote assignments are active
• THEN the state becomes draining, no new targets or assignments are admitted, and existing workers retain their result path

Scenario: Drain reaches completion

• WHEN no live remote assignments remain and every accepted result bundle has reached database commit
• THEN the reconciler advances the state to drained

Scenario: Pending targets remain

• WHEN pending queue targets remain but all issued assignments and pre-commit bundles are resolved
• THEN drain may still become drained

Scenario: Projection work remains

• WHEN projection or keycheck work remains after its result bundle is database-committed
• THEN that work does not prevent the control state from becoming drained

Requirement: Discovery process observability

Supervisor and the operations console SHALL expose each discovery producer's source, role, lifecycle state, last cycle result, last successful discovery time, next scheduled run, and bounded safe error category.

Scenario: Provider rejects credentials

• WHEN a discovery producer receives a provider authorization error
• THEN operations state reports the source and safe authorization category without exposing the credential or provider response body containing secrets

Scenario: Producer is stopped

• WHEN an operator stops a managed producer through a typed Supervisor action
• THEN structured state identifies it as stopped without changing the persisted discovery pause flag