truf-server/openspec/changes/add-worker-operator-experience/specs/worker-progress-deadlines/spec.md

ADDED Requirements

Requirement: Canonical assignment phase model

The worker SHALL represent assignment execution with one versioned phase/event model shared by local status, local history, server progress, and administrative views.

Scenario: Assignment completes normally

• WHEN a slot claims, executes, stages, uploads, and receives acceptance for an assignment
• THEN it SHALL emit monotonic phase events sufficient to reconstruct the time spent from assigned through awaiting_receipt

Scenario: Process restarts during an assignment

• WHEN a worker restarts with persisted slot state
• THEN recovered events SHALL continue from the persisted sequence and SHALL record recovery without rewriting the prior timeline

Requirement: Complete scan-stage watchdog

The worker SHALL enforce one hard scan-stage deadline across permit acquisition, local preparation/resolution, data acquisition, scanner execution, filtering, cleanup, and result staging by supervising the complete execution unit outside the controller process.

Scenario: Scanner child exceeds the deadline

• WHEN the assignment runner remains active at the scan-stage deadline
• THEN the controller SHALL terminate its complete process tree, release/detach local resources, and produce a timeout result identifying the final phase

Scenario: Cleanup blocks after scanner exit

• WHEN scanner execution has ended but cleanup or staging remains blocked at the deadline
• THEN the same hard deadline SHALL terminate the runner and SHALL prevent the slot from remaining occupied until assignment expiry

Scenario: Permit acquisition consumes the budget

• WHEN no scan permit is acquired before the scan-stage deadline
• THEN the worker SHALL produce a phase-specific timeout result without starting the scanner

Requirement: Non-renewing server progress

The Worker API SHALL accept idempotent monotonic progress events for the current reservation while preserving the original immutable assignment and queue deadlines.

Scenario: Progress is accepted

• WHEN the assigned device submits the next valid event sequence for its unresolved reservation
• THEN the server SHALL persist the event/latest phase and SHALL NOT alter assignment expiry or ownership

Scenario: Duplicate progress is retried

• WHEN an already accepted event sequence is submitted again
• THEN the server SHALL return the prior acceptance without creating a duplicate timeline event

Scenario: Progress cannot reach the server

• WHEN local phase transitions occur during a temporary connection failure
• THEN execution SHALL continue under the fixed deadline and events SHALL remain available locally for ordered retry

Requirement: Observable deadline semantics

Assignments SHALL carry distinct effective target-scan, result-upload, and end-to-end assignment deadlines, and every operator/admin view SHALL label them by those meanings.

Scenario: Operator inspects active work

• WHEN status or admin renders an active reservation
• THEN it SHALL show the effective scan deadline, assignment deadline, time remaining, and current phase without conflating them

Scenario: Assignment expires

• WHEN the immutable assignment deadline passes without an accepted terminal result
• THEN expiry evidence SHALL include the last accepted phase and last-progress timestamp when available

Requirement: Global and per-source assignment policy

The managed runtime configuration SHALL provide a global assignment TTL fallback and optional explicit overrides for GitLab, DockerHub, and HuggingFace, selected by the server at issuance.

Scenario: Source override exists

• WHEN a DockerHub assignment is issued and a DockerHub assignment TTL override is configured
• THEN its immutable expiry SHALL use the override and the assignment SHALL report that effective policy

Scenario: Source override is absent

• WHEN an assignment is issued for a source without an override
• THEN the global assignment TTL SHALL be used

Scenario: Invalid deadline policy is previewed

• WHEN an effective assignment deadline cannot cover its source scan timeout, upload deadline, and required handoff margin
• THEN managed configuration preview SHALL reject the candidate with a field-specific explanation

Requirement: Phase duration percentiles

The server SHALL expose p50, p95, and p99 durations by source, phase, outcome, and selected time window, based only on completed observations appropriate to that metric.

Scenario: Administrator reviews DockerHub latency

• WHEN duration metrics are requested for DockerHub
• THEN the result SHALL separate end-to-end, scanning, cleanup, bundling, and upload percentiles and SHALL report sample counts

Scenario: Insufficient samples exist

• WHEN a percentile does not have the configured minimum sample count
• THEN the UI/API SHALL label it insufficient rather than presenting it as a stable policy recommendation