sashatrask
2.2 KiB
2.2 KiB
Why
The scanner is now broad enough that small reliability issues and conservative defaults are limiting throughput: source uptime resets from Windows state-file races, large artifacts are skipped too aggressively, and CI/package discovery often spends cycles on low-value or already-known targets. This change stabilizes source execution and widens coverage incrementally without introducing new queues or a scheduler rewrite.
What Changes
- Make runner state persistence resilient to transient Windows file-lock races so sources do not crash when supervisor or dashboard reads state concurrently.
- Increase configurable size limits for package, Postman, and CI artifacts in a controlled way while keeping worker counts conservative.
- Refine discovery queries so repository/package metadata searches focus on provider, host, and framework terms instead of generic secret words.
- Improve
package_gitdiscovery quality by broadening metadata extraction and canonicalization while preserving the current queue model. - Improve CI source target selection by parsing more seed formats and scanning more relevant GitHub Actions and GitLab CI repositories per cycle.
- Continue the provider-specific detector plus keychecker pattern, including context-based routing for generic key formats.
Capabilities
New Capabilities
scanner-runtime-stability: resilient source state persistence and restart behavior for supervised scanner processes.scan-coverage-sizing: configurable artifact size coverage for packages, Postman artifacts, and CI logs/artifacts.source-discovery-targeting: higher-signal source queries, package git discovery, and CI seed target selection.provider-key-validation: provider-specific custom detectors, keycheckers, and context routing for ambiguous key formats.
Modified Capabilities
Impact
- Affected code:
app/console_runner.py,app/supervisor.py,app/dashboard.py,app/scanner.py,app/scanner_db.py,app/config.yaml, and selectedapp/keycheckers/**modules. - Affected runtime data:
runtime/state/runner_state_*.json,runtime/queues/*, scanner logs, target scan records, and keycheck results. - No breaking changes to command-line entry points or existing queue file formats are intended.