sashatrask
29 lines
2.2 KiB
Markdown
29 lines
2.2 KiB
Markdown
## Why
|
|
|
|
The scanner is now broad enough that small reliability issues and conservative defaults are limiting throughput: source uptime resets from Windows state-file races, large artifacts are skipped too aggressively, and CI/package discovery often spends cycles on low-value or already-known targets. This change stabilizes source execution and widens coverage incrementally without introducing new queues or a scheduler rewrite.
|
|
|
|
## What Changes
|
|
|
|
- Make runner state persistence resilient to transient Windows file-lock races so sources do not crash when supervisor or dashboard reads state concurrently.
|
|
- Increase configurable size limits for package, Postman, and CI artifacts in a controlled way while keeping worker counts conservative.
|
|
- Refine discovery queries so repository/package metadata searches focus on provider, host, and framework terms instead of generic secret words.
|
|
- Improve `package_git` discovery quality by broadening metadata extraction and canonicalization while preserving the current queue model.
|
|
- Improve CI source target selection by parsing more seed formats and scanning more relevant GitHub Actions and GitLab CI repositories per cycle.
|
|
- Continue the provider-specific detector plus keychecker pattern, including context-based routing for generic key formats.
|
|
|
|
## Capabilities
|
|
|
|
### New Capabilities
|
|
- `scanner-runtime-stability`: resilient source state persistence and restart behavior for supervised scanner processes.
|
|
- `scan-coverage-sizing`: configurable artifact size coverage for packages, Postman artifacts, and CI logs/artifacts.
|
|
- `source-discovery-targeting`: higher-signal source queries, package git discovery, and CI seed target selection.
|
|
- `provider-key-validation`: provider-specific custom detectors, keycheckers, and context routing for ambiguous key formats.
|
|
|
|
### Modified Capabilities
|
|
|
|
## Impact
|
|
|
|
- Affected code: `app/console_runner.py`, `app/supervisor.py`, `app/dashboard.py`, `app/scanner.py`, `app/scanner_db.py`, `app/config.yaml`, and selected `app/keycheckers/**` modules.
|
|
- Affected runtime data: `runtime/state/runner_state_*.json`, `runtime/queues/*`, scanner logs, target scan records, and keycheck results.
|
|
- No breaking changes to command-line entry points or existing queue file formats are intended.
|