truf-server/openspec/changes/add-postman-source/proposal.md

Why

Public Postman collections and environments are a high-signal source for leaked API credentials because they often preserve request auth settings, headers, variables, and example payloads close to real API usage. The existing scanner already supports multi-source discovery, queues, TruffleHog filesystem scans, token rotation, and observability, so adding Postman can reuse the current architecture while expanding coverage beyond repositories, packages, containers, and HuggingFace Spaces.

What Changes

• Add a new postman source that discovers, queues, scans, and records Postman collection/environment artifacts.
• Seed Postman targets from GitHub code search using public *.postman_collection.json and *.postman_environment.json files.
• Support a one-time backfill mode that scans up to the GitHub Search API result limit per query while filtering out artifacts older than a configured age window.
• Support a daily tail mode that fetches recently indexed pages and stops early when all targets on consecutive pages are already known.
• Use the configured GitHub auth pool for Postman discovery, rotating across tokens and sleeping when all tokens are rate-limited.
• Add durable Postman artifact caching so targets discovered from GitHub, npm, and PyPI can be scanned after temporary extraction directories are removed.
• Harvest Postman artifacts from npm and PyPI packages during existing package extraction flows and enqueue them into the shared Postman queue.
• Add Postman-aware result enrichment that classifies credentials using TruffleHog findings plus Postman auth/header/query/body/environment context.

Capabilities

New Capabilities

• postman-source: Discovery, queueing, scanning, caching, and enrichment for Postman collection and environment artifacts.

Modified Capabilities

• None.

Impact

• Affected scanner paths: app/scanner.py, app/console_runner.py, app/scanner_db.py, app/dashboard.py, and app/config.yaml.
• Adds runtime files under runtime/queues/ for todo_postman.txt and checked_postman.txt.
• Adds durable artifact storage under a runtime Postman cache directory.
• Uses existing GitHub auth pools from secrets.yaml; no new secret format is required for GitHub discovery.
• Uses existing TruffleHog filesystem scanning and keychecker follow-up flows; no breaking changes to current sources are expected.