Initial server source import
This commit is contained in:
sashatrask
@@ -0,0 +1,253 @@
|
||||
# Production edge deployment
|
||||
|
||||
This opt-in deployment keeps PostgreSQL, supervisor control, the standalone dashboard,
|
||||
the Worker API, and its typed admin backend on the runtime container's loopback. The
|
||||
root-owned deployment profile selects one of two exact Caddy topologies. Both keep the
|
||||
private Caddy admin API on an unpublished Unix socket and preserve the same Worker API,
|
||||
admin authentication, operator attribution, denylist, and header contract.
|
||||
|
||||
## Deployment profiles
|
||||
|
||||
If `/etc/truf/deployment-profile` is absent, `standalone-edge-v1` is selected. The
|
||||
standalone profile publishes runtime TCP 443 and gives the managed edge only
|
||||
`NET_BIND_SERVICE`.
|
||||
|
||||
For a host whose existing root-owned Caddy must remain the sole owner of ports 80/443,
|
||||
install the shared profile before running the host-agent installer:
|
||||
|
||||
```sh
|
||||
printf '%s\n' shared-host-edge-v1 | sudo install -m 0444 -o root -g root /dev/stdin /etc/truf/deployment-profile
|
||||
```
|
||||
|
||||
`shared-host-edge-v1` runs the runtime in the host network namespace with no Docker
|
||||
published ports. The managed Truf edge shares that namespace, has no capabilities, and
|
||||
binds plain HTTP only at `127.0.0.1:18766`. The existing host Caddy imports the fixed
|
||||
route-only `deploy/edge/host-caddy-shared.caddy` snippet inside the reviewed public site.
|
||||
Install that import before any catch-all handler. It handles only `/api/v1/worker/*` and
|
||||
the exact random admin prefix; it does not define a listener, TLS policy, global option,
|
||||
or route for another application. The host agent never restarts or reconfigures host
|
||||
Caddy or X-UI.
|
||||
|
||||
Profile changes are maintenance operations: stop the host agent first, require no active
|
||||
apply or failed hold, install the exact root-owned mode-0444 value, validate the selected
|
||||
Compose projection and host-Caddy configuration, then restart the agent. Never expose
|
||||
profile selection through the admin or host-agent request.
|
||||
|
||||
### Choosing a topology
|
||||
|
||||
Use `standalone-edge-v1` on a dedicated host where the managed edge can own public TCP
|
||||
443. The request path is:
|
||||
|
||||
```text
|
||||
Internet -> managed Caddy :443 -> private runtime :8766
|
||||
```
|
||||
|
||||
Use `shared-host-edge-v1` only when an existing root-owned Caddy must remain the sole
|
||||
owner of public ports and TLS. The request path is:
|
||||
|
||||
```text
|
||||
Internet -> host Caddy :443 -> 127.0.0.1:18766 -> managed Caddy -> private runtime :8766
|
||||
```
|
||||
|
||||
Shared-host mode adds a one-time integration boundary, not a second public edge. The
|
||||
operator installs the fixed route snippet, places its import before every catch-all,
|
||||
supplies the independent ingress marker, and validates the complete host Caddy
|
||||
configuration. After that bootstrap, runtime restart and document apply use the same
|
||||
host-agent lifecycle as standalone mode. The host agent never owns the host Caddy
|
||||
configuration or service lifecycle.
|
||||
|
||||
These are the only supported production topologies. Nginx, Traefik, an arbitrary Caddy
|
||||
layout, or an ad-hoc Compose override is not equivalent to either profile. Add and test a
|
||||
new exact deployment profile instead of translating private headers approximately. The
|
||||
host agent validates the selected fixed Compose projection and rejects metadata drift.
|
||||
|
||||
The shared-host projection currently carries the constrained-host runtime limits declared
|
||||
in `compose.shared-host.yaml`. A materially different CPU or memory envelope also requires
|
||||
a reviewed profile change; do not hide it in an unvalidated local override.
|
||||
|
||||
### End-to-end host bootstrap
|
||||
|
||||
The repository provides fixed deployment components, not a universal VPS installer,
|
||||
Ansible role, public image registry, or infrastructure module. Bootstrap a new host from
|
||||
one reviewed release checkout as follows:
|
||||
|
||||
1. Install the reviewed Linux, Docker Engine and Compose plugin, systemd, Python 3, and
|
||||
fail2ban prerequisites; provision DNS and the selected TLS ownership boundary.
|
||||
2. Install the release checkout root-owned at `/opt/truf` and choose exactly one deployment
|
||||
profile before installing the host agent.
|
||||
3. In shared-host mode, install the fixed host-Caddy import, validate the complete host
|
||||
configuration, and prove an unavailable loopback edge cannot fall through to another
|
||||
application.
|
||||
4. Create the protected edge directories, denylist state, and mode-0600 edge environment
|
||||
described below. Generate independent admin, edge, and shared-ingress values rather
|
||||
than copying values from another host.
|
||||
5. Install and validate the fixed host agent. Its first install seeds an absent active
|
||||
config from `app/config.linux.yaml` and an absent secrets document as an empty mapping;
|
||||
repeat installation never replaces active documents.
|
||||
6. Install every trusted worker-package manifest referenced by the runtime config beneath
|
||||
`/etc/truf/worker-packages` with the exact ownership and mode described below.
|
||||
7. Review the private active config and secrets, then build `runtime` and `edge` from the
|
||||
same checkout with the exact base and selected profile Compose files.
|
||||
8. Start the stack, explicitly enable Worker API and admin only after their private
|
||||
configuration is complete, and verify PostgreSQL, runtime, edge, HTTPS, admin, Worker
|
||||
API, host-agent, and unrelated host applications.
|
||||
|
||||
Initial host bootstrap is therefore intentionally more manual than later operation.
|
||||
Normal config apply, restart, rollback, status, and audit are performed through the typed
|
||||
control plane and fixed host agent after this trust boundary is established.
|
||||
|
||||
## Host agent and fixed runtime paths
|
||||
|
||||
Install the root-owned checkout at `/opt/truf`. Before invoking the host-agent installer,
|
||||
prepare the edge state below and create the complete protected environment file that its
|
||||
fixed combined-Compose validation consumes.
|
||||
|
||||
UID/GID 10001 is the numeric edge identity. The denylist directory is mounted, rather than
|
||||
its file, so atomic replacement remains visible in the container.
|
||||
|
||||
```sh
|
||||
sudo install -d -o 10001 -g 10001 -m 0700 /var/log/truf-edge
|
||||
sudo install -d -o root -g 10001 -m 2750 /etc/truf-edge/denylist
|
||||
sudo install -m 0640 -o root -g 10001 deploy/edge/admin-denylist.caddy /etc/truf-edge/denylist/admin-denylist.caddy
|
||||
sudo install -d -o root -g root -m 0700 /var/lib/truf-edge
|
||||
sudo install -m 0750 -o root -g root deploy/fail2ban/truf_caddy_admin_denylist.py /usr/local/sbin/truf-caddy-admin-denylist
|
||||
```
|
||||
|
||||
Create `/etc/truf-edge/edge.env` as root with mode 0600. Generate a new admin segment
|
||||
with `openssl rand -hex 32`. It must be exactly 64 lowercase hex characters (256 random
|
||||
bits). Generate the bcrypt value interactively with the pinned edge image's
|
||||
`caddy hash-password` command; never put the plaintext password in a command, file, or
|
||||
Compose variable.
|
||||
|
||||
```dotenv
|
||||
TRUF_EDGE_HOST=edge.example.net
|
||||
TRUF_ADMIN_PREFIX=replace_with_64_lowercase_hex_characters
|
||||
TRUF_ADMIN_USER=operator
|
||||
TRUF_ADMIN_PASSWORD_HASH='$2a$14$replace_with_a_real_caddy_bcrypt_hash'
|
||||
TRUF_ADMIN_EDGE_MARKER=replace_with_a_second_independent_64_character_hex_secret
|
||||
TRUF_SHARED_INGRESS_MARKER=replace_with_a_third_independent_64_character_hex_secret
|
||||
TRUF_EDGE_AUTH_LOG_DIR=/var/log/truf-edge
|
||||
TRUF_EDGE_DENYLIST_DIR=/etc/truf-edge/denylist
|
||||
```
|
||||
|
||||
`TRUF_SHARED_INGRESS_MARKER` is required only by `shared-host-edge-v1`. Host Caddy strips
|
||||
any inbound transit/private headers, injects this marker and its observed client address,
|
||||
and proxies to loopback. The managed edge rejects a missing marker before trusting that
|
||||
address and removes the marker before proxying to the application.
|
||||
|
||||
Now install and validate the fixed host agent. The installer creates the fixed candidate,
|
||||
result, PostgreSQL socket, and active-document paths and enables
|
||||
`/run/truf/host-agent.sock`; Compose refuses to create missing bind sources.
|
||||
|
||||
```sh
|
||||
sudo /usr/bin/python3 -I -S -B /opt/truf/deploy/host-agent/truf_host_agent_install.py install
|
||||
sudo /usr/bin/python3 -I -S -B /opt/truf/deploy/host-agent/truf_host_agent_install.py validate
|
||||
```
|
||||
|
||||
The active `/etc/truf/runtime/config.yaml` and `secrets.yaml` are UID/GID 10001 mode 0600
|
||||
documents and are never overwritten by repeat installation. Any package manifest referenced
|
||||
by the config must be installed beneath `/etc/truf/worker-packages` as a root-owned,
|
||||
root:root mode 0644 regular file before validation. The runtime maps that immutable authority
|
||||
read-only at `/data/worker-packages`; do not place manifests in the private active-document
|
||||
directory.
|
||||
|
||||
Automatic TLS remains the default. A deployment that must use operator-provided
|
||||
certificates can mount a root-owned, non-link `*.caddy` file under `/etc/caddy/tls` and
|
||||
set `TRUF_EDGE_TLS_INCLUDE` to that absolute container path in a reviewed Compose
|
||||
override. The include should contain only the site's `tls CERT KEY` directive. Never use
|
||||
the repository's localhost test certificate or key in a deployment.
|
||||
|
||||
The normal `compose.yaml` remains private and unchanged. Confirm the host-agent socket is
|
||||
active and rerun installer validation immediately before starting production edge. Always
|
||||
supply the base file, the exact selected profile file, and the protected environment file.
|
||||
For standalone:
|
||||
|
||||
```sh
|
||||
docker compose --env-file /etc/truf-edge/edge.env -f compose.yaml -f compose.edge.yaml build runtime edge
|
||||
docker compose --env-file /etc/truf-edge/edge.env -f compose.yaml -f compose.edge.yaml up -d
|
||||
```
|
||||
|
||||
For shared host:
|
||||
|
||||
```sh
|
||||
docker compose --env-file /etc/truf-edge/edge.env -f compose.yaml -f compose.shared-host.yaml build runtime edge
|
||||
docker compose --env-file /etc/truf-edge/edge.env -f compose.yaml -f compose.shared-host.yaml up -d
|
||||
```
|
||||
|
||||
Before starting shared host, validate the complete existing host Caddy configuration with
|
||||
the snippet import in place. A matching request must fail at that Truf route if the
|
||||
loopback edge is unavailable; it must never fall through to X-UI or another upstream.
|
||||
|
||||
Provisioning creates the private `/data/managed-files` namespace in the named data volume.
|
||||
Each configured writable root must be a reviewed immediate child such as
|
||||
`/data/managed-files/exports`, created with UID/GID 10001 and mode 0700 while the runtime is
|
||||
stopped. Arbitrary host bind paths are not managed-file roots.
|
||||
|
||||
Worker admission remains disabled by `app/config.linux.yaml`. Configure the private
|
||||
runtime config's worker sources, compatibility profiles, and hashed device credentials
|
||||
before explicitly enabling `supervisor.worker_api.enabled`. The edge does not enable it.
|
||||
The typed admin backend is disabled independently under `supervisor.worker_api.admin`.
|
||||
Set its exact `origin` to `https://TRUF_EDGE_HOST`, set `edge_marker` to the same independent
|
||||
256-bit value as `TRUF_ADMIN_EDGE_MARKER`, then explicitly enable it. Both authenticated
|
||||
surfaces share the private runtime loopback port 8766. Caddy strips any inbound
|
||||
`X-Truf-Admin-Edge` and `X-Truf-Admin-Operator`, sets the configured marker and the
|
||||
authenticated Basic-auth username only after authentication, and rewrites the public
|
||||
random prefix to the private `/admin-internal` backend path. The backend accepts the
|
||||
operator identity only together with the private marker.
|
||||
Worker API requests receive neither private admin header.
|
||||
|
||||
Build and client bootstrap instructions for Windows and Linux remote workers are in
|
||||
`docs/remote-worker-operations.md`. Worker executables and images must be produced from a
|
||||
reviewed release checkout; operators must not assemble Python, Git, TruffleHog, detector
|
||||
policy, or dependencies manually on each worker.
|
||||
|
||||
## Fail2ban
|
||||
|
||||
Install the host files under their conventional names and enable fail2ban plus the expiry
|
||||
timer. The jail counts only redacted `admin_auth_failure` JSON records. An initial Basic
|
||||
challenge without credentials, Worker API authentication failures, and unrelated 404s do
|
||||
not enter that log. The action changes only the matcher imported inside the secret admin
|
||||
route; it does not create firewall rules and therefore does not block workers sharing an IP.
|
||||
|
||||
```sh
|
||||
sudo install -m 0644 deploy/fail2ban/filter.d-truf-admin-auth.conf /etc/fail2ban/filter.d/truf-admin-auth.conf
|
||||
sudo install -m 0644 deploy/fail2ban/jail.d-truf-admin-auth.local /etc/fail2ban/jail.d/truf-admin-auth.local
|
||||
sudo install -m 0644 deploy/fail2ban/action.d-truf-caddy-admin-denylist.conf /etc/fail2ban/action.d/truf-caddy-admin-denylist.conf
|
||||
sudo install -m 0644 deploy/fail2ban/fail2ban.d-truf-persistence.local /etc/fail2ban/fail2ban.d/truf-persistence.local
|
||||
sudo install -m 0644 deploy/systemd/truf-caddy-admin-denylist-expire.service /etc/systemd/system/
|
||||
sudo install -m 0644 deploy/systemd/truf-caddy-admin-denylist-expire.timer /etc/systemd/system/
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now fail2ban truf-caddy-admin-denylist-expire.timer
|
||||
sudo fail2ban-client status truf-admin-auth
|
||||
```
|
||||
|
||||
Fail2ban persists jail state in `/var/lib/fail2ban/fail2ban.sqlite3`. The updater persists
|
||||
canonical IPs and expiry timestamps in private
|
||||
`/var/lib/truf-edge/admin-denylist.json`. It validates the complete Caddyfile in the running
|
||||
edge container, reloads it through the private admin endpoint, and restores/reloads the
|
||||
previous state if a command fails. Reload uses Caddy's private `/run/caddy-admin.sock` inside the edge
|
||||
container. The socket is not mounted or published. Shared mode renders denylist matchers
|
||||
against the marker-authenticated client address; standalone mode uses the direct peer.
|
||||
|
||||
## SSH recovery
|
||||
|
||||
Use fail2ban's normal unban first so its database and Caddy agree:
|
||||
|
||||
```sh
|
||||
sudo fail2ban-client set truf-admin-auth unbanip 203.0.113.10
|
||||
sudo /usr/local/sbin/truf-caddy-admin-denylist status
|
||||
sudo /usr/local/sbin/truf-caddy-admin-denylist expire
|
||||
```
|
||||
|
||||
If fail2ban is unavailable, run the updater's explicit unban over SSH:
|
||||
|
||||
```sh
|
||||
sudo /usr/local/sbin/truf-caddy-admin-denylist unban 203.0.113.10
|
||||
```
|
||||
|
||||
For recovery from a damaged generated snippet, stop the expiry timer and fail2ban, restore
|
||||
`deploy/edge/admin-denylist.caddy` to `/etc/truf-edge/denylist/admin-denylist.caddy`, then
|
||||
run Caddy validation and reload through the private Unix admin socket only after validation
|
||||
succeeds. Reconcile each
|
||||
remaining address with the updater before re-enabling the services. Do not use a global
|
||||
firewall ban as a shortcut.
|
||||
Reference in New Issue
Block a user