feat(worker): add authenticated bounded PREPARE HTTP executor tracer

This commit is contained in:
devops committed 2026-10-23 13:49:38 +00:00
1 parent 58d3794050
commit aa99216c49
5 files changed
+742

No files matched your search

+28
View File
@@ -0,0 +1,28 @@
# Bounded PREPARE worker transport (not an LLM pilot)
This is an opt-in Nest dynamic module and one-shot deterministic Node executor. Nothing imports it into AppModule. No live service, credential, dispatch schedule or deployment is created.
## Required parent wiring
- Import `WorkerTransportModule.register(config)` from `backend/src/orchestration-worker/worker-transport.module.ts` in the parent application only after review. Default disabled; provide empty grants until secret provisioning. The module owns its own PrepareWorkflowService/config; do not broaden existing orchestration providers.
- Provision >=32 random-byte service credential through a mounted secret `/run/secrets/prepare_worker_credential` to the pilot, and provision only its SHA256 in server grants. Never accept identity/role from request bodies, JWT browser users, headers other than this credential, or a snapshot inventory.
- `grants` is an explicit digest -> `{id,role}` mapping with independent `workerIds`/`stageCiIds`. Worker and CI identities/digests cannot overlap. Use a separate CI-only secret for `STAGE_CI`; do not give it to this executor.
- Fixed config: absolute private writable `artifactDirectory`, authenticated HTTPS `artifactBaseUrl` pointing to `/internal/prepare-worker/artifacts`, separate persistent artifact volume. The fixture host `isolated.invalid` is NOT a real uploaded artifact URI. TLS routing/URI reachability must be verified by parent Stage CI.
- Restrict routes to service network, TLS, ingress body limit (8KiB recommended), rate limit, no browser CORS credentials, no public user JWT access. The module validates strict DTO fields, but production ingress/body limits/rate controls remain parent's wiring.
- Run `node infra/hermes-worker/pilot.mjs` with explicit `PREPARE_TASK_ID` UUID and `PREPARE_WORKER_BASE_URL` HTTPS origin. It consumes exactly one task/lease, has 5s per HTTP timeout, does not retry, and exits on SIGINT/SIGTERM. No polling or global dispatch. Disable all egress except backend; do not mount shell/Docker/browser credentials or repo write access. Loopback HTTP is available only via the exported test API option, not CLI.
## Contract and bounds
POST `/internal/prepare-worker/claim`: `{taskId}` -> `{lease:null|{...}}`; heartbeat: `{taskId,runId,leaseToken}`; complete: same binding plus `report` (max 2048 chars). Server accepts only byte-exact deterministic tracer JSON derived from the immutable run; it computes SHA256, writes real content, and chooses artifact URI/cost (0 cents), not client-supplied costs/URIs/pass flags. It does **not** inspect repository contents or validate application acceptance criteria. QA here means this harmless tracer succeeded, not application QA.
Core service still owns transactional Gate1, scope, kill checks, task locks, lease fencing, reservations and VERIFYING event. Transport requires exactly 20-cent task budgets and explicit task IDs. One expired unknown-cost reservation exhausts this pilot budget; do not release it to permit retries. Interrupt means cancellation/non-RUNNING state or STOP_AGENTS; heartbeat before compute, before complete, and atomic completion recheck. File writes may leave unregistered orphan content if the final transaction is denied; retrieval requires a registered QA artifact and verifies its bytes against the requested digest.
POST `/internal/prepare-worker/stage-evidence` has a separate strict CI DTO `{taskId,runId,commitSha,sha256,evidenceSha256,uri}` and calls independently authenticated core Stage evidence ingestion. This tracer never submits it. Valid CI evidence retrieval/provenance remains CI's responsibility. No Gate2 approval/release transport exists.
Shared department budgets are not implemented by core and are not claimed here. Do not expand beyond a single bounded pilot until aggregate budgeting and live provisioning are reviewed.
## Real isolated verification
Run `bash infra/hermes-worker/test-isolated.sh green`. It creates no-network disposable PostgreSQL16, applies only the three repository v4 migrations, copies readonly source and Node22 dependency donor into private writable storage, regenerates Prisma, runs actual Nest TCP HTTP and a child Node executor, then full backend compilation. Docker DB/anonymous volume removed on exit. No live tables/containers/volumes, RELEASE_AUTHORIZE, push or deploy touched.
Evidence: `/opt/services/canina/validation/v4-worker-transport/`: `red.log` (missing module initial RED), `integrity-red.log` (real 200 vs expected409 corruption regression), `green.log`, `build.log`, `generate.log`, `tracer-evidence.json` (actual artifact content/hash). Final tests cover credential/body identity denial, disabled/overlap config, Gate1 no-dispatch, competing claims, expired/token fencing, retained20-cent budget, interrupt/kill rechecks, CI separation, real executor to VERIFYING, authenticated artifact GET and tamper rejection. Fixture commit is 40 `a` characters; this is not a repository QA claim or live Stage deployment.
+37
View File
@@ -0,0 +1,37 @@
import { readFile } from 'node:fs/promises';
import { pathToFileURL } from 'node:url';
/** One explicit task, one attempt, fixed computation; no shell, prompt, browser or LLM. */
export async function runOnce({baseUrl, credential, taskId, allowLoopbackHttp=false, signal}) {
const url = new URL(baseUrl);
if (url.username || url.password || url.search || url.hash || (url.protocol !== 'https:' && !(allowLoopbackHttp && url.protocol === 'http:' && ['127.0.0.1','[::1]'].includes(url.hostname)))) throw new Error('HTTPS service origin required');
if (!credential || credential.length < 32) throw new Error('Service credential required');
async function post(route, body) {
signal?.throwIfAborted();
const timeout=AbortSignal.timeout(5000);
const r=await fetch(new URL('/internal/prepare-worker/'+route,url), {method:'POST',redirect:'error',signal:signal ? AbortSignal.any([signal,timeout]) : timeout,headers:{authorization:'Bearer '+credential,'content-type':'application/json'},body:JSON.stringify(body)});
const text=await r.text();
if(text.length>8192) throw new Error('Oversized service response');
if(!r.ok) throw new Error('Worker request denied: '+r.status);
return JSON.parse(text);
}
const {lease}=await post('claim',{taskId});
if(!lease) return {state:'IDLE'};
if(lease.taskId!==taskId || lease.environment!=='staging' || lease.maxCostCents!==20 || !/^[a-f0-9]{40}$/.test(lease.commitSha)) throw new Error('Unbounded lease rejected');
const binding={taskId,runId:lease.runId,leaseToken:lease.leaseToken};
await post('heartbeat',binding); // stop/interrupt recheck before any computation
signal?.throwIfAborted();
const report=JSON.stringify({kind:'DETERMINISTIC_PREPARE_TRACER',schemaVersion:1,taskId,runId:lease.runId,commitSha:lease.commitSha,environment:'staging',llmPilot:false,browserExecuted:false,shellExecuted:false,deploymentExecuted:false,checks:['lease-bound identity','deterministic JSON serialization'],actualCents:0});
await post('heartbeat',binding); // stop/interrupt recheck immediately before completion
return post('complete',{...binding,report});
}
if (process.argv[1] && import.meta.url===pathToFileURL(process.argv[1]).href) {
const controller=new AbortController();
process.once('SIGINT',()=>controller.abort());
process.once('SIGTERM',()=>controller.abort());
try {
const credential=(await readFile('/run/secrets/prepare_worker_credential','utf8')).trim();
const result=await runOnce({baseUrl:process.env.PREPARE_WORKER_BASE_URL,credential,taskId:process.env.PREPARE_TASK_ID,signal:controller.signal});
console.log(JSON.stringify(result));
} catch { console.error('Bounded PREPARE tracer failed; no completion assumed'); process.exitCode=1; }
}
+23
View File
@@ -0,0 +1,23 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
E="${WORKER_EVIDENCE_DIR:-/opt/services/canina/validation/v4-worker-transport}"
mkdir -p "$E"
DB="v4-worker-test-${RANDOM}"
trap 'docker rm -fv "$DB" >/dev/null 2>&1 || true' EXIT
docker run -d --name "$DB" --network none -e POSTGRES_PASSWORD=disposable -e POSTGRES_DB=worker postgres:16-alpine >/dev/null
for n in $(seq 1 60); do if docker exec "$DB" pg_isready -U postgres >/dev/null 2>&1; then break; fi; sleep 1; done
for migration in 20261022000000_add_orchestration_v4_core 20261022190000_harden_orchestration_ledger 20261023170000_prepare_workflow_outbox; do
docker exec -i "$DB" psql -v ON_ERROR_STOP=1 -U postgres -d worker < "$ROOT/backend/prisma/migrations/$migration/migration.sql" > "$E/$migration.log"
done
docker run --rm --userns=host --network "container:$DB" -e DATABASE_URL=postgresql://postgres:disposable@127.0.0.1:5432/worker -e WORKER_TRANSPORT_DB=1 -e TRACER_EVIDENCE_PATH=/evidence/tracer-evidence.json -e TMPDIR=/work/tmp -v "$ROOT:/source:ro" -v /opt/services/canina/worktrees/orchestration-v4-runtime/backend/node_modules:/deps:ro -v "$E:/evidence" node:22-bookworm bash -c '
set -euo pipefail
mkdir -p /work/backend /work/infra /work/tmp
cp -a /source/backend/. /work/backend/
cp -a /source/infra/. /work/infra/
cp -a /deps /work/backend/node_modules
cd /work/backend
./node_modules/.bin/prisma generate > /evidence/generate.log 2>&1
npm test -- --runInBand orchestration-worker > /evidence/'"${1:-green}"'.log 2>&1
npm run build > /evidence/build.log 2>&1
'