Add opt-in CI Stage credential provisioning and restricted SQLite inventory export
This commit is contained in:
1 parent
4b147b0091
commit
aa23c7ef91
6 files changed
+552
No files matched your search
@@ -0,0 +1,145 @@
|
||||
# Opt-in Stage runtime provisioning (parent wiring contract)
|
||||
|
||||
This provisions **snapshot inventory** and credentials for the existing PREPARE
|
||||
transport. It does NOT create a real LLM executor. The current deterministic worker
|
||||
is still `llmPilot:false`. No provider credentials are collected, no cron installed,
|
||||
no task claimed and no worker polling/service started by the provisioner.
|
||||
|
||||
## CI-only host invocation
|
||||
|
||||
Parent owns workflow/deploy.sh/AppModule; none are changed here. Run only through
|
||||
the existing pinned authenticated SSH CI boundary, under its Stage environment
|
||||
approval and serialized rollout lock. `CI=true` is an accident guard, not an auth
|
||||
boundary. Provisioner checks clean HEAD equals the full approved commit; no branch
|
||||
promotion, push, Production, RELEASE_AUTHORIZE, Docker or DB writes are performed.
|
||||
Only the CI account should be able to invoke this on a root-owned trusted parent.
|
||||
|
||||
From the clean pinned release checkout on the host (not the dirty Stage checkout):
|
||||
|
||||
```sh
|
||||
CI=true STAGE_PROVISION_ENABLED=true STAGE_PROVISION_APPROVED_SHA="$DEPLOY_SHA" \
|
||||
python3 scripts/provision-stage-runtime.py --uid 1000 --gid 1000
|
||||
```
|
||||
|
||||
Default output root is `/opt/services/canina/stage-runtime` and its parent must
|
||||
already exist. First creation is staged and atomically renamed. Repeating validates
|
||||
existing state without rotating credentials. Corrupt/missing/unsafe files fail
|
||||
closed, not repaired. CI MUST serialize execution and prevent untrusted edits to
|
||||
source/output parents; this is not a hostile-filesystem sandbox. `--root` is for
|
||||
CI-reviewed test/Stage locations only, not arbitrary browser input. Secure backup
|
||||
and separately approved rotation are parent/operator responsibilities.
|
||||
|
||||
- `secrets/` 0700, files 0400 owned by the runtime UID (1000 in the current backend
|
||||
image); `hermes_username/password/signing_key` are new independent random values.
|
||||
- `prepare_worker_grants` contains SHA256 only, exact separate service identities
|
||||
`stage-prepare-worker`/WORKFLOW_WORKER and `stage-evidence-ci`/STAGE_CI. These are
|
||||
transport service identities, **not invented DB admin users**.
|
||||
- Raw `credentials/prepare_worker_token` and `credentials/stage_ci_token` stay in a
|
||||
CI-owner-only 0700 directory, 0400 files. Do NOT mount either into inventory or
|
||||
backend, log them, upload them as artifacts or send them to browsers. A later
|
||||
reviewed one-shot executor receives only its own token through a private file.
|
||||
- `artifacts/` is 0700 owned by backend UID; backend alone writes it. Persistent
|
||||
local disk is not object storage or external publication evidence.
|
||||
- `hermes_runtime_grants` is `{}`: deny all. No live admin query was needed/performed.
|
||||
Parent must separately query actual Stage admin IDs read-only, review exact ID to
|
||||
profile mapping and provision a private replacement via configuration management
|
||||
before claiming panel access. Never infer identities from usernames or JWT input.
|
||||
Deliberately changed grants require separate review; default provisioner refuses
|
||||
nonempty runtime grants on repeat.
|
||||
- Compose file-backed secrets preserve host permissions/ownership; `uid/gid/mode`
|
||||
declarations do not fix bind mount ownership. Verify effective host IDs if Docker
|
||||
uses userns remapping; do not chmod 0444 as a workaround.
|
||||
|
||||
## Consistent restricted export
|
||||
|
||||
After approval, the SAME CI invocation may explicitly request one profile's source
|
||||
SQLite DB. Resolve its real `HERMES_HOME/state.db` through reviewed host mounts;
|
||||
never guess a gateway volume path or mount the original home into inventory.
|
||||
|
||||
```sh
|
||||
CI=true STAGE_PROVISION_ENABLED=true STAGE_PROVISION_APPROVED_SHA="$DEPLOY_SHA" \
|
||||
python3 scripts/provision-stage-runtime.py --uid 1000 --gid 1000 \
|
||||
--snapshot-db "$REVIEWED_STATE_DB" --profile default \
|
||||
--snapshot-name "$UNIQUE_CI_CAPTURE_ID"
|
||||
```
|
||||
|
||||
SQLite online `Connection.backup()` reads committed WAL frames into memory (never
|
||||
`cp state.db`, never `immutable=1`). A 30-second callback deadline bounds busy
|
||||
backup. Only explicit source/time/token/cost columns from `sessions` are projected.
|
||||
Private session IDs become capture-local ordinal IDs; sources use an enum; no
|
||||
messages, titles, prompts, user/chat IDs, paths, URLs, model config, credentials,
|
||||
config.yaml, auth.json, .env, cron commands, memory or skills are exported. Unknown
|
||||
schema/nonnumeric metadata fails. One profile per immutable capture; there is no
|
||||
multi-DB atomicity claim. A source DB may be large: backup is memory-bound, so size
|
||||
and host resources need operator review before a real capture.
|
||||
|
||||
Output directory is new/immutable (0500, 0400 files), never overwritten:
|
||||
`inventory.json`, exact three-field `snapshot-manifest.json` and `export-report.json`.
|
||||
Manifest `capturedAt` is measured immediately after successful backup, not startup,
|
||||
mtime or an example. Report records actual projected row count, inventory digest,
|
||||
method/exclusions; it is not a signature or proof of source authenticity. No live
|
||||
capture was performed in this change. Never reuse fixture manifests as evidence.
|
||||
|
||||
Projected HTTP readers never consult gateway config, state DB or cron files:
|
||||
model/provider are explicit `unknown`, context length is a placeholder zero,
|
||||
cron arrays empty mean **not captured**, session activity false is a placeholder,
|
||||
profile inventory doesn't establish live process availability. Status retains
|
||||
`liveProcess:unknown`. Usage days are relative to actual capture time. Snapshot
|
||||
fresh/stale provenance from the existing adapter remains mandatory. No real LLM
|
||||
pilot readiness follows from this inventory responding 200.
|
||||
|
||||
## Exact parent compose/application wiring
|
||||
|
||||
`infra/hermes-runtime/compose.stage-provision.yml` is an opt-in merge fragment,
|
||||
NOT called by the existing deploy script or ci-apply.sh. Parent must include it
|
||||
in **every** Stage compose command after provisioning, using the existing stable
|
||||
Compose project identity and existing release isolation/volume rules:
|
||||
|
||||
```sh
|
||||
export STAGE_RUNTIME_SERVE_FILE="$PINNED_RELEASE/infra/hermes-runtime/serve.py"
|
||||
export STAGE_RUNTIME_SNAPSHOT="/opt/services/canina/stage-runtime/snapshots/$UNIQUE_CI_CAPTURE_ID"
|
||||
# Only inside approved CI rollout, never manually:
|
||||
# docker compose -p <existing-stage-project> -f <parent-stage-compose> \
|
||||
# -f "$PINNED_RELEASE/infra/hermes-runtime/compose.stage-provision.yml" \
|
||||
# --profile stage-runtime-inventory <parent-approved command>
|
||||
```
|
||||
|
||||
The fragment adds backend `PREPARE_WORKER_ENABLED=true`, fixed-path secrets
|
||||
`/run/secrets/prepare_worker_grants`, `/run/secrets/hermes_runtime_grants`,
|
||||
`hermes_username/password`, and artifact bind at
|
||||
`/var/lib/canina/prepare-artifacts`. It adds a project-scoped **internal** network
|
||||
between backend and inventory only, alias `canina-hermes-inventory` (not the live
|
||||
`canina-hermes`). No host port, Docker socket or live gateway volume. Inventory is
|
||||
read-only, nonroot, no capabilities, restricted tmpfs and pinned image, mounts only
|
||||
its immutable projected export and script. `create_host_path:false` fails on missing
|
||||
exports/storage. Original compose.readonly.yml/ci-apply.sh remain legacy review-only;
|
||||
do not combine those with this provisioning fragment.
|
||||
|
||||
AppModule must keep AuthModule/JwtStrategy, import
|
||||
`RuntimeReadModule.registerFromFile('/run/secrets/hermes_runtime_grants')` for reads,
|
||||
and spread `...registerPrepareWorker()` for the opt-in worker transport (verify
|
||||
existing AppModule before adding duplicates). The latter defaults to no module
|
||||
unless exact env `true`; enabling compose alone cannot add missing imports. Routes
|
||||
remain service-authenticated. Artifact base is fixed to
|
||||
`https://stageapi.canina.ir/api/internal/prepare-worker/artifacts`; parent must verify
|
||||
its actual global `/api` prefix/proxy/authenticated retrieval after CI deployment.
|
||||
No URL here is asserted externally reachable by disposable tests.
|
||||
|
||||
Parent still must implement/review a bounded one-shot real LLM executor separately;
|
||||
this commit intentionally adds neither model/provider secrets nor automatic polling.
|
||||
After CI rollout verify backend UID/secret readability by names only, deny-all reads,
|
||||
worker unauth denial, snapshot provenance and authenticated artifact digest readback
|
||||
before a separately authorized task. Do not call RELEASE_AUTHORIZE.
|
||||
|
||||
## Executed disposable validation
|
||||
|
||||
Pinned Hermes image, `--network none`, repository mounted read-only; private
|
||||
TemporaryDirectory fixtures only. RED observed missing provisioner, missing exporter,
|
||||
wrong projected profile and acceptance of malformed credential; GREEN: four
|
||||
provision/export/security/CI-guard unittest cases; projected HTTP auth/profile/session/
|
||||
wrong-profile/model/cron assertions; existing exact-image HTTP acceptance, TCP serve
|
||||
mutation denial and authenticated WebSocket 1008 denial. Compose merged config
|
||||
`--quiet` passed without applying it. An initial test mount mismatch was corrected
|
||||
(`/source` for repo-relative tests, `/contract` for legacy tests). No application
|
||||
backend code changed; full backend suites not rerun. No host credentials, artifact
|
||||
storage, snapshots, DB users, services or networks were provisioned live.
|
||||
@@ -0,0 +1,66 @@
|
||||
# Parent merges this file with its pinned Stage compose; CI opt-in only.
|
||||
services:
|
||||
backend_stage:
|
||||
environment:
|
||||
PREPARE_WORKER_ENABLED: "true"
|
||||
secrets:
|
||||
- hermes_username
|
||||
- hermes_password
|
||||
- hermes_runtime_grants
|
||||
- prepare_worker_grants
|
||||
volumes:
|
||||
- type: bind
|
||||
source: /opt/services/canina/stage-runtime/artifacts
|
||||
target: /var/lib/canina/prepare-artifacts
|
||||
bind:
|
||||
create_host_path: false
|
||||
networks:
|
||||
- hermes_inventory_backend
|
||||
runtime-inventory:
|
||||
profiles: [stage-runtime-inventory]
|
||||
image: nousresearch/hermes-agent@sha256:9774f4f39a9bb8c2f68ce728ed5e99ddbad282163be56764afacf88ed952b784
|
||||
user: "1000:1000"
|
||||
entrypoint: [/opt/hermes/.venv/bin/python3, /runtime/serve.py]
|
||||
restart: unless-stopped
|
||||
read_only: true
|
||||
environment:
|
||||
HERMES_HOME: /inventory
|
||||
HERMES_SERVE_HEADLESS: "1"
|
||||
HERMES_REQUIRE_PROJECTED_INVENTORY: "1"
|
||||
PYTHONDONTWRITEBYTECODE: "1"
|
||||
tmpfs: [/tmp:rw,noexec,nosuid,size=64m,mode=1777]
|
||||
volumes:
|
||||
- type: bind
|
||||
source: ${STAGE_RUNTIME_SERVE_FILE:?absolute pinned serve.py required}
|
||||
target: /runtime/serve.py
|
||||
read_only: true
|
||||
bind:
|
||||
create_host_path: false
|
||||
- type: bind
|
||||
source: ${STAGE_RUNTIME_SNAPSHOT:?approved immutable projected snapshot required}
|
||||
target: /inventory
|
||||
read_only: true
|
||||
bind:
|
||||
create_host_path: false
|
||||
secrets: [hermes_username, hermes_password, hermes_signing_key]
|
||||
networks:
|
||||
hermes_inventory_backend:
|
||||
aliases: [canina-hermes-inventory]
|
||||
security_opt: [no-new-privileges:true]
|
||||
cap_drop: [ALL]
|
||||
pids_limit: 128
|
||||
mem_limit: 1g
|
||||
networks:
|
||||
hermes_inventory_backend:
|
||||
internal: true
|
||||
secrets:
|
||||
hermes_username:
|
||||
file: /opt/services/canina/stage-runtime/secrets/hermes_username
|
||||
hermes_password:
|
||||
file: /opt/services/canina/stage-runtime/secrets/hermes_password
|
||||
hermes_signing_key:
|
||||
file: /opt/services/canina/stage-runtime/secrets/hermes_signing_key
|
||||
hermes_runtime_grants:
|
||||
file: /opt/services/canina/stage-runtime/secrets/hermes_runtime_grants
|
||||
prepare_worker_grants:
|
||||
file: /opt/services/canina/stage-runtime/secrets/prepare_worker_grants
|
||||
@@ -48,6 +48,48 @@ def build_app():
|
||||
'liveGatewayState':'unknown','liveActiveAgents':None}
|
||||
# Place before installed catch-all UI route.
|
||||
web_server.app.router.routes.insert(0, web_server.app.router.routes.pop())
|
||||
projected=Path(os.environ['HERMES_HOME'])/'inventory.json'
|
||||
if os.environ.get('HERMES_REQUIRE_PROJECTED_INVENTORY')=='1' and not projected.is_file():
|
||||
raise RuntimeError('Projected inventory required')
|
||||
if projected.exists():
|
||||
# Immutable exporter projection, not installed config/session/cron readers.
|
||||
if projected.is_symlink() or projected.stat().st_size>32*1024*1024:
|
||||
raise RuntimeError('Invalid projected inventory')
|
||||
inventory=json.loads(projected.read_text())
|
||||
if set(inventory)!={'schemaVersion','profile','sessions'} or inventory['schemaVersion']!=1 or not isinstance(inventory['sessions'],list):
|
||||
raise RuntimeError('Invalid projected inventory')
|
||||
profile=inventory['profile']
|
||||
import re
|
||||
if not isinstance(profile,str) or not re.fullmatch('[a-zA-Z0-9][a-zA-Z0-9_-]{0,63}',profile):
|
||||
raise RuntimeError('Invalid projected inventory')
|
||||
replaced={'/api/profiles','/api/sessions','/api/analytics/usage','/api/cron/jobs','/api/model/info'}
|
||||
web_server.app.router.routes[:]=[r for r in web_server.app.router.routes if getattr(r,'path',None) not in replaced]
|
||||
async def projected_read(request:Request):
|
||||
from fastapi import HTTPException
|
||||
path=request.url.path
|
||||
requested=request.query_params.get('profile',profile)
|
||||
if requested!=profile: raise HTTPException(status_code=404)
|
||||
if path=='/api/profiles':
|
||||
return {'profiles':[{'name':profile,'is_default':profile=='default','gateway_running':False}]}
|
||||
if path=='/api/model/info':
|
||||
return {'model':'unknown','provider':'unknown','effective_context_length':0}
|
||||
if path=='/api/cron/jobs': return [] # Cron commands deliberately not captured.
|
||||
try:
|
||||
limit=int(request.query_params.get('limit','20'));offset=int(request.query_params.get('offset','0'))
|
||||
days=int(request.query_params.get('days','30'))
|
||||
if not 1<=limit<=100 or not 0<=offset<=10000 or not 1<=days<=365: raise ValueError()
|
||||
except ValueError: raise HTTPException(status_code=400)
|
||||
rows=inventory['sessions']
|
||||
if path=='/api/sessions':
|
||||
return {'total':len(rows),'sessions':[{k:r[k] for k in ['id','profile','source','is_active']} for r in list(reversed(rows))[offset:offset+limit]],'storage':{}}
|
||||
captured=datetime.fromisoformat(snapshot_provenance()['capturedAt'].replace('Z','+00:00')).timestamp()
|
||||
filtered=[r for r in rows if captured-days*86400<=r['startedAt']<=captured]
|
||||
return {'period_days':days,'totals':{'total_input':sum(r['inputTokens'] for r in filtered),
|
||||
'total_output':sum(r['outputTokens'] for r in filtered),'total_sessions':len(filtered),
|
||||
'total_estimated_cost':sum(r['estimatedCost'] for r in filtered)}}
|
||||
for path in replaced:
|
||||
web_server.app.add_api_route(path,projected_read,methods=['GET'])
|
||||
web_server.app.router.routes.insert(0,web_server.app.router.routes.pop())
|
||||
register_global_provider(BasicAuthProvider(username=secret('hermes_username'),password_hash=hash_password(secret('hermes_password')),secret=secret('hermes_signing_key').encode(),ttl_seconds=300))
|
||||
web_server.app.state.auth_required=True
|
||||
web_server.app.state.bound_host='0.0.0.0'
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
"""Projected inventory uses no gateway config or session store."""
|
||||
import os, tempfile, json, importlib.util
|
||||
from pathlib import Path
|
||||
os.environ['HERMES_HOME'] = tempfile.mkdtemp()
|
||||
os.environ['HERMES_SERVE_HEADLESS'] = '1'
|
||||
root = Path(os.environ['HERMES_HOME'])
|
||||
from datetime import datetime, timezone
|
||||
(root/'snapshot-manifest.json').write_text(json.dumps({'schemaVersion':1,'source':'snapshot','capturedAt':datetime.now(timezone.utc).isoformat().replace('+00:00','Z')}))
|
||||
(root/'inventory.json').write_text(json.dumps({'schemaVersion':1,'profile':'seo','sessions':[{'id':'snapshot-0','profile':'seo','source':'cli','startedAt':1,'inputTokens':4,'outputTokens':6,'estimatedCost':0.01,'is_active':False}]}))
|
||||
spec=importlib.util.spec_from_file_location('serve',Path(__file__).parent/'serve.py');m=importlib.util.module_from_spec(spec);spec.loader.exec_module(m)
|
||||
m.secret=lambda name:'projected-test-only-'+name+'x'*32
|
||||
from fastapi.testclient import TestClient
|
||||
c=TestClient(m.build_app(),base_url='http://localhost')
|
||||
assert c.get('/api/profiles').status_code==401
|
||||
r=c.post('/auth/password-login',json={'provider':'basic','username':m.secret('hermes_username'),'password':m.secret('hermes_password')});assert r.status_code==200
|
||||
r=c.get('/api/profiles');assert [p['name'] for p in r.json()['profiles']]==['seo'],r.text
|
||||
r=c.get('/api/sessions?profile=seo');assert r.json()['total']==1 and r.json()['sessions'][0]['id']=='snapshot-0',r.text
|
||||
r=c.get('/api/sessions?profile=default');assert r.status_code==404,r.text
|
||||
r=c.get('/api/model/info?profile=seo');assert r.json()['model']=='unknown',r.text
|
||||
r=c.get('/api/cron/jobs?profile=seo');assert r.json()==[],r.text
|
||||
assert not (root/'state.db').exists()
|
||||
print('PASS projected profile/session/model/cron HTTP; no state DB/config used')
|
||||
@@ -0,0 +1,75 @@
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sqlite3
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
SPEC = importlib.util.spec_from_file_location('provision', ROOT / 'scripts/provision-stage-runtime.py')
|
||||
|
||||
class ProvisionTests(unittest.TestCase):
|
||||
def load(self):
|
||||
self.assertTrue(SPEC.origin and Path(SPEC.origin).exists(), 'Stage provisioner missing')
|
||||
m = importlib.util.module_from_spec(SPEC); SPEC.loader.exec_module(m)
|
||||
return m
|
||||
|
||||
def test_idempotent_dedicated_credentials_deny_all_runtime(self):
|
||||
m = self.load()
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
root = Path(d) / 'target'
|
||||
m.provision(root, 1000, 1000)
|
||||
first = {p.name: p.read_bytes() for p in (root / 'secrets').iterdir()}
|
||||
m.provision(root, 1000, 1000)
|
||||
self.assertEqual(first, {p.name: p.read_bytes() for p in (root / 'secrets').iterdir()})
|
||||
self.assertEqual(json.loads(first['hermes_runtime_grants']), {})
|
||||
config = json.loads(first['prepare_worker_grants'])
|
||||
self.assertNotEqual(config['workerIds'], config['stageCiIds'])
|
||||
self.assertEqual(len({g['credentialSha256'] for g in config['grants']}), 2)
|
||||
for p in (root / 'secrets').iterdir(): self.assertEqual(p.stat().st_mode & 0o777, 0o400)
|
||||
self.assertEqual((root / 'artifacts').stat().st_mode & 0o777, 0o700)
|
||||
|
||||
def test_consistent_wal_export_projects_only_allowlisted_inventory(self):
|
||||
m = self.load()
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
root = Path(d); db = root / 'state.db'
|
||||
c = sqlite3.connect(db)
|
||||
c.execute('pragma journal_mode=WAL')
|
||||
c.execute('create table sessions(id text,source text,started_at real,input_tokens integer,output_tokens integer,estimated_cost_usd real,system_prompt text,title text)')
|
||||
c.execute("insert into sessions values('private-person-id','cli',100,4,6,0.01,'NEVER_COPY_PROMPT','NEVER_COPY_TITLE')")
|
||||
c.execute('create table messages(content text)')
|
||||
c.execute("insert into messages values('NEVER_COPY_MESSAGE')"); c.commit()
|
||||
(root / 'config.yaml').write_text('NEVER_COPY_KEY')
|
||||
target = root / 'snapshot'
|
||||
self.assertTrue(hasattr(m, 'export_snapshot'), 'Consistent inventory exporter missing')
|
||||
m.export_snapshot(db, target, 'default', 1000, 1000)
|
||||
text = ''.join(p.read_text() for p in target.iterdir())
|
||||
for secret in ['NEVER_COPY', 'private-person-id', 'system_prompt', 'messages']:
|
||||
self.assertNotIn(secret, text)
|
||||
data = json.loads((target / 'inventory.json').read_text())
|
||||
self.assertEqual(data['sessions'][0]['inputTokens'], 4)
|
||||
self.assertEqual(data['sessions'][0]['source'], 'cli')
|
||||
self.assertEqual(set(p.name for p in target.iterdir()), {'inventory.json','snapshot-manifest.json','export-report.json'})
|
||||
self.assertEqual(json.loads((target / 'snapshot-manifest.json').read_text())['source'], 'snapshot')
|
||||
with self.assertRaises(Exception): m.export_snapshot(db, target, 'default', 1000, 1000)
|
||||
c.close()
|
||||
|
||||
def test_rejects_unsafe_paths_and_modified_credentials(self):
|
||||
m = self.load()
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
root = Path(d) / 'target'; m.provision(root, 1000, 1000)
|
||||
p = root / 'secrets/hermes_password'
|
||||
p.chmod(0o600); p.write_text('compromised'); p.chmod(0o400)
|
||||
with self.assertRaises(Exception): m.provision(root, 1000, 1000)
|
||||
|
||||
def test_cli_rejects_non_ci_before_mutation(self):
|
||||
import subprocess
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
target = Path(d) / 'target'
|
||||
env = {**os.environ, 'CI':'false'}
|
||||
r = subprocess.run(['python3', str(SPEC.origin), '--root', str(target)], env=env, capture_output=True)
|
||||
self.assertNotEqual(r.returncode, 0)
|
||||
self.assertFalse(target.exists())
|
||||
|
||||
if __name__ == '__main__': unittest.main()
|
||||
@@ -0,0 +1,202 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Opt-in CI Stage provisioning. No Docker, DB writes, service start or polling."""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import secrets
|
||||
import shutil
|
||||
import stat
|
||||
import subprocess
|
||||
import tempfile
|
||||
|
||||
BASE_URL = 'https://stageapi.canina.ir/api/internal/prepare-worker/artifacts'
|
||||
|
||||
|
||||
def safe_path(path):
|
||||
path = Path(path)
|
||||
if not path.is_absolute() or '..' in path.parts:
|
||||
raise ValueError('Absolute canonical path required')
|
||||
for part in [*reversed(path.parents), path]:
|
||||
if part.is_symlink():
|
||||
raise ValueError('Symlink path rejected')
|
||||
if part.exists() and not part.is_dir():
|
||||
raise ValueError('Directory required')
|
||||
return path
|
||||
|
||||
|
||||
def write_private(path, value, uid, gid):
|
||||
fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o400)
|
||||
with os.fdopen(fd, 'w') as f:
|
||||
f.write(value)
|
||||
f.flush(); os.fsync(f.fileno())
|
||||
os.fchown(f.fileno(), uid, gid)
|
||||
|
||||
|
||||
def provision(root, uid, gid):
|
||||
root = safe_path(root)
|
||||
if root.exists():
|
||||
# Idempotence never silently rotates or repairs potentially compromised state.
|
||||
validate(root, uid, gid)
|
||||
return
|
||||
if not root.parent.is_dir():
|
||||
raise ValueError('Provision parent must already exist')
|
||||
staging = Path(tempfile.mkdtemp(prefix='.stage-provision-', dir=root.parent))
|
||||
try:
|
||||
for name, mode, owner in [('secrets', 0o700, uid), ('credentials', 0o700, os.getuid()), ('artifacts', 0o700, uid), ('snapshots', 0o700, os.getuid())]:
|
||||
p = staging / name; p.mkdir(mode=mode); os.chown(p, owner, gid)
|
||||
values = {name: secrets.token_urlsafe(48) for name in ['hermes_username', 'hermes_password', 'hermes_signing_key', 'prepare_worker_token', 'stage_ci_token']}
|
||||
config = {'grants': [
|
||||
{'id': 'stage-prepare-worker', 'role': 'WORKFLOW_WORKER', 'credentialSha256': hashlib.sha256(values['prepare_worker_token'].encode()).hexdigest()},
|
||||
{'id': 'stage-evidence-ci', 'role': 'STAGE_CI', 'credentialSha256': hashlib.sha256(values['stage_ci_token'].encode()).hexdigest()}],
|
||||
'workerIds': ['stage-prepare-worker'], 'stageCiIds': ['stage-evidence-ci'], 'artifactBaseUrl': BASE_URL}
|
||||
for name in ['hermes_username', 'hermes_password', 'hermes_signing_key']:
|
||||
write_private(staging / 'secrets' / name, values[name], uid, gid)
|
||||
for name in ['prepare_worker_token', 'stage_ci_token']:
|
||||
write_private(staging / 'credentials' / name, values[name], os.getuid(), os.getgid())
|
||||
for name, value in [('hermes_runtime_grants', {}), ('prepare_worker_grants', config)]:
|
||||
write_private(staging / 'secrets' / name, json.dumps(value), uid, gid)
|
||||
# Root traversal only; each child controls its own readers.
|
||||
staging.chmod(0o711)
|
||||
validate(staging, uid, gid)
|
||||
# Serialized CI invocation required; refuse a concurrent winner.
|
||||
os.rename(staging, root)
|
||||
finally:
|
||||
if staging.exists(): shutil.rmtree(staging)
|
||||
|
||||
|
||||
def validate(root, uid, gid):
|
||||
expected = {'secrets', 'credentials', 'artifacts', 'snapshots'}
|
||||
if set(p.name for p in root.iterdir()) != expected or root.stat().st_mode & 0o077 != 0o011:
|
||||
raise ValueError('Unexpected provision root')
|
||||
for folder in expected:
|
||||
p = safe_path(root / folder); st = p.stat()
|
||||
if stat.S_IMODE(st.st_mode) != 0o700 or st.st_uid != (uid if folder in {'secrets', 'artifacts'} else os.getuid()):
|
||||
raise ValueError('Unsafe provision directory')
|
||||
names = {'hermes_username', 'hermes_password', 'hermes_signing_key', 'hermes_runtime_grants', 'prepare_worker_grants'}
|
||||
if {p.name for p in (root / 'secrets').iterdir()} != names:
|
||||
raise ValueError('Unexpected secrets')
|
||||
if {p.name for p in (root / 'credentials').iterdir()} != {'prepare_worker_token', 'stage_ci_token'}:
|
||||
raise ValueError('Unexpected credentials')
|
||||
for directory, owner in [('secrets', uid), ('credentials', os.getuid())]:
|
||||
for p in (root / directory).iterdir():
|
||||
st = p.lstat()
|
||||
if not stat.S_ISREG(st.st_mode) or st.st_nlink != 1 or st.st_uid != owner or stat.S_IMODE(st.st_mode) != 0o400 or not 1 <= st.st_size <= 16384:
|
||||
raise ValueError('Unsafe credential file')
|
||||
for name in ['hermes_username', 'hermes_password', 'hermes_signing_key']:
|
||||
if not re.fullmatch('[A-Za-z0-9_-]{64}', (root / 'secrets' / name).read_text()):
|
||||
raise ValueError('Invalid dedicated credential')
|
||||
config = json.loads((root / 'secrets/prepare_worker_grants').read_text())
|
||||
if set(config) != {'grants', 'workerIds', 'stageCiIds', 'artifactBaseUrl'} or len(config['grants']) != 2:
|
||||
raise ValueError('Unexpected worker config')
|
||||
for index, name in enumerate(['prepare_worker_token', 'stage_ci_token']):
|
||||
grant = config['grants'][index]
|
||||
if set(grant) != {'id', 'role', 'credentialSha256'} or grant['id'] != ['stage-prepare-worker', 'stage-evidence-ci'][index]:
|
||||
raise ValueError('Unexpected worker identity')
|
||||
if not re.fullmatch('[A-Za-z0-9_-]{64}', (root / 'credentials' / name).read_text()):
|
||||
raise ValueError('Invalid service credential')
|
||||
if hashlib.sha256((root / 'credentials' / name).read_bytes()).hexdigest() != grant['credentialSha256']:
|
||||
raise ValueError('Credential digest mismatch')
|
||||
if config['workerIds'] != ['stage-prepare-worker'] or config['stageCiIds'] != ['stage-evidence-ci'] or config['artifactBaseUrl'] != BASE_URL or [g['role'] for g in config['grants']] != ['WORKFLOW_WORKER', 'STAGE_CI']:
|
||||
raise ValueError('Unexpected grants')
|
||||
if json.loads((root / 'secrets/hermes_runtime_grants').read_text()) != {}:
|
||||
raise ValueError('Runtime grants require separate verified-ID review')
|
||||
|
||||
|
||||
def export_snapshot(database, destination, profile, uid, gid):
|
||||
"""SQLite online backup -> explicit metadata projection; raw backup never leaves RAM."""
|
||||
import sqlite3
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
if not re.fullmatch('[a-zA-Z0-9][a-zA-Z0-9_-]{0,63}', profile):
|
||||
raise ValueError('Invalid profile')
|
||||
database = Path(database)
|
||||
safe_path(database.parent)
|
||||
st = database.lstat()
|
||||
if not stat.S_ISREG(st.st_mode) or st.st_nlink != 1:
|
||||
raise ValueError('Regular unlinked database required')
|
||||
destination = safe_path(destination)
|
||||
if destination.exists() or not destination.parent.is_dir():
|
||||
raise ValueError('New snapshot destination required')
|
||||
# mode=ro includes committed WAL frames. Never immutable=1 on a live WAL database.
|
||||
src = sqlite3.connect(database.as_uri() + '?mode=ro', uri=True, timeout=5)
|
||||
backup = sqlite3.connect(':memory:')
|
||||
deadline = time.monotonic() + 30
|
||||
def progress(status, remaining, total):
|
||||
if time.monotonic() > deadline: raise ValueError('Backup deadline exceeded')
|
||||
try:
|
||||
src.backup(backup, pages=256, progress=progress, sleep=0.05)
|
||||
captured = datetime.now(timezone.utc).isoformat().replace('+00:00', 'Z')
|
||||
# Fail closed on schema drift. No SELECT *, private IDs, titles, prompts, messages,
|
||||
# config, provider credentials, cron commands, filesystem paths or attachments.
|
||||
rows = backup.execute('SELECT source,started_at,input_tokens,output_tokens,estimated_cost_usd FROM sessions ORDER BY started_at').fetchall()
|
||||
if len(rows) > 100000: raise ValueError('Inventory exceeds bound')
|
||||
allowed_sources = {'cli', 'telegram', 'discord', 'slack', 'api', 'web', 'cron'}
|
||||
sessions = []
|
||||
for index, (source, started, inputs, outputs, cost) in enumerate(rows):
|
||||
import math
|
||||
numbers = [started, inputs or 0, outputs or 0, cost or 0]
|
||||
if any(not isinstance(n, (int, float)) or not math.isfinite(n) or n < 0 for n in numbers):
|
||||
raise ValueError('Invalid inventory numeric metadata')
|
||||
sessions.append({'id': 'snapshot-' + str(index), 'profile': profile,
|
||||
'source': source if source in allowed_sources else 'other',
|
||||
'startedAt': started, 'inputTokens': inputs or 0,
|
||||
'outputTokens': outputs or 0, 'estimatedCost': cost or 0,
|
||||
'is_active': False})
|
||||
inventory = {'schemaVersion': 1, 'profile': profile, 'sessions': sessions}
|
||||
payload = json.dumps(inventory, allow_nan=False, separators=(',', ':'))
|
||||
if len(payload.encode()) > 32 * 1024 * 1024: raise ValueError('Inventory exceeds bound')
|
||||
finally:
|
||||
backup.close(); src.close()
|
||||
staging = Path(tempfile.mkdtemp(prefix='.snapshot-', dir=destination.parent))
|
||||
try:
|
||||
manifest = {'schemaVersion': 1, 'source': 'snapshot', 'capturedAt': captured}
|
||||
report = {'schemaVersion': 1, 'method': 'sqlite-online-backup-to-memory-allowlist-projection',
|
||||
'capturedAt': captured, 'sessions': len(sessions),
|
||||
'inventorySha256': hashlib.sha256(payload.encode()).hexdigest(),
|
||||
'excluded': ['credentials', 'config', 'prompts', 'transcripts', 'private-identities', 'cron-commands'],
|
||||
'liveExecutor': False}
|
||||
for name, value in [('inventory.json', payload), ('snapshot-manifest.json', json.dumps(manifest)), ('export-report.json', json.dumps(report))]:
|
||||
write_private(staging / name, value, uid, gid)
|
||||
os.chown(staging, uid, gid)
|
||||
staging.chmod(0o500)
|
||||
os.rename(staging, destination)
|
||||
finally:
|
||||
if staging.exists():
|
||||
staging.chmod(0o700); shutil.rmtree(staging)
|
||||
|
||||
|
||||
def ci_gate():
|
||||
sha = os.environ.get('STAGE_PROVISION_APPROVED_SHA', '')
|
||||
if os.environ.get('CI') != 'true' or os.environ.get('STAGE_PROVISION_ENABLED') != 'true' or not re.fullmatch('[0-9a-f]{40}', sha):
|
||||
raise ValueError('Approved Stage CI invocation required')
|
||||
repo = Path(__file__).resolve().parents[1]
|
||||
def git(*args):
|
||||
return subprocess.check_output(['git', '-C', str(repo), *args], text=True).strip()
|
||||
if git('rev-parse', 'HEAD') != sha or git('status', '--porcelain'):
|
||||
raise ValueError('Clean approved commit required')
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--root', type=Path, default=Path('/opt/services/canina/stage-runtime'))
|
||||
parser.add_argument('--uid', type=int, default=1000)
|
||||
parser.add_argument('--gid', type=int, default=1000)
|
||||
parser.add_argument('--snapshot-db', type=Path)
|
||||
parser.add_argument('--snapshot-name')
|
||||
parser.add_argument('--profile', default='default')
|
||||
args = parser.parse_args()
|
||||
ci_gate()
|
||||
if args.uid <= 0 or args.gid <= 0: raise ValueError('Non-root runtime identity required')
|
||||
if bool(args.snapshot_db) != bool(args.snapshot_name) or (args.snapshot_name and not re.fullmatch('[a-zA-Z0-9_-]{1,80}', args.snapshot_name)):
|
||||
raise ValueError('Paired snapshot database and safe unique name required')
|
||||
provision(args.root, args.uid, args.gid)
|
||||
if args.snapshot_db:
|
||||
export_snapshot(args.snapshot_db, args.root / 'snapshots' / args.snapshot_name, args.profile, args.uid, args.gid)
|
||||
print('Stage credentials and artifact storage validated; no services started')
|
||||
|
||||
if __name__ == '__main__':
|
||||
try: main()
|
||||
except Exception: raise SystemExit('Stage provisioning rejected (details suppressed)')
|
||||
Reference in new issue
Block a user