Compare commits

...
9 changed files with 450 additions and 0 deletions

No files matched your search

+78
View File
@@ -0,0 +1,78 @@
# Real bounded Tech/Dev Hermes pilot (experimental, NOT enabled)
Owned files only in `infra/hermes-pilot`; no existing core, module, tracer, profile or runtime config changed. No push/deploy/RELEASE_AUTHORIZE. Do not wire this into the existing tracer completion endpoint: that endpoint deliberately accepts only deterministic JSON.
## Fixed task and boundaries
Task `tech-dev-clamp-quantity-v1`: produce one <=2048-byte Python `clamp_quantity(quantity)` implementation, integer range 0..99. No input/browser prompt is accepted. The model has **zero tools**, no shell/browser/Git/deploy credentials. Trusted code alone writes `quantity.py`, runs separate AST-allowlisted QA (9 boundary cases), and commits an isolated, remote-free fixture repository. This is real generated-code QA, not application QA. Source commit is recorded as provenance, not represented as the fixture's base commit.
`infer.py` uses installed `run_agent.AIAgent`, verified from the installed signature and `agent/agent_init.py`; `enabled_toolsets=[]` is additionally enforced by asserting empty `agent.tools`. Existing approved `OPENAI_API_KEY` is read in memory only; no messaging/API-server credentials/auth pools copied. Separate temporary HERMES_HOME, no memories/context/background review/fallback. Provider/model pinned to current canina-hermes `custom`, `https://nr.parsaaghayi.ir/v1`, `Free`; no provider installation or account changes. Existing configs remain untouched. Model output is untrusted and never directly shell-executed; QA rejects all syntax except the single nested min/max return expression before isolated Python execution.
Installed image identity: `sha256:373ddd5e6649f525b17993e1c885cdf1352127f8e98fd329140a73237125e52f`, install stamp commit `818c13be1dc4fd28987e1e881a9408224afd4535`, v0.21.6. Discovery: `/opt/hermes/hermes chat --help` supports `--oneshot`, `--max-turns`, `--run-budget`; `/opt/hermes/.venv/bin/python` loads installed API. Official skills and https://hermes-agent.nousresearch.com/docs/llms.txt consulted. The initially guessed developer API docs URL returned404; installed source/signature is authoritative for this pinned build.
## PREPARE mapping — parent integration required
`executor.execute(lease, artifact_root, infer, heartbeat)` only accepts a fresh staging, first-attempt, 20-cent, 40hex-source lease and exact lease fields; heartbeat failure aborts before inference and before QA/commit. Trusted adapter must obtain this lease via existing authenticated claim AFTER independent Gate1/READY, verify the explicitly allowlisted task identity/scope/criteria server-side, and supply heartbeat connected to the authenticated backend. A caller-created dict is NOT an authorization. Never expose this Python interface to browser requests.
## Heartbeat runner and parent API (this revision)
After independent READY/Gate1, allowlist and provider-cap admission, parent constructs
`runner = executor.pinned_inference(executor.IMAGE)` and calls
`executor.execute(authenticated_lease, artifact_root, runner, authenticated_heartbeat)`.
Only trusted Python may construct runners. Callback receives an unchanged COPY of the
original lease binding (taskId/runId/leaseToken), must use authenticated server transport,
and must raise on any non-2xx, ownership/fencing/expiry denial or malformed response.
Explicit `False` also denies; `None` means success for existing raise-on-error adapters.
Callback MUST impose <=5s network timeout, no retries; a blocked callback thread cannot
be forcibly killed by Python. It is daemonized; its result cannot restart inference or
extend a local lease. Parent must serialize executions per server lease; the runner is
single-use and concurrent reuse of that runner is locked out. This is not a browser API.
Immediate initial heartbeat; dedicated ongoing thread waits 5s between acknowledged
calls. With required <=5s transport ceiling, heartbeat starts are <=10s apart. A stalled
heartbeat (10s without acknowledgment), exception or denial aborts actual subprocess
inference, not just its caller. No leaseUntil mutation/local extension occurs. Fresh
server-authorized heartbeats retain fencing; this code cannot create authorization.
Guard checks also fence QA/artifact commit/report return. A denial during QA prevents
report return; any partial local fixture is not accepted completion evidence.
Monotonic wall ceiling 90s with cancellation reserve: work aborts at85s, remote kill
command <=2s and local reap <=1s; bounded QA3s/Git1s. Pinned runner uses one model
call, max_iterations1/max_tokens160, run_budget80s, tools0, no retries. An independent
in-container timeout85s survives loss of the Docker client. Cancellation addresses
the actual in-container process session via a random PID file plus cancellation
marker (handles cancellation before PID publication); it kills that process group.
Cancellation transport failure raises and prevents success, but cannot prove immediate
remote death; independent inner timeout is the fallback. Small random marker files
in container /tmp are intentional fencing tombstones, not configuration changes.
`rehearse.py` remains synthetic, never a live lease. It uses this cancellable runner
but is NOT admission evidence. Do not run it until provider-cap approval. Existing
20-cent reservation remains UNKNOWN, never zero, and no completion/claim/release or
backend transport is implemented here. Existing tracer endpoint still rejects model
reports; parent must separately implement authenticated allowlisted artifact ingestion.
No changes to installed Hermes or runtime/profile configuration.
## Actual evidence and blockers
Evidence directory `/opt/services/canina/validation/v4-real-pilot`:
- `red.log`: missing executor assertion failed before implementation.
- `inference-private.log`: actual installed Hermes/provider inference exit0 under75s diagnostic ceiling; generated source saved, not fabricated. This ceiling was ONLY diagnostic, not accepted as a PREPARE lease run.
- `model-artifact/quantity.py`: actual model artifact; independent QA passes9 cases, SHA256 `0125246b551896c5f9ee98fabeb02982a6690a9974c0eb24ef3ae676d46380d5`; isolated Git commit `95cb3d27d636ae817d6a778e8053e2f87b50e57b`.
- `actual.log`: bounded full rehearsal failed with inner timeout exit137 at18s; no completion/QA success was assumed. Previous client-only timeout was corrected to inner process kill. **Installed model/startup cannot currently complete reliably inside30s PREPARE lease without authenticated ongoing heartbeat.** Do not extend lease locally or call the75s diagnostic a bounded lease success.
Contract tests: `python3 infra/hermes-pilot/test_contract.py`. Parent still owns backend integration, authenticated heartbeat orchestration, allowlist admission, billing caps, signed artifact transport and Stage CI. This commit is a real generation/independent-QA proof and fail-closed executor seam, not a live autonomous Tech department or released product.
## Verification of heartbeat revision
`python3 -m unittest discover -s infra/hermes-pilot -v`: 11 passing tests, including
real OS child/descendant cancellation, ongoing rejection, stalled heartbeat, wall
ceiling, concurrent runner reuse, immutable lease and unknown-cost preservation.
The artifact success test is explicitly a deterministic process fixture, NOT model
inference. TDD red observed for missing ongoing runner and explicit false denial;
additional cancellation/concurrency regression cases were added and executed.
Previously generated real model artifact was rechecked by current AST allowlist QA:
9 cases pass, SHA256 `0125246b551896c5f9ee98fabeb02982a6690a9974c0eb24ef3ae676d46380d5`.
No new real inference was run: README provider pricing/cap gate remains unresolved.
Docker cancellation path is implemented but not runtime-exercised here; OS process
group cancellation is exercised. No live backend/claim/release, push or deploy.
+64
View File
@@ -0,0 +1,64 @@
"""Fixed Tech/Dev pilot. Adapter must supply a fresh authenticated PREPARE lease."""
import datetime, hashlib, json, os, pathlib, re, subprocess, tempfile, time
TASK='tech-dev-clamp-quantity-v1'
IMAGE='sha256:373ddd5e6649f525b17993e1c885cdf1352127f8e98fd329140a73237125e52f'
def validate_lease(lease):
if (set(lease) != {'taskId','runId','leaseToken','leaseUntil','commitSha','environment','maxCostCents','attempt'}
or lease['environment']!='staging' or lease['maxCostCents']!=20 or lease['attempt']!=1
or not re.fullmatch('[a-f0-9]{40}',lease['commitSha'])):
raise ValueError('Bounded PREPARE lease required')
for k in ('taskId','runId','leaseToken'):
if not re.fullmatch('[a-f0-9-]{36}',lease[k]): raise ValueError('Invalid lease binding')
expiry=datetime.datetime.fromisoformat(lease['leaseUntil'].replace('Z','+00:00')).timestamp()
if not 5 < expiry-time.time() <= 31: raise ValueError('Fresh 30-second lease required')
return lease
def _execute(lease, artifact_root, infer, heartbeat):
"""No claim/approval generation. heartbeat must call trusted server; errors abort.
infer is the pinned runner, not a browser-supplied command or prompt.
Completion is deliberately NOT sent: existing transport accepts tracer JSON only.
"""
validate_lease(lease)
heartbeat(lease)
root=pathlib.Path(tempfile.mkdtemp(prefix='real-pilot-',dir=artifact_root))
def git(*args):
return subprocess.check_output(['git','-C',str(root),*args],stderr=subprocess.DEVNULL,text=True,timeout=1).strip()
git('init','-q'); git('config','user.email','pilot@invalid'); git('config','user.name','Bounded Hermes Pilot')
(root/'README').write_text('Disposable fixture; PREPARE source '+lease['commitSha']+'\n')
git('add','README'); git('commit','-qm','Seed isolated pilot fixture')
base=git('rev-parse','HEAD')
# One cancellable inference, no tools; ongoing authenticated guard.
source=infer()
heartbeat(lease)
if not isinstance(source,str) or len(source.encode())>2048: raise ValueError('Invalid generated artifact')
(root/'quantity.py').write_text(source)
qa=subprocess.check_output(['python3','-I',str(pathlib.Path(__file__).with_name('qa.py')),str(root)],timeout=3,text=True)
heartbeat(lease)
git('add','quantity.py'); git('commit','-qm','Hermes generated quantity clamp; independent QA passed')
patch=git('diff',base,'HEAD','--','quantity.py')+'\n'
(root/'artifact.patch').write_text(patch)
report={'kind':'REAL_HERMES_TECH_DEV_PILOT','task':TASK,'taskId':lease['taskId'],'runId':lease['runId'],
'sourceCommit':lease['commitSha'],'artifactCommit':git('rev-parse','HEAD'),'baseCommit':base,
'patchSha256':hashlib.sha256(patch.encode()).hexdigest(),'qa':json.loads(qa),
'costCents':None,'costStatus':'UNKNOWN_RETAIN_FULL_20_CENT_RESERVATION',
'gate1':'EXTERNAL_LEASE_REQUIRED','published':False,'deploymentExecuted':False}
(root/'evidence.json').write_text(json.dumps(report,indent=2)+'\n')
return root,report
from runtime import LeaseGuard, ProcessInference, pinned_inference
def execute(lease, artifact_root, infer, heartbeat):
validate_lease(lease)
with LeaseGuard(lease, heartbeat) as guard:
guard.check()
if not isinstance(infer, ProcessInference):
raise TypeError("Cancellable ProcessInference required")
def generate():
response = json.loads(infer.run(guard))
if not isinstance(response,dict) or response.get("tools") != 0 or response.get("model") != "Free":
raise ValueError("Pinned tool-less model required")
return response["source"]
result = _execute(lease, artifact_root, generate, lambda _: guard.check())
guard.check()
return result
+22
View File
@@ -0,0 +1,22 @@
"""Executed only inside pinned canina-hermes image by controlled runner."""
import contextlib, io, json, os, pathlib, tempfile
# Copy ONE existing inference credential in memory; no auth pools or messaging keys.
from dotenv import dotenv_values
key=dotenv_values('/opt/data/.env').get('OPENAI_API_KEY')
if not key: raise RuntimeError('Existing OPENAI_API_KEY unavailable')
home=tempfile.TemporaryDirectory(prefix='canina-real-pilot-')
os.environ.clear()
os.environ.update({'PATH':'/usr/bin:/bin','HOME':home.name,'HERMES_HOME':home.name,'OPENAI_API_KEY':key,'OPENAI_BASE_URL':'https://nr.parsaaghayi.ir/v1','HERMES_DISABLE_PLUGINS':'1'})
# Installed API verified by inspect.signature at pinned image commit 818c13be.
with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()):
from run_agent import AIAgent
agent=AIAgent(model='Free',provider='custom',base_url='https://nr.parsaaghayi.ir/v1',api_key=key,
enabled_toolsets=[],max_iterations=1,max_tokens=160,run_budget_seconds=80,
skip_context_files=True,skip_memory=True,skip_background_review=True,load_soul_identity=False,
quiet_mode=True,fallback_model={},cwd=home.name)
if agent.tools: raise RuntimeError('Tool-less invariant failed')
result=agent.run_conversation('Write only Python source, no markdown: define clamp_quantity(quantity) returning integer quantity clamped to inclusive 0..99. Use one return with nested min/max calls only. No annotations, comments, imports, decorators, docstrings or other statements.')
final=result.get('final_response')
if not isinstance(final,str) or not final.strip(): raise RuntimeError('No model artifact; result keys='+','.join(result.keys()))
print(json.dumps({'source':final.strip()+'\n','model':'Free','tools':0}))
home.cleanup()
+26
View File
@@ -0,0 +1,26 @@
"""Independent structural + behavior QA. Never executes arbitrary model Python."""
import ast, hashlib, json, pathlib, sys
def verify(root):
p=pathlib.Path(root)/'quantity.py'
if p.is_symlink() or not p.is_file() or p.stat().st_size>2048: raise ValueError('Invalid file')
raw=p.read_bytes(); tree=ast.parse(raw)
if len(tree.body)!=1 or not isinstance(tree.body[0],ast.FunctionDef): raise ValueError('Single function required')
fn=tree.body[0]
if fn.name!='clamp_quantity' or fn.decorator_list or fn.returns or fn.args.defaults or fn.args.kw_defaults or fn.args.kwonlyargs or fn.args.posonlyargs or fn.args.vararg or fn.args.kwarg or len(fn.args.args)!=1 or fn.args.args[0].arg!='quantity' or fn.args.args[0].annotation or len(fn.body)!=1 or not isinstance(fn.body[0],ast.Return): raise ValueError('Unexpected function structure')
def expr(n):
if isinstance(n,ast.Constant) and type(n.value) is int and 0<=n.value<=99: return
if isinstance(n,ast.Name) and n.id=='quantity': return
if isinstance(n,ast.Call) and isinstance(n.func,ast.Name) and n.func.id in ('min','max') and len(n.args)==2 and not n.keywords:
for a in n.args: expr(a)
return
raise ValueError('Nonallowlisted syntax')
expr(fn.body[0].value)
scope={'__builtins__':{},'min':min,'max':max}
exec(compile(tree,'generated-quantity','exec'),scope)
cases=[-1000,-1,0,1,50,98,99,100,10**30]
for q in cases:
actual=scope['clamp_quantity'](q)
if type(actual) is not int or actual!=max(0,min(q,99)): raise ValueError('Behavior QA failed')
return {'passed':True,'cases':len(cases),'sha256':hashlib.sha256(raw).hexdigest(),'scope':'quantity clamp fixture only'}
if __name__=='__main__': print(json.dumps(verify(sys.argv[1])))
+18
View File
@@ -0,0 +1,18 @@
"""Disposable model/artifact exercise, NOT a live Gate1 approval or service completion."""
import datetime, json, pathlib, subprocess, time, uuid
from executor import execute, IMAGE, pinned_inference
def main():
image=subprocess.check_output(['docker','inspect','--format','{{.Image}}','canina-hermes'],text=True).strip()
if image!=IMAGE: raise RuntimeError('Pinned installed image mismatch')
infer=pinned_inference(IMAGE)
deadline=time.monotonic()+90
lease={k:str(uuid.uuid4()) for k in ('taskId','runId','leaseToken')}
lease.update(leaseUntil=(datetime.datetime.now(datetime.timezone.utc)+datetime.timedelta(seconds=30)).isoformat(),commitSha=subprocess.check_output(['git','rev-parse','HEAD'],text=True).strip(),environment='staging',maxCostCents=20,attempt=1)
def fixture_heartbeat(binding):
if time.monotonic()>deadline: raise TimeoutError('Fixture hard deadline')
root,report=execute(lease,'/opt/services/canina/validation/v4-real-pilot',infer,fixture_heartbeat)
report['executionContext']='DISPOSABLE_FIXTURE_NOT_LIVE_PREPARE_LEASE'
(root/'evidence.json').write_text(json.dumps(report,indent=2)+'\n')
print(json.dumps({'artifactDirectory':str(root),'report':report}))
if __name__=='__main__': main()
+127
View File
@@ -0,0 +1,127 @@
"""Trusted subprocess boundary; no claim, lease extension or completion transport."""
import json
import os
import pathlib
import signal
import subprocess
import tempfile
import threading
import time
import uuid
HEARTBEAT_INTERVAL = 5.0
HEARTBEAT_TIMEOUT = 5.0
MAX_WALL_SECONDS = 90.0
class LeaseGuard:
def __init__(self, lease, heartbeat):
self.lease, self.heartbeat = lease, heartbeat
self.stop = threading.Event()
self.ready = threading.Event()
self.error = None
self.started = time.monotonic()
self.last = self.started
self.thread = threading.Thread(target=self._run, daemon=True)
def _run(self):
while not self.stop.is_set():
try:
if self.heartbeat(dict(self.lease)) is False:
raise PermissionError("Authenticated heartbeat denied")
self.last = time.monotonic()
self.ready.set()
except BaseException as exc:
self.error = exc
self.ready.set()
return
if self.stop.wait(HEARTBEAT_INTERVAL):
return
def check(self):
if self.error is not None:
raise self.error
now = time.monotonic()
if now - self.started >= MAX_WALL_SECONDS - 5:
raise TimeoutError('Pilot wall deadline (cancellation reserve included)')
if now - self.last >= HEARTBEAT_INTERVAL + HEARTBEAT_TIMEOUT:
raise TimeoutError('Authenticated heartbeat stalled')
def __enter__(self):
self.thread.start()
if not self.ready.wait(HEARTBEAT_TIMEOUT):
self.stop.set()
raise TimeoutError('Initial authenticated heartbeat stalled')
try:
self.check()
except BaseException:
self.stop.set()
raise
return self
def __exit__(self, *args):
self.stop.set()
self.thread.join(timeout=.05)
class ProcessInference:
"""Trusted argv only. cancel_remote MUST kill the in-container process group."""
def __init__(self, argv, input_text=None, cancel_remote=None):
self.argv, self.input_text, self.cancel_remote = argv, input_text, cancel_remote
self.used = False
self.lock = threading.Lock()
def run(self, guard):
with self.lock:
if self.used:
raise RuntimeError('Exactly one inference allowed; no retry')
self.used = True
guard.check()
# Files avoid pipe deadlocks and unbounded captured output in memory.
with tempfile.TemporaryFile(mode='w+') as incoming, tempfile.TemporaryFile(mode='w+') as out, tempfile.TemporaryFile(mode='w+') as err:
incoming.write(self.input_text or '')
incoming.seek(0)
child = subprocess.Popen(self.argv, stdin=incoming, stdout=out, stderr=err, start_new_session=True)
try:
while child.poll() is None:
guard.check()
time.sleep(.02)
guard.check()
if child.returncode:
raise RuntimeError('Inference failed (exit %d)' % child.returncode)
out.seek(0)
result = out.read(4097)
if len(result) > 4096:
raise ValueError('Oversized inference response')
return result
finally:
# Always cancel remote, even when docker exec itself failed.
try:
if self.cancel_remote:
self.cancel_remote()
finally:
try:
os.killpg(child.pid, signal.SIGKILL)
except ProcessLookupError:
pass
child.wait(timeout=1)
def pinned_inference(image):
actual = subprocess.check_output(['docker','inspect','--format','{{.Image}}','canina-hermes'], text=True, timeout=2).strip()
if actual != image:
raise RuntimeError('Pinned installed image mismatch')
token = uuid.uuid4().hex
pidfile = '/tmp/canina-pilot-' + token + '.pid'
script = pathlib.Path(__file__).with_name('infer.py').read_text()
# Session leader is created INSIDE the container. Inner kill is independent of client.
wrapper = ('import os,signal,subprocess,sys; '
'p=subprocess.Popen(["timeout","--signal=KILL","85","/opt/hermes/.venv/bin/python","-"],stdin=sys.stdin,start_new_session=True); '
'open(%r,"w").write(str(p.pid)); '
'os.path.exists(%r) and os.killpg(p.pid,signal.SIGKILL); sys.exit(p.wait())') % (pidfile, pidfile+'.cancel')
cancel = ('import os,signal; p=%r; '
'\nopen(p+".cancel","w").close()\ntry:\n pid=int(open(p).read()); os.killpg(pid,signal.SIGKILL)\n'
'except (FileNotFoundError,ProcessLookupError): pass\n'
'finally:\n'
' try: os.unlink(p)\n except FileNotFoundError: pass\n') % pidfile
def cancel_remote():
subprocess.run(['docker','exec','canina-hermes','/opt/hermes/.venv/bin/python','-c',cancel], check=True, timeout=2, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
return ProcessInference(['docker','exec','-i','canina-hermes','/opt/hermes/.venv/bin/python','-c',wrapper], script, cancel_remote)
+68
View File
@@ -0,0 +1,68 @@
import json, pathlib, sys, tempfile, threading, time, unittest
from unittest.mock import patch
sys.path.insert(0,str(pathlib.Path(__file__).parent))
import executor as m
import runtime
from test_contract import lease
class Cancellation(unittest.TestCase):
def test_wall_deadline_kills_descendant(self):
with tempfile.TemporaryDirectory() as root:
marker=pathlib.Path(root,'late')
child=f'import pathlib,time; time.sleep(.7); pathlib.Path({str(marker)!r}).touch()'
code=f'import subprocess,sys,time; subprocess.Popen([sys.executable,"-c",{child!r}]); time.sleep(5)'
with patch.object(runtime,'MAX_WALL_SECONDS',5.2):
with self.assertRaises(TimeoutError): m.execute(lease(),root,m.ProcessInference([sys.executable,'-c',code]),lambda _: None)
time.sleep(.8)
self.assertFalse(marker.exists())
def test_stalled_heartbeat_cancels_child(self):
with tempfile.TemporaryDirectory() as root:
marker=pathlib.Path(root,'late'); release=threading.Event(); calls=[]
def heartbeat(_):
calls.append(time.monotonic())
if len(calls)>1: release.wait(3)
code=f'import time,pathlib; time.sleep(.7); pathlib.Path({str(marker)!r}).touch()'
try:
with patch.object(runtime,'HEARTBEAT_INTERVAL',.05),patch.object(runtime,'HEARTBEAT_TIMEOUT',.1):
with self.assertRaises(TimeoutError): m.execute(lease(),root,m.ProcessInference([sys.executable,'-c',code]),heartbeat)
finally: release.set()
time.sleep(.8); self.assertFalse(marker.exists())
def test_fixture_artifact_qa_and_cost_reservation(self):
# Explicit deterministic PROCESS fixture, never reported as real model inference.
source='def clamp_quantity(quantity):\n return min(99, max(0, quantity))\n'
response=json.dumps(dict(source=source,model='Free',tools=0))
beats=[]; original=lease(); snapshot=dict(original)
def heartbeat(binding):
beats.append(time.monotonic()); binding['leaseUntil']='untrusted mutation'
with tempfile.TemporaryDirectory() as root:
infer=m.ProcessInference([sys.executable,'-c',f'import time; time.sleep(.25); print({response!r})'])
with patch.object(runtime,'HEARTBEAT_INTERVAL',.05):
_,report=m.execute(original,root,infer,heartbeat)
self.assertEqual(original,snapshot)
self.assertGreater(len(beats),2)
self.assertIsNone(report['costCents'])
self.assertEqual(report['costStatus'],'UNKNOWN_RETAIN_FULL_20_CENT_RESERVATION')
with self.assertRaises(RuntimeError):
with runtime.LeaseGuard(lease(),lambda _:None) as guard: infer.run(guard)
def test_concurrent_reuse_rejected(self):
infer=m.ProcessInference([sys.executable,"-c","import time; time.sleep(.3); print(1)"])
errors=[]
with runtime.LeaseGuard(lease(),lambda _:None) as guard:
t=threading.Thread(target=lambda:infer.run(guard)); t.start()
time.sleep(.05)
with self.assertRaises(RuntimeError): infer.run(guard)
t.join()
def test_explicit_false_heartbeat_denied(self):
with tempfile.TemporaryDirectory() as root:
with self.assertRaises(PermissionError):
m.execute(lease(),root,m.ProcessInference([sys.executable,"-c","print(1)"]),lambda _:False)
def test_callable_not_cancellable_rejected(self):
with tempfile.TemporaryDirectory() as root:
with self.assertRaises(TypeError): m.execute(lease(),root,lambda:'',lambda _:None)
if __name__=='__main__': unittest.main()
+23
View File
@@ -0,0 +1,23 @@
import datetime, importlib.util, pathlib, tempfile, unittest, uuid
p=pathlib.Path(__file__).with_name('executor.py')
spec=importlib.util.spec_from_file_location('pilot',p); m=importlib.util.module_from_spec(spec); spec.loader.exec_module(m)
qs=importlib.util.spec_from_file_location('qa',p.with_name('qa.py')); qa=importlib.util.module_from_spec(qs); qs.loader.exec_module(qa)
def lease():
d={k:str(uuid.uuid4()) for k in ('taskId','runId','leaseToken')}; d.update(environment='staging',maxCostCents=20,attempt=1,commitSha='a'*40,leaseUntil=(datetime.datetime.now(datetime.timezone.utc)+datetime.timedelta(seconds=30)).isoformat()); return d
class Contract(unittest.TestCase):
def test_missing_lease_rejected(self):
with self.assertRaises(ValueError): m.validate_lease({})
def test_bounds(self):
for k,v in [('environment','production'),('attempt',2),('maxCostCents',21),('commitSha','main'),('leaseUntil','2000-01-01T00:00:00+00:00')]:
d=lease(); d[k]=v
with self.assertRaises(ValueError): m.validate_lease(d)
def test_heartbeat_denial_prevents_inference(self):
with tempfile.TemporaryDirectory() as root:
def deny(_): raise PermissionError('killed')
with self.assertRaises(PermissionError): m.execute(lease(),root,lambda:self.fail('must not infer'),deny)
def test_qa_rejects_malicious_and_wrong_behavior(self):
for source in ['import os\n','def clamp_quantity(quantity):\n return quantity\n','def clamp_quantity(quantity):\n return __import__("os").system("true")\n']:
with tempfile.TemporaryDirectory() as root:
pathlib.Path(root,'quantity.py').write_text(source)
with self.assertRaises(ValueError): qa.verify(root)
if __name__=='__main__': unittest.main()
+24
View File
@@ -0,0 +1,24 @@
import datetime, pathlib, sys, tempfile, threading, time, unittest
from unittest.mock import patch
sys.path.insert(0, str(pathlib.Path(__file__).parent))
import executor as m
import runtime
from test_contract import lease
class Heartbeats(unittest.TestCase):
def test_ongoing_denial_kills_real_child(self):
with tempfile.TemporaryDirectory() as root:
marker=pathlib.Path(root,'late')
code=f'import time,pathlib; time.sleep(1); pathlib.Path({str(marker)!r}).touch()'
beats=[]
def beat(binding):
beats.append(time.monotonic())
if len(beats)>1: raise PermissionError('revoked')
with patch.object(runtime,'HEARTBEAT_INTERVAL',.1):
with self.assertRaises(PermissionError):
m.execute(lease(),root,m.ProcessInference([sys.executable,'-c',code]),beat)
time.sleep(1.1)
self.assertFalse(marker.exists())
self.assertGreaterEqual(len(beats),2)
if __name__=='__main__': unittest.main()