Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
209339dc57 |
No files matched your search
@@ -0,0 +1,24 @@
|
||||
name: Repair Canina Hermes Git route
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- ops/hermes-git-routing-20261106
|
||||
jobs:
|
||||
routing:
|
||||
runs-on: canina
|
||||
steps:
|
||||
- name: Apply scoped runtime routing without restart
|
||||
env:
|
||||
REPAIR_SHA: ${{ github.sha }}
|
||||
REPAIR_REF: ${{ github.ref }}
|
||||
run: |
|
||||
set -eu
|
||||
[ "$REPAIR_REF" = refs/heads/ops/hermes-git-routing-20261106 ]
|
||||
case "$REPAIR_SHA" in *[!0-9a-f]*|'') exit 2 ;; esac
|
||||
[ "${#REPAIR_SHA}" -eq 40 ]
|
||||
export http_proxy= https_proxy= HTTP_PROXY= HTTPS_PROXY=
|
||||
wget --no-proxy -qO repair.py "https://git.parsaaghayi.ir/parsa/canina/raw/commit/${REPAIR_SHA}/infra/hermes-git-routing/repair.py"
|
||||
python3 repair.py
|
||||
docker exec -i -u 10000 -e HOME=/opt/data -e HERMES_HOME=/opt/data canina-hermes python3 - --apply < repair.py
|
||||
docker exec -u 10000 -e HOME=/opt/data -e HERMES_HOME=/opt/data canina-hermes git ls-remote ssh://git@git.parsaaghayi.ir:222/parsa/canina.git refs/heads/develop
|
||||
docker exec -u 10000 -e HOME=/opt/data -e HERMES_HOME=/opt/data canina-hermes git ls-remote canina-gitea:parsa/canina.git refs/heads/develop
|
||||
@@ -1,25 +0,0 @@
|
||||
name: Owned disk maintenance only
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- ops/bounded-disk-20261119
|
||||
jobs:
|
||||
maintenance:
|
||||
runs-on: canina
|
||||
steps:
|
||||
- name: Test and install bounded maintenance without app deployment
|
||||
env:
|
||||
SHA: ${{ github.sha }}
|
||||
run: |
|
||||
set -eu
|
||||
umask 077
|
||||
case "$SHA" in *[!0-9a-f]*|'') exit 2;; esac
|
||||
[ "${#SHA}" -eq 40 ]
|
||||
mkdir -p disk-maintenance
|
||||
cd disk-maintenance
|
||||
export no_proxy=git.parsaaghayi.ir,localhost,127.0.0.1
|
||||
for name in managed-disk.py test-managed-disk.py install-managed-disk.py; do
|
||||
wget --no-proxy -qO "$name" "https://git.parsaaghayi.ir/parsa/canina/raw/commit/$SHA/scripts/$name"
|
||||
done
|
||||
python3 -B test-managed-disk.py
|
||||
tar -cf - managed-disk.py test-managed-disk.py install-managed-disk.py | ssh -F /dev/null -i /root/.ssh/id_ed25519 -p 2234 -o BatchMode=yes -o StrictHostKeyChecking=yes -o ConnectTimeout=15 devops@172.17.0.1 'sudo -n bash -c '\''set -eu; umask 077; d=$(mktemp -d /opt/services/canina/tmp/managed-disk-ci.XXXXXXXX); trap "rm -rf -- $d" EXIT; tar -xf - -C "$d"; GITHUB_ACTIONS=true python3 -B "$d/install-managed-disk.py"; python3 -B /opt/services/canina/.managed-disk/managed-disk.py; python3 -B /opt/services/canina/.managed-disk/managed-disk.py'\'''
|
||||
@@ -1,29 +0,0 @@
|
||||
# Owned disk maintenance
|
||||
|
||||
Installed only by `.gitea/workflows/managed-disk.yml` on the dedicated ops branch. This workflow never calls an application deployment. The daily system timer manages only unused >24h Docker builder cache (no --all, images, volumes or logs) and explicit marked test children under `/opt/services/canina/.managed-disk/tests`. Unmarked directories, Git/env/dependency-bearing trees, symlinks, mounts and open files are preserved. Tests use temporary roots, not privileged installer fixtures.
|
||||
|
||||
Shared heavy work entry point:
|
||||
|
||||
```
|
||||
python3 -B /opt/services/canina/.managed-disk/managed-disk.py build COMMAND ARG...
|
||||
```
|
||||
|
||||
Holds `.managed-disk/heavy.lock` for the entire subprocess and requires >=20GiB available immediately before execution. This is an installed opt-in guard, NOT a global reservation: existing application build dispatchers do not yet acquire it. Migration of each dispatcher belongs in its reviewed release CI. Do not claim shared serialization is complete until all producers adopt it. Current lower headroom intentionally rejects new guarded heavy builds.
|
||||
|
||||
No existing test artifacts are automatically marked. New disposable tests may create `.disposable-test` containing exactly `canina-managed-test-v1\n`; expiry is seven days. Tests must acquire the same lock when creating/using that namespace. Existing source/donor/archive paths are outside this policy.
|
||||
|
||||
## Container log gate
|
||||
|
||||
Fresh inspection confirms older containers including Traefik, Nexus and databases have `json-file` with empty options. The daemon already has defaults; these do not retrofit existing containers. The exact per-service Compose option for the next separately authorized release is:
|
||||
|
||||
```yaml
|
||||
logging:
|
||||
driver: json-file
|
||||
options:
|
||||
max-size: "10m"
|
||||
max-file: "3"
|
||||
```
|
||||
|
||||
Do not apply a broad override or recreate these services now. Traefik/DB/VPN downtime is not authorized. No live log truncation is permitted. Each affected service's managed Compose source must receive the scoped option through its own existing release path, followed by explicit downtime approval and a verified recreation. This document is a configuration recommendation, not an installed log-rotation change.
|
||||
|
||||
No historical backup staging, cloud history, named volume, source checkout or protected review image is retired by this maintenance.
|
||||
@@ -0,0 +1,54 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Narrow Canina gateway Git routing repair; run as UID10000 only."""
|
||||
import hashlib,json,os,pathlib,subprocess,sys,tempfile
|
||||
HOME=pathlib.Path('/opt/data')
|
||||
BLOCK='''\n# Managed Canina repository SSH route (no credential provisioning).
|
||||
Match originalhost git.parsaaghayi.ir user git
|
||||
Port 222
|
||||
IdentityFile /opt/data/.ssh/canina_gitea
|
||||
IdentitiesOnly yes
|
||||
StrictHostKeyChecking yes
|
||||
UserKnownHostsFile /opt/data/.ssh/gitea_known_hosts
|
||||
ConnectTimeout 10
|
||||
'''
|
||||
GUIDANCE='''# Canina execution preflight
|
||||
|
||||
This gateway runs as UID10000 with HOME=HERMES_HOME=/opt/data; Docker root is not the worker identity.
|
||||
Before reporting Git credentials missing, run read-only `git ls-remote canina-gitea:parsa/canina.git refs/heads/develop` from the actual worker. The dedicated existing key is /opt/data/.ssh/canina_gitea; pinned trust is /opt/data/.ssh/gitea_known_hosts. Never print/copy private keys, disable host verification, request a new key/token without testing the existing route, or embed HTTPS tokens in URLs. Browser vault login tools are for website forms, not SSH key provisioning.
|
||||
Raw repository URL: ssh://git@git.parsaaghayi.ir:222/parsa/canina.git. Diagnose URL/SSH host matching independently from repository permissions. A successful read does not establish push permission or CI publication.
|
||||
Use isolated worktrees and exact tool exit/results. Delegation dispatched is not completed; wait for delivered results, and distinguish failed access, implementation, tests, CI and live acceptance. Do not report feature completion from plans, tool presence or another agent's unsupported claim. Do not replay unrelated feature requests while repairing Hermes.
|
||||
Preserve owner approval gates, existing sessions and all unrelated profiles. All code/config repair changes must be tracked in Git and released through CI/CD; never restart gateways or deploy site features merely to test access.
|
||||
'''
|
||||
def config_probe(path,host,user='git'):
|
||||
r=subprocess.run(['ssh','-F',str(path),'-G','-l',user,host],capture_output=True,text=True,timeout=10,check=True)
|
||||
return dict(line.split(' ',1) for line in r.stdout.splitlines() if ' ' in line and line.split(' ',1)[0] in {'port','identitiesonly','stricthostkeychecking','identityfile','userknownhostsfile','hostname'})
|
||||
def test():
|
||||
with tempfile.TemporaryDirectory() as t:
|
||||
p=pathlib.Path(t)/'config'; p.write_text('Host canina-gitea\n HostName git.parsaaghayi.ir\n User git\n Port 222\n IdentityFile /opt/data/.ssh/canina_gitea\n IdentitiesOnly yes\n StrictHostKeyChecking yes\n UserKnownHostsFile /opt/data/.ssh/gitea_known_hosts\n'+BLOCK)
|
||||
d=config_probe(p,'git.parsaaghayi.ir'); assert d['port']=='222' and d['identitiesonly']=='yes' and d['stricthostkeychecking']=='true' and d['identityfile']=='/opt/data/.ssh/canina_gitea',d
|
||||
assert config_probe(p,'unrelated.example')['port']=='22'
|
||||
assert config_probe(p,'git.parsaaghayi.ir','other')['port']=='22'
|
||||
assert config_probe(p,'canina-gitea')['port']=='222'
|
||||
print('PASS: raw git route, alias route, unrelated host and unrelated user isolation')
|
||||
def apply():
|
||||
assert os.getuid()==10000 and os.getenv('HERMES_HOME')=='/opt/data'
|
||||
cfg=HOME/'.ssh/config'; before=cfg.read_text(); guide=HOME/'.hermes.md'
|
||||
assert 'Host canina-gitea\n' in before
|
||||
assert not guide.exists() or guide.read_text()==GUIDANCE,'unmanaged guidance exists'
|
||||
assert 'Match originalhost git.parsaaghayi.ir' not in before or BLOCK in before,'unmanaged raw host route'
|
||||
if BLOCK not in before:
|
||||
assert 'Host git.parsaaghayi.ir' not in before
|
||||
backup=HOME/'.ssh/config.before-canina-routing-20261106'
|
||||
if backup.exists(): assert backup.read_text()==before,'backup conflict'
|
||||
else:
|
||||
fd=os.open(backup,os.O_WRONLY|os.O_CREAT|os.O_EXCL,0o600)
|
||||
with os.fdopen(fd,'w') as f: f.write(before)
|
||||
tmp=cfg.with_suffix('.repair-new'); tmp.write_text(before+BLOCK); tmp.chmod(cfg.stat().st_mode & 0o777); os.replace(tmp,cfg)
|
||||
if not guide.exists():
|
||||
fd=os.open(guide,os.O_WRONLY|os.O_CREAT|os.O_EXCL,0o600)
|
||||
with os.fdopen(fd,'w') as f:f.write(GUIDANCE)
|
||||
d=config_probe(cfg,'git.parsaaghayi.ir'); assert d['port']=='222' and d['identityfile']=='/opt/data/.ssh/canina_gitea',d
|
||||
print(json.dumps({'status':'applied','uid':os.getuid(),'config_sha256':hashlib.sha256(cfg.read_bytes()).hexdigest(),'guidance_sha256':hashlib.sha256(guide.read_bytes()).hexdigest(),'restart':False}))
|
||||
if __name__=='__main__':
|
||||
test()
|
||||
if '--apply' in sys.argv: apply()
|
||||
@@ -1,19 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Host maintenance only; never deploy/recreate an application."""
|
||||
import os,pathlib,subprocess
|
||||
if os.environ.get('GITHUB_ACTIONS')!='true' or os.getuid()!=0 or pathlib.Path('/.dockerenv').exists():raise SystemExit('Authenticated host CI required')
|
||||
r=pathlib.Path('/opt/services/canina/.managed-disk')
|
||||
if r.is_symlink() or r.resolve()!=r:raise SystemExit('unsafe destination')
|
||||
r.mkdir(mode=0o700,exist_ok=True);(r/'tests').mkdir(mode=0o700,exist_ok=True)
|
||||
for n in ['managed-disk.py','test-managed-disk.py']:
|
||||
p=r/n
|
||||
if p.is_symlink():raise SystemExit('unsafe destination file')
|
||||
p.write_bytes(pathlib.Path(__file__).with_name(n).read_bytes());p.chmod(0o600)
|
||||
(r/'heavy.lock').touch(mode=0o600,exist_ok=True)
|
||||
subprocess.run(['python3','-B',str(r/'test-managed-disk.py')],check=True)
|
||||
u=pathlib.Path('/etc/systemd/system')
|
||||
(u/'canina-managed-disk.service').write_text('[Unit]\nDescription=Bounded owned disposable disk retention\n[Service]\nType=oneshot\nExecStart=/usr/bin/python3 -B /opt/services/canina/.managed-disk/managed-disk.py\nNice=15\nIOSchedulingClass=idle\nNoNewPrivileges=true\n')
|
||||
(u/'canina-managed-disk.timer').write_text('[Unit]\nDescription=Daily conservative disposable retention\n[Timer]\nOnCalendar=*-*-* 07:30:00 UTC\nRandomizedDelaySec=15min\nPersistent=true\n[Install]\nWantedBy=timers.target\n')
|
||||
subprocess.run(['systemctl','daemon-reload'],check=True)
|
||||
subprocess.run(['systemctl','enable','--now','canina-managed-disk.timer'],check=True)
|
||||
subprocess.run(['systemctl','is-active','--quiet','canina-managed-disk.timer'],check=True)
|
||||
@@ -1,56 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Owned opt-in test expiry, conservative unused cache, and shared build guard."""
|
||||
import pathlib,os,time,shutil,fcntl,subprocess,json,sys
|
||||
BASE=pathlib.Path('/opt/services/canina/.managed-disk')
|
||||
RESERVE=20*1024**3
|
||||
|
||||
def retire(root,now=None,protected=()):
|
||||
root=pathlib.Path(root);now=time.time() if now is None else now
|
||||
if root.is_symlink() or root.resolve()!=root:raise RuntimeError('unsafe root')
|
||||
removed=[]
|
||||
for p in sorted(root.iterdir()):
|
||||
marker=p/'.disposable-test'
|
||||
if p.is_symlink() or not p.is_dir() or p.resolve()!=p:continue
|
||||
if not marker.is_file() or marker.is_symlink() or marker.read_text()!='canina-managed-test-v1\n':continue
|
||||
if now-p.stat().st_mtime<7*86400:continue
|
||||
if any(v==str(p) or v.startswith(str(p)+'/') or str(p).startswith(v.rstrip('/')+'/') for v in protected):continue
|
||||
if os.path.ismount(p):continue
|
||||
# Unknown/source-bearing contents are never eligible; symlinks are never followed.
|
||||
if any(x.name in {'.git','.env','node_modules','vendor'} or x.is_symlink() or os.path.ismount(x) for x in p.rglob('*')):continue
|
||||
users=[]
|
||||
for proc in pathlib.Path('/proc').glob('[0-9]*'):
|
||||
for q in [proc/'cwd',proc/'root',*(proc/'fd').glob('*')]:
|
||||
try:
|
||||
v=os.readlink(q)
|
||||
if v==str(p) or v.startswith(str(p)+'/'):users.append(v)
|
||||
except OSError:pass
|
||||
if users:continue
|
||||
shutil.rmtree(p);removed.append(p.name)
|
||||
return removed
|
||||
|
||||
def busy():
|
||||
for p in pathlib.Path('/proc').glob('[0-9]*'):
|
||||
try:
|
||||
args=(p/'cmdline').read_bytes().replace(b'\0',b' ').decode(errors='replace')
|
||||
if any(x in args for x in ['docker build','compose build','next build','npm ci','npm install','composer install','rclone rcat']):return True
|
||||
except OSError:continue
|
||||
return False
|
||||
|
||||
def main():
|
||||
if BASE.is_symlink() or BASE.resolve()!=BASE:raise SystemExit('unsafe state root')
|
||||
with (BASE/'heavy.lock').open('a') as lock:
|
||||
fcntl.flock(lock,fcntl.LOCK_EX|fcntl.LOCK_NB)
|
||||
if sys.argv[1:2]==['build']:
|
||||
free=shutil.disk_usage(BASE).free
|
||||
if free<RESERVE:raise SystemExit('Insufficient available capacity: 20GiB required')
|
||||
if not sys.argv[2:]:raise SystemExit('command required')
|
||||
raise SystemExit(subprocess.call(sys.argv[2:]))
|
||||
if busy():raise SystemExit('Active heavy work: retention deferred')
|
||||
before=shutil.disk_usage(BASE).free
|
||||
ids=subprocess.check_output(['docker','ps','-aq'],text=True).split()
|
||||
containers=json.loads(subprocess.check_output(['docker','inspect',*ids],text=True)) if ids else []
|
||||
protected=[m['Source'] for c in containers for m in c['Mounts']]
|
||||
removed=retire(BASE/'tests',protected=protected)
|
||||
p=subprocess.run(['docker','builder','prune','--force','--filter','until=24h'],capture_output=True,text=True,timeout=300,check=True)
|
||||
print(json.dumps({'removed_tests':removed,'docker_cache_result':p.stdout.strip(),'available_before':before,'available_after':shutil.disk_usage(BASE).free}))
|
||||
if __name__=='__main__':main()
|
||||
@@ -1,25 +0,0 @@
|
||||
import importlib.util,pathlib,tempfile,unittest,os,time
|
||||
spec=importlib.util.spec_from_file_location('disk',pathlib.Path(__file__).with_name('managed-disk.py'))
|
||||
class Tests(unittest.TestCase):
|
||||
def test_expiry_and_escape(self):
|
||||
self.assertTrue(pathlib.Path(spec.origin).exists(),'managed retention implementation missing')
|
||||
m=importlib.util.module_from_spec(spec);spec.loader.exec_module(m)
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
r=pathlib.Path(d);old=r/'old';old.mkdir();(old/'.disposable-test').write_text('canina-managed-test-v1\n');(old/'result').write_text('fixture');os.utime(old,(1,1))
|
||||
unknown=r/'unknown';unknown.mkdir();os.utime(unknown,(1,1))
|
||||
outside=r/'source';outside.mkdir();(outside/'important').write_text('preserve');os.symlink(outside,r/'escape')
|
||||
self.assertEqual(m.retire(r,now=time.time()),['old']);self.assertEqual(m.retire(r,now=time.time()),[])
|
||||
self.assertTrue((outside/'important').exists());self.assertTrue(unknown.exists());self.assertTrue((r/'escape').is_symlink())
|
||||
def test_open_artifact_is_preserved(self):
|
||||
m=importlib.util.module_from_spec(spec);spec.loader.exec_module(m)
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
r=pathlib.Path(d);p=r/'busy';p.mkdir();(p/'.disposable-test').write_text('canina-managed-test-v1\n');f=(p/'result').open('w');os.utime(p,(1,1))
|
||||
try:self.assertEqual(m.retire(r),[])
|
||||
finally:f.close()
|
||||
self.assertEqual(m.retire(r),['busy'])
|
||||
def test_mounted_artifact_is_preserved(self):
|
||||
m=importlib.util.module_from_spec(spec);spec.loader.exec_module(m)
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
r=pathlib.Path(d);p=r/'mounted';p.mkdir();(p/'.disposable-test').write_text('canina-managed-test-v1\n');os.utime(p,(1,1))
|
||||
self.assertEqual(m.retire(r,protected=[str(p/'result')]),[])
|
||||
if __name__=='__main__':unittest.main()
|
||||
Reference in new issue
Block a user