Commit Graph

29 Commits

Author SHA1 Message Date
S
66febf69bb Persist configure_runners values incrementally 2026-03-01 08:15:32 -05:00
S
f4a6b04d14 feat: rework runner config to INI format with full field support
Replace pipe-delimited runners.conf with INI-style sections supporting
host resolution, container images, repo-scoped tokens, resource limits,
capacity, and SSH key passthrough. All defaults pulled from .env.

- Add INI parsing helpers (ini_list_sections, ini_get, ini_set) to common.sh
- Add SSH key support (UNRAID_SSH_KEY, FEDORA_SSH_KEY) to ssh_exec/scp_to
- Add .env vars: RUNNER_DEFAULT_IMAGE, RUNNER_DEFAULT_CAPACITY,
  RUNNER_DEFAULT_DATA_PATH, LOCAL_RUNNER_DATA_PATH, LOCAL_REGISTRY
- Rewrite manage_runner.sh with host/image/token resolution and resource limits
- Rewrite configure_runners.sh wizard for INI format with all 9 fields
- Update phase3 scripts to use ini_list_sections instead of pipe parsing
- Add runners.conf INI validation to preflight.sh (check 5b)
- Update templates to use resolved labels, capacity, and deploy resources

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 23:14:46 -05:00
S
fcd966f97d feat: add interactive runners.conf configuration wizard
Replaces manual pipe-delimited file editing with a guided setup
script matching the configure_env.sh UX pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 22:21:14 -05:00
S
5ce3a234f3 fix: harden phase auth and failure handling 2026-02-28 22:09:21 -05:00
S
0e0aeda658 feat: extract .env validators to common.sh and add validate_env()
Move 10 validation functions from configure_env.sh to lib/common.sh as
shared utilities. Define variable-to-validator mapping using parallel
arrays (bash 3.2 compatible). validate_env() checks all ~50 .env
variables against their expected format and reports all failures at once.

Wired into preflight.sh (Check 6b) and bitwarden_to_env.sh (post-restore).
configure_env.sh now sources validators from common.sh instead of
defining its own copies.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 22:08:01 -05:00
S
743f1281e6 chore: fix shellcheck findings across migration scripts 2026-02-28 21:39:23 -05:00
S
572e4c151c chore: snapshot current workspace changes 2026-02-28 21:35:03 -05:00
S
088e355962 docs: add README.md and USAGE_GUIDE.md
README covers architecture, 9-phase pipeline, file structure, design
decisions with rationale (bash over Ansible, single control plane,
envsubst templates, check-before-act idempotency, SQLite, mirror
marking vs archiving), and compromises (shared credentials, 3-repo
limit, syntactic workflow migration, no automatic rollback, timeout
polling, unencrypted backups, Docker socket exposure).

USAGE_GUIDE covers the happy path (automated and manual), resuming
after failure, edge cases (rate limits, token expiry, large repos,
port conflicts, DNS, Certbot, SSH, runner offline, invalid YAML),
rollback procedures (full, partial, single-phase, with cleanup),
verification commands for each rollback scenario, day-to-day ops
(version updates, token rotation, adding repos, mirror sync, SSL
renewal), backup/restore, runner management, and troubleshooting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 21:06:36 -05:00
S
316d318b5e feat: add cross-host SSH trust, state-aware teardown, and configurable migration polling
- Add setup/cross_host_ssh.sh to establish ed25519 SSH trust between
  Unraid and Fedora (required by backup/restore scripts for direct SCP)
- Add ssh_key and authorized_key cleanup handlers to setup/cleanup.sh
- Rewrite phase8 cutover to mark GitHub repos as mirrors instead of
  archiving them (archived repos reject push mirror writes), with a
  JSON state snapshot of pre-cutover settings (description, homepage,
  wiki, projects, Pages) for exact restoration on teardown
- Rewrite phase8 teardown to restore from state snapshot with fallback
  to legacy "— was:" description parsing
- Make migration polling configurable via MIGRATION_POLL_INTERVAL_SEC
  and MIGRATION_POLL_TIMEOUT_SEC in .env (was hardcoded 120s/3s)
- Fix preflight SSL validation: check SSL_MODE instead of always
  requiring SSL_EMAIL, add conditional checks per SSL_MODE
- Add preflight checks 23-24: cross-host SSH connectivity
- Add --start-from range validation and cross_host_ssh.sh to run_all.sh

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 20:50:41 -05:00
S
dc08375ad0 fix: address multiple bugs from code review
- teardown_all.sh: replace `yes |` pipeline with `< <(yes)` process
  substitution to avoid SIGPIPE (exit 141) false failures under pipefail
- phase6_teardown.sh: extract push mirror `.id` instead of `.remote_name`
  to match the DELETE /push_mirrors/{id} API contract
- phase5_migrate_pipelines.sh: expand sed regex from `[a-z_]*` to
  `[a-z_.]*` to handle nested GitHub contexts like
  `github.event.pull_request.number`
- lib/common.sh: render_template now requires explicit variable list to
  prevent envsubst from eating Nginx variables ($host, $proxy_add_...)
- backup scripts: remove MacBook relay, use direct Unraid↔Fedora SCP;
  fix dump path to write to /data/ (mounted volume) instead of /tmp/
  (container-only); add unzip -t integrity verification
- preflight.sh: add --skip-port-checks flag for resuming with
  --start-from (ports already bound by earlier phases)
- run_all.sh: update run_step to pass extra args; use --skip-port-checks
  when --start-from > 1
- post-checks (phase4/7/9): wrap API calls in helper functions with
  >/dev/null redirection instead of passing -o /dev/null as API data
- phase8: replace GitHub archiving with [MIRROR] description marking
  and disable wiki/projects/Pages (archived repos reject push mirrors)
- restore_to_primary.sh: add require_vars for Fedora SSH variables

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 20:18:35 -05:00
S
07d27f7a9c feat: add version checking and install manifest tracking
Add minimum version validation for all dependencies across local and
remote machines (jq>=1.6, curl>=7.70, git>=2.30, docker>=20.0,
compose>=2.0, shellcheck>=0.8, gh>=2.0). Setup scripts now record
every install action to .manifests/<host>.manifest files, enabling
full rollback via setup/cleanup.sh. teardown_all.sh gains --cleanup
flag to chain prerequisite removal after phase teardowns.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 19:35:09 -06:00
S
720197bb10 feat: add OS compatibility checks before running platform-specific logic
- lib/common.sh: add require_local_os, require_remote_os, require_remote_pkg_manager
- setup/macbook.sh: require macOS (Darwin)
- setup/unraid.sh: require remote is Linux
- setup/fedora.sh: require remote is Linux + has dnf (RPM-based)
- manage_runner.sh: native runner add/remove requires macOS
- run_all.sh: control plane must be macOS
- preflight.sh: 3 new checks (1: local=macOS, 2: Unraid=Linux, 3: Fedora=Linux+dnf)
- phase5_migrate_pipelines.sh: fix sed -i to be portable (no macOS-only syntax)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 19:00:13 -06:00
S
40fe847755 feat: add orchestration (run_all.sh, teardown_all.sh)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:33:51 -06:00
S
e643caa0b0 feat: add backup and restore scripts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:32:34 -06:00
S
ea5524949a feat: add Phase 9 — Security Scanning
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:30:38 -06:00
S
9379b95a41 feat: add Phase 8 — Cutover (HTTPS + Archive GitHub)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:29:14 -06:00
S
3179390af9 feat: add Phase 7 — Branch Protection
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:27:14 -06:00
S
058b85e146 feat: add Phase 6 — GitHub Push Mirrors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:27:13 -06:00
S
6e3b4c66d1 feat: add Phase 5 — Migrate Pipelines (GitHub → Gitea Actions)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:25:04 -06:00
S
e2ab5ee108 feat: add Phase 4 — Migrate Repos + Fedora mirrors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:21:50 -06:00
S
6b82752d9e feat: add Phase 3 — Runners + manage_runner.sh
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:20:12 -06:00
S
eaffb97144 feat: add Phase 2 — Gitea on Fedora (backup instance)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:16:32 -06:00
S
63f708e556 feat: add Phase 1 — Gitea on Unraid
- phase1_gitea_unraid.sh: 9-step deploy (dirs, docker-compose, app.ini,
  container start, wait, admin user, API token, save to .env, create org).
  Every step has idempotency check — running twice changes nothing.
- phase1_post_check.sh: 5 independent verification checks
- phase1_teardown.sh: stop container + optionally remove data, with prompts

Also adds inline comments to lib/common.sh and preflight.sh explaining
WHY decisions were made (SSH flags, API tmpfile pattern, port checks, etc.)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:12:02 -06:00
S
da9f56cce9 feat: add preflight validation
16 checks: .env exists, runners.conf exists, 24 required vars non-empty,
SSH to Unraid/Fedora, Docker + docker-compose on both servers, ports free,
DNS resolution, GitHub token valid, GitHub repos exist, Nginx running,
Nginx conf dir writable. Runs ALL checks even if earlier ones fail.
Exits 0 only if all 16 pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:08:37 -06:00
S
5f043cbb45 feat: add setup scripts (configure_env, macbook, unraid, fedora)
- configure_env.sh: interactive 50-prompt wizard with progress [N/50],
  input validation (IP, port, email, path, URL, bool, password, ssl_mode),
  conditional SSL prompts based on SSL_MODE, summary with masked passwords
- macbook.sh: Homebrew packages, envsubst, Xcode CLI tools, SSH tests
- unraid.sh: Docker verify, docker-compose + jq static binary install
- fedora.sh: Docker CE + compose plugin install, jq, docker group setup

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:07:34 -06:00
S
6b0e4de464 feat: add configuration templates
- docker-compose-gitea.yml.tpl: Gitea + SQLite container
- app.ini.tpl: Gitea config (INSTALL_LOCK, Actions enabled, no registration)
- docker-compose-runner.yml.tpl: act_runner Docker container (Linux)
- runner-config.yaml.tpl: act_runner config (capacity=1, timeout=3h)
- com.gitea.runner.plist.tpl: macOS launchd service for native runner
- nginx-gitea.conf.tpl: Nginx reverse proxy with SSL/WebSocket support
- workflows/security-scan.yml.tpl: Semgrep + Trivy + Gitleaks workflow

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:03:54 -06:00
S
f32e200c64 feat: add API contracts
Document all Gitea REST API v1 and GitHub REST API endpoints used across
phases 1-9. Each endpoint includes: method, path, request/response schemas,
status codes, and which script uses it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:02:28 -06:00
S
d2c0730068 feat: add shared library (lib/common.sh)
17 functions: logging (info/warn/error/success/step/phase_header),
env management (load_env/save_env_var/require_vars), SSH wrappers
(ssh_exec/ssh_check/scp_to), API wrappers (gitea_api/gitea_backup_api/
github_api), template rendering, and polling (wait_for_http/wait_for_ssh).

All logs go to stderr, JSON data to stdout. Shellcheck clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:01:28 -06:00
S
e4ed5c5879 init: project structure, .gitignore, .env.example, runners.conf.example
- .gitignore: excludes .env, runners.conf, certs, temp files, editor files
- .env.example: all configuration variables with sections and descriptions
- runners.conf.example: dynamic runner definition format (pipe-delimited)
- PLAN.md: comprehensive implementation plan with DoD for all 18 milestones
- CLAUDE.md: project conventions and instructions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 14:59:17 -06:00