privacy-in-design/gitea-migration
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
ccca97a4b395e7e8f9313736a5018ff431ac25ca
gitea-migration/PLAN.md

63 KiB
Raw Blame History

Gitea Migration Toolkit — Implementation Plan

Note

: This is the original implementation plan. Some sections describe the initial Nginx/SSL_MODE/3-repo architecture that has since been replaced by Caddy/TLS_MODE/dynamic repos with macvlan networking. See README.md and .env.example for the current architecture.

Context

Migrating GitHub repos to self-hosted Gitea (Unraid primary, Fedora backup mirror, GitHub as push mirror). All automation runs from MacBook, SSHing into remote machines. Scripts must be idempotent, .env-driven, with preflight + post-checks + teardown per phase.

Architecture Decisions

Decision Choice Rationale
Execution model All scripts run from MacBook, SSH into remotes Single control plane, no agent installs needed
Gitea deployment Docker Compose on both Unraid + Fedora Scriptable, reproducible, version-pinned
Runner deployment (Linux) Docker container via docker-compose Consistent with Gitea deployment
Runner deployment (macOS) Native binary + launchd plist Docker Desktop on Mac is heavyweight; native binary is lighter and more reliable
Idempotency pattern Check-before-act (query state, skip if already done) Every create operation first checks if the resource exists
Template rendering .tpl files + envsubst Keeps config templates separate from script logic
Error handling set -euo pipefail + trap cleanup Fail fast, no silent errors
API interaction Shared gitea_api() curl wrapper with JSON/jq Consistent auth, error checking, response parsing
HTTPS proxy Caddy reverse proxy with Cloudflare DNS-01 or existing certs Automatic TLS with zero-touch renewal; each host gets a dedicated macvlan IP

File Structure

gitea-migration/
├── .env.example
├── .env
├── runners.conf.example
├── runners.conf
├── PLAN.md
├── lib/
│   └── common.sh
├── setup/
│   ├── configure_env.sh
│   ├── configure_runners.sh
│   ├── macbook.sh
│   ├── unraid.sh
│   ├── fedora.sh
│   ├── cross_host_ssh.sh
│   ├── env_to_bitwarden.sh
│   ├── bitwarden_to_env.sh
│   └── cleanup.sh
├── templates/
│   ├── docker-compose-gitea.yml.tpl
│   ├── docker-compose-runner.yml.tpl
│   ├── app.ini.tpl
│   ├── runner-config.yaml.tpl
│   ├── com.gitea.runner.plist.tpl
│   ├── Caddyfile.tpl
│   ├── docker-compose-caddy.yml.tpl
│   ├── com.gitea.runner.newsyslog.conf.tpl
│   └── workflows/
│       └── security-scan.yml.tpl
├── backup/
│   ├── backup_primary.sh
│   └── restore_to_primary.sh
├── contracts/
│   └── gitea-api.md
├── preflight.sh
├── phase1_gitea_unraid.sh
├── phase1_post_check.sh
├── phase1_teardown.sh
├── phase2_gitea_fedora.sh
├── phase2_post_check.sh
├── phase2_teardown.sh
├── phase3_runners.sh
├── phase3_post_check.sh
├── phase3_teardown.sh
├── phase4_migrate_repos.sh
├── phase4_post_check.sh
├── phase4_teardown.sh
├── phase5_migrate_pipelines.sh
├── phase5_post_check.sh
├── phase5_teardown.sh
├── phase6_github_mirrors.sh
├── phase6_post_check.sh
├── phase6_teardown.sh
├── phase7_branch_protection.sh
├── phase7_post_check.sh
├── phase7_teardown.sh
├── phase8_cutover.sh
├── phase8_post_check.sh
├── phase8_teardown.sh
├── phase9_security.sh
├── phase9_post_check.sh
├── phase9_teardown.sh
├── manage_runner.sh
├── run_all.sh
└── teardown_all.sh

Implementation Tracker

1. Foundation

# File Description Status
1.1 lib/common.sh Shared functions: logging, SSH, API wrappers, template rendering, checks DONE
1.2 .env.example All env vars: TLS_MODE, macvlan networking, DB support, Caddy config DONE
1.3 contracts/gitea-api.md Gitea REST API endpoints used across all phases DONE

2. Templates

# File Description Status
2.1 templates/docker-compose-gitea.yml.tpl Gitea + DB docker-compose (sqlite3/mysql/postgres/mssql) DONE
2.2 templates/app.ini.tpl Gitea custom config (INSTALL_LOCK, Actions enabled, etc.) DONE
2.3 templates/docker-compose-runner.yml.tpl act_runner docker-compose (Linux) DONE
2.4 templates/runner-config.yaml.tpl act_runner config DONE
2.5 templates/com.gitea.runner.plist.tpl macOS launchd service for act_runner DONE
2.6 templates/com.gitea.runner.newsyslog.conf.tpl macOS log rotation for native runner DONE
2.7 templates/Caddyfile.tpl + docker-compose-caddy.yml.tpl Caddy reverse proxy with Cloudflare DNS-01 DONE
2.8 templates/workflows/security-scan.yml.tpl Semgrep + Trivy + Gitleaks workflow DONE

3. Machine Setup

# File Description Status
3.1 setup/configure_env.sh Interactive wizard: prompts for each .env var, writes to .env DONE
3.2 setup/macbook.sh Homebrew, jq, curl, envsubst, git, Xcode CLI Tools, shellcheck, gh DONE
3.3 setup/unraid.sh Verify Docker, install docker-compose + jq (static binary) DONE
3.4 setup/fedora.sh Install Docker CE, compose plugin, jq, enable systemd services DONE
3.5 setup/configure_runners.sh Interactive runner definition wizard, writes runners.conf DONE
3.6 setup/cross_host_ssh.sh SSH key exchange between Unraid and Fedora DONE
3.7 setup/env_to_bitwarden.sh Export .env to Bitwarden JSON import format DONE
3.8 setup/bitwarden_to_env.sh Restore .env from Bitwarden CLI DONE
3.9 setup/cleanup.sh Manifest-driven rollback of setup scripts DONE

4. Preflight

# File Description Status
4.1 preflight.sh Validate .env, SSH, Docker, IPs, DNS, GitHub token, Caddy, repos DONE

5. Phase 1 — Gitea on Unraid

# File Description Status
5.1 phase1_gitea_unraid.sh Deploy Gitea container, create admin user + token + org DONE
5.2 phase1_post_check.sh Verify Gitea HTTP 200, admin auth, token valid, org exists DONE
5.3 phase1_teardown.sh docker-compose down, optionally remove data DONE

6. Phase 2 — Gitea on Fedora

# File Description Status
6.1 phase2_gitea_fedora.sh Deploy Gitea container on Fedora, create admin user + token DONE
6.2 phase2_post_check.sh Verify Fedora Gitea HTTP 200, admin auth, token valid DONE
6.3 phase2_teardown.sh docker-compose down on Fedora DONE

7. Phase 3 — Runners

# File Description Status
7.1 runners.conf.example Runner definition format + example entries DONE
7.2 manage_runner.sh Add/remove/list runners dynamically (reads runners.conf) DONE
7.3 phase3_runners.sh Get registration token, deploy all runners defined in runners.conf DONE
7.4 phase3_post_check.sh Verify all runners from runners.conf are online in Gitea admin DONE
7.5 phase3_teardown.sh Stop + deregister all runners from runners.conf DONE

8. Phase 4 — Migrate Repos + Fedora Mirrors

# File Description Status
8.1 phase4_migrate_repos.sh Import repos from GitHub, set up Fedora pull mirrors DONE
8.2 phase4_post_check.sh Verify repos on primary + mirror repos on Fedora DONE
8.3 phase4_teardown.sh Delete repos from primary + Fedora DONE

9. Phase 5 — Migrate Pipelines

# File Description Status
9.1 phase5_migrate_pipelines.sh Copy .github/workflows/ → .gitea/workflows/, apply compat fixes DONE
9.2 phase5_post_check.sh Verify workflows visible in Gitea Actions UI DONE
9.3 phase5_teardown.sh Remove .gitea/workflows/ from repos DONE

10. Phase 6 — GitHub Push Mirrors

# File Description Status
10.1 phase6_github_mirrors.sh Configure push mirrors from Gitea → GitHub DONE
10.2 phase6_post_check.sh Verify mirror config, trigger sync, check GitHub DONE
10.3 phase6_teardown.sh Remove push mirror config DONE

11. Phase 7 — Branch Protection

# File Description Status
11.1 phase7_branch_protection.sh Set up branch protection rules on all repos DONE
11.2 phase7_post_check.sh Verify protection rules exist DONE
11.3 phase7_teardown.sh Delete branch protection rules DONE

12. Phase 8 — Cutover (HTTPS + Archive GitHub)

# File Description Status
12.1 phase8_cutover.sh Caddy HTTPS reverse proxy + mark GitHub repos as mirrors DONE
12.2 phase8_post_check.sh Verify HTTPS, repos accessible, mirrors working DONE
12.3 phase8_teardown.sh Remove Caddy stack, restore GitHub repo settings DONE

13. Phase 9 — Security Scanning

# File Description Status
13.1 phase9_security.sh Deploy security workflow (Semgrep+Trivy+Gitleaks) to all repos DONE
13.2 phase9_post_check.sh Verify workflows exist, dry-run passes, branch protection updated DONE
13.3 phase9_teardown.sh Remove security workflows DONE

14. Backup & Restore (post-migration operational scripts)

# File Description Status
14.1 backup/backup_primary.sh Run gitea dump on Unraid (DB + repos + config + users), SCP archive to Fedora DONE
14.2 backup/restore_to_primary.sh Restore a gitea dump archive to Unraid (fresh or existing instance) DONE

15. Orchestration

# File Description Status
15.1 run_all.sh Run setup → preflight → phases 1-9 sequentially, --start-from=N DONE
15.2 teardown_all.sh Run teardowns in reverse, --through=N DONE

16. Project Init

# File Description Status
16.1 Git repo init + .gitignore Initialize git repo, ignore .env and temp files DONE
16.2 CLAUDE.md Project-specific instructions for this codebase DONE

17. Validation

# File Description Status
17.1 Shellcheck all .sh files Must pass with no errors DONE
17.2 bash -n syntax check all scripts Verify syntax without executing DONE

Definition of Done — Every Item

1.1 — lib/common.sh

Depends on: nothing Steps:

  1. Write shell library with set -euo pipefail
  2. Implement every function listed below
  3. Every function handles errors (non-zero exit, clear message)

Functions and their contracts:

Function Inputs Behavior Returns
load_env none Sources .env, exports all vars. Exits 1 if .env missing. 0 on success
save_env_var KEY VALUE key name, value string If KEY exists in .env, replaces its line. If not, appends. Must not corrupt other lines. 0 on success
require_vars VAR1 VAR2... variable names For each: checks if exported and non-empty. Exits 1 naming the first missing var. 0 if all set
log_info MSG message string Prints [INFO] MSG to stderr in blue —
log_warn MSG message string Prints [WARN] MSG to stderr in yellow —
log_error MSG message string Prints [ERROR] MSG to stderr in red —
log_success MSG message string Prints [OK] MSG to stderr in green —
log_step N MSG step number, message Prints [N] MSG to stderr —
phase_header NUM NAME phase number, name Prints \n=== Phase NUM: NAME ===\n to stderr —
ssh_exec HOST_KEY CMD host key (e.g. UNRAID), command string Reads ${HOST_KEY}_IP, ${HOST_KEY}_SSH_USER, ${HOST_KEY}_SSH_PORT from env. Runs command via SSH with -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new. Streams stdout/stderr. Exits with remote exit code. remote exit code
ssh_check HOST_KEY host key Same as ssh_exec but runs true. Returns 0 if reachable, 1 if not. Does NOT exit on failure. 0 or 1
scp_to HOST_KEY SRC DST host key, local file path, remote file path Copies local file to remote via SCP. Exits 1 if transfer fails. 0 on success
gitea_api METHOD PATH [DATA] HTTP method, API path (e.g. /orgs), optional JSON body Calls curl -s -X METHOD ${GITEA_INTERNAL_URL}/api/v1${PATH} with Authorization: token ${GITEA_ADMIN_TOKEN}. Exits 1 if HTTP status >= 400 (prints status + response body). Returns JSON response on stdout. JSON on stdout
gitea_backup_api METHOD PATH [DATA] same Same as gitea_api but uses GITEA_BACKUP_INTERNAL_URL + GITEA_BACKUP_ADMIN_TOKEN JSON on stdout
github_api METHOD PATH [DATA] same Calls https://api.github.com${PATH} with Authorization: token ${GITHUB_TOKEN} JSON on stdout
render_template SRC DEST .tpl file path, output path Runs envsubst with all exported vars. Writes to DEST. 0 on success
wait_for_http URL MAX_SECS URL, timeout in seconds Polls with curl -sf -o /dev/null every 2 seconds. Exits 1 with timeout message if MAX_SECS exceeded. 0 when URL returns 200
wait_for_ssh HOST_KEY MAX_SECS host key, timeout Polls ssh_check every 2 seconds. Exits 1 on timeout. 0 when SSH connects

Done when:

  • Every function above exists with exact signature
  • shellcheck lib/common.sh passes with zero warnings
  • bash -n lib/common.sh passes
  • Sourcing the file (source lib/common.sh) does NOT execute anything — functions only, no side effects at source time
  • save_env_var tested: set a var, read it back, value matches. Set it again with different value, only one line exists for that key.
  • require_vars tested: prints the missing var's name, not a generic error
  • ssh_exec uses ConnectTimeout to avoid hanging forever
  • API functions return raw JSON on stdout (not mixed with log messages — logs go to stderr)

1.3 — contracts/gitea-api.md

Depends on: knowing all API calls used across phases 1-9 Steps:

  1. For every Gitea API call in the project, document: method, path, request body, expected response, which script uses it
  2. For every GitHub API call, same treatment
  3. Cross-reference: every gitea_api/github_api call in any script MUST have a matching entry

API endpoints to document (complete list):

Endpoint Used in
POST /api/v1/users/{user}/tokens Phase 1, Phase 2 (generate admin token)
GET /api/v1/user Preflight (validate token)
POST /api/v1/orgs Phase 1 (create org)
GET /api/v1/orgs/{org} Phase 1 (check if org exists)
POST /api/v1/repos/migrate Phase 4 (import repos + Fedora mirrors)
GET /api/v1/repos/{owner}/{repo} Phase 4 post-check (verify repo exists)
GET /api/v1/admin/runners/registration-token Phase 3 (get runner token)
GET /api/v1/admin/runners Phase 3 post-check (list runners)
POST /api/v1/repos/{owner}/{repo}/push_mirrors Phase 6 (create push mirror)
GET /api/v1/repos/{owner}/{repo}/push_mirrors Phase 6 post-check
POST /api/v1/repos/{owner}/{repo}/push_mirrors-sync Phase 6 post-check (trigger sync)
POST /api/v1/repos/{owner}/{repo}/branch_protections Phase 7
GET /api/v1/repos/{owner}/{repo}/branch_protections Phase 7 post-check
DELETE /api/v1/repos/{owner}/{repo}/branch_protections/{name} Phase 7 teardown
DELETE /api/v1/repos/{owner}/{repo} Phase 4 teardown
GET /api/v1/repos/{owner}/{repo}/actions/workflows Phase 5 post-check
GitHub: GET /user Preflight (validate GitHub token)
GitHub: GET /repos/{owner}/{repo} Preflight (verify repos exist)
GitHub: PATCH /repos/{owner}/{repo} Phase 8 (archive repos)

Done when:

  • Every endpoint above is documented with: method, full path, request body schema, response schema, HTTP status codes
  • Every gitea_api/github_api call in the codebase has a corresponding entry
  • No endpoint is used in code but missing from the contract

2.1 — templates/docker-compose-gitea.yml.tpl

Depends on: .env vars defined Produces: Docker Compose file for Gitea + SQLite on a single host

Template must include:

  • Gitea image pinned to $GITEA_VERSION
  • Container name: gitea
  • Volumes: $DATA_PATH/data:/data, $DATA_PATH/config:/data/gitea/conf
  • Networks: macvlan with $GITEA_CONTAINER_IP (dedicated LAN IP, no port mapping)
  • Environment: USER_UID=1000, USER_GID=1000
  • Restart policy: unless-stopped
  • No database service for sqlite3; conditional DB service block for external DBs

Variables used: GITEA_VERSION, DATA_PATH, GITEA_CONTAINER_IP (+ DB vars if external DB)

Done when:

  • render_template produces valid YAML (test with python3 -c "import yaml; yaml.safe_load(open('output.yml'))" or yq)
  • No unexpanded $VAR or ${VAR} tokens in rendered output
  • Container name is gitea (hardcoded, not variable — needed for docker exec)
  • Volume paths use the variable, not hardcoded paths

2.2 — templates/app.ini.tpl

Depends on: .env vars defined Produces: Gitea configuration file

Must set:

  • [security] INSTALL_LOCK = true — skip install wizard
  • [server] ROOT_URL = https://$GITEA_DOMAIN/
  • [server] SSH_DOMAIN = $GITEA_DOMAIN
  • [server] DOMAIN = $GITEA_DOMAIN
  • [database] DB_TYPE = $GITEA_DB_TYPE
  • [database] PATH = /data/gitea/gitea.db
  • [service] DISABLE_REGISTRATION = true — no public signups
  • [actions] ENABLED = true — enable Gitea Actions
  • [repository] DEFAULT_BRANCH = main
  • [mailer] section left unconfigured (can be added later)

Done when:

  • Rendered output is valid INI format
  • INSTALL_LOCK = true is present (critical — without it, Gitea shows install wizard)
  • ENABLED = true under [actions] (critical — without it, runners can't connect)
  • No unexpanded variables

2.3 — templates/docker-compose-runner.yml.tpl

Depends on: .env vars, runner config Produces: Docker Compose file for act_runner on Linux

Must include:

  • act_runner image: gitea/act_runner:$ACT_RUNNER_VERSION
  • Container name: gitea-runner-$RUNNER_NAME
  • Volume: Docker socket mount /var/run/docker.sock:/var/run/docker.sock (needed to spawn job containers)
  • Volume: $RUNNER_DATA_PATH:/data
  • Environment: GITEA_INSTANCE_URL, GITEA_RUNNER_REGISTRATION_TOKEN, GITEA_RUNNER_NAME, GITEA_RUNNER_LABELS
  • Restart policy: unless-stopped

Done when:

  • Rendered output is valid YAML
  • Docker socket is mounted (without this, runner can't create job containers)
  • Runner name and labels come from variables (not hardcoded)

2.4 — templates/runner-config.yaml.tpl

Depends on: act_runner config schema Produces: act_runner configuration file

Must set:

  • runner.name: $RUNNER_NAME
  • runner.labels: $RUNNER_LABELS
  • runner.capacity: 1 (one concurrent job per runner — safe default)
  • runner.timeout: 3h (max job duration)
  • cache.enabled: true

Done when:

  • Valid YAML after rendering
  • Labels format is correct for act_runner (comma-separated label:scheme pairs)

2.5 — templates/com.gitea.runner.plist.tpl

Depends on: macOS launchd plist schema Produces: launchd service definition for act_runner on macOS

Must include:

  • Label: com.gitea.runner.$RUNNER_NAME
  • ProgramArguments: path to act_runner binary, daemon subcommand
  • WorkingDirectory: $RUNNER_DATA_PATH
  • RunAtLoad: true
  • KeepAlive: true
  • StandardOutPath / StandardErrorPath: log file locations

Done when:

  • Valid XML plist (test with plutil -lint)
  • Binary path matches where manage_runner.sh installs it
  • KeepAlive ensures runner restarts if it crashes

2.6 — templates/Caddyfile.tpl + docker-compose-caddy.yml.tpl

Depends on: .env vars (GITEA_DOMAIN, TLS_BLOCK, GITEA_CONTAINER_IP, CADDY_DATA_PATH, CADDY_CONTAINER_IP) Produces: Caddy reverse proxy config + Docker Compose for Caddy container

Caddyfile must include:

  • ${GITEA_DOMAIN} as the site address
  • ${TLS_BLOCK} placeholder (script sets to tls { dns cloudflare {env.CF_API_TOKEN} } or tls /path/cert /path/key)
  • reverse_proxy ${GITEA_CONTAINER_IP}:3000

Docker Compose must include:

  • slothcroissant/caddy-cloudflaredns:latest image
  • Volume mounts for Caddyfile, data, and config
  • macvlan network with static IP (CADDY_CONTAINER_IP)
  • Conditional CF_API_TOKEN env var and cert volume mounts based on TLS mode

Done when:

  • Caddy starts and obtains TLS certificate
  • HTTPS proxy works to Gitea
  • HTTP redirects to HTTPS

2.7 — templates/workflows/security-scan.yml.tpl

Depends on: Gitea Actions workflow syntax Produces: Shared security workflow for all repos

Must include:

  • on: [pull_request] trigger
  • Three jobs: semgrep, trivy, gitleaks
  • Semgrep: returntocorp/semgrep:$SEMGREP_VERSION Docker image, semgrep scan --config auto .
  • Trivy: aquasec/trivy:$TRIVY_VERSION, trivy fs --exit-code 1 --severity HIGH,CRITICAL .
  • Gitleaks: zricethezav/gitleaks:$GITLEAKS_VERSION, gitleaks detect --source . --exit-code 1
  • All three jobs must report as status checks (default behavior in Gitea Actions)

Done when:

  • Valid YAML after rendering
  • All three tools pinned to version vars (not latest hardcoded)
  • Each job exits non-zero on findings (so branch protection can block merge)
  • runs-on labels match what Linux runners advertise in runners.conf

3.1 — setup/configure_env.sh

Depends on: .env.example exists Runs: locally on MacBook (interactive — requires terminal input) Purpose: Guided wizard that prompts for every required .env variable, validates input, and writes a complete .env file.

Behavior:

  1. If .env already exists, load current values as defaults (shown in brackets, press Enter to keep)
  2. If .env does not exist, copy .env.example to .env first
  3. Walk through each required variable in section order, prompting with:
    • Progress indicator: [12/47] ── GITEA SHARED CREDENTIALS ──────────────────
    • Variable name and description (from .env.example comments)
    • Current value if any (from existing .env): UNRAID_IP [192.168.1.10]:
    • Empty prompt if no current value: UNRAID_IP (Static IP of Unraid server):
  4. Basic input validation per variable:
    • IP addresses: regex match ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$
    • Ports: integer between 1-65535
    • Passwords: minimum 8 characters
    • Emails: contains @
    • Paths: starts with /
    • URLs: starts with http:// or https://
  5. After all prompts, write values to .env preserving the file structure (comments, sections)
  6. Do NOT prompt for auto-populated vars (GITEA_ADMIN_TOKEN, GITEA_BACKUP_ADMIN_TOKEN, GITEA_RUNNER_REGISTRATION_TOKEN)
  7. Do NOT prompt for vars with defaults unless user wants to change them — show default, press Enter to accept
  8. Print summary at end showing all configured values (passwords masked)

Variable prompt order (matches .env.example sections):

# Variable Validation Default
1-5 UNRAID_IP, UNRAID_SSH_USER, UNRAID_SSH_PORT, UNRAID_GITEA_DATA_PATH, UNRAID_SSH_KEY IP, non-empty, port, path, optional —, —, 22, —, —
6-10 FEDORA_IP, FEDORA_SSH_USER, FEDORA_SSH_PORT, FEDORA_GITEA_DATA_PATH, FEDORA_SSH_KEY IP, non-empty, port, path, optional —, —, 22, —, —
11-24 Macvlan networking: *_MACVLAN_PARENT, *_MACVLAN_SUBNET, *_MACVLAN_GATEWAY, *_MACVLAN_IP_RANGE, *_GITEA_IP, *_DB_IP, UNRAID_CADDY_IP (per host) non-empty, non-empty, IP, non-empty, IP, optional, IP —
25-30 GITEA_ADMIN_USER, GITEA_ADMIN_PASSWORD, GITEA_ADMIN_EMAIL, GITEA_ORG_NAME, GITEA_INSTANCE_NAME, GITEA_DB_TYPE non-empty, password, email, non-empty, non-empty, db_type —, —, —, —, —, sqlite3
31-35 (conditional, only if DB_TYPE != sqlite3) GITEA_DB_HOST, GITEA_DB_PORT, GITEA_DB_NAME, GITEA_DB_USER, GITEA_DB_PASSWD non-empty, port, non-empty, non-empty, password —, auto, gitea, gitea, —
36-37 GITEA_VERSION, ACT_RUNNER_VERSION Non-empty 1.25, 0.3.0
38-39 GITEA_DOMAIN, GITEA_INTERNAL_URL Non-empty, URL —
40-43 GITEA_BACKUP_INTERNAL_URL, GITEA_BACKUP_MIRROR_INTERVAL, BACKUP_STORAGE_PATH, BACKUP_RETENTION_COUNT URL, non-empty, path, integer —, 8h, —, 5
44-45 GITHUB_USERNAME, GITHUB_TOKEN Non-empty —
46 "How many repos?" + N × repo names → REPO_NAMES positive integer, non-empty —
47-53 MIGRATE_ISSUES, MIGRATE_LABELS, MIGRATE_MILESTONES, MIGRATE_WIKI, MIGRATION_POLL_INTERVAL_SEC, MIGRATION_POLL_TIMEOUT_SEC, GITHUB_MIRROR_INTERVAL bool, bool, bool, bool, positive_integer, positive_integer, non-empty false, true, false, false, 3, 600, 8h
54-55 RUNNER_DEFAULT_IMAGE, LOCAL_REGISTRY non-empty, optional catthehacker/ubuntu:act-latest, —
56-58 TLS_MODE, CADDY_DOMAIN, CADDY_DATA_PATH tls_mode, non-empty, path cloudflare, —, —
59-61 (conditional) CLOUDFLARE_API_TOKEN or SSL_CERT_PATH + SSL_KEY_PATH non-empty / path —
62-64 PROTECTED_BRANCH, REQUIRE_PR_REVIEW, REQUIRED_APPROVALS Non-empty, bool, integer main, false, 1
65-68 SEMGREP_VERSION, TRIVY_VERSION, GITLEAKS_VERSION, SECURITY_FAIL_ON_ERROR Non-empty, non-empty, non-empty, bool latest, latest, latest, true

Done when:

  • Each prompt shows progress: [N/~68] with section header when entering a new section
  • Running with no existing .env walks through all prompts and produces a valid .env
  • TLS prompts are conditional: if TLS_MODE=cloudflare, prompt for CLOUDFLARE_API_TOKEN only; if TLS_MODE=existing, prompt for SSL_CERT_PATH and SSL_KEY_PATH only
  • DB prompts are conditional: if GITEA_DB_TYPE is not sqlite3, prompt for host/port/name/user/password
  • Running with an existing .env shows current values and only overwrites what user changes
  • Invalid input (bad IP, path not starting with /, password too short) re-prompts with error message
  • Auto-populated vars are skipped entirely (no prompt, no overwrite)
  • Summary at end shows all values (passwords masked with ****)
  • .env file preserves comment structure from .env.example
  • shellcheck setup/configure_env.sh passes

3.2 — setup/macbook.sh

Depends on: nothing (first script to run) Runs: locally on MacBook

Steps:

  1. Check for Homebrew → if missing, print install command and exit (don't auto-install — it's interactive)
  2. brew install jq curl gettext shellcheck gh — skip any already installed
  3. Verify built-in tools exist: ssh, git, scp, envsubst (from gettext)
  4. Check Xcode CLI Tools: xcode-select -p → if fails, run xcode-select --install and wait
  5. Print summary of what was installed vs already present

Idempotency: Running twice produces no errors and no redundant installs.

Done when:

  • Running on a Mac where all tools exist prints "All prerequisites satisfied" and exits 0
  • Running on a Mac missing jq installs it via brew and exits 0
  • Script does NOT install Homebrew automatically (security — user should do it themselves)
  • envsubst --version works after script runs (this is the most commonly missing tool)
  • shellcheck setup/macbook.sh passes

3.3 — setup/unraid.sh

Depends on: SSH access to Unraid (.env vars: UNRAID_IP, UNRAID_SSH_USER, UNRAID_SSH_PORT) Runs: from MacBook via SSH into Unraid

Steps:

  1. ssh_check UNRAID — fail if can't connect
  2. Check Docker: ssh_exec UNRAID "docker --version" — if missing, exit 1 with message "Docker not found on Unraid. Install it via Unraid's Docker settings."
  3. Check docker-compose: try docker compose version, then docker-compose --version. If neither works, download standalone docker-compose binary to /usr/local/bin/
  4. Check jq: ssh_exec UNRAID "jq --version" — if missing, download static binary from GitHub releases to /usr/local/bin/jq, chmod +x
  5. Verify data path: ssh_exec UNRAID "mkdir -p $UNRAID_GITEA_DATA_PATH && touch $UNRAID_GITEA_DATA_PATH/.write-test && rm $UNRAID_GITEA_DATA_PATH/.write-test"

Done when:

  • docker --version works on Unraid via SSH
  • docker compose version OR docker-compose --version works on Unraid via SSH
  • jq --version works on Unraid via SSH
  • UNRAID_GITEA_DATA_PATH exists and is writable
  • Script does NOT attempt to install Docker on Unraid (could break Unraid's custom setup)
  • shellcheck setup/unraid.sh passes

3.4 — setup/fedora.sh

Depends on: SSH access to Fedora (.env vars: FEDORA_IP, FEDORA_SSH_USER, FEDORA_SSH_PORT) Runs: from MacBook via SSH into Fedora

Steps:

  1. ssh_check FEDORA — fail if can't connect
  2. Check Docker: docker --version — if missing:
    • sudo dnf -y install dnf-plugins-core
    • sudo dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo
    • sudo dnf -y install docker-ce docker-ce-cli containerd.io docker-compose-plugin
    • sudo systemctl enable --now docker
    • sudo usermod -aG docker $FEDORA_SSH_USER
    • Print warning: "User added to docker group. You may need to re-login for this to take effect."
  3. Check jq: if missing, sudo dnf -y install jq
  4. Verify Docker works: docker run --rm hello-world (must succeed without sudo after group membership)
  5. Verify data path writable (same as Unraid step)

Done when:

  • docker --version works on Fedora via SSH
  • docker compose version works on Fedora via SSH
  • jq --version works on Fedora via SSH
  • docker run --rm hello-world succeeds without sudo (if it requires sudo, the docker group membership hasn't taken effect — script must warn about re-login)
  • FEDORA_GITEA_DATA_PATH exists and is writable
  • shellcheck setup/fedora.sh passes

4.1 — preflight.sh

Depends on: setup scripts completed, .env populated Runs: locally on MacBook Purpose: Pure validation. Installs nothing. Exits 0 only if EVERYTHING is ready.

Checks (each prints PASS/FAIL with specific message):

# Check Pass condition Fail message
1 .env exists File present in project root ".env not found. Copy .env.example to .env and fill in values."
2 runners.conf exists File present in project root "runners.conf not found. Copy runners.conf.example to runners.conf."
3 Required .env vars set Every var in the list below is non-empty "Missing required var: VAR_NAME"

Check #3 — Required variables (must be non-empty):

Section Variable
Unraid UNRAID_IP
Unraid UNRAID_SSH_USER
Unraid UNRAID_GITEA_DATA_PATH
Fedora FEDORA_IP
Fedora FEDORA_SSH_USER
Fedora FEDORA_GITEA_DATA_PATH
Shared creds GITEA_ADMIN_USER
Shared creds GITEA_ADMIN_PASSWORD
Shared creds GITEA_ADMIN_EMAIL
Shared creds GITEA_ORG_NAME
Shared creds GITEA_INSTANCE_NAME
Primary GITEA_DOMAIN
Primary GITEA_INTERNAL_URL
Backup GITEA_BACKUP_INTERNAL_URL
Backup BACKUP_STORAGE_PATH
Repos GITHUB_USERNAME
Repos GITHUB_TOKEN
Repos REPO_NAMES
Mirror GITHUB_TOKEN
TLS/Caddy TLS_MODE
TLS/Caddy CADDY_DOMAIN
TLS/Caddy CADDY_DATA_PATH

Not checked (have defaults or auto-populated): UNRAID_SSH_PORT, FEDORA_SSH_PORT, GITEA_DB_TYPE, GITEA_VERSION, ACT_RUNNER_VERSION, GITEA_BACKUP_MIRROR_INTERVAL, BACKUP_RETENTION_COUNT, MIGRATE_*, GITHUB_MIRROR_INTERVAL, PROTECTED_BRANCH, REQUIRE_PR_REVIEW, REQUIRED_APPROVALS, SEMGREP_VERSION, TRIVY_VERSION, GITLEAKS_VERSION, SECURITY_FAIL_ON_ERROR, GITEA_ADMIN_TOKEN, GITEA_BACKUP_ADMIN_TOKEN, GITEA_RUNNER_REGISTRATION_TOKEN | 4 | SSH to Unraid | ssh_check UNRAID returns 0 | "Cannot SSH to Unraid at $UNRAID_IP. Run setup/unraid.sh or check SSH config." | | 5 | SSH to Fedora | ssh_check FEDORA returns 0 | Same pattern | | 6 | Docker on Unraid | ssh_exec UNRAID "docker --version" exits 0 | "Docker not found on Unraid. Run setup/unraid.sh." | | 7 | Docker on Fedora | Same | Same | | 8 | docker-compose on Unraid | ssh_exec UNRAID "docker compose version" or docker-compose --version | "docker-compose not found on Unraid. Run setup/unraid.sh." | | 9 | docker-compose on Fedora | Same | Same | | 10 | Container IPs available | Ping-check UNRAID_GITEA_IP, UNRAID_CADDY_IP, FEDORA_GITEA_IP — warn if responding | "IP $ip is already responding to ping (may be in use)." | | 14 | DNS resolves | python3 socket.getaddrinfo($GITEA_DOMAIN) returns $UNRAID_IP | "$GITEA_DOMAIN does not resolve to $UNRAID_IP." | | 15 | GitHub token valid | curl https://api.github.com/user returns 200 | "GitHub token invalid. Check GITHUB_TOKEN in .env." | | 16 | GitHub repos exist | For each repo in REPO_NAMES: curl /repos/$GITHUB_USERNAME/$repo returns 200 | "GitHub repo $repo not found under $GITHUB_USERNAME." | | 17 | Caddy data path writable | ssh_exec UNRAID "test -w $CADDY_DATA_PATH" or parent dir writable | "Caddy data path $CADDY_DATA_PATH not writable on Unraid." | | 18-20 | Tool minimum versions | Local, Unraid, Fedora: jq>=1.6, curl>=7.70, git>=2.30, docker>=20, compose>=2 | Version-specific error messages | | 21-22 | Cross-host SSH | Unraid→Fedora and Fedora→Unraid SSH with key auth | "Cannot SSH between hosts. Run setup/cross_host_ssh.sh." |

Exit behavior: Runs ALL checks (doesn't stop at first failure). Prints summary at end. Exits 0 if all pass, 1 if any fail.

Done when:

  • Every check in the table above is implemented
  • Failed checks point to the correct setup script or config to fix
  • All checks run even if earlier ones fail (user sees full picture)
  • Exit code is 1 if ANY check fails, 0 only if ALL pass
  • shellcheck preflight.sh passes

5.1 — phase1_gitea_unraid.sh

Depends on: preflight passed, templates exist Produces: Running Gitea instance on Unraid, admin user, API token in .env, org created require_vars: UNRAID_IP, UNRAID_SSH_USER, UNRAID_SSH_PORT, UNRAID_GITEA_DATA_PATH, UNRAID_MACVLAN_PARENT, UNRAID_MACVLAN_SUBNET, UNRAID_MACVLAN_GATEWAY, UNRAID_MACVLAN_IP_RANGE, UNRAID_GITEA_IP, GITEA_ADMIN_USER, GITEA_ADMIN_PASSWORD, GITEA_ADMIN_EMAIL, GITEA_ORG_NAME, GITEA_INSTANCE_NAME, GITEA_DB_TYPE, GITEA_VERSION, GITEA_INTERNAL_URL, GITEA_DOMAIN (+ UNRAID_DB_IP + DB vars if DB_TYPE != sqlite3)

Steps with idempotency:

# Action Idempotency check (skip if true)
1 Create data dirs on Unraid ssh_exec UNRAID "test -d $UNRAID_GITEA_DATA_PATH/data"
2 Render + SCP docker-compose-gitea.yml.tpl ssh_exec UNRAID "test -f $UNRAID_GITEA_DATA_PATH/docker-compose.yml"
3 Render + SCP app.ini.tpl ssh_exec UNRAID "test -f $UNRAID_GITEA_DATA_PATH/config/app.ini"
4 docker-compose up -d on Unraid ssh_exec UNRAID "docker ps --filter name=gitea --format '{{.Status}}'" contains "Up"
5 Wait for Gitea HTTP ready curl -sf $GITEA_INTERNAL_URL/api/v1/version returns 200
6 Create admin user via docker exec gitea gitea admin user create curl -sf -u $GITEA_ADMIN_USER:$GITEA_ADMIN_PASSWORD $GITEA_INTERNAL_URL/api/v1/user returns 200
7 Generate API token GITEA_ADMIN_TOKEN is non-empty in .env AND gitea_api GET /user returns 200
8 Save token to .env Token is in .env file
9 Create org gitea_api GET /orgs/$GITEA_ORG_NAME returns 200

Done when:

  • curl $GITEA_INTERNAL_URL returns Gitea HTML page
  • curl -H "Authorization: token $GITEA_ADMIN_TOKEN" $GITEA_INTERNAL_URL/api/v1/user returns admin user JSON
  • GITEA_ADMIN_TOKEN is written to .env and is non-empty
  • Org exists: gitea_api GET /orgs/$GITEA_ORG_NAME returns 200
  • Running the script again changes nothing (all steps skip with "already exists" messages)
  • shellcheck phase1_gitea_unraid.sh passes

5.2 — phase1_post_check.sh

Depends on: phase1 completed Purpose: Independent verification that phase 1 succeeded (can be run separately)

Checks:

  • Gitea responds at $GITEA_INTERNAL_URL with HTTP 200
  • Admin user authenticates: curl -u user:pass .../api/v1/user returns 200
  • API token works: gitea_api GET /user returns 200 with correct username
  • Org exists: gitea_api GET /orgs/$GITEA_ORG_NAME returns 200
  • Gitea Actions enabled: gitea_api GET /api/v1/settings/api or check app.ini

Done when:

  • Runs all checks, prints PASS/FAIL for each
  • Exits 0 only if ALL pass

5.3 — phase1_teardown.sh

Depends on: phase1 was run Destructive: yes — prompts for confirmation

Steps:

  1. Prompt: "This will stop Gitea on Unraid. Continue? [y/N]"
  2. ssh_exec UNRAID "cd $UNRAID_GITEA_DATA_PATH && docker-compose down"
  3. Prompt: "Remove all Gitea data ($UNRAID_GITEA_DATA_PATH)? This is irreversible. [y/N]"
  4. If confirmed: ssh_exec UNRAID "rm -rf $UNRAID_GITEA_DATA_PATH"
  5. Clear GITEA_ADMIN_TOKEN from .env: save_env_var GITEA_ADMIN_TOKEN ""

Done when:

  • Gitea container is stopped and removed
  • Data is only deleted if user explicitly confirms
  • GITEA_ADMIN_TOKEN is cleared from .env
  • Running against an already-torn-down instance doesn't error

6.1 — phase2_gitea_fedora.sh

Depends on: preflight passed, templates exist Produces: Running Gitea instance on Fedora, admin user, API token in .env require_vars: FEDORA_IP, FEDORA_SSH_USER, FEDORA_SSH_PORT, FEDORA_GITEA_DATA_PATH, FEDORA_MACVLAN_PARENT, FEDORA_MACVLAN_SUBNET, FEDORA_MACVLAN_GATEWAY, FEDORA_MACVLAN_IP_RANGE, FEDORA_GITEA_IP, GITEA_ADMIN_USER, GITEA_ADMIN_PASSWORD, GITEA_ADMIN_EMAIL, GITEA_DB_TYPE, GITEA_VERSION, GITEA_BACKUP_INTERNAL_URL (+ FEDORA_DB_IP + DB vars if DB_TYPE != sqlite3)

Identical to phase1 except:

  • Target: Fedora (uses FEDORA_IP, FEDORA_SSH_USER, FEDORA_SSH_PORT, FEDORA_GITEA_DATA_PATH, FEDORA_GITEA_IP)
  • Uses GITEA_BACKUP_INTERNAL_URL for API calls
  • Saves token as GITEA_BACKUP_ADMIN_TOKEN
  • Uses same admin credentials (GITEA_ADMIN_USER/GITEA_ADMIN_PASSWORD/GITEA_ADMIN_EMAIL)
  • Does NOT create an org (mirrors will be under admin user's namespace)

Done when:

  • Same criteria as 5.1 but targeting Fedora URLs/paths
  • GITEA_BACKUP_ADMIN_TOKEN is in .env and works
  • No org was created on Fedora instance

6.2 — phase2_post_check.sh

Same as 5.2 but targeting Fedora instance. No org check.

6.3 — phase2_teardown.sh

Same as 5.3 but targeting Fedora. Clears GITEA_BACKUP_ADMIN_TOKEN.

7.2 — manage_runner.sh

Depends on: lib/common.sh, runners.conf, templates Purpose: Standalone tool to add/remove/list individual runners

Subcommands:

manage_runner.sh add --name <runner_name>:

  1. Read runner entry from runners.conf by name
  2. If type=docker:
    • Render docker-compose-runner.yml.tpl with runner's vars
    • SCP to $DATA_PATH/docker-compose.yml on runner host
    • Render runner-config.yaml.tpl, SCP to $DATA_PATH/config.yaml
    • ssh_exec HOST "cd $DATA_PATH && docker-compose up -d"
    • wait_for_http on Gitea API for runner to appear
  3. If type=native:
    • Download act_runner binary for host's OS/arch to $DATA_PATH/act_runner
    • Run $DATA_PATH/act_runner register --no-interactive --instance $GITEA_INTERNAL_URL --token $GITEA_RUNNER_REGISTRATION_TOKEN --name $NAME --labels $LABELS
    • Render launchd plist, copy to ~/Library/LaunchAgents/
    • launchctl load ~/Library/LaunchAgents/com.gitea.runner.$NAME.plist

manage_runner.sh remove --name <runner_name>:

  1. Read runner entry from runners.conf
  2. If type=docker: ssh_exec HOST "cd $DATA_PATH && docker-compose down", optionally rm data
  3. If type=native: launchctl unload plist, rm binary + plist
  4. Deregister from Gitea via API if possible (or just let it go offline)

manage_runner.sh list:

  1. Read all entries from runners.conf
  2. For each: query Gitea API for runner status
  3. Print table: name, host, labels, type, status (online/offline)

Done when:

  • add with a docker-type runner: container is running, runner appears in Gitea admin panel
  • add with a native-type runner: launchd service is loaded, runner appears in Gitea admin panel
  • remove stops the runner and it disappears from Gitea admin (or shows offline)
  • list shows all runners with current status
  • add on an already-deployed runner prints "already running" and exits 0
  • remove on a non-existent runner prints warning and exits 0
  • shellcheck manage_runner.sh passes

7.3 — phase3_runners.sh

Depends on: Phase 1 completed (Gitea running), runners.conf populated Produces: All runners from runners.conf deployed and registered require_vars: GITEA_INTERNAL_URL, GITEA_ADMIN_TOKEN (auto), ACT_RUNNER_VERSION

Steps:

  1. Get registration token: gitea_api GET /admin/runners/registration-token
  2. save_env_var GITEA_RUNNER_REGISTRATION_TOKEN <token>
  3. For each entry in runners.conf: call manage_runner.sh add --name <name>

Done when:

  • GITEA_RUNNER_REGISTRATION_TOKEN is in .env
  • Every runner in runners.conf is deployed and shows "online" in Gitea admin
  • Running again skips all already-deployed runners

7.4 — phase3_post_check.sh

Checks:

  • For each runner in runners.conf: runner exists in Gitea admin API response
  • For each runner in runners.conf: status is "online" (not just registered but idle/active)
  • Runner count in Gitea matches line count in runners.conf

7.5 — phase3_teardown.sh

For each runner in runners.conf: manage_runner.sh remove --name <name>. Clears GITEA_RUNNER_REGISTRATION_TOKEN.

8.1 — phase4_migrate_repos.sh

Depends on: Phase 1 + Phase 2 completed (both Gitea instances running) Produces: All repos on Unraid primary under org + pull mirrors on Fedora require_vars: GITEA_ADMIN_TOKEN (auto), GITEA_BACKUP_ADMIN_TOKEN (auto), GITEA_INTERNAL_URL, GITEA_BACKUP_INTERNAL_URL, GITEA_ORG_NAME, GITEA_ADMIN_USER, GITEA_ADMIN_PASSWORD, GITHUB_USERNAME, GITHUB_TOKEN, REPO_NAMES, MIGRATE_ISSUES, MIGRATE_LABELS, MIGRATE_MILESTONES, MIGRATE_WIKI, GITEA_BACKUP_MIRROR_INTERVAL

Steps:

For each repo in REPO_NAMES:

# Action API call Idempotency check
1 Import repo from GitHub to Unraid gitea_api POST /repos/migrate with clone_addr=https://github.com/$GITHUB_USERNAME/$REPO.git, auth_token=$GITHUB_TOKEN, repo_owner=$GITEA_ORG_NAME, repo_name=$REPO, mirror=false, issues=$MIGRATE_ISSUES, labels=$MIGRATE_LABELS, milestones=$MIGRATE_MILESTONES, wiki=$MIGRATE_WIKI gitea_api GET /repos/$GITEA_ORG_NAME/$REPO returns 200
2 Wait for migration Poll gitea_api GET /repos/$GITEA_ORG_NAME/$REPO until empty=false (repo has content) —
3 Create pull mirror on Fedora gitea_backup_api POST /repos/migrate with clone_addr=$GITEA_INTERNAL_URL/$GITEA_ORG_NAME/$REPO.git, auth_username=$GITEA_ADMIN_USER, auth_password=$GITEA_ADMIN_PASSWORD, repo_owner=$GITEA_ADMIN_USER, mirror=true, mirror_interval=$GITEA_BACKUP_MIRROR_INTERVAL gitea_backup_api GET /repos/$GITEA_ADMIN_USER/$REPO returns 200

Done when:

  • All repos exist under $GITEA_ORG_NAME on Unraid with commits
  • Each repo's default branch matches the GitHub source
  • All mirror repos exist on Fedora under admin user
  • Fedora mirrors show mirror=true in API response
  • Fedora mirrors have synced at least once (has commits)
  • Running again skips all existing repos

8.2 — phase4_post_check.sh

Checks:

  • Each repo exists on primary: gitea_api GET /repos/$ORG/$REPO returns 200
  • Each repo has commits: gitea_api GET /repos/$ORG/$REPO/commits?limit=1 returns at least 1 commit
  • Default branch matches source: compare default_branch field from Gitea vs GitHub API
  • Each mirror repo exists on Fedora: gitea_backup_api GET /repos/$ADMIN/$REPO returns 200
  • Each mirror has mirror: true in response

8.3 — phase4_teardown.sh

  1. For each repo in REPO_NAMES: gitea_api DELETE /repos/$GITEA_ORG_NAME/$REPO
  2. For each mirror: gitea_backup_api DELETE /repos/$ADMIN/$REPO
  3. Prompt before each deletion

9.1 — phase5_migrate_pipelines.sh

Depends on: Phase 4 completed (repos exist on Gitea) Produces: .gitea/workflows/ directory in each repo with adapted workflows require_vars: GITEA_ADMIN_TOKEN (auto), GITEA_INTERNAL_URL, GITEA_ORG_NAME, GITEA_ADMIN_USER, REPO_NAMES

Steps for each repo:

  1. Clone repo from Gitea to temp dir: git clone $GITEA_INTERNAL_URL/$GITEA_ORG_NAME/$REPO.git /tmp/gitea-migration-$REPO
  2. Check if .github/workflows/ exists — if not, log warning "No GitHub workflows found" and skip
  3. Create .gitea/workflows/ directory
  4. Copy all .yml files from .github/workflows/ to .gitea/workflows/
  5. Apply compatibility fixes in each copied file:
    • Replace github.repository → gitea.repository in expressions
    • Replace github.event → gitea.event in expressions
    • Replace github.token → gitea.token
    • Replace github.server_url → gitea.server_url
    • Keep actions/checkout@v4 as-is (compatible with Gitea)
    • Add comment at top: # Migrated from GitHub Actions — review for Gitea compatibility
  6. git add .gitea/, git commit -m "Migrate workflows to Gitea Actions", git push
  7. Clean up temp dir

Idempotency: Skip repo if .gitea/workflows/ already exists with files.

Known limitations (document in script output):

  • GitHub-specific marketplace actions may not work in Gitea — script logs warnings but doesn't block
  • Self-hosted runner tool caches may differ — user may need to install tools manually
  • OIDC/secrets need to be re-configured in Gitea settings

Done when:

  • Each repo that had .github/workflows/ now has .gitea/workflows/ with adapted files
  • Context variable replacements applied (github.* → gitea.*)
  • Each adapted workflow file has the migration comment header
  • Repos without workflows are skipped with a warning (not an error)
  • Running again skips repos that already have .gitea/workflows/

9.2 — phase5_post_check.sh

Checks:

  • Each repo has .gitea/workflows/ directory (check via API: gitea_api GET /repos/$ORG/$REPO/contents/.gitea/workflows)
  • At least one .yml file in that directory
  • Gitea Actions tab shows workflows (may require checking the Gitea web UI or API for action runs)

9.3 — phase5_teardown.sh

For each repo: clone, rm -rf .gitea/workflows, commit, push. Only if .gitea/workflows/ exists.

10.1 — phase6_github_mirrors.sh

Depends on: Phase 4 completed, GitHub mirror token set Produces: Push mirrors from Gitea → GitHub configured for all repos require_vars: GITEA_ADMIN_TOKEN (auto), GITEA_INTERNAL_URL, GITEA_ORG_NAME, GITHUB_USERNAME, GITHUB_TOKEN, GITHUB_MIRROR_INTERVAL, REPO_NAMES

Steps for each repo:

  1. Check if push mirror already exists: gitea_api GET /repos/$ORG/$REPO/push_mirrors — skip if non-empty
  2. Create push mirror: gitea_api POST /repos/$ORG/$REPO/push_mirrors with:
    • remote_address: https://github.com/$GITHUB_USERNAME/$REPO.git
    • remote_username: $GITHUB_USERNAME
    • remote_password: $GITHUB_TOKEN
    • interval: $GITHUB_MIRROR_INTERVAL
    • sync_on_commit: true
  3. Trigger initial sync: gitea_api POST /repos/$ORG/$REPO/push_mirrors-sync
  4. Disable GitHub Actions: github_api PATCH /repos/$GITHUB_USERNAME/$REPO with {"has_projects": false} — note: disabling Actions requires setting has_actions: false which may not be in the API. If not possible via API, print manual instructions.

Done when:

  • Each repo has exactly one push mirror configured
  • gitea_api GET /repos/$ORG/$REPO/push_mirrors returns the mirror config
  • After triggering sync, GitHub repo has the same latest commit as Gitea
  • Running again skips repos that already have mirrors

10.2 — phase6_post_check.sh

  • Each repo: push mirror exists in API response
  • Each repo: trigger sync and verify GitHub's latest commit SHA matches Gitea's HEAD

10.3 — phase6_teardown.sh

For each repo: gitea_api DELETE /repos/$ORG/$REPO/push_mirrors/{id}. Get mirror ID from list endpoint first.

11.1 — phase7_branch_protection.sh

Depends on: Phase 4 completed (repos exist) Produces: Branch protection on $PROTECTED_BRANCH for all repos require_vars: GITEA_ADMIN_TOKEN (auto), GITEA_INTERNAL_URL, GITEA_ORG_NAME, REPO_NAMES, PROTECTED_BRANCH, REQUIRE_PR_REVIEW, REQUIRED_APPROVALS

Steps for each repo:

  1. Check if protection exists: gitea_api GET /repos/$ORG/$REPO/branch_protections/$PROTECTED_BRANCH — skip if 200
  2. Create protection: gitea_api POST /repos/$ORG/$REPO/branch_protections with:
    • branch_name: $PROTECTED_BRANCH
    • enable_push: false
    • enable_push_whitelist: false
    • require_signed_commits: false
    • enable_status_check: true
    • enable_approvals_whitelist: $REQUIRE_PR_REVIEW
    • required_approvals: $REQUIRED_APPROVALS

Done when:

  • Each repo: gitea_api GET /repos/$ORG/$REPO/branch_protections returns the rule
  • Direct pushes to $PROTECTED_BRANCH are blocked
  • Running again skips existing protections

11.2 / 11.3 — post-check and teardown

Post-check: verify protection rules via API. Teardown: gitea_api DELETE /repos/$ORG/$REPO/branch_protections/$PROTECTED_BRANCH.

12.1 — phase8_cutover.sh

Depends on: macvlan network created (Phase 1), all prior phases Produces: HTTPS access to Gitea via Caddy, GitHub repos marked as mirrors require_vars: UNRAID_IP, UNRAID_SSH_USER, UNRAID_GITEA_IP, UNRAID_CADDY_IP, GITEA_INTERNAL_URL, GITEA_DOMAIN, GITEA_ADMIN_TOKEN (auto), GITEA_ORG_NAME, TLS_MODE, CADDY_DOMAIN, CADDY_DATA_PATH, GITHUB_USERNAME, GITHUB_TOKEN, REPO_NAMES. Conditional: CLOUDFLARE_API_TOKEN (if TLS_MODE=cloudflare), SSL_CERT_PATH + SSL_KEY_PATH (if TLS_MODE=existing).

Steps with idempotency:

# Action Detail Idempotency check (skip if true)
1 Create Caddy dirs ssh_exec UNRAID "mkdir -p $CADDY_DATA_PATH/{data,config}" test -d $CADDY_DATA_PATH
2 Deploy Caddyfile Render Caddyfile.tpl with TLS_BLOCK (cloudflare DNS-01 or existing cert paths), SCP to $CADDY_DATA_PATH/Caddyfile test -f $CADDY_DATA_PATH/Caddyfile
3 Deploy Caddy docker-compose Render docker-compose-caddy.yml.tpl, SCP to $CADDY_DATA_PATH/docker-compose.yml test -f $CADDY_DATA_PATH/docker-compose.yml
4 Start Caddy docker compose up -d in $CADDY_DATA_PATH Caddy container already running
5 Wait for HTTPS Poll https://$GITEA_DOMAIN/api/v1/version with retries until cert is obtained —
6 Mark GitHub repos as mirrors Save pre-cutover state to .manifests/phase8_github_repo_state.json, update description to [MIRROR], disable wiki/projects/Pages GitHub repo description starts with [MIRROR]

TLS mode handling:

  • cloudflare: Caddyfile uses tls { dns cloudflare {env.CF_API_TOKEN} }, docker-compose passes CF_API_TOKEN env var
  • existing: Caddyfile uses tls /path/to/cert /path/to/key, docker-compose mounts cert/key as volumes

Done when:

  • https://$GITEA_DOMAIN returns valid HTTPS response
  • HTTP requests redirect to HTTPS (301)
  • SSL certificate is valid (openssl check)
  • All repos accessible via HTTPS API
  • GitHub repos marked with [MIRROR] description prefix

12.2 — phase8_post_check.sh

  • HTTPS works with valid cert: curl -sf https://$GITEA_DOMAIN/api/v1/version returns 200
  • HTTP redirects to HTTPS: curl -sI http://$GITEA_DOMAIN/ returns 301
  • Certificate is valid: openssl s_client returns non-empty issuer
  • All repos accessible: API call to each repo returns 200
  • GitHub repos marked as mirrors: description starts with [MIRROR]

12.3 — phase8_teardown.sh

  1. Stop + remove Caddy container: docker compose down in $CADDY_DATA_PATH
  2. Remove Caddy config files: rm -f $CADDY_DATA_PATH/docker-compose.yml $CADDY_DATA_PATH/Caddyfile
  3. Optionally remove Caddy TLS data: rm -rf $CADDY_DATA_PATH/data $CADDY_DATA_PATH/config
  4. Restore GitHub repo settings from saved Phase 8 state snapshot (description, homepage, wiki, projects, Pages). Falls back to parsing [MIRROR] ... — was: ORIGINAL if snapshot missing.

13.1 — phase9_security.sh

Depends on: Phase 5 completed (repos have .gitea/workflows/) Produces: Security scan workflow in all repos, branch protection updated require_vars: GITEA_ADMIN_TOKEN (auto), GITEA_INTERNAL_URL, GITEA_ORG_NAME, GITEA_ADMIN_USER, REPO_NAMES, SEMGREP_VERSION, TRIVY_VERSION, GITLEAKS_VERSION, SECURITY_FAIL_ON_ERROR, PROTECTED_BRANCH

Steps for each repo:

  1. Clone from Gitea to temp dir
  2. Render security-scan.yml.tpl to .gitea/workflows/security-scan.yml
  3. git add, git commit -m "Add security scanning workflow", git push
  4. If SECURITY_FAIL_ON_ERROR=true: update branch protection to require status checks from security jobs
    • gitea_api PATCH /repos/$ORG/$REPO/branch_protections/$PROTECTED_BRANCH with status_check_contexts: ["semgrep", "trivy", "gitleaks"]

Idempotency: Skip if security-scan.yml already exists in repo.

Done when:

  • Each repo has .gitea/workflows/security-scan.yml
  • Workflow file references correct tool versions from .env
  • If SECURITY_FAIL_ON_ERROR=true: branch protection includes the three status checks
  • Creating a test PR triggers the security workflow (manual verification)

13.2 / 13.3 — post-check and teardown

Post-check: verify file exists in each repo via API, verify branch protection includes status checks. Teardown: remove file, update branch protection to remove status checks.

14.1 — backup/backup_primary.sh

Depends on: Phase 1 completed Produces: gitea-dump-*.zip archive on Fedora require_vars: UNRAID_IP, UNRAID_SSH_USER, UNRAID_SSH_PORT, UNRAID_GITEA_DATA_PATH, FEDORA_IP, FEDORA_SSH_USER, FEDORA_SSH_PORT, BACKUP_STORAGE_PATH, BACKUP_RETENTION_COUNT

Steps:

  1. ssh_exec UNRAID "docker exec -u git gitea gitea dump -c /data/gitea/conf/app.ini -f /tmp/gitea-dump-$(date +%Y%m%d-%H%M%S).zip"
  2. SCP dump from Unraid /tmp/ to $BACKUP_STORAGE_PATH/ on Fedora
  3. Remove dump from Unraid /tmp/
  4. Prune old backups: ssh_exec FEDORA "ls -t $BACKUP_STORAGE_PATH/gitea-dump-*.zip | tail -n +$((BACKUP_RETENTION_COUNT+1)) | xargs rm -f"
  5. Print: backup file name, size, path on Fedora, remaining backup count

What's in the dump (verify these are captured):

  • SQLite database (users, tokens, SSH keys, OAuth, webhooks, org/team membership, issues, PRs)
  • All git repositories
  • app.ini config

Done when:

  • Zip file exists on Fedora at $BACKUP_STORAGE_PATH/
  • Zip contains: gitea-db.sql (or gitea.db for SQLite), repos/ directory, app.ini
  • Old backups beyond retention count are deleted
  • Dump file on Unraid /tmp/ is cleaned up

14.2 — backup/restore_to_primary.sh

Depends on: A backup archive exists Produces: Restored Gitea instance on Unraid require_vars: UNRAID_IP, UNRAID_SSH_USER, UNRAID_SSH_PORT, UNRAID_GITEA_DATA_PATH, GITEA_INTERNAL_URL, GITEA_ADMIN_USER, GITEA_ADMIN_PASSWORD

Steps:

  1. Accept --archive <path> (path on Fedora or local)
  2. Prompt: "This will REPLACE all Gitea data on Unraid. Continue? [y/N]"
  3. If archive is on Fedora: SCP to Unraid /tmp/
  4. Stop Gitea: ssh_exec UNRAID "cd $UNRAID_GITEA_DATA_PATH && docker-compose down"
  5. Back up current data (safety): ssh_exec UNRAID "mv $UNRAID_GITEA_DATA_PATH/data $UNRAID_GITEA_DATA_PATH/data.pre-restore"
  6. Extract archive: ssh_exec UNRAID "cd $UNRAID_GITEA_DATA_PATH && unzip /tmp/gitea-dump-*.zip"
  7. Move extracted files to correct locations (data, config)
  8. Start Gitea: ssh_exec UNRAID "cd $UNRAID_GITEA_DATA_PATH && docker-compose up -d"
  9. wait_for_http $GITEA_INTERNAL_URL 60
  10. Verify admin login works: curl -sf -u $GITEA_ADMIN_USER:$GITEA_ADMIN_PASSWORD $GITEA_INTERNAL_URL/api/v1/user
  11. Regenerate API token (old token from dump may conflict): create new token via basic auth, save_env_var GITEA_ADMIN_TOKEN

Done when:

  • Gitea is running with restored data
  • Admin can log in
  • All repos are accessible
  • All users from the backup exist
  • New API token is generated and saved to .env
  • Pre-restore data is preserved (not deleted) in case restore went wrong

15.1 — run_all.sh

Depends on: all scripts exist Steps:

  1. Parse args: --start-from=N (default 0), --skip-setup
  2. Define execution order array:
    • Step 0 — Setup:
      1. setup/configure_env.sh — interactive env wizard (populates .env)
      2. setup/macbook.sh — install MacBook prerequisites
      3. setup/unraid.sh — install Unraid prerequisites (via SSH)
      4. setup/fedora.sh — install Fedora prerequisites (via SSH)
    • Step P — Preflight: preflight.sh (validates everything before proceeding)
    • Steps 1-9 — Phases: phaseN_*.sh + phaseN_post_check.sh
  3. Execute sequentially from start point
  4. On any script exit code != 0: stop, print which step failed, print summary of what completed
  5. On success: print full summary with checkmarks
  6. --start-from=N (where N >= 1): skips setup + preflight only for phases. Still runs preflight.sh to validate .env vars (but NOT configure_env.sh or machine setup — assumes those were already done).

Done when:

  • Running without args executes: configure_env → macbook → unraid → fedora → preflight → phase 1-9
  • --start-from=3 skips phases 1-2 but still runs preflight, then starts at phase 3
  • --skip-setup skips configure_env + machine setup scripts, starts at preflight
  • Failure in any step stops execution (doesn't continue to next phase)
  • Summary at end shows pass/fail for each step that ran

15.2 — teardown_all.sh

Steps:

  1. Parse args: --through=N (default 1 = tear down everything)
  2. Execute in REVERSE order: phase9_teardown → phase8_teardown → ... → phaseN_teardown
  3. Each teardown prompts for confirmation (unless --yes flag)

Done when:

  • --through=5 tears down phases 5-9 but leaves 1-4 intact
  • Each phase teardown runs independently (doesn't fail because a later phase is already torn down)
  • --yes skips all confirmation prompts

16.1 — Git repo + .gitignore

.gitignore must contain:

# Secrets — never commit
.env
runners.conf
*.pem
*.key
*.crt

# macOS
.DS_Store

# Temp files from script runs
/tmp/
*.log

# Backup archives
*.zip

# Editor / IDE
.vscode/
.idea/
*.swp
*~

Done when:

  • git init completed
  • .gitignore excludes .env, runners.conf, temp files
  • .env.example and runners.conf.example are NOT ignored (they're templates)

16.2 — CLAUDE.md

Project-specific instructions for AI assistants working on this codebase.

Done when:

  • Documents: project structure, how to run scripts, .env setup, script conventions
  • Notes the set -euo pipefail + shellcheck requirement for all scripts

17.1 / 17.2 — Validation

Done when:

  • shellcheck *.sh lib/*.sh setup/*.sh backup/*.sh exits 0 with zero warnings
  • bash -n on every .sh file exits 0
  • No file has unexpanded template variables (grep for \$[A-Z_] in rendered outputs should find nothing unexpected)

Git Commit Milestones

Each milestone is a git commit. Commit after all files in that group pass shellcheck and bash -n. Do not batch multiple milestones into one commit.

# Commit message Files included Tracker sections
1 init: project structure, .gitignore, .env.example, runners.conf.example .gitignore, .env.example, runners.conf.example, PLAN.md, CLAUDE.md 16.1, 16.2
2 feat: add shared library (lib/common.sh) lib/common.sh 1.1
3 feat: add API contracts contracts/gitea-api.md 1.3
4 feat: add configuration templates templates/*.tpl, templates/workflows/*.tpl 2.1–2.7
5 feat: add setup scripts (configure_env, macbook, unraid, fedora) setup/configure_env.sh, setup/macbook.sh, setup/unraid.sh, setup/fedora.sh 3.1–3.4
6 feat: add preflight validation preflight.sh 4.1
7 feat: add Phase 1 — Gitea on Unraid phase1_gitea_unraid.sh, phase1_post_check.sh, phase1_teardown.sh 5.1–5.3
8 feat: add Phase 2 — Gitea on Fedora phase2_gitea_fedora.sh, phase2_post_check.sh, phase2_teardown.sh 6.1–6.3
9 feat: add Phase 3 — Runners phase3_runners.sh, phase3_post_check.sh, phase3_teardown.sh, manage_runner.sh 7.2–7.5
10 feat: add Phase 4 — Migrate repos + Fedora mirrors phase4_migrate_repos.sh, phase4_post_check.sh, phase4_teardown.sh 8.1–8.3
11 feat: add Phase 5 — Migrate pipelines phase5_migrate_pipelines.sh, phase5_post_check.sh, phase5_teardown.sh 9.1–9.3
12 feat: add Phase 6 — GitHub push mirrors phase6_github_mirrors.sh, phase6_post_check.sh, phase6_teardown.sh 10.1–10.3
13 feat: add Phase 7 — Branch protection phase7_branch_protection.sh, phase7_post_check.sh, phase7_teardown.sh 11.1–11.3
14 feat: add Phase 8 — Cutover (HTTPS + archive GitHub) phase8_cutover.sh, phase8_post_check.sh, phase8_teardown.sh 12.1–12.3
15 feat: add Phase 9 — Security scanning phase9_security.sh, phase9_post_check.sh, phase9_teardown.sh 13.1–13.3
16 feat: add backup and restore scripts backup/backup_primary.sh, backup/restore_to_primary.sh 14.1–14.2
17 feat: add orchestration (run_all.sh, teardown_all.sh) run_all.sh, teardown_all.sh 15.1–15.2
18 chore: shellcheck + syntax validation fixes Any files fixed during validation 17.1–17.2
Reference in New Issue View Git Blame Copy Permalink