gitea-migration/USAGE_GUIDE.md

Usage Guide

Step-by-step instructions for running the migration, handling failures, rolling back, and day-to-day operations.

---

Table of Contents

1. Before You Start
2. Happy Path: Full Migration
3. Resuming After Failure
4. Edge Cases
5. Rollback Procedures
6. Verifying Rollback Success
7. Day-to-Day Operations
8. Backup and Restore
9. Runner Management
10. Troubleshooting

---

Before You Start

1. Network and DNS

Before running anything, confirm:

• MacBook can SSH to Unraid: ssh user@UNRAID_IP
• MacBook can SSH to Fedora: ssh user@FEDORA_IP
• A DNS A record exists for your Gitea domain pointing to UNRAID_IP
• Macvlan container IPs are available on the LAN (not already in use)
• If using TLS_MODE=cloudflare: a Cloudflare API token with Zone:DNS:Edit permission

2. Passwordless sudo on remote hosts

The setup and phase scripts run sudo commands on Unraid and Fedora over SSH (non-interactive, no TTY). If sudo requires a password, it will fail with: a terminal is required to read the password.

Enable temporarily (on each remote host):

# SSH in interactively
ssh user@HOST_IP

# Create a drop-in sudoers file
sudo visudo -f /etc/sudoers.d/temp-nopasswd
# Add this line (replace YOUR_USER with the SSH username):
YOUR_USER ALL=(ALL) NOPASSWD: ALL
# Save and exit, then verify:
sudo -n true && echo "OK — passwordless sudo works"

Disable after migration is complete:

sudo rm /etc/sudoers.d/temp-nopasswd

3. GitHub Tokens

You need one GitHub Personal Access Token:

Token	Scope	Used By
GITHUB_TOKEN	repo (read+write)	Migration, push mirrors, preflight validation

4. Configuration

# Option A: Interactive wizard (recommended for first time)
./setup/configure_env.sh

# Option B: Manual
cp .env.example .env
cp runners.conf.example runners.conf
# Edit both files with your values

# Optional: interactive runner definition wizard
./setup/configure_runners.sh

The wizard validates every input (IP format, port ranges, URL format, password length) and shows your current values in brackets so you can press Enter to keep them.

---

Happy Path: Full Migration

Automated (recommended)

./run_all.sh

This runs the full pipeline: setup scripts, preflight checks, and all 9 phases with post-checks after each phase. It stops on the first failure and prints a summary of what completed.

Expected output on success:

>>> Running: Setup MacBook
>>> Running: Setup Unraid
>>> Running: Setup Fedora
>>> Running: Cross-host SSH trust
>>> Running: Preflight checks
>>> Running: Phase 1: Gitea on Unraid
>>> Running: Phase 1 — post-check
>>> Running: Phase 2: Gitea on Fedora
>>> Running: Phase 2 — post-check
...
>>> Running: Phase 9: Security
>>> Running: Phase 9 — post-check

===== Execution Summary =====
  Setup MacBook
  Setup Unraid
  Setup Fedora
  Cross-host SSH trust
  Preflight checks
  Phase 1: Gitea on Unraid
  Phase 1 — post-check
  ...
  Phase 9: Security
  Phase 9 — post-check

Migration complete! Gitea is live.

Manual (phase by phase)

If you prefer to run each phase individually and inspect results:

# 1. Setup (once)
./setup/configure_env.sh
./setup/macbook.sh
./setup/unraid.sh
./setup/fedora.sh
./setup/cross_host_ssh.sh

# 2. Validate
./preflight.sh

# 3. Run each phase
./phase1_gitea_unraid.sh && ./phase1_post_check.sh
./phase2_gitea_fedora.sh && ./phase2_post_check.sh
./phase3_runners.sh       && ./phase3_post_check.sh
./phase4_migrate_repos.sh && ./phase4_post_check.sh
./phase5_migrate_pipelines.sh && ./phase5_post_check.sh
./phase6_github_mirrors.sh    && ./phase6_post_check.sh
./phase7_branch_protection.sh && ./phase7_post_check.sh
./phase8_cutover.sh           && ./phase8_post_check.sh
./phase9_security.sh          && ./phase9_post_check.sh

Skip setup (already done)

./run_all.sh --skip-setup

What to verify when it's done

After the full migration completes:

1. HTTPS access: Open https://YOUR_DOMAIN in a browser — you should see the Gitea login page with a valid SSL certificate.
2. Repository content: Log in as admin, navigate to your org, confirm all repos have commits, branches, and (if enabled) issues/labels.
3. CI pipelines: Create a test PR in one repo. Gitea Actions should run the migrated workflows. If Phase 9 ran, security scans (Semgrep, Trivy, Gitleaks) should also trigger.
4. GitHub mirrors: Check your GitHub repos — the description should start with [MIRROR] and the latest commits should match Gitea.
5. Fedora mirrors: SSH into the Fedora Gitea instance and verify repos exist under the admin user namespace.
6. Runners: Run ./manage_runner.sh list — all runners should show status online or idle.

---

Resuming After Failure

Starting from a specific phase

If Phase 5 failed but Phases 1-4 completed successfully:

./run_all.sh --start-from=5

This runs preflight (with --skip-port-checks since Gitea is already running on those ports), then phases 5 through 9.

Re-running a single phase

Every phase is idempotent. If Phase 4 failed on the second repo, just re-run it:

./phase4_migrate_repos.sh

It will skip the first repo (already exists) and retry the second.

Re-running a post-check

Post-checks are read-only and can be run at any time:

./phase4_post_check.sh

Preflight after partial migration

When resuming from a later phase, Gitea is already running on ports 3000. Use:

./preflight.sh --skip-port-checks

run_all.sh --start-from=N does this automatically when N > 1.

---

Edge Cases

GitHub API rate limit hit during migration

Symptom: Phase 4 or Phase 6 fails with HTTP 403 and a message about rate limits.

Fix: Wait for the rate limit window to reset (usually 1 hour), then re-run the failed phase. The phase is idempotent — repos that already migrated will be skipped.

GitHub token expires mid-migration

Symptom: API calls fail with HTTP 401.

Fix: Generate a new token on GitHub, update GITHUB_TOKEN in .env, then re-run the failed phase.

Large repos time out during migration

Symptom: Phase 4 logs "Timeout waiting for REPO migration to complete".

Fix: Increase the timeout in .env:

MIGRATION_POLL_TIMEOUT_SEC=1800    # 30 minutes instead of default 10

Then re-run Phase 4. Already-migrated repos will be skipped.

Container IP already in use

Symptom: Preflight check 13 warns that a macvlan IP is already responding to ping.

Fix: Either release the conflicting IP or change the container IP in .env (UNRAID_GITEA_IP, UNRAID_CADDY_IP, or FEDORA_GITEA_IP).

DNS doesn't resolve to Unraid IP

Symptom: Preflight check 14 fails.

Fix: Add or update your DNS A record. If using a local DNS server or /etc/hosts, ensure the record points to UNRAID_IP. DNS propagation can take minutes to hours.

Caddy fails to start or obtain TLS certificate in Phase 8

Symptom: Phase 8 times out waiting for HTTPS to become available.

Common causes (TLS_MODE=cloudflare):

• CLOUDFLARE_API_TOKEN is invalid or lacks Zone:DNS:Edit permission
• DNS zone is not managed by Cloudflare
• Caddy container cannot reach Cloudflare API (firewall/proxy)

Common causes (TLS_MODE=existing):

• SSL_CERT_PATH or SSL_KEY_PATH doesn't exist on the Unraid host
• Certificate is expired or doesn't match GITEA_DOMAIN

Fix: Check Caddy container logs: ssh USER@UNRAID_IP "docker logs caddy". For Cloudflare issues, verify the API token at dash.cloudflare.com. For existing certs, verify the paths exist and the cert matches the domain.

SSH to remote host fails

Symptom: Preflight check 7 or 8 fails, or any phase fails with "SSH config incomplete".

Fix: Verify SSH key-based auth works manually:

ssh -o BatchMode=yes USER@IP "echo ok"

If this fails, add your public key to the remote host's ~/.ssh/authorized_keys.

Cross-host SSH fails (Unraid cannot reach Fedora or vice versa)

Symptom: Preflight checks 21-22 fail, or backup/backup_primary.sh fails on the SCP step.

Fix: Run the cross-host SSH setup:

./setup/cross_host_ssh.sh

This generates ed25519 keys on each host and distributes public keys to the other host's authorized_keys.

Docker group membership not active on Fedora

Symptom: setup/fedora.sh warns "Docker requires sudo" or Phase 2/3 fails with permission errors.

Fix: Log out and back in to the Fedora machine (SSH session), then re-run the failed script. Adding a user to the docker group requires a new login session to take effect.

Runner shows as offline after deployment

Symptom: Phase 3 post-check reports a runner as offline.

Fix:

• For Docker runners: ssh USER@HOST "docker logs gitea-runner-RUNNER_NAME" to check logs
• For native runners: cat ~/gitea-runner/runner.err.log
• For boot-mode native runners: service management requires sudo launchctl list | grep com.gitea.runner
• Common cause: the runner registration token expired. Clear it and re-run Phase 3:

# In .env, clear:
GITEA_RUNNER_REGISTRATION_TOKEN=
# Then:
./phase3_runners.sh

Workflow files have invalid YAML after migration

Symptom: Gitea Actions fails to parse workflows in Phase 5 post-check or when triggering a PR.

Fix: Phase 5's sed replacements are syntactic — they can't break valid YAML, but the original file may have had GitHub-specific syntax that Gitea doesn't support. Clone the repo and fix the workflow files manually:

git clone https://YOUR_DOMAIN/ORG/REPO.git
cd REPO
# Edit .gitea/workflows/*.yml
git commit -am "Fix workflow compatibility"
git push

---

Rollback Procedures

Full rollback (everything)

./teardown_all.sh

Tears down all phases from 9 to 1, prompting for confirmation at each phase. This:

• Removes security workflows from repos
• Restores GitHub repo settings (description, homepage, wiki, projects, Pages)
• Removes Caddy container, config, and TLS data
• Removes branch protection rules
• Removes push mirrors and re-enables GitHub Actions
• Removes .gitea/workflows/ from repos
• Deletes repos from primary and mirrors from Fedora
• Stops and removes runner containers
• Stops Gitea on both Unraid and Fedora, optionally removes all data

Non-interactive full rollback

./teardown_all.sh --yes

Answers "yes" to all confirmation prompts. Use for automated testing.

Partial rollback (keep infrastructure, remove config)

To tear down Phases 5-9 but keep Gitea and repos (Phases 1-4):

./teardown_all.sh --through=5

This removes security workflows, Caddy, branch protection, push mirrors, and Gitea workflows — but leaves the Gitea instances running with the migrated repos.

Teardown a single phase

./phase6_teardown.sh

Each teardown script is safe to run independently and prompts before destructive actions.

Full teardown including prerequisites

./teardown_all.sh --cleanup

After tearing down all phases, this also runs setup/cleanup.sh which reads .manifests/ and uninstalls everything the setup scripts installed (brew packages on macOS, dnf packages on Fedora, static binaries on Unraid, SSH keys from cross-host setup).

Preview what cleanup would do

./setup/cleanup.sh --dry-run

Cleanup a specific host

./setup/cleanup.sh --host=fedora
./setup/cleanup.sh --host=unraid
./setup/cleanup.sh --host=macbook

---

Verifying Rollback Success

After a rollback, run these checks to confirm everything was reversed cleanly.

After Phase 1-2 teardown (Gitea removed)

# Verify Gitea is not running
ssh USER@UNRAID_IP "docker ps | grep gitea"     # Should return nothing
ssh USER@FEDORA_IP "docker ps | grep gitea"     # Should return nothing

# Verify data is removed (if you confirmed deletion)
ssh USER@UNRAID_IP "ls /path/to/gitea/data"     # Should fail: No such file
ssh USER@FEDORA_IP "ls /path/to/gitea/data"     # Should fail: No such file

# Verify tokens are cleared
grep "GITEA_ADMIN_TOKEN=" .env                   # Should be empty
grep "GITEA_BACKUP_ADMIN_TOKEN=" .env            # Should be empty

After Phase 3 teardown (runners removed)

# Docker runners
ssh USER@UNRAID_IP "docker ps | grep gitea-runner"   # Should return nothing
ssh USER@FEDORA_IP "docker ps | grep gitea-runner"   # Should return nothing

# Native runner (macOS) — login agent
launchctl list | grep com.gitea.runner                # Should return nothing
# Native runner (macOS) — boot daemon (if boot=true was used)
sudo launchctl list | grep com.gitea.runner           # Should return nothing

# Registration token cleared
grep "GITEA_RUNNER_REGISTRATION_TOKEN=" .env          # Should be empty

After Phase 4 teardown (repos deleted)

If Gitea is still running (you only tore down Phase 4+), verify via API:

# Should all return 404
curl -sf -H "Authorization: token $TOKEN" \
  http://UNRAID_IP:3000/api/v1/repos/ORG/REPO_NAME

curl -sf -H "Authorization: token $BACKUP_TOKEN" \
  http://FEDORA_IP:3000/api/v1/repos/ADMIN/REPO_NAME

After Phase 6 teardown (push mirrors removed)

# Should return empty array []
curl -sf -H "Authorization: token $TOKEN" \
  http://UNRAID_IP:3000/api/v1/repos/ORG/REPO_NAME/push_mirrors

# GitHub Actions should be re-enabled
# Check at: https://github.com/USERNAME/REPO/settings/actions

After Phase 8 teardown (HTTPS removed, GitHub restored)

# HTTPS should no longer work
curl -sf https://YOUR_DOMAIN/api/v1/version       # Should fail

# Caddy container and config removed
ssh USER@UNRAID_IP "docker ps --filter name=caddy --format '{{.Status}}'"  # Should be empty
ssh USER@UNRAID_IP "test -f /path/to/caddy/Caddyfile && echo exists || echo gone"
# Should print: gone

# GitHub repos should be restored
# Check that description no longer starts with [MIRROR]
curl -sf -H "Authorization: token $GH_TOKEN" \
  https://api.github.com/repos/USERNAME/REPO | jq '.description'

After full teardown + cleanup

# Verify setup manifests are cleared
ls .manifests/    # Directory should be empty or not exist

# Verify brew packages removed (macOS)
brew list jq      # Should fail if jq was installed by setup

# Verify cross-host SSH keys removed
ssh USER@UNRAID_IP "test -f ~/.ssh/id_ed25519 && echo exists || echo gone"
ssh USER@FEDORA_IP "test -f ~/.ssh/id_ed25519 && echo exists || echo gone"

---

Day-to-Day Operations

Updating Gitea version

1. Edit GITEA_VERSION in .env
2. SSH to Unraid: cd /path/to/gitea && docker compose pull && docker compose up -d
3. Repeat on Fedora if you want the backup instance to match

Rotating API tokens

# Delete old token in Gitea web UI (Site Administration > Applications)
# Then clear and regenerate:
sed -i 's/^GITEA_ADMIN_TOKEN=.*/GITEA_ADMIN_TOKEN=/' .env
./phase1_gitea_unraid.sh    # Step 7 will generate a new token

Adding a new repository

1. Append the new repo name to REPO_NAMES in .env (space-separated): REPO_NAMES="repo1 repo2 new-repo"
2. Run phases 4-9 (existing repos will be skipped, only the new one gets processed)

Checking mirror sync status

# Push mirrors (Gitea to GitHub)
curl -sf -H "Authorization: token $TOKEN" \
  http://UNRAID_IP:3000/api/v1/repos/ORG/REPO/push_mirrors | jq '.[].last_update'

# Trigger manual push sync
curl -sf -X POST -H "Authorization: token $TOKEN" \
  http://UNRAID_IP:3000/api/v1/repos/ORG/REPO/push_mirrors-sync

TLS certificate renewal

When TLS_MODE=cloudflare, Caddy handles certificate renewal automatically — no manual action needed. Caddy renews certificates 30 days before expiry.

When TLS_MODE=existing, replace the cert/key files at SSL_CERT_PATH and SSL_KEY_PATH on the Unraid host, then restart Caddy:

ssh USER@UNRAID_IP "cd /path/to/caddy && docker compose restart"

---

Backup and Restore

Creating a backup

./backup/backup_primary.sh

This:

1. Runs gitea dump inside the Gitea container on Unraid
2. SCPs the dump directly from Unraid to Fedora (not through MacBook)
3. Verifies archive integrity on Fedora
4. Cleans up the dump from Unraid
5. Prunes old backups beyond BACKUP_RETENTION_COUNT

The dump contains the SQLite database (users, tokens, SSH keys, org structure, issues), all git repositories, and the app.ini config.

Restoring from backup

# From a backup stored on Fedora:
./backup/restore_to_primary.sh --archive /path/on/fedora/gitea-dump-20260228-120000.zip

# From a local file on the MacBook:
./backup/restore_to_primary.sh --archive ./gitea-dump-20260228-120000.zip

This is destructive — it replaces all Gitea data on Unraid. The script:

1. Stops the Gitea container
2. Renames the current data/ directory to data.pre-restore-TIMESTAMP as a safety net
3. Extracts the archive
4. Restarts Gitea
5. Verifies admin login
6. Regenerates the API token (old tokens from the dump may be stale)

Verifying a restore

./phase1_post_check.sh       # Verify Gitea is running with correct admin
./phase4_post_check.sh       # Verify all repos exist with commits

Scheduling automatic backups

Add a cron job on the MacBook (or any machine with SSH access to both servers):

# Daily at 2 AM
0 2 * * * cd /path/to/gitea-migration && ./backup/backup_primary.sh >> /var/log/gitea-backup.log 2>&1

Backing up .env to Bitwarden

The .env file is gitignored (it contains passwords and tokens) but is needed for ongoing operations — teardown, runner management, backup/restore, and token rotation. If you lose it, you lose the ability to manage the migration.

Export .env to Bitwarden:

./setup/env_to_bitwarden.sh -o bitwarden-import.json

This creates a Bitwarden-importable JSON file with a secure note called gitea-migration-env. Each .env variable becomes a custom field — passwords and tokens are marked as hidden fields.

Import it: Bitwarden Web Vault → Tools → Import Data → Format: Bitwarden (json) → Upload the file. Delete bitwarden-import.json from disk afterward.

Restore .env from Bitwarden:

bw unlock                              # unlock your vault first
./setup/bitwarden_to_env.sh --bw       # fetches just the one item
./preflight.sh                         # validate the restored .env

The script uses .env.example as a template to preserve section headers and comments. It automatically validates all restored values (IP formats, port ranges, email addresses, booleans, etc.) and warns about any failures. Run preflight afterward to confirm connectivity works.

Important: Do not use bw export (full vault export) to get the data — it dumps your entire vault to a plaintext JSON file on disk. The --bw flag fetches only the gitea-migration-env item.

After cleanup: teardown_all.sh --cleanup uninstalls the bw CLI from your Mac, but the secure note remains in your Bitwarden vault. Reinstall with brew install bitwarden-cli if you need to restore later.

---

Runner Management

Listing all runners

./manage_runner.sh list

Output:

NAME                 HOST             LABELS     TYPE     CAP    STATUS
----                 ----             ------     ----     ---    ------
unraid-runner        192.168.1.10     linux      docker   2      online
fedora-runner        192.168.1.20     linux      docker   2      idle
macbook-runner       local            macos      native   1      online

Adding a runner

Define it in runners.conf (INI format):

[new-runner]
host     = custom
type     = docker
data_path = /opt/gitea-runner
labels   = linux
default_image = catthehacker/ubuntu:act-latest
repos    = all
capacity = 2
ssh_host = 192.168.1.30
ssh_user = user
ssh_port = 22

Then deploy:

./manage_runner.sh add --name new-runner

Removing a runner

./manage_runner.sh remove --name new-runner

For Docker runners, this stops and removes the container. For native runners, this unloads the launchd service, removes the plist, and optionally deletes the runner data directory. If the native runner was deployed with boot = true, removal uses sudo for launchctl unload and plist deletion.

Runner types

Type	Deployed On	How It Works
docker	Linux hosts	Docker Compose container, auto-registers via environment variables
native	macOS (login)	Binary + launchd agent in ~/Library/LaunchAgents/ — starts when user logs in
native + boot=true	macOS (boot)	Binary + launchd daemon in /Library/LaunchDaemons/ — starts at boot before login

Boot vs login startup (native runners)

Native macOS runners support two startup modes controlled by the boot field in runners.conf:

Setting	Plist Location	Starts	Sudo Required	Use Case
boot = false (default)	~/Library/LaunchAgents/	At user login	No	Developer workstation, shared Mac
boot = true	/Library/LaunchDaemons/	At boot (before login)	Yes	Headless Mac mini, dedicated CI machine

When boot = true, manage_runner.sh uses sudo for:

• Copying the plist to /Library/LaunchDaemons/
• Running launchctl load / launchctl unload
• Removing the plist on teardown

The plist includes a <key>UserName</key> entry so the daemon runs as the deploying user, not root. Log rotation via newsyslog always requires sudo regardless of boot mode.

---

Troubleshooting

"Missing required var: X"

A required .env variable is empty. Open .env and fill in the missing value, or re-run ./setup/configure_env.sh.

"SSH config incomplete for HOST"

The *_IP or *_SSH_USER variables for the target host are empty in .env.

"API POST /path returned HTTP 409"

The resource already exists (conflict). This usually means idempotency is working — the script should skip this operation and continue. If it doesn't, check for a bug in the idempotency check.

"API POST /path returned HTTP 422"

Validation error from Gitea. Common causes:

• Token name already exists (Phase 1/2 token generation after a partial teardown)
• Branch protection rule name conflict
• Invalid JSON payload

Check the error response body in the log output for details.

"Timeout waiting for HTTP 200 at URL"

Gitea didn't start within the timeout window (120 seconds). Check:

• Docker container logs: ssh USER@HOST "docker logs gitea"
• Disk space: ssh USER@HOST "df -h"
• Port conflicts: ssh USER@HOST "ss -tlnp | grep 3000"

"curl failed for METHOD URL"

Network-level failure (connection refused, DNS resolution failed, timeout). Verify:

• The target service is running
• Firewall rules allow the connection
• The URL in .env is correct (check for typos in IP/port)

Reading logs

All script output goes to stderr with colored prefixes:

• [INFO] — progress messages (blue)
• [OK] — success confirmations (green)
• [WARN] — non-fatal issues (yellow)
• [ERROR] — failures (red)

To capture logs to a file while still seeing them:

./run_all.sh 2>&1 | tee migration.log

Checking what scripts installed

# See all install manifests
ls .manifests/

# See what was installed on a specific host
cat .manifests/macbook.manifest
cat .manifests/unraid.manifest
cat .manifests/fedora.manifest

# Preview cleanup without doing it
./setup/cleanup.sh --dry-run