1275 lines
63 KiB
Markdown
1275 lines
63 KiB
Markdown
# Gitea Migration Toolkit — Implementation Plan
|
||
|
||
> **Note**: This is the original implementation plan. Some sections describe the initial Nginx/SSL_MODE/3-repo architecture that has since been replaced by Caddy/TLS_MODE/dynamic repos with macvlan networking. See README.md and .env.example for the current architecture.
|
||
|
||
## Context
|
||
Migrating GitHub repos to self-hosted Gitea (Unraid primary, Fedora backup mirror, GitHub as push mirror). All automation runs from MacBook, SSHing into remote machines. Scripts must be idempotent, .env-driven, with preflight + post-checks + teardown per phase.
|
||
|
||
---
|
||
|
||
## Architecture Decisions
|
||
|
||
| Decision | Choice | Rationale |
|
||
|---|---|---|
|
||
| Execution model | All scripts run from MacBook, SSH into remotes | Single control plane, no agent installs needed |
|
||
| Gitea deployment | Docker Compose on both Unraid + Fedora | Scriptable, reproducible, version-pinned |
|
||
| Runner deployment (Linux) | Docker container via docker-compose | Consistent with Gitea deployment |
|
||
| Runner deployment (macOS) | Native binary + launchd plist | Docker Desktop on Mac is heavyweight; native binary is lighter and more reliable |
|
||
| Idempotency pattern | Check-before-act (query state, skip if already done) | Every create operation first checks if the resource exists |
|
||
| Template rendering | `.tpl` files + `envsubst` | Keeps config templates separate from script logic |
|
||
| Error handling | `set -euo pipefail` + trap cleanup | Fail fast, no silent errors |
|
||
| API interaction | Shared `gitea_api()` curl wrapper with JSON/jq | Consistent auth, error checking, response parsing |
|
||
| HTTPS proxy | Caddy reverse proxy with Cloudflare DNS-01 or existing certs | Automatic TLS with zero-touch renewal; each host gets a dedicated macvlan IP |
|
||
|
||
---
|
||
|
||
## File Structure
|
||
|
||
```
|
||
gitea-migration/
|
||
├── .env.example
|
||
├── .env
|
||
├── runners.conf.example
|
||
├── runners.conf
|
||
├── PLAN.md
|
||
├── lib/
|
||
│ └── common.sh
|
||
├── setup/
|
||
│ ├── configure_env.sh
|
||
│ ├── configure_runners.sh
|
||
│ ├── macbook.sh
|
||
│ ├── unraid.sh
|
||
│ ├── fedora.sh
|
||
│ ├── cross_host_ssh.sh
|
||
│ ├── env_to_bitwarden.sh
|
||
│ ├── bitwarden_to_env.sh
|
||
│ └── cleanup.sh
|
||
├── templates/
|
||
│ ├── docker-compose-gitea.yml.tpl
|
||
│ ├── docker-compose-runner.yml.tpl
|
||
│ ├── app.ini.tpl
|
||
│ ├── runner-config.yaml.tpl
|
||
│ ├── com.gitea.runner.plist.tpl
|
||
│ ├── Caddyfile.tpl
|
||
│ ├── docker-compose-caddy.yml.tpl
|
||
│ ├── com.gitea.runner.newsyslog.conf.tpl
|
||
│ └── workflows/
|
||
│ └── security-scan.yml.tpl
|
||
├── backup/
|
||
│ ├── backup_primary.sh
|
||
│ └── restore_to_primary.sh
|
||
├── contracts/
|
||
│ └── gitea-api.md
|
||
├── preflight.sh
|
||
├── phase1_gitea_unraid.sh
|
||
├── phase1_post_check.sh
|
||
├── phase1_teardown.sh
|
||
├── phase2_gitea_fedora.sh
|
||
├── phase2_post_check.sh
|
||
├── phase2_teardown.sh
|
||
├── phase3_runners.sh
|
||
├── phase3_post_check.sh
|
||
├── phase3_teardown.sh
|
||
├── phase4_migrate_repos.sh
|
||
├── phase4_post_check.sh
|
||
├── phase4_teardown.sh
|
||
├── phase5_migrate_pipelines.sh
|
||
├── phase5_post_check.sh
|
||
├── phase5_teardown.sh
|
||
├── phase6_github_mirrors.sh
|
||
├── phase6_post_check.sh
|
||
├── phase6_teardown.sh
|
||
├── phase7_branch_protection.sh
|
||
├── phase7_post_check.sh
|
||
├── phase7_teardown.sh
|
||
├── phase8_cutover.sh
|
||
├── phase8_post_check.sh
|
||
├── phase8_teardown.sh
|
||
├── phase9_security.sh
|
||
├── phase9_post_check.sh
|
||
├── phase9_teardown.sh
|
||
├── manage_runner.sh
|
||
├── run_all.sh
|
||
└── teardown_all.sh
|
||
```
|
||
|
||
---
|
||
|
||
## Implementation Tracker
|
||
|
||
### 1. Foundation
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 1.1 | `lib/common.sh` | Shared functions: logging, SSH, API wrappers, template rendering, checks | DONE |
|
||
| 1.2 | `.env.example` | All env vars: TLS_MODE, macvlan networking, DB support, Caddy config | DONE |
|
||
| 1.3 | `contracts/gitea-api.md` | Gitea REST API endpoints used across all phases | DONE |
|
||
|
||
### 2. Templates
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 2.1 | `templates/docker-compose-gitea.yml.tpl` | Gitea + DB docker-compose (sqlite3/mysql/postgres/mssql) | DONE |
|
||
| 2.2 | `templates/app.ini.tpl` | Gitea custom config (INSTALL_LOCK, Actions enabled, etc.) | DONE |
|
||
| 2.3 | `templates/docker-compose-runner.yml.tpl` | act_runner docker-compose (Linux) | DONE |
|
||
| 2.4 | `templates/runner-config.yaml.tpl` | act_runner config | DONE |
|
||
| 2.5 | `templates/com.gitea.runner.plist.tpl` | macOS launchd service for act_runner | DONE |
|
||
| 2.6 | `templates/com.gitea.runner.newsyslog.conf.tpl` | macOS log rotation for native runner | DONE |
|
||
| 2.7 | `templates/Caddyfile.tpl` + `docker-compose-caddy.yml.tpl` | Caddy reverse proxy with Cloudflare DNS-01 | DONE |
|
||
| 2.8 | `templates/workflows/security-scan.yml.tpl` | Semgrep + Trivy + Gitleaks workflow | DONE |
|
||
|
||
### 3. Machine Setup
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 3.1 | `setup/configure_env.sh` | Interactive wizard: prompts for each .env var, writes to .env | DONE |
|
||
| 3.2 | `setup/macbook.sh` | Homebrew, jq, curl, envsubst, git, Xcode CLI Tools, shellcheck, gh | DONE |
|
||
| 3.3 | `setup/unraid.sh` | Verify Docker, install docker-compose + jq (static binary) | DONE |
|
||
| 3.4 | `setup/fedora.sh` | Install Docker CE, compose plugin, jq, enable systemd services | DONE |
|
||
| 3.5 | `setup/configure_runners.sh` | Interactive runner definition wizard, writes runners.conf | DONE |
|
||
| 3.6 | `setup/cross_host_ssh.sh` | SSH key exchange between Unraid and Fedora | DONE |
|
||
| 3.7 | `setup/env_to_bitwarden.sh` | Export .env to Bitwarden JSON import format | DONE |
|
||
| 3.8 | `setup/bitwarden_to_env.sh` | Restore .env from Bitwarden CLI | DONE |
|
||
| 3.9 | `setup/cleanup.sh` | Manifest-driven rollback of setup scripts | DONE |
|
||
|
||
### 4. Preflight
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 4.1 | `preflight.sh` | Validate .env, SSH, Docker, IPs, DNS, GitHub token, Caddy, repos | DONE |
|
||
|
||
### 5. Phase 1 — Gitea on Unraid
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 5.1 | `phase1_gitea_unraid.sh` | Deploy Gitea container, create admin user + token + org | DONE |
|
||
| 5.2 | `phase1_post_check.sh` | Verify Gitea HTTP 200, admin auth, token valid, org exists | DONE |
|
||
| 5.3 | `phase1_teardown.sh` | docker-compose down, optionally remove data | DONE |
|
||
|
||
### 6. Phase 2 — Gitea on Fedora
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 6.1 | `phase2_gitea_fedora.sh` | Deploy Gitea container on Fedora, create admin user + token | DONE |
|
||
| 6.2 | `phase2_post_check.sh` | Verify Fedora Gitea HTTP 200, admin auth, token valid | DONE |
|
||
| 6.3 | `phase2_teardown.sh` | docker-compose down on Fedora | DONE |
|
||
|
||
### 7. Phase 3 — Runners
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 7.1 | `runners.conf.example` | Runner definition format + example entries | DONE |
|
||
| 7.2 | `manage_runner.sh` | Add/remove/list runners dynamically (reads runners.conf) | DONE |
|
||
| 7.3 | `phase3_runners.sh` | Get registration token, deploy all runners defined in runners.conf | DONE |
|
||
| 7.4 | `phase3_post_check.sh` | Verify all runners from runners.conf are online in Gitea admin | DONE |
|
||
| 7.5 | `phase3_teardown.sh` | Stop + deregister all runners from runners.conf | DONE |
|
||
|
||
### 8. Phase 4 — Migrate Repos + Fedora Mirrors
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 8.1 | `phase4_migrate_repos.sh` | Import repos from GitHub, set up Fedora pull mirrors | DONE |
|
||
| 8.2 | `phase4_post_check.sh` | Verify repos on primary + mirror repos on Fedora | DONE |
|
||
| 8.3 | `phase4_teardown.sh` | Delete repos from primary + Fedora | DONE |
|
||
|
||
### 9. Phase 5 — Migrate Pipelines
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 9.1 | `phase5_migrate_pipelines.sh` | Copy .github/workflows/ → .gitea/workflows/, apply compat fixes | DONE |
|
||
| 9.2 | `phase5_post_check.sh` | Verify workflows visible in Gitea Actions UI | DONE |
|
||
| 9.3 | `phase5_teardown.sh` | Remove .gitea/workflows/ from repos | DONE |
|
||
|
||
### 10. Phase 6 — GitHub Push Mirrors
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 10.1 | `phase6_github_mirrors.sh` | Configure push mirrors from Gitea → GitHub | DONE |
|
||
| 10.2 | `phase6_post_check.sh` | Verify mirror config, trigger sync, check GitHub | DONE |
|
||
| 10.3 | `phase6_teardown.sh` | Remove push mirror config | DONE |
|
||
|
||
### 11. Phase 7 — Branch Protection
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 11.1 | `phase7_branch_protection.sh` | Set up branch protection rules on all repos | DONE |
|
||
| 11.2 | `phase7_post_check.sh` | Verify protection rules exist | DONE |
|
||
| 11.3 | `phase7_teardown.sh` | Delete branch protection rules | DONE |
|
||
|
||
### 12. Phase 8 — Cutover (HTTPS + Archive GitHub)
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 12.1 | `phase8_cutover.sh` | Caddy HTTPS reverse proxy + mark GitHub repos as mirrors | DONE |
|
||
| 12.2 | `phase8_post_check.sh` | Verify HTTPS, repos accessible, mirrors working | DONE |
|
||
| 12.3 | `phase8_teardown.sh` | Remove Caddy stack, restore GitHub repo settings | DONE |
|
||
|
||
### 13. Phase 9 — Security Scanning
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 13.1 | `phase9_security.sh` | Deploy security workflow (Semgrep+Trivy+Gitleaks) to all repos | DONE |
|
||
| 13.2 | `phase9_post_check.sh` | Verify workflows exist, dry-run passes, branch protection updated | DONE |
|
||
| 13.3 | `phase9_teardown.sh` | Remove security workflows | DONE |
|
||
|
||
### 14. Backup & Restore (post-migration operational scripts)
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 14.1 | `backup/backup_primary.sh` | Run `gitea dump` on Unraid (DB + repos + config + users), SCP archive to Fedora | DONE |
|
||
| 14.2 | `backup/restore_to_primary.sh` | Restore a `gitea dump` archive to Unraid (fresh or existing instance) | DONE |
|
||
|
||
### 15. Orchestration
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 15.1 | `run_all.sh` | Run setup → preflight → phases 1-9 sequentially, --start-from=N | DONE |
|
||
| 15.2 | `teardown_all.sh` | Run teardowns in reverse, --through=N | DONE |
|
||
|
||
### 16. Project Init
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 16.1 | Git repo init + .gitignore | Initialize git repo, ignore .env and temp files | DONE |
|
||
| 16.2 | `CLAUDE.md` | Project-specific instructions for this codebase | DONE |
|
||
|
||
### 17. Validation
|
||
|
||
| # | File | Description | Status |
|
||
|---|------|-------------|--------|
|
||
| 17.1 | Shellcheck all `.sh` files | Must pass with no errors | DONE |
|
||
| 17.2 | `bash -n` syntax check all scripts | Verify syntax without executing | DONE |
|
||
|
||
---
|
||
|
||
## Definition of Done — Every Item
|
||
|
||
### 1.1 — `lib/common.sh`
|
||
|
||
**Depends on**: nothing
|
||
**Steps**:
|
||
1. Write shell library with `set -euo pipefail`
|
||
2. Implement every function listed below
|
||
3. Every function handles errors (non-zero exit, clear message)
|
||
|
||
**Functions and their contracts**:
|
||
|
||
| Function | Inputs | Behavior | Returns |
|
||
|----------|--------|----------|---------|
|
||
| `load_env` | none | Sources `.env`, exports all vars, and derives `GITEA_INTERNAL_URL`/`GITEA_BACKUP_INTERNAL_URL` from `UNRAID_GITEA_IP`/`FEDORA_GITEA_IP`. Exits 1 if `.env` missing. | 0 on success |
|
||
| `save_env_var KEY VALUE` | key name, value string | If KEY exists in `.env`, replaces its line. If not, appends. Must not corrupt other lines. | 0 on success |
|
||
| `require_vars VAR1 VAR2...` | variable names | For each: checks if exported and non-empty. Exits 1 naming the **first** missing var. | 0 if all set |
|
||
| `log_info MSG` | message string | Prints `[INFO] MSG` to stderr in blue | — |
|
||
| `log_warn MSG` | message string | Prints `[WARN] MSG` to stderr in yellow | — |
|
||
| `log_error MSG` | message string | Prints `[ERROR] MSG` to stderr in red | — |
|
||
| `log_success MSG` | message string | Prints `[OK] MSG` to stderr in green | — |
|
||
| `log_step N MSG` | step number, message | Prints ` [N] MSG` to stderr | — |
|
||
| `phase_header NUM NAME` | phase number, name | Prints `\n=== Phase NUM: NAME ===\n` to stderr | — |
|
||
| `ssh_exec HOST_KEY CMD` | host key (e.g. UNRAID), command string | Reads `${HOST_KEY}_IP`, `${HOST_KEY}_SSH_USER`, `${HOST_KEY}_SSH_PORT` from env. Runs command via SSH with `-o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new`. Streams stdout/stderr. Exits with remote exit code. | remote exit code |
|
||
| `ssh_check HOST_KEY` | host key | Same as ssh_exec but runs `true`. Returns 0 if reachable, 1 if not. Does NOT exit on failure. | 0 or 1 |
|
||
| `scp_to HOST_KEY SRC DST` | host key, local file path, remote file path | Copies local file to remote via SCP. Exits 1 if transfer fails. | 0 on success |
|
||
| `gitea_api METHOD PATH [DATA]` | HTTP method, API path (e.g. `/orgs`), optional JSON body | Calls `curl -s -X METHOD ${GITEA_INTERNAL_URL}/api/v1${PATH}` with `Authorization: token ${GITEA_ADMIN_TOKEN}`. Exits 1 if HTTP status >= 400 (prints status + response body). Returns JSON response on stdout. | JSON on stdout |
|
||
| `gitea_backup_api METHOD PATH [DATA]` | same | Same as gitea_api but uses `GITEA_BACKUP_INTERNAL_URL` + `GITEA_BACKUP_ADMIN_TOKEN` | JSON on stdout |
|
||
| `github_api METHOD PATH [DATA]` | same | Calls `https://api.github.com${PATH}` with `Authorization: token ${GITHUB_TOKEN}` | JSON on stdout |
|
||
| `render_template SRC DEST` | .tpl file path, output path | Runs `envsubst` with all exported vars. Writes to DEST. | 0 on success |
|
||
| `wait_for_http URL MAX_SECS` | URL, timeout in seconds | Polls with `curl -sf -o /dev/null` every 2 seconds. Exits 1 with timeout message if MAX_SECS exceeded. | 0 when URL returns 200 |
|
||
| `wait_for_ssh HOST_KEY MAX_SECS` | host key, timeout | Polls `ssh_check` every 2 seconds. Exits 1 on timeout. | 0 when SSH connects |
|
||
|
||
**Done when**:
|
||
- [ ] Every function above exists with exact signature
|
||
- [ ] `shellcheck lib/common.sh` passes with zero warnings
|
||
- [ ] `bash -n lib/common.sh` passes
|
||
- [ ] Sourcing the file (`source lib/common.sh`) does NOT execute anything — functions only, no side effects at source time
|
||
- [ ] `save_env_var` tested: set a var, read it back, value matches. Set it again with different value, only one line exists for that key.
|
||
- [ ] `require_vars` tested: prints the missing var's name, not a generic error
|
||
- [ ] `ssh_exec` uses ConnectTimeout to avoid hanging forever
|
||
- [ ] API functions return raw JSON on stdout (not mixed with log messages — logs go to stderr)
|
||
|
||
---
|
||
|
||
### 1.3 — `contracts/gitea-api.md`
|
||
|
||
**Depends on**: knowing all API calls used across phases 1-9
|
||
**Steps**:
|
||
1. For every Gitea API call in the project, document: method, path, request body, expected response, which script uses it
|
||
2. For every GitHub API call, same treatment
|
||
3. Cross-reference: every `gitea_api`/`github_api` call in any script MUST have a matching entry
|
||
|
||
**API endpoints to document** (complete list):
|
||
|
||
| Endpoint | Used in |
|
||
|----------|---------|
|
||
| `POST /api/v1/users/{user}/tokens` | Phase 1, Phase 2 (generate admin token) |
|
||
| `GET /api/v1/user` | Preflight (validate token) |
|
||
| `POST /api/v1/orgs` | Phase 1 (create org) |
|
||
| `GET /api/v1/orgs/{org}` | Phase 1 (check if org exists) |
|
||
| `POST /api/v1/repos/migrate` | Phase 4 (import repos + Fedora mirrors) |
|
||
| `GET /api/v1/repos/{owner}/{repo}` | Phase 4 post-check (verify repo exists) |
|
||
| `GET /api/v1/admin/runners/registration-token` | Phase 3 (get runner token) |
|
||
| `GET /api/v1/admin/runners` | Phase 3 post-check (list runners) |
|
||
| `POST /api/v1/repos/{owner}/{repo}/push_mirrors` | Phase 6 (create push mirror) |
|
||
| `GET /api/v1/repos/{owner}/{repo}/push_mirrors` | Phase 6 post-check |
|
||
| `POST /api/v1/repos/{owner}/{repo}/push_mirrors-sync` | Phase 6 post-check (trigger sync) |
|
||
| `POST /api/v1/repos/{owner}/{repo}/branch_protections` | Phase 7 |
|
||
| `GET /api/v1/repos/{owner}/{repo}/branch_protections` | Phase 7 post-check |
|
||
| `DELETE /api/v1/repos/{owner}/{repo}/branch_protections/{name}` | Phase 7 teardown |
|
||
| `DELETE /api/v1/repos/{owner}/{repo}` | Phase 4 teardown |
|
||
| `GET /api/v1/repos/{owner}/{repo}/actions/workflows` | Phase 5 post-check |
|
||
| GitHub: `GET /user` | Preflight (validate GitHub token) |
|
||
| GitHub: `GET /repos/{owner}/{repo}` | Preflight (verify repos exist) |
|
||
| GitHub: `PATCH /repos/{owner}/{repo}` | Phase 8 (archive repos) |
|
||
|
||
**Done when**:
|
||
- [ ] Every endpoint above is documented with: method, full path, request body schema, response schema, HTTP status codes
|
||
- [ ] Every `gitea_api`/`github_api` call in the codebase has a corresponding entry
|
||
- [ ] No endpoint is used in code but missing from the contract
|
||
|
||
---
|
||
|
||
### 2.1 — `templates/docker-compose-gitea.yml.tpl`
|
||
|
||
**Depends on**: .env vars defined
|
||
**Produces**: Docker Compose file for Gitea + SQLite on a single host
|
||
|
||
**Template must include**:
|
||
- Gitea image pinned to `$GITEA_VERSION`
|
||
- Container name: `gitea`
|
||
- Volumes: `$DATA_PATH/data:/data`, `$DATA_PATH/config:/data/gitea/conf`
|
||
- Networks: macvlan with `$GITEA_CONTAINER_IP` (dedicated LAN IP, no port mapping)
|
||
- Environment: `USER_UID=1000`, `USER_GID=1000`
|
||
- Restart policy: `unless-stopped`
|
||
- No database service for sqlite3; conditional DB service block for external DBs
|
||
|
||
**Variables used**: `GITEA_VERSION`, `DATA_PATH`, `GITEA_CONTAINER_IP` (+ DB vars if external DB)
|
||
|
||
**Done when**:
|
||
- [ ] `render_template` produces valid YAML (test with `python3 -c "import yaml; yaml.safe_load(open('output.yml'))"` or `yq`)
|
||
- [ ] No unexpanded `$VAR` or `${VAR}` tokens in rendered output
|
||
- [ ] Container name is `gitea` (hardcoded, not variable — needed for `docker exec`)
|
||
- [ ] Volume paths use the variable, not hardcoded paths
|
||
|
||
---
|
||
|
||
### 2.2 — `templates/app.ini.tpl`
|
||
|
||
**Depends on**: .env vars defined
|
||
**Produces**: Gitea configuration file
|
||
|
||
**Must set**:
|
||
- `[security] INSTALL_LOCK = true` — skip install wizard
|
||
- `[server] ROOT_URL = https://$GITEA_DOMAIN/`
|
||
- `[server] SSH_DOMAIN = $GITEA_DOMAIN`
|
||
- `[server] DOMAIN = $GITEA_DOMAIN`
|
||
- `[database] DB_TYPE = $GITEA_DB_TYPE`
|
||
- `[database] PATH = /data/gitea/gitea.db`
|
||
- `[service] DISABLE_REGISTRATION = true` — no public signups
|
||
- `[actions] ENABLED = true` — enable Gitea Actions
|
||
- `[repository] DEFAULT_BRANCH = main`
|
||
- `[mailer]` section left unconfigured (can be added later)
|
||
|
||
**Done when**:
|
||
- [ ] Rendered output is valid INI format
|
||
- [ ] `INSTALL_LOCK = true` is present (critical — without it, Gitea shows install wizard)
|
||
- [ ] `ENABLED = true` under `[actions]` (critical — without it, runners can't connect)
|
||
- [ ] No unexpanded variables
|
||
|
||
---
|
||
|
||
### 2.3 — `templates/docker-compose-runner.yml.tpl`
|
||
|
||
**Depends on**: .env vars, runner config
|
||
**Produces**: Docker Compose file for act_runner on Linux
|
||
|
||
**Must include**:
|
||
- act_runner image: `gitea/act_runner:$ACT_RUNNER_VERSION`
|
||
- Container name: `gitea-runner-$RUNNER_NAME`
|
||
- Volume: Docker socket mount `/var/run/docker.sock:/var/run/docker.sock` (needed to spawn job containers)
|
||
- Volume: `$RUNNER_DATA_PATH:/data`
|
||
- Environment: `GITEA_INSTANCE_URL`, `GITEA_RUNNER_REGISTRATION_TOKEN`, `GITEA_RUNNER_NAME`, `GITEA_RUNNER_LABELS`
|
||
- Restart policy: `unless-stopped`
|
||
|
||
**Done when**:
|
||
- [ ] Rendered output is valid YAML
|
||
- [ ] Docker socket is mounted (without this, runner can't create job containers)
|
||
- [ ] Runner name and labels come from variables (not hardcoded)
|
||
|
||
---
|
||
|
||
### 2.4 — `templates/runner-config.yaml.tpl`
|
||
|
||
**Depends on**: act_runner config schema
|
||
**Produces**: act_runner configuration file
|
||
|
||
**Must set**:
|
||
- `runner.name: $RUNNER_NAME`
|
||
- `runner.labels: $RUNNER_LABELS`
|
||
- `runner.capacity: 1` (one concurrent job per runner — safe default)
|
||
- `runner.timeout: 3h` (max job duration)
|
||
- `cache.enabled: true`
|
||
|
||
**Done when**:
|
||
- [ ] Valid YAML after rendering
|
||
- [ ] Labels format is correct for act_runner (comma-separated `label:scheme` pairs)
|
||
|
||
---
|
||
|
||
### 2.5 — `templates/com.gitea.runner.plist.tpl`
|
||
|
||
**Depends on**: macOS launchd plist schema
|
||
**Produces**: launchd service definition for act_runner on macOS
|
||
|
||
**Must include**:
|
||
- Label: `com.gitea.runner.$RUNNER_NAME`
|
||
- ProgramArguments: path to act_runner binary, `daemon` subcommand
|
||
- WorkingDirectory: `$RUNNER_DATA_PATH`
|
||
- RunAtLoad: true
|
||
- KeepAlive: true
|
||
- StandardOutPath / StandardErrorPath: log file locations
|
||
|
||
**Done when**:
|
||
- [ ] Valid XML plist (test with `plutil -lint`)
|
||
- [ ] Binary path matches where `manage_runner.sh` installs it
|
||
- [ ] KeepAlive ensures runner restarts if it crashes
|
||
|
||
---
|
||
|
||
### 2.6 — `templates/Caddyfile.tpl` + `docker-compose-caddy.yml.tpl`
|
||
|
||
**Depends on**: .env vars (`GITEA_DOMAIN`, `TLS_BLOCK`, `GITEA_CONTAINER_IP`, `CADDY_DATA_PATH`, `CADDY_CONTAINER_IP`)
|
||
**Produces**: Caddy reverse proxy config + Docker Compose for Caddy container
|
||
|
||
**Caddyfile must include**:
|
||
- `${GITEA_DOMAIN}` as the site address
|
||
- `${TLS_BLOCK}` placeholder (script sets to `tls { dns cloudflare {env.CF_API_TOKEN} }` or `tls /path/cert /path/key`)
|
||
- `reverse_proxy ${GITEA_CONTAINER_IP}:3000`
|
||
|
||
**Docker Compose must include**:
|
||
- `slothcroissant/caddy-cloudflaredns:latest` image
|
||
- Volume mounts for Caddyfile, data, and config
|
||
- macvlan network with static IP (`CADDY_CONTAINER_IP`)
|
||
- Conditional `CF_API_TOKEN` env var and cert volume mounts based on TLS mode
|
||
|
||
**Done when**:
|
||
- [ ] Caddy starts and obtains TLS certificate
|
||
- [ ] HTTPS proxy works to Gitea
|
||
- [ ] HTTP redirects to HTTPS
|
||
|
||
---
|
||
|
||
### 2.7 — `templates/workflows/security-scan.yml.tpl`
|
||
|
||
**Depends on**: Gitea Actions workflow syntax
|
||
**Produces**: Shared security workflow for all repos
|
||
|
||
**Must include**:
|
||
- `on: [pull_request]` trigger
|
||
- Three jobs: `semgrep`, `trivy`, `gitleaks`
|
||
- Semgrep: `returntocorp/semgrep:$SEMGREP_VERSION` Docker image, `semgrep scan --config auto .`
|
||
- Trivy: `aquasec/trivy:$TRIVY_VERSION`, `trivy fs --exit-code 1 --severity HIGH,CRITICAL .`
|
||
- Gitleaks: `zricethezav/gitleaks:$GITLEAKS_VERSION`, `gitleaks detect --source . --exit-code 1`
|
||
- All three jobs must report as status checks (default behavior in Gitea Actions)
|
||
|
||
**Done when**:
|
||
- [ ] Valid YAML after rendering
|
||
- [ ] All three tools pinned to version vars (not `latest` hardcoded)
|
||
- [ ] Each job exits non-zero on findings (so branch protection can block merge)
|
||
- [ ] `runs-on` labels match what Linux runners advertise in `runners.conf`
|
||
|
||
---
|
||
|
||
### 3.1 — `setup/configure_env.sh`
|
||
|
||
**Depends on**: `.env.example` exists
|
||
**Runs**: locally on MacBook (interactive — requires terminal input)
|
||
**Purpose**: Guided wizard that prompts for every required `.env` variable, validates input, and writes a complete `.env` file.
|
||
|
||
**Behavior**:
|
||
1. If `.env` already exists, load current values as defaults (shown in brackets, press Enter to keep)
|
||
2. If `.env` does not exist, copy `.env.example` to `.env` first
|
||
3. Walk through each required variable in section order, prompting with:
|
||
- Progress indicator: `[12/47] ── GITEA SHARED CREDENTIALS ──────────────────`
|
||
- Variable name and description (from `.env.example` comments)
|
||
- Current value if any (from existing `.env`): `UNRAID_IP [192.168.1.10]: `
|
||
- Empty prompt if no current value: `UNRAID_IP (Static IP of Unraid server): `
|
||
4. Basic input validation per variable:
|
||
- IP addresses: regex match `^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$`
|
||
- Ports: integer between 1-65535
|
||
- Passwords: minimum 8 characters
|
||
- Emails: contains `@`
|
||
- Paths: starts with `/`
|
||
- Optional paths: empty or starts with `/` or `~/`
|
||
5. After all prompts, write values to `.env` preserving the file structure (comments, sections)
|
||
6. Do NOT prompt for auto-populated vars (`GITEA_ADMIN_TOKEN`, `GITEA_BACKUP_ADMIN_TOKEN`, `GITEA_RUNNER_REGISTRATION_TOKEN`) or derived vars (`GITEA_INTERNAL_URL`, `GITEA_BACKUP_INTERNAL_URL`)
|
||
7. Do NOT prompt for vars with defaults unless user wants to change them — show default, press Enter to accept
|
||
8. Print summary at end showing all configured values (passwords masked)
|
||
|
||
**Variable prompt order** (matches `.env.example` sections):
|
||
|
||
| # | Variable | Validation | Default |
|
||
|---|----------|------------|---------|
|
||
| 1-5 | `UNRAID_IP`, `UNRAID_SSH_USER`, `UNRAID_SSH_PORT`, `UNRAID_GITEA_DATA_PATH`, `UNRAID_SSH_KEY` | IP, non-empty, port, path, optional | —, —, 22, —, — |
|
||
| 6-10 | `FEDORA_IP`, `FEDORA_SSH_USER`, `FEDORA_SSH_PORT`, `FEDORA_GITEA_DATA_PATH`, `FEDORA_SSH_KEY` | IP, non-empty, port, path, optional | —, —, 22, —, — |
|
||
| 11-22 | Macvlan networking: `*_MACVLAN_PARENT`, `*_MACVLAN_SUBNET`, `*_MACVLAN_GATEWAY`, `*_MACVLAN_IP_RANGE`, `*_GITEA_IP`, `UNRAID_CADDY_IP` (per host) | non-empty, non-empty, IP, non-empty, IP, IP | — |
|
||
| 25-30 | `GITEA_ADMIN_USER`, `GITEA_ADMIN_PASSWORD`, `GITEA_ADMIN_EMAIL`, `GITEA_ORG_NAME`, `GITEA_INSTANCE_NAME`, `GITEA_DB_TYPE` | non-empty, password, email, non-empty, non-empty, db_type | —, —, —, —, —, sqlite3 |
|
||
| 29-32 | *(conditional, only if DB_TYPE != sqlite3)* `GITEA_DB_PORT`, `GITEA_DB_NAME`, `GITEA_DB_USER`, `GITEA_DB_PASSWD` | port, non-empty, non-empty, password | auto, gitea, gitea, — |
|
||
| 36-37 | `GITEA_VERSION`, `ACT_RUNNER_VERSION` | Non-empty | 1.25, 0.3.0 |
|
||
| 38 | `GITEA_DOMAIN` | Non-empty | — |
|
||
| 39-41 | `GITEA_BACKUP_MIRROR_INTERVAL`, `BACKUP_STORAGE_PATH`, `BACKUP_RETENTION_COUNT` | non-empty, path, integer | 8h, —, 5 |
|
||
| 42-43 | `GITHUB_USERNAME`, `GITHUB_TOKEN` | Non-empty | — |
|
||
| 44 | "How many repos?" + N × repo names → `REPO_NAMES` | positive integer, non-empty | — |
|
||
| 45-51 | `MIGRATE_ISSUES`, `MIGRATE_LABELS`, `MIGRATE_MILESTONES`, `MIGRATE_WIKI`, `MIGRATION_POLL_INTERVAL_SEC`, `MIGRATION_POLL_TIMEOUT_SEC`, `GITHUB_MIRROR_INTERVAL` | bool, bool, bool, bool, positive_integer, positive_integer, non-empty | false, true, false, false, 3, 600, 8h |
|
||
| 52-53 | `RUNNER_DEFAULT_IMAGE`, `LOCAL_REGISTRY` | non-empty, optional | catthehacker/ubuntu:act-latest, — |
|
||
| 54-56 | `TLS_MODE`, `CADDY_DOMAIN`, `CADDY_DATA_PATH` | tls_mode, non-empty, path | cloudflare, —, — |
|
||
| 57-59 | *(conditional)* `CLOUDFLARE_API_TOKEN` or `SSL_CERT_PATH` + `SSL_KEY_PATH` | non-empty / path | — |
|
||
| 60-62 | `PROTECTED_BRANCH`, `REQUIRE_PR_REVIEW`, `REQUIRED_APPROVALS` | Non-empty, bool, integer | main, false, 1 |
|
||
| 63-66 | `SEMGREP_VERSION`, `TRIVY_VERSION`, `GITLEAKS_VERSION`, `SECURITY_FAIL_ON_ERROR` | Non-empty, non-empty, non-empty, bool | latest, latest, latest, true |
|
||
|
||
**Done when**:
|
||
- [x] Each prompt shows progress: `[N/~66]` with section header when entering a new section
|
||
- [ ] Running with no existing `.env` walks through all prompts and produces a valid `.env`
|
||
- [ ] TLS prompts are conditional: if `TLS_MODE=cloudflare`, prompt for `CLOUDFLARE_API_TOKEN` only; if `TLS_MODE=existing`, prompt for `SSL_CERT_PATH` and `SSL_KEY_PATH` only
|
||
- [ ] DB prompts are conditional: if `GITEA_DB_TYPE` is not `sqlite3`, prompt for host/port/name/user/password
|
||
- [ ] Running with an existing `.env` shows current values and only overwrites what user changes
|
||
- [ ] Invalid input (bad IP, path not starting with `/`, password too short) re-prompts with error message
|
||
- [ ] Auto-populated vars are skipped entirely (no prompt, no overwrite)
|
||
- [ ] Summary at end shows all values (passwords masked with `****`)
|
||
- [ ] `.env` file preserves comment structure from `.env.example`
|
||
- [ ] `shellcheck setup/configure_env.sh` passes
|
||
|
||
---
|
||
|
||
### 3.2 — `setup/macbook.sh`
|
||
|
||
**Depends on**: nothing (first script to run)
|
||
**Runs**: locally on MacBook
|
||
|
||
**Steps**:
|
||
1. Check for Homebrew → if missing, print install command and exit (don't auto-install — it's interactive)
|
||
2. `brew install jq curl gettext shellcheck gh` — skip any already installed
|
||
3. Verify built-in tools exist: `ssh`, `git`, `scp`, `envsubst` (from gettext)
|
||
4. Check Xcode CLI Tools: `xcode-select -p` → if fails, run `xcode-select --install` and wait
|
||
5. Print summary of what was installed vs already present
|
||
|
||
**Idempotency**: Running twice produces no errors and no redundant installs.
|
||
|
||
**Done when**:
|
||
- [ ] Running on a Mac where all tools exist prints "All prerequisites satisfied" and exits 0
|
||
- [ ] Running on a Mac missing `jq` installs it via brew and exits 0
|
||
- [ ] Script does NOT install Homebrew automatically (security — user should do it themselves)
|
||
- [ ] `envsubst --version` works after script runs (this is the most commonly missing tool)
|
||
- [ ] `shellcheck setup/macbook.sh` passes
|
||
|
||
---
|
||
|
||
### 3.3 — `setup/unraid.sh`
|
||
|
||
**Depends on**: SSH access to Unraid (.env vars: UNRAID_IP, UNRAID_SSH_USER, UNRAID_SSH_PORT)
|
||
**Runs**: from MacBook via SSH into Unraid
|
||
|
||
**Steps**:
|
||
1. `ssh_check UNRAID` — fail if can't connect
|
||
2. Check Docker: `ssh_exec UNRAID "docker --version"` — if missing, exit 1 with message "Docker not found on Unraid. Install it via Unraid's Docker settings."
|
||
3. Check docker-compose: try `docker compose version`, then `docker-compose --version`. If neither works, download standalone docker-compose binary to `/usr/local/bin/`
|
||
4. Check jq: `ssh_exec UNRAID "jq --version"` — if missing, download static binary from GitHub releases to `/usr/local/bin/jq`, chmod +x
|
||
5. Verify data path: `ssh_exec UNRAID "mkdir -p $UNRAID_GITEA_DATA_PATH && touch $UNRAID_GITEA_DATA_PATH/.write-test && rm $UNRAID_GITEA_DATA_PATH/.write-test"`
|
||
|
||
**Done when**:
|
||
- [ ] `docker --version` works on Unraid via SSH
|
||
- [ ] `docker compose version` OR `docker-compose --version` works on Unraid via SSH
|
||
- [ ] `jq --version` works on Unraid via SSH
|
||
- [ ] `UNRAID_GITEA_DATA_PATH` exists and is writable
|
||
- [ ] Script does NOT attempt to install Docker on Unraid (could break Unraid's custom setup)
|
||
- [ ] `shellcheck setup/unraid.sh` passes
|
||
|
||
---
|
||
|
||
### 3.4 — `setup/fedora.sh`
|
||
|
||
**Depends on**: SSH access to Fedora (.env vars: FEDORA_IP, FEDORA_SSH_USER, FEDORA_SSH_PORT)
|
||
**Runs**: from MacBook via SSH into Fedora
|
||
|
||
**Steps**:
|
||
1. `ssh_check FEDORA` — fail if can't connect
|
||
2. Check Docker: `docker --version` — if missing:
|
||
- `sudo dnf -y install dnf-plugins-core`
|
||
- `sudo dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo`
|
||
- `sudo dnf -y install docker-ce docker-ce-cli containerd.io docker-compose-plugin`
|
||
- `sudo systemctl enable --now docker`
|
||
- `sudo usermod -aG docker $FEDORA_SSH_USER`
|
||
- Print warning: "User added to docker group. You may need to re-login for this to take effect."
|
||
3. Check jq: if missing, `sudo dnf -y install jq`
|
||
4. Verify Docker works: `docker run --rm hello-world` (must succeed without sudo after group membership)
|
||
5. Verify data path writable (same as Unraid step)
|
||
|
||
**Done when**:
|
||
- [ ] `docker --version` works on Fedora via SSH
|
||
- [ ] `docker compose version` works on Fedora via SSH
|
||
- [ ] `jq --version` works on Fedora via SSH
|
||
- [ ] `docker run --rm hello-world` succeeds **without sudo** (if it requires sudo, the docker group membership hasn't taken effect — script must warn about re-login)
|
||
- [ ] `FEDORA_GITEA_DATA_PATH` exists and is writable
|
||
- [ ] `shellcheck setup/fedora.sh` passes
|
||
|
||
---
|
||
|
||
### 4.1 — `preflight.sh`
|
||
|
||
**Depends on**: setup scripts completed, `.env` populated
|
||
**Runs**: locally on MacBook
|
||
**Purpose**: Pure validation. Installs nothing. Exits 0 only if EVERYTHING is ready.
|
||
|
||
**Checks** (each prints PASS/FAIL with specific message):
|
||
|
||
| # | Check | Pass condition | Fail message |
|
||
|---|-------|----------------|--------------|
|
||
| 1 | `.env` exists | File present in project root | ".env not found. Copy .env.example to .env and fill in values." |
|
||
| 2 | `runners.conf` exists | File present in project root | "runners.conf not found. Copy runners.conf.example to runners.conf." |
|
||
| 3 | Required .env vars set | Every var in the list below is non-empty | "Missing required var: VAR_NAME" |
|
||
|
||
**Check #3 — Required variables** (must be non-empty):
|
||
|
||
| Section | Variable |
|
||
|---------|----------|
|
||
| Unraid | `UNRAID_IP` |
|
||
| Unraid | `UNRAID_SSH_USER` |
|
||
| Unraid | `UNRAID_GITEA_DATA_PATH` |
|
||
| Fedora | `FEDORA_IP` |
|
||
| Fedora | `FEDORA_SSH_USER` |
|
||
| Fedora | `FEDORA_GITEA_DATA_PATH` |
|
||
| Shared creds | `GITEA_ADMIN_USER` |
|
||
| Shared creds | `GITEA_ADMIN_PASSWORD` |
|
||
| Shared creds | `GITEA_ADMIN_EMAIL` |
|
||
| Shared creds | `GITEA_ORG_NAME` |
|
||
| Shared creds | `GITEA_INSTANCE_NAME` |
|
||
| Primary | `GITEA_DOMAIN` |
|
||
| Backup | `BACKUP_STORAGE_PATH` |
|
||
| Repos | `GITHUB_USERNAME` |
|
||
| Repos | `GITHUB_TOKEN` |
|
||
| Repos | `REPO_NAMES` |
|
||
| Mirror | `GITHUB_TOKEN` |
|
||
| TLS/Caddy | `TLS_MODE` |
|
||
| TLS/Caddy | `CADDY_DOMAIN` |
|
||
| TLS/Caddy | `CADDY_DATA_PATH` |
|
||
|
||
**Not checked** (have defaults or auto-populated):
|
||
`UNRAID_SSH_PORT`, `FEDORA_SSH_PORT`, `GITEA_DB_TYPE`, `GITEA_VERSION`, `ACT_RUNNER_VERSION`, `GITEA_INTERNAL_URL`, `GITEA_BACKUP_INTERNAL_URL`, `GITEA_BACKUP_MIRROR_INTERVAL`, `BACKUP_RETENTION_COUNT`, `MIGRATE_*`, `GITHUB_MIRROR_INTERVAL`, `PROTECTED_BRANCH`, `REQUIRE_PR_REVIEW`, `REQUIRED_APPROVALS`, `SEMGREP_VERSION`, `TRIVY_VERSION`, `GITLEAKS_VERSION`, `SECURITY_FAIL_ON_ERROR`, `GITEA_ADMIN_TOKEN`, `GITEA_BACKUP_ADMIN_TOKEN`, `GITEA_RUNNER_REGISTRATION_TOKEN`
|
||
| 4 | SSH to Unraid | `ssh_check UNRAID` returns 0 | "Cannot SSH to Unraid at $UNRAID_IP. Run setup/unraid.sh or check SSH config." |
|
||
| 5 | SSH to Fedora | `ssh_check FEDORA` returns 0 | Same pattern |
|
||
| 6 | Docker on Unraid | `ssh_exec UNRAID "docker --version"` exits 0 | "Docker not found on Unraid. Run setup/unraid.sh." |
|
||
| 7 | Docker on Fedora | Same | Same |
|
||
| 8 | docker-compose on Unraid | `ssh_exec UNRAID "docker compose version"` or `docker-compose --version` | "docker-compose not found on Unraid. Run setup/unraid.sh." |
|
||
| 9 | docker-compose on Fedora | Same | Same |
|
||
| 10 | Container IPs available | Ping-check `UNRAID_GITEA_IP`, `UNRAID_CADDY_IP`, `FEDORA_GITEA_IP` — warn if responding | "IP $ip is already responding to ping (may be in use)." |
|
||
| 14 | DNS resolves | `python3 socket.getaddrinfo($GITEA_DOMAIN)` returns `$UNRAID_IP` | "$GITEA_DOMAIN does not resolve to $UNRAID_IP." |
|
||
| 15 | GitHub token valid | `curl https://api.github.com/user` returns 200 | "GitHub token invalid. Check GITHUB_TOKEN in .env." |
|
||
| 16 | GitHub repos exist | For each repo in `REPO_NAMES`: `curl /repos/$GITHUB_USERNAME/$repo` returns 200 | "GitHub repo $repo not found under $GITHUB_USERNAME." |
|
||
| 17 | Caddy data path writable | `ssh_exec UNRAID "test -w $CADDY_DATA_PATH"` or parent dir writable | "Caddy data path $CADDY_DATA_PATH not writable on Unraid." |
|
||
| 18-20 | Tool minimum versions | Local, Unraid, Fedora: jq>=1.6, curl>=7.70, git>=2.30, docker>=20, compose>=2 | Version-specific error messages |
|
||
| 21-22 | Cross-host SSH | Unraid→Fedora and Fedora→Unraid SSH with key auth | "Cannot SSH between hosts. Run setup/cross_host_ssh.sh." |
|
||
|
||
**Exit behavior**: Runs ALL checks (doesn't stop at first failure). Prints summary at end. Exits 0 if all pass, 1 if any fail.
|
||
|
||
**Done when**:
|
||
- [x] Every check in the table above is implemented
|
||
- [x] Failed checks point to the correct setup script or config to fix
|
||
- [ ] All checks run even if earlier ones fail (user sees full picture)
|
||
- [ ] Exit code is 1 if ANY check fails, 0 only if ALL pass
|
||
- [ ] `shellcheck preflight.sh` passes
|
||
|
||
---
|
||
|
||
### 5.1 — `phase1_gitea_unraid.sh`
|
||
|
||
**Depends on**: preflight passed, templates exist
|
||
**Produces**: Running Gitea instance on Unraid, admin user, API token in .env, org created
|
||
**`require_vars`**: `UNRAID_IP`, `UNRAID_SSH_USER`, `UNRAID_SSH_PORT`, `UNRAID_GITEA_DATA_PATH`, `UNRAID_COMPOSE_DIR`, `UNRAID_GITEA_IP`, `GITEA_ADMIN_USER`, `GITEA_ADMIN_PASSWORD`, `GITEA_ADMIN_EMAIL`, `GITEA_ORG_NAME`, `GITEA_INSTANCE_NAME`, `GITEA_DB_TYPE`, `GITEA_VERSION`, `GITEA_DOMAIN` (+ DB vars if DB_TYPE != sqlite3)
|
||
|
||
**Steps with idempotency**:
|
||
|
||
| # | Action | Idempotency check (skip if true) |
|
||
|---|--------|----------------------------------|
|
||
| 1 | Create data dirs on Unraid | `ssh_exec UNRAID "test -d $UNRAID_GITEA_DATA_PATH/data"` |
|
||
| 2 | Render + SCP docker-compose-gitea.yml.tpl | `ssh_exec UNRAID "test -f $UNRAID_GITEA_DATA_PATH/docker-compose.yml"` |
|
||
| 3 | Render + SCP app.ini.tpl | `ssh_exec UNRAID "test -f $UNRAID_GITEA_DATA_PATH/config/app.ini"` |
|
||
| 4 | `docker-compose up -d` on Unraid | `ssh_exec UNRAID "docker ps --filter name=gitea --format '{{.Status}}'"` contains "Up" |
|
||
| 5 | Wait for Gitea HTTP ready | `curl -sf $GITEA_INTERNAL_URL/api/v1/version` returns 200 |
|
||
| 6 | Create admin user via `docker exec gitea gitea admin user create` | `curl -sf -u $GITEA_ADMIN_USER:$GITEA_ADMIN_PASSWORD $GITEA_INTERNAL_URL/api/v1/user` returns 200 |
|
||
| 7 | Generate API token | `GITEA_ADMIN_TOKEN` is non-empty in .env AND `gitea_api GET /user` returns 200 |
|
||
| 8 | Save token to .env | Token is in .env file |
|
||
| 9 | Create org | `gitea_api GET /orgs/$GITEA_ORG_NAME` returns 200 |
|
||
|
||
**Done when**:
|
||
- [ ] `curl $GITEA_INTERNAL_URL` returns Gitea HTML page
|
||
- [ ] `curl -H "Authorization: token $GITEA_ADMIN_TOKEN" $GITEA_INTERNAL_URL/api/v1/user` returns admin user JSON
|
||
- [ ] `GITEA_ADMIN_TOKEN` is written to `.env` and is non-empty
|
||
- [ ] Org exists: `gitea_api GET /orgs/$GITEA_ORG_NAME` returns 200
|
||
- [ ] Running the script again changes nothing (all steps skip with "already exists" messages)
|
||
- [ ] `shellcheck phase1_gitea_unraid.sh` passes
|
||
|
||
---
|
||
|
||
### 5.2 — `phase1_post_check.sh`
|
||
|
||
**Depends on**: phase1 completed
|
||
**Purpose**: Independent verification that phase 1 succeeded (can be run separately)
|
||
|
||
**Checks**:
|
||
- [ ] Gitea responds at `$GITEA_INTERNAL_URL` with HTTP 200
|
||
- [ ] Admin user authenticates: `curl -u user:pass .../api/v1/user` returns 200
|
||
- [ ] API token works: `gitea_api GET /user` returns 200 with correct username
|
||
- [ ] Org exists: `gitea_api GET /orgs/$GITEA_ORG_NAME` returns 200
|
||
- [ ] Gitea Actions enabled: `gitea_api GET /api/v1/settings/api` or check app.ini
|
||
|
||
**Done when**:
|
||
- [ ] Runs all checks, prints PASS/FAIL for each
|
||
- [ ] Exits 0 only if ALL pass
|
||
|
||
---
|
||
|
||
### 5.3 — `phase1_teardown.sh`
|
||
|
||
**Depends on**: phase1 was run
|
||
**Destructive**: yes — prompts for confirmation
|
||
|
||
**Steps**:
|
||
1. Prompt: "This will stop Gitea on Unraid. Continue? [y/N]"
|
||
2. `ssh_exec UNRAID "cd $UNRAID_GITEA_DATA_PATH && docker-compose down"`
|
||
3. Prompt: "Remove all Gitea data ($UNRAID_GITEA_DATA_PATH)? This is irreversible. [y/N]"
|
||
4. If confirmed: `ssh_exec UNRAID "rm -rf $UNRAID_GITEA_DATA_PATH"`
|
||
5. Clear `GITEA_ADMIN_TOKEN` from .env: `save_env_var GITEA_ADMIN_TOKEN ""`
|
||
|
||
**Done when**:
|
||
- [ ] Gitea container is stopped and removed
|
||
- [ ] Data is only deleted if user explicitly confirms
|
||
- [ ] `GITEA_ADMIN_TOKEN` is cleared from .env
|
||
- [ ] Running against an already-torn-down instance doesn't error
|
||
|
||
---
|
||
|
||
### 6.1 — `phase2_gitea_fedora.sh`
|
||
|
||
**Depends on**: preflight passed, templates exist
|
||
**Produces**: Running Gitea instance on Fedora, admin user, API token in .env
|
||
**`require_vars`**: `FEDORA_IP`, `FEDORA_SSH_USER`, `FEDORA_SSH_PORT`, `FEDORA_GITEA_DATA_PATH`, `FEDORA_COMPOSE_DIR`, `FEDORA_MACVLAN_PARENT`, `FEDORA_MACVLAN_SUBNET`, `FEDORA_MACVLAN_GATEWAY`, `FEDORA_MACVLAN_IP_RANGE`, `FEDORA_GITEA_IP`, `GITEA_ADMIN_USER`, `GITEA_ADMIN_PASSWORD`, `GITEA_ADMIN_EMAIL`, `GITEA_DB_TYPE`, `GITEA_VERSION` (+ DB vars if DB_TYPE != sqlite3)
|
||
|
||
**Identical to phase1 except**:
|
||
- Target: Fedora (uses `FEDORA_IP`, `FEDORA_SSH_USER`, `FEDORA_SSH_PORT`, `FEDORA_GITEA_DATA_PATH`, `FEDORA_GITEA_IP`)
|
||
- Uses `GITEA_BACKUP_INTERNAL_URL` (derived from `FEDORA_GITEA_IP`) for API calls
|
||
- Saves token as `GITEA_BACKUP_ADMIN_TOKEN`
|
||
- Uses same admin credentials (`GITEA_ADMIN_USER`/`GITEA_ADMIN_PASSWORD`/`GITEA_ADMIN_EMAIL`)
|
||
- Does NOT create an org (mirrors will be under admin user's namespace)
|
||
|
||
**Done when**:
|
||
- [ ] Same criteria as 5.1 but targeting Fedora URLs/paths
|
||
- [ ] `GITEA_BACKUP_ADMIN_TOKEN` is in .env and works
|
||
- [ ] No org was created on Fedora instance
|
||
|
||
---
|
||
|
||
### 6.2 — `phase2_post_check.sh`
|
||
|
||
Same as 5.2 but targeting Fedora instance. No org check.
|
||
|
||
---
|
||
|
||
### 6.3 — `phase2_teardown.sh`
|
||
|
||
Same as 5.3 but targeting Fedora. Clears `GITEA_BACKUP_ADMIN_TOKEN`.
|
||
|
||
---
|
||
|
||
### 7.2 — `manage_runner.sh`
|
||
|
||
**Depends on**: `lib/common.sh`, `runners.conf`, templates
|
||
**Purpose**: Standalone tool to add/remove/list individual runners
|
||
|
||
**Subcommands**:
|
||
|
||
**`manage_runner.sh add --name <runner_name>`**:
|
||
1. Read runner entry from `runners.conf` by name
|
||
2. If `type=docker`:
|
||
- Render `docker-compose-runner.yml.tpl` with runner's vars
|
||
- SCP to `$DATA_PATH/docker-compose.yml` on runner host
|
||
- Render `runner-config.yaml.tpl`, SCP to `$DATA_PATH/config.yaml`
|
||
- `ssh_exec HOST "cd $DATA_PATH && docker-compose up -d"`
|
||
- `wait_for_http` on Gitea API for runner to appear
|
||
3. If `type=native`:
|
||
- Download `act_runner` binary for host's OS/arch to `$DATA_PATH/act_runner`
|
||
- Run `$DATA_PATH/act_runner register --no-interactive --instance $GITEA_INTERNAL_URL --token $GITEA_RUNNER_REGISTRATION_TOKEN --name $NAME --labels $LABELS`
|
||
- Render launchd plist, copy to `~/Library/LaunchAgents/`
|
||
- `launchctl load ~/Library/LaunchAgents/com.gitea.runner.$NAME.plist`
|
||
|
||
**`manage_runner.sh remove --name <runner_name>`**:
|
||
1. Read runner entry from `runners.conf`
|
||
2. If `type=docker`: `ssh_exec HOST "cd $DATA_PATH && docker-compose down"`, optionally rm data
|
||
3. If `type=native`: `launchctl unload` plist, rm binary + plist
|
||
4. Deregister from Gitea via API if possible (or just let it go offline)
|
||
|
||
**`manage_runner.sh list`**:
|
||
1. Read all entries from `runners.conf`
|
||
2. For each: query Gitea API for runner status
|
||
3. Print table: name, host, labels, type, status (online/offline)
|
||
|
||
**Done when**:
|
||
- [ ] `add` with a docker-type runner: container is running, runner appears in Gitea admin panel
|
||
- [ ] `add` with a native-type runner: launchd service is loaded, runner appears in Gitea admin panel
|
||
- [ ] `remove` stops the runner and it disappears from Gitea admin (or shows offline)
|
||
- [ ] `list` shows all runners with current status
|
||
- [ ] `add` on an already-deployed runner prints "already running" and exits 0
|
||
- [ ] `remove` on a non-existent runner prints warning and exits 0
|
||
- [ ] `shellcheck manage_runner.sh` passes
|
||
|
||
---
|
||
|
||
### 7.3 — `phase3_runners.sh`
|
||
|
||
**Depends on**: Phase 1 completed (Gitea running), `runners.conf` populated
|
||
**Produces**: All runners from `runners.conf` deployed and registered
|
||
**`require_vars`**: `GITEA_INTERNAL_URL`, `GITEA_ADMIN_TOKEN` *(auto)*, `ACT_RUNNER_VERSION`
|
||
|
||
**Steps**:
|
||
1. Get registration token: `gitea_api GET /admin/runners/registration-token`
|
||
2. `save_env_var GITEA_RUNNER_REGISTRATION_TOKEN <token>`
|
||
3. For each entry in `runners.conf`: call `manage_runner.sh add --name <name>`
|
||
|
||
**Done when**:
|
||
- [ ] `GITEA_RUNNER_REGISTRATION_TOKEN` is in .env
|
||
- [ ] Every runner in `runners.conf` is deployed and shows "online" in Gitea admin
|
||
- [ ] Running again skips all already-deployed runners
|
||
|
||
---
|
||
|
||
### 7.4 — `phase3_post_check.sh`
|
||
|
||
**Checks**:
|
||
- [ ] For each runner in `runners.conf`: runner exists in Gitea admin API response
|
||
- [ ] For each runner in `runners.conf`: status is "online" (not just registered but idle/active)
|
||
- [ ] Runner count in Gitea matches line count in `runners.conf`
|
||
|
||
---
|
||
|
||
### 7.5 — `phase3_teardown.sh`
|
||
|
||
For each runner in `runners.conf`: `manage_runner.sh remove --name <name>`. Clears `GITEA_RUNNER_REGISTRATION_TOKEN`.
|
||
|
||
---
|
||
|
||
### 8.1 — `phase4_migrate_repos.sh`
|
||
|
||
**Depends on**: Phase 1 + Phase 2 completed (both Gitea instances running)
|
||
**Produces**: All repos on Unraid primary under org + pull mirrors on Fedora
|
||
**`require_vars`**: `GITEA_ADMIN_TOKEN` *(auto)*, `GITEA_BACKUP_ADMIN_TOKEN` *(auto)*, `GITEA_INTERNAL_URL`, `GITEA_BACKUP_INTERNAL_URL`, `GITEA_ORG_NAME`, `GITEA_ADMIN_USER`, `GITEA_ADMIN_PASSWORD`, `GITHUB_USERNAME`, `GITHUB_TOKEN`, `REPO_NAMES`, `MIGRATE_ISSUES`, `MIGRATE_LABELS`, `MIGRATE_MILESTONES`, `MIGRATE_WIKI`, `GITEA_BACKUP_MIRROR_INTERVAL`
|
||
|
||
**Steps**:
|
||
|
||
For each repo in `REPO_NAMES`:
|
||
|
||
| # | Action | API call | Idempotency check |
|
||
|---|--------|----------|-------------------|
|
||
| 1 | Import repo from GitHub to Unraid | `gitea_api POST /repos/migrate` with `clone_addr=https://github.com/$GITHUB_USERNAME/$REPO.git`, `auth_token=$GITHUB_TOKEN`, `repo_owner=$GITEA_ORG_NAME`, `repo_name=$REPO`, `mirror=false`, `issues=$MIGRATE_ISSUES`, `labels=$MIGRATE_LABELS`, `milestones=$MIGRATE_MILESTONES`, `wiki=$MIGRATE_WIKI` | `gitea_api GET /repos/$GITEA_ORG_NAME/$REPO` returns 200 |
|
||
| 2 | Wait for migration | Poll `gitea_api GET /repos/$GITEA_ORG_NAME/$REPO` until `empty=false` (repo has content) | — |
|
||
| 3 | Create pull mirror on Fedora | `gitea_backup_api POST /repos/migrate` with `clone_addr=$GITEA_INTERNAL_URL/$GITEA_ORG_NAME/$REPO.git`, `auth_username=$GITEA_ADMIN_USER`, `auth_password=$GITEA_ADMIN_PASSWORD`, `repo_owner=$GITEA_ADMIN_USER`, `mirror=true`, `mirror_interval=$GITEA_BACKUP_MIRROR_INTERVAL` | `gitea_backup_api GET /repos/$GITEA_ADMIN_USER/$REPO` returns 200 |
|
||
|
||
**Done when**:
|
||
- [ ] All repos exist under `$GITEA_ORG_NAME` on Unraid with commits
|
||
- [ ] Each repo's default branch matches the GitHub source
|
||
- [ ] All mirror repos exist on Fedora under admin user
|
||
- [ ] Fedora mirrors show `mirror=true` in API response
|
||
- [ ] Fedora mirrors have synced at least once (has commits)
|
||
- [ ] Running again skips all existing repos
|
||
|
||
---
|
||
|
||
### 8.2 — `phase4_post_check.sh`
|
||
|
||
**Checks**:
|
||
- [ ] Each repo exists on primary: `gitea_api GET /repos/$ORG/$REPO` returns 200
|
||
- [ ] Each repo has commits: `gitea_api GET /repos/$ORG/$REPO/commits?limit=1` returns at least 1 commit
|
||
- [ ] Default branch matches source: compare `default_branch` field from Gitea vs GitHub API
|
||
- [ ] Each mirror repo exists on Fedora: `gitea_backup_api GET /repos/$ADMIN/$REPO` returns 200
|
||
- [ ] Each mirror has `mirror: true` in response
|
||
|
||
---
|
||
|
||
### 8.3 — `phase4_teardown.sh`
|
||
|
||
1. For each repo in `REPO_NAMES`: `gitea_api DELETE /repos/$GITEA_ORG_NAME/$REPO`
|
||
2. For each mirror: `gitea_backup_api DELETE /repos/$ADMIN/$REPO`
|
||
3. Prompt before each deletion
|
||
|
||
---
|
||
|
||
### 9.1 — `phase5_migrate_pipelines.sh`
|
||
|
||
**Depends on**: Phase 4 completed (repos exist on Gitea)
|
||
**Produces**: `.gitea/workflows/` directory in each repo with adapted workflows
|
||
**`require_vars`**: `GITEA_ADMIN_TOKEN` *(auto)*, `GITEA_INTERNAL_URL`, `GITEA_ORG_NAME`, `GITEA_ADMIN_USER`, `REPO_NAMES`
|
||
|
||
**Steps for each repo**:
|
||
1. Clone repo from Gitea to temp dir: `git clone $GITEA_INTERNAL_URL/$GITEA_ORG_NAME/$REPO.git /tmp/gitea-migration-$REPO`
|
||
2. Check if `.github/workflows/` exists — if not, log warning "No GitHub workflows found" and skip
|
||
3. Create `.gitea/workflows/` directory
|
||
4. Copy all `.yml` files from `.github/workflows/` to `.gitea/workflows/`
|
||
5. Apply compatibility fixes in each copied file:
|
||
- Replace `github.repository` → `gitea.repository` in expressions
|
||
- Replace `github.event` → `gitea.event` in expressions
|
||
- Replace `github.token` → `gitea.token`
|
||
- Replace `github.server_url` → `gitea.server_url`
|
||
- Keep `actions/checkout@v4` as-is (compatible with Gitea)
|
||
- Add comment at top: `# Migrated from GitHub Actions — review for Gitea compatibility`
|
||
6. `git add .gitea/`, `git commit -m "Migrate workflows to Gitea Actions"`, `git push`
|
||
7. Clean up temp dir
|
||
|
||
**Idempotency**: Skip repo if `.gitea/workflows/` already exists with files.
|
||
|
||
**Known limitations** (document in script output):
|
||
- GitHub-specific marketplace actions may not work in Gitea — script logs warnings but doesn't block
|
||
- Self-hosted runner tool caches may differ — user may need to install tools manually
|
||
- OIDC/secrets need to be re-configured in Gitea settings
|
||
|
||
**Done when**:
|
||
- [ ] Each repo that had `.github/workflows/` now has `.gitea/workflows/` with adapted files
|
||
- [ ] Context variable replacements applied (`github.*` → `gitea.*`)
|
||
- [ ] Each adapted workflow file has the migration comment header
|
||
- [ ] Repos without workflows are skipped with a warning (not an error)
|
||
- [ ] Running again skips repos that already have `.gitea/workflows/`
|
||
|
||
---
|
||
|
||
### 9.2 — `phase5_post_check.sh`
|
||
|
||
**Checks**:
|
||
- [ ] Each repo has `.gitea/workflows/` directory (check via API: `gitea_api GET /repos/$ORG/$REPO/contents/.gitea/workflows`)
|
||
- [ ] At least one `.yml` file in that directory
|
||
- [ ] Gitea Actions tab shows workflows (may require checking the Gitea web UI or API for action runs)
|
||
|
||
---
|
||
|
||
### 9.3 — `phase5_teardown.sh`
|
||
|
||
For each repo: clone, `rm -rf .gitea/workflows`, commit, push. Only if `.gitea/workflows/` exists.
|
||
|
||
---
|
||
|
||
### 10.1 — `phase6_github_mirrors.sh`
|
||
|
||
**Depends on**: Phase 4 completed, GitHub mirror token set
|
||
**Produces**: Push mirrors from Gitea → GitHub configured for all repos
|
||
**`require_vars`**: `GITEA_ADMIN_TOKEN` *(auto)*, `GITEA_INTERNAL_URL`, `GITEA_ORG_NAME`, `GITHUB_USERNAME`, `GITHUB_TOKEN`, `GITHUB_MIRROR_INTERVAL`, `REPO_NAMES`
|
||
|
||
**Steps for each repo**:
|
||
1. Check if push mirror already exists: `gitea_api GET /repos/$ORG/$REPO/push_mirrors` — skip if non-empty
|
||
2. Create push mirror: `gitea_api POST /repos/$ORG/$REPO/push_mirrors` with:
|
||
- `remote_address`: `https://github.com/$GITHUB_USERNAME/$REPO.git`
|
||
- `remote_username`: `$GITHUB_USERNAME`
|
||
- `remote_password`: `$GITHUB_TOKEN`
|
||
- `interval`: `$GITHUB_MIRROR_INTERVAL`
|
||
- `sync_on_commit`: true
|
||
3. Trigger initial sync: `gitea_api POST /repos/$ORG/$REPO/push_mirrors-sync`
|
||
4. Disable GitHub Actions: `github_api PATCH /repos/$GITHUB_USERNAME/$REPO` with `{"has_projects": false}` — note: disabling Actions requires setting `has_actions: false` which may not be in the API. If not possible via API, print manual instructions.
|
||
|
||
**Done when**:
|
||
- [ ] Each repo has exactly one push mirror configured
|
||
- [ ] `gitea_api GET /repos/$ORG/$REPO/push_mirrors` returns the mirror config
|
||
- [ ] After triggering sync, GitHub repo has the same latest commit as Gitea
|
||
- [ ] Running again skips repos that already have mirrors
|
||
|
||
---
|
||
|
||
### 10.2 — `phase6_post_check.sh`
|
||
|
||
- [ ] Each repo: push mirror exists in API response
|
||
- [ ] Each repo: trigger sync and verify GitHub's latest commit SHA matches Gitea's HEAD
|
||
|
||
---
|
||
|
||
### 10.3 — `phase6_teardown.sh`
|
||
|
||
For each repo: `gitea_api DELETE /repos/$ORG/$REPO/push_mirrors/{id}`. Get mirror ID from list endpoint first.
|
||
|
||
---
|
||
|
||
### 11.1 — `phase7_branch_protection.sh`
|
||
|
||
**Depends on**: Phase 4 completed (repos exist)
|
||
**Produces**: Branch protection on `$PROTECTED_BRANCH` for all repos
|
||
**`require_vars`**: `GITEA_ADMIN_TOKEN` *(auto)*, `GITEA_INTERNAL_URL`, `GITEA_ORG_NAME`, `REPO_NAMES`, `PROTECTED_BRANCH`, `REQUIRE_PR_REVIEW`, `REQUIRED_APPROVALS`
|
||
|
||
**Steps for each repo**:
|
||
1. Check if protection exists: `gitea_api GET /repos/$ORG/$REPO/branch_protections/$PROTECTED_BRANCH` — skip if 200
|
||
2. Create protection: `gitea_api POST /repos/$ORG/$REPO/branch_protections` with:
|
||
- `branch_name`: `$PROTECTED_BRANCH`
|
||
- `enable_push`: false
|
||
- `enable_push_whitelist`: false
|
||
- `require_signed_commits`: false
|
||
- `enable_status_check`: true
|
||
- `enable_approvals_whitelist`: `$REQUIRE_PR_REVIEW`
|
||
- `required_approvals`: `$REQUIRED_APPROVALS`
|
||
|
||
**Done when**:
|
||
- [ ] Each repo: `gitea_api GET /repos/$ORG/$REPO/branch_protections` returns the rule
|
||
- [ ] Direct pushes to `$PROTECTED_BRANCH` are blocked
|
||
- [ ] Running again skips existing protections
|
||
|
||
---
|
||
|
||
### 11.2 / 11.3 — post-check and teardown
|
||
|
||
Post-check: verify protection rules via API.
|
||
Teardown: `gitea_api DELETE /repos/$ORG/$REPO/branch_protections/$PROTECTED_BRANCH`.
|
||
|
||
---
|
||
|
||
### 12.1 — `phase8_cutover.sh`
|
||
|
||
**Depends on**: macvlan network created (Phase 1), all prior phases
|
||
**Produces**: HTTPS access to Gitea via Caddy, GitHub repos marked as mirrors
|
||
**`require_vars`**: `UNRAID_IP`, `UNRAID_SSH_USER`, `UNRAID_GITEA_IP`, `UNRAID_CADDY_IP`, `GITEA_INTERNAL_URL`, `GITEA_DOMAIN`, `GITEA_ADMIN_TOKEN` *(auto)*, `GITEA_ORG_NAME`, `TLS_MODE`, `CADDY_DOMAIN`, `CADDY_DATA_PATH`, `GITHUB_USERNAME`, `GITHUB_TOKEN`, `REPO_NAMES`. Conditional: `CLOUDFLARE_API_TOKEN` (if `TLS_MODE=cloudflare`), `SSL_CERT_PATH` + `SSL_KEY_PATH` (if `TLS_MODE=existing`).
|
||
|
||
**Steps with idempotency**:
|
||
|
||
| # | Action | Detail | Idempotency check (skip if true) |
|
||
|---|--------|--------|----------------------------------|
|
||
| 1 | Create Caddy dirs | `ssh_exec UNRAID "mkdir -p $CADDY_DATA_PATH/{data,config}"` | `test -d $CADDY_DATA_PATH` |
|
||
| 2 | Deploy Caddyfile | Render `Caddyfile.tpl` with TLS_BLOCK (cloudflare DNS-01 or existing cert paths), SCP to `$CADDY_DATA_PATH/Caddyfile` | `test -f $CADDY_DATA_PATH/Caddyfile` |
|
||
| 3 | Deploy Caddy docker-compose | Render `docker-compose-caddy.yml.tpl`, SCP to `$CADDY_DATA_PATH/docker-compose.yml` | `test -f $CADDY_DATA_PATH/docker-compose.yml` |
|
||
| 4 | Start Caddy | `docker compose up -d` in `$CADDY_DATA_PATH` | Caddy container already running |
|
||
| 5 | Wait for HTTPS | Poll `https://$GITEA_DOMAIN/api/v1/version` with retries until cert is obtained | — |
|
||
| 6 | Mark GitHub repos as mirrors | Save pre-cutover state to `.manifests/phase8_github_repo_state.json`, update description to `[MIRROR]`, disable wiki/projects/Pages | GitHub repo description starts with `[MIRROR]` |
|
||
|
||
**TLS mode handling**:
|
||
- `cloudflare`: Caddyfile uses `tls { dns cloudflare {env.CF_API_TOKEN} }`, docker-compose passes `CF_API_TOKEN` env var
|
||
- `existing`: Caddyfile uses `tls /path/to/cert /path/to/key`, docker-compose mounts cert/key as volumes
|
||
|
||
**Done when**:
|
||
- [ ] `https://$GITEA_DOMAIN` returns valid HTTPS response
|
||
- [ ] HTTP requests redirect to HTTPS (301)
|
||
- [ ] SSL certificate is valid (openssl check)
|
||
- [ ] All repos accessible via HTTPS API
|
||
- [ ] GitHub repos marked with `[MIRROR]` description prefix
|
||
|
||
---
|
||
|
||
### 12.2 — `phase8_post_check.sh`
|
||
|
||
- [ ] HTTPS works with valid cert: `curl -sf https://$GITEA_DOMAIN/api/v1/version` returns 200
|
||
- [ ] HTTP redirects to HTTPS: `curl -sI http://$GITEA_DOMAIN/` returns 301
|
||
- [ ] Certificate is valid: `openssl s_client` returns non-empty issuer
|
||
- [ ] All repos accessible: API call to each repo returns 200
|
||
- [ ] GitHub repos marked as mirrors: description starts with `[MIRROR]`
|
||
|
||
---
|
||
|
||
### 12.3 — `phase8_teardown.sh`
|
||
|
||
1. Stop + remove Caddy container: `docker compose down` in `$CADDY_DATA_PATH`
|
||
2. Remove Caddy config files: `rm -f $CADDY_DATA_PATH/docker-compose.yml $CADDY_DATA_PATH/Caddyfile`
|
||
3. Optionally remove Caddy TLS data: `rm -rf $CADDY_DATA_PATH/data $CADDY_DATA_PATH/config`
|
||
4. Restore GitHub repo settings from saved Phase 8 state snapshot (description, homepage, wiki, projects, Pages). Falls back to parsing `[MIRROR] ... — was: ORIGINAL` if snapshot missing.
|
||
|
||
---
|
||
|
||
### 13.1 — `phase9_security.sh`
|
||
|
||
**Depends on**: Phase 5 completed (repos have `.gitea/workflows/`)
|
||
**Produces**: Security scan workflow in all repos, branch protection updated
|
||
**`require_vars`**: `GITEA_ADMIN_TOKEN` *(auto)*, `GITEA_INTERNAL_URL`, `GITEA_ORG_NAME`, `GITEA_ADMIN_USER`, `REPO_NAMES`, `SEMGREP_VERSION`, `TRIVY_VERSION`, `GITLEAKS_VERSION`, `SECURITY_FAIL_ON_ERROR`, `PROTECTED_BRANCH`
|
||
|
||
**Steps for each repo**:
|
||
1. Clone from Gitea to temp dir
|
||
2. Render `security-scan.yml.tpl` to `.gitea/workflows/security-scan.yml`
|
||
3. `git add`, `git commit -m "Add security scanning workflow"`, `git push`
|
||
4. If `SECURITY_FAIL_ON_ERROR=true`: update branch protection to require status checks from security jobs
|
||
- `gitea_api PATCH /repos/$ORG/$REPO/branch_protections/$PROTECTED_BRANCH` with `status_check_contexts: ["semgrep", "trivy", "gitleaks"]`
|
||
|
||
**Idempotency**: Skip if `security-scan.yml` already exists in repo.
|
||
|
||
**Done when**:
|
||
- [ ] Each repo has `.gitea/workflows/security-scan.yml`
|
||
- [ ] Workflow file references correct tool versions from .env
|
||
- [ ] If `SECURITY_FAIL_ON_ERROR=true`: branch protection includes the three status checks
|
||
- [ ] Creating a test PR triggers the security workflow (manual verification)
|
||
|
||
---
|
||
|
||
### 13.2 / 13.3 — post-check and teardown
|
||
|
||
Post-check: verify file exists in each repo via API, verify branch protection includes status checks.
|
||
Teardown: remove file, update branch protection to remove status checks.
|
||
|
||
---
|
||
|
||
### 14.1 — `backup/backup_primary.sh`
|
||
|
||
**Depends on**: Phase 1 completed
|
||
**Produces**: `gitea-dump-*.zip` archive on Fedora
|
||
**`require_vars`**: `UNRAID_IP`, `UNRAID_SSH_USER`, `UNRAID_SSH_PORT`, `UNRAID_GITEA_DATA_PATH`, `FEDORA_IP`, `FEDORA_SSH_USER`, `FEDORA_SSH_PORT`, `BACKUP_STORAGE_PATH`, `BACKUP_RETENTION_COUNT`
|
||
|
||
**Steps**:
|
||
1. `ssh_exec UNRAID "docker exec -u git gitea gitea dump -c /data/gitea/conf/app.ini -f /tmp/gitea-dump-$(date +%Y%m%d-%H%M%S).zip"`
|
||
2. SCP dump from Unraid `/tmp/` to `$BACKUP_STORAGE_PATH/` on Fedora
|
||
3. Remove dump from Unraid `/tmp/`
|
||
4. Prune old backups: `ssh_exec FEDORA "ls -t $BACKUP_STORAGE_PATH/gitea-dump-*.zip | tail -n +$((BACKUP_RETENTION_COUNT+1)) | xargs rm -f"`
|
||
5. Print: backup file name, size, path on Fedora, remaining backup count
|
||
|
||
**What's in the dump** (verify these are captured):
|
||
- SQLite database (users, tokens, SSH keys, OAuth, webhooks, org/team membership, issues, PRs)
|
||
- All git repositories
|
||
- app.ini config
|
||
|
||
**Done when**:
|
||
- [ ] Zip file exists on Fedora at `$BACKUP_STORAGE_PATH/`
|
||
- [ ] Zip contains: `gitea-db.sql` (or `gitea.db` for SQLite), `repos/` directory, `app.ini`
|
||
- [ ] Old backups beyond retention count are deleted
|
||
- [ ] Dump file on Unraid `/tmp/` is cleaned up
|
||
|
||
---
|
||
|
||
### 14.2 — `backup/restore_to_primary.sh`
|
||
|
||
**Depends on**: A backup archive exists
|
||
**Produces**: Restored Gitea instance on Unraid
|
||
**`require_vars`**: `UNRAID_IP`, `UNRAID_SSH_USER`, `UNRAID_SSH_PORT`, `UNRAID_GITEA_DATA_PATH`, `GITEA_INTERNAL_URL`, `GITEA_ADMIN_USER`, `GITEA_ADMIN_PASSWORD`
|
||
|
||
**Steps**:
|
||
1. Accept `--archive <path>` (path on Fedora or local)
|
||
2. Prompt: "This will REPLACE all Gitea data on Unraid. Continue? [y/N]"
|
||
3. If archive is on Fedora: SCP to Unraid `/tmp/`
|
||
4. Stop Gitea: `ssh_exec UNRAID "cd $UNRAID_GITEA_DATA_PATH && docker-compose down"`
|
||
5. Back up current data (safety): `ssh_exec UNRAID "mv $UNRAID_GITEA_DATA_PATH/data $UNRAID_GITEA_DATA_PATH/data.pre-restore"`
|
||
6. Extract archive: `ssh_exec UNRAID "cd $UNRAID_GITEA_DATA_PATH && unzip /tmp/gitea-dump-*.zip"`
|
||
7. Move extracted files to correct locations (data, config)
|
||
8. Start Gitea: `ssh_exec UNRAID "cd $UNRAID_GITEA_DATA_PATH && docker-compose up -d"`
|
||
9. `wait_for_http $GITEA_INTERNAL_URL 60`
|
||
10. Verify admin login works: `curl -sf -u $GITEA_ADMIN_USER:$GITEA_ADMIN_PASSWORD $GITEA_INTERNAL_URL/api/v1/user`
|
||
11. Regenerate API token (old token from dump may conflict): create new token via basic auth, `save_env_var GITEA_ADMIN_TOKEN`
|
||
|
||
**Done when**:
|
||
- [ ] Gitea is running with restored data
|
||
- [ ] Admin can log in
|
||
- [ ] All repos are accessible
|
||
- [ ] All users from the backup exist
|
||
- [ ] New API token is generated and saved to .env
|
||
- [ ] Pre-restore data is preserved (not deleted) in case restore went wrong
|
||
|
||
---
|
||
|
||
### 15.1 — `run_all.sh`
|
||
|
||
**Depends on**: all scripts exist
|
||
**Steps**:
|
||
1. Parse args: `--start-from=N` (default 0), `--skip-setup`
|
||
2. Define execution order array:
|
||
- **Step 0 — Setup**:
|
||
1. `setup/configure_env.sh` — interactive env wizard (populates `.env`)
|
||
2. `setup/macbook.sh` — install MacBook prerequisites
|
||
3. `setup/unraid.sh` — install Unraid prerequisites (via SSH)
|
||
4. `setup/fedora.sh` — install Fedora prerequisites (via SSH)
|
||
- **Step P — Preflight**: `preflight.sh` (validates everything before proceeding)
|
||
- **Steps 1-9 — Phases**: `phaseN_*.sh` + `phaseN_post_check.sh`
|
||
3. Execute sequentially from start point
|
||
4. On any script exit code != 0: stop, print which step failed, print summary of what completed
|
||
5. On success: print full summary with checkmarks
|
||
6. **`--start-from=N`** (where N >= 1): skips setup + preflight only for phases. Still runs `preflight.sh` to validate .env vars (but NOT `configure_env.sh` or machine setup — assumes those were already done).
|
||
|
||
**Done when**:
|
||
- [ ] Running without args executes: configure_env → macbook → unraid → fedora → preflight → phase 1-9
|
||
- [ ] `--start-from=3` skips phases 1-2 but still runs preflight, then starts at phase 3
|
||
- [ ] `--skip-setup` skips configure_env + machine setup scripts, starts at preflight
|
||
- [ ] Failure in any step stops execution (doesn't continue to next phase)
|
||
- [ ] Summary at end shows pass/fail for each step that ran
|
||
|
||
---
|
||
|
||
### 15.2 — `teardown_all.sh`
|
||
|
||
**Steps**:
|
||
1. Parse args: `--through=N` (default 1 = tear down everything)
|
||
2. Execute in REVERSE order: phase9_teardown → phase8_teardown → ... → phaseN_teardown
|
||
3. Each teardown prompts for confirmation (unless `--yes` flag)
|
||
|
||
**Done when**:
|
||
- [ ] `--through=5` tears down phases 5-9 but leaves 1-4 intact
|
||
- [ ] Each phase teardown runs independently (doesn't fail because a later phase is already torn down)
|
||
- [ ] `--yes` skips all confirmation prompts
|
||
|
||
---
|
||
|
||
### 16.1 — Git repo + .gitignore
|
||
|
||
**`.gitignore` must contain**:
|
||
```
|
||
# Secrets — never commit
|
||
.env
|
||
runners.conf
|
||
*.pem
|
||
*.key
|
||
*.crt
|
||
|
||
# macOS
|
||
.DS_Store
|
||
|
||
# Temp files from script runs
|
||
/tmp/
|
||
*.log
|
||
|
||
# Backup archives
|
||
*.zip
|
||
|
||
# Editor / IDE
|
||
.vscode/
|
||
.idea/
|
||
*.swp
|
||
*~
|
||
```
|
||
|
||
**Done when**:
|
||
- [ ] `git init` completed
|
||
- [ ] `.gitignore` excludes `.env`, `runners.conf`, temp files
|
||
- [ ] `.env.example` and `runners.conf.example` are NOT ignored (they're templates)
|
||
|
||
---
|
||
|
||
### 16.2 — `CLAUDE.md`
|
||
|
||
Project-specific instructions for AI assistants working on this codebase.
|
||
|
||
**Done when**:
|
||
- [ ] Documents: project structure, how to run scripts, .env setup, script conventions
|
||
- [ ] Notes the `set -euo pipefail` + shellcheck requirement for all scripts
|
||
|
||
---
|
||
|
||
### 17.1 / 17.2 — Validation
|
||
|
||
**Done when**:
|
||
- [ ] `shellcheck *.sh lib/*.sh setup/*.sh backup/*.sh` exits 0 with zero warnings
|
||
- [ ] `bash -n` on every `.sh` file exits 0
|
||
- [ ] No file has unexpanded template variables (grep for `\$[A-Z_]` in rendered outputs should find nothing unexpected)
|
||
|
||
---
|
||
|
||
## Git Commit Milestones
|
||
|
||
Each milestone is a git commit. Commit **after** all files in that group pass `shellcheck` and `bash -n`. Do not batch multiple milestones into one commit.
|
||
|
||
| # | Commit message | Files included | Tracker sections |
|
||
|---|---------------|----------------|-----------------|
|
||
| 1 | `init: project structure, .gitignore, .env.example, runners.conf.example` | `.gitignore`, `.env.example`, `runners.conf.example`, `PLAN.md`, `CLAUDE.md` | 16.1, 16.2 |
|
||
| 2 | `feat: add shared library (lib/common.sh)` | `lib/common.sh` | 1.1 |
|
||
| 3 | `feat: add API contracts` | `contracts/gitea-api.md` | 1.3 |
|
||
| 4 | `feat: add configuration templates` | `templates/*.tpl`, `templates/workflows/*.tpl` | 2.1–2.7 |
|
||
| 5 | `feat: add setup scripts (configure_env, macbook, unraid, fedora)` | `setup/configure_env.sh`, `setup/macbook.sh`, `setup/unraid.sh`, `setup/fedora.sh` | 3.1–3.4 |
|
||
| 6 | `feat: add preflight validation` | `preflight.sh` | 4.1 |
|
||
| 7 | `feat: add Phase 1 — Gitea on Unraid` | `phase1_gitea_unraid.sh`, `phase1_post_check.sh`, `phase1_teardown.sh` | 5.1–5.3 |
|
||
| 8 | `feat: add Phase 2 — Gitea on Fedora` | `phase2_gitea_fedora.sh`, `phase2_post_check.sh`, `phase2_teardown.sh` | 6.1–6.3 |
|
||
| 9 | `feat: add Phase 3 — Runners` | `phase3_runners.sh`, `phase3_post_check.sh`, `phase3_teardown.sh`, `manage_runner.sh` | 7.2–7.5 |
|
||
| 10 | `feat: add Phase 4 — Migrate repos + Fedora mirrors` | `phase4_migrate_repos.sh`, `phase4_post_check.sh`, `phase4_teardown.sh` | 8.1–8.3 |
|
||
| 11 | `feat: add Phase 5 — Migrate pipelines` | `phase5_migrate_pipelines.sh`, `phase5_post_check.sh`, `phase5_teardown.sh` | 9.1–9.3 |
|
||
| 12 | `feat: add Phase 6 — GitHub push mirrors` | `phase6_github_mirrors.sh`, `phase6_post_check.sh`, `phase6_teardown.sh` | 10.1–10.3 |
|
||
| 13 | `feat: add Phase 7 — Branch protection` | `phase7_branch_protection.sh`, `phase7_post_check.sh`, `phase7_teardown.sh` | 11.1–11.3 |
|
||
| 14 | `feat: add Phase 8 — Cutover (HTTPS + archive GitHub)` | `phase8_cutover.sh`, `phase8_post_check.sh`, `phase8_teardown.sh` | 12.1–12.3 |
|
||
| 15 | `feat: add Phase 9 — Security scanning` | `phase9_security.sh`, `phase9_post_check.sh`, `phase9_teardown.sh` | 13.1–13.3 |
|
||
| 16 | `feat: add backup and restore scripts` | `backup/backup_primary.sh`, `backup/restore_to_primary.sh` | 14.1–14.2 |
|
||
| 17 | `feat: add orchestration (run_all.sh, teardown_all.sh)` | `run_all.sh`, `teardown_all.sh` | 15.1–15.2 |
|
||
| 18 | `chore: shellcheck + syntax validation fixes` | Any files fixed during validation | 17.1–17.2 |
|