feat: add phase 7.5 Nginx to Caddy migration script and update usage guide

README.md

@@ -55,6 +55,7 @@ The entire process is driven from a MacBook over SSH. Nothing is installed on th
 | 6 | `phase6_github_mirrors.sh` | Configure push mirrors from Gitea to GitHub, disable GitHub Actions |
 | 7 | `phase7_branch_protection.sh` | Apply branch protection rules to all repos |
 | 8 | `phase8_cutover.sh` | Deploy Caddy HTTPS reverse proxy (Cloudflare DNS-01 or existing certs), mark GitHub repos as mirrors |
+| 7.5 (optional) | `phase7_5_nginx_to_caddy.sh` | One-time multi-domain Nginx -> Caddy migration helper (canary/full), supports `sintheus.com` + `privacyindesign.com` in one Caddy |
 | 9 | `phase9_security.sh` | Deploy Semgrep + Trivy + Gitleaks security scanning workflows |
 
 Each phase has three scripts: the main script, a `_post_check.sh` that independently verifies success, and a `_teardown.sh` that cleanly reverses the phase.
@@ -96,6 +97,8 @@ gitea-migration/
 ├── run_all.sh                # Full pipeline orchestration
 ├── post-migration-check.sh   # Read-only infrastructure state check
 ├── teardown_all.sh           # Reverse teardown (9 to 1)
+├── phase7_5_nginx_to_caddy.sh # Optional one-time Nginx -> Caddy consolidation step
+├── TODO.md                   # Phase 7.5 migration context, backlog, and DoD
 ├── manage_runner.sh          # Dynamic runner add/remove/list
 ├── phase{1-9}_*.sh           # Main phase scripts
 ├── phase{1-9}_post_check.sh  # Verification scripts

TODO.md

@@ -0,0 +1,111 @@
+# TODO — Phase 7.5 Nginx -> Caddy Consolidation
+
+## Why this exists
+
+This file captures the decisions and migration context for the one-time "phase 7.5"
+work so we do not lose reasoning between sessions.
+
+## What happened so far
+
+1. The original `phase8_cutover.sh` was designed for one wildcard zone
+   (`*.${CADDY_DOMAIN}`), mainly for Gitea cutover.
+2. The homelab currently has two active DNS zones in scope:
+   - `sintheus.com` (legacy services behind Nginx)
+   - `privacyindesign.com` (new Gitea public endpoint)
+3. Decision made: run a one-time migration where a single Caddy instance serves
+   both zones, then gradually retire Nginx.
+4. Implemented: `phase7_5_nginx_to_caddy.sh` to generate/deploy a multi-domain
+   Caddyfile and run canary/full rollout modes.
+
+## Current design decisions
+
+1. Public ingress should be HTTPS-only for all migrated hostnames.
+2. Backend scheme is mixed for now:
+   - Keep `http://` upstream where service does not yet have TLS.
+   - Keep `https://` where already available.
+3. End-to-end HTTPS is a target state, not an immediate requirement.
+4. A strict toggle exists in phase 7.5:
+   - `--strict-backend-https` fails if any upstream is `http://`.
+5. Canary-first rollout:
+   - first migration target is `tower.sintheus.com`.
+
+## Host map and backend TLS status
+
+### Canary scope (default mode)
+
+- `tower.sintheus.com -> https://192.168.1.82:443` (TLS backend; cert verify skipped)
+- `${GITEA_DOMAIN} -> http://${UNRAID_GITEA_IP}:3000` (HTTP backend for now)
+
+### Full migration scope
+
+- `ai.sintheus.com -> http://192.168.1.82:8181`
+- `photos.sintheus.com -> http://192.168.1.222:2283`
+- `fin.sintheus.com -> http://192.168.1.233:8096`
+- `disk.sintheus.com -> http://192.168.1.52:80`
+- `pi.sintheus.com -> http://192.168.1.4:80`
+- `plex.sintheus.com -> http://192.168.1.111:32400`
+- `sync.sintheus.com -> http://192.168.1.119:8384`
+- `syno.sintheus.com -> https://100.108.182.16:5001` (verify skipped)
+- `tower.sintheus.com -> https://192.168.1.82:443` (verify skipped)
+- `${GITEA_DOMAIN} -> http://${UNRAID_GITEA_IP}:3000`
+
+## Definition of done (phase 7.5)
+
+Phase 7.5 is done only when all are true:
+
+1. Caddy is running on Unraid with generated multi-domain config.
+2. Canary host `tower.sintheus.com` is reachable over HTTPS through Caddy.
+3. Canary routing is proven by at least one path:
+   - `curl --resolve` tests, or
+   - split-DNS/hosts override, or
+   - intentional DNS cutover.
+4. Legacy Nginx remains available for non-migrated hosts during canary.
+5. No critical regressions observed for at least 24 hours on canary traffic.
+
+## Definition of done (final state after full migration)
+
+1. All selected domains route to Caddy through the intended ingress path:
+   - LAN-only: split-DNS/private resolution to Caddy, or
+   - public: DNS to WAN ingress that forwards 443 to Caddy.
+2. Caddy serves valid certificates for both zones.
+3. Functional checks pass for each service (UI load, API, websocket/streaming where relevant).
+4. Nginx is no longer on the request path for migrated domains.
+5. Long-term target: all backends upgraded to `https://` and strict mode passes.
+
+## What remains to happen
+
+1. Run canary:
+   - `./phase7_5_nginx_to_caddy.sh --mode=canary`
+2. Route canary traffic to Caddy using one method:
+   - `curl --resolve` for zero-DNS-change testing, or
+   - split-DNS/private DNS, or
+   - explicit DNS cutover if desired.
+3. Observe errors/latency/app behavior for at least 24 hours.
+4. If canary is clean, run full:
+   - `./phase7_5_nginx_to_caddy.sh --mode=full`
+5. Move remaining routes in batches (DNS or split-DNS, depending on ingress model).
+6. Validate each app after each batch.
+7. After everything is stable, plan Nginx retirement.
+8. Later hardening pass:
+   - enable TLS on each backend service one by one
+   - flip each corresponding upstream to `https://`
+   - finally run `--strict-backend-https` and require it to pass.
+
+## Risks and why mixed backend HTTP is acceptable short-term
+
+1. Risk: backend HTTP is unencrypted on LAN.
+   - Mitigation: traffic stays on trusted local network, temporary state only.
+2. Risk: if strict mode is enabled too early, rollout blocks.
+   - Mitigation: keep strict mode off until backend TLS coverage improves.
+3. Risk: moving all DNS at once can create broad outage.
+   - Mitigation: canary-first and batch DNS cutover.
+
+## Operational notes
+
+1. If Caddyfile already exists, phase 7.5 backs it up as:
+   - `${CADDY_DATA_PATH}/Caddyfile.pre_phase7_5.<timestamp>`
+2. Compose stack path for Caddy:
+   - `${UNRAID_COMPOSE_DIR}/caddy/docker-compose.yml`
+3. Script does not change Cloudflare DNS records automatically.
+   - DNS updates are intentional/manual to keep blast radius controlled.
+4. Do not set public Cloudflare proxied records to private `192.168.x.x` addresses.

USAGE_GUIDE.md

@@ -154,6 +154,23 @@ If you prefer to run each phase individually and inspect results:
 ./phase9_security.sh          && ./phase9_post_check.sh
 ```
 
+### Optional Phase 7.5 (one-time Nginx -> Caddy migration)
+
+Use this only if you want one Caddy instance to serve both legacy and new domains.
+
+```bash
+# Canary first (default): tower.sintheus.com + Gitea domain
+./phase7_5_nginx_to_caddy.sh --mode=canary
+
+# Full host map cutover
+./phase7_5_nginx_to_caddy.sh --mode=full
+
+# Enforce strict end-to-end TLS for all upstreams
+./phase7_5_nginx_to_caddy.sh --mode=full --strict-backend-https
+```
+
+Detailed migration context, rationale, and next actions are tracked in `TODO.md`.
+
 ### Skip setup (already done)
 
 ```bash

phase7_5_nginx_to_caddy.sh

@@ -0,0 +1,326 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# =============================================================================
+# phase7_5_nginx_to_caddy.sh — One-time Nginx -> Caddy migration cutover helper
+#
+# Goals:
+#   - Serve both sintheus.com and privacyindesign.com hostnames from one Caddy
+#   - Keep public ingress HTTPS-only
+#   - Support canary-first rollout (default: tower.sintheus.com only)
+#   - Preserve current mixed backend schemes (http/https) unless strict mode is enabled
+#
+# Usage examples:
+#   ./phase7_5_nginx_to_caddy.sh
+#   ./phase7_5_nginx_to_caddy.sh --mode=full
+#   ./phase7_5_nginx_to_caddy.sh --mode=full --strict-backend-https
+#   ./phase7_5_nginx_to_caddy.sh --mode=canary --yes
+# =============================================================================
+
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
+source "${SCRIPT_DIR}/lib/common.sh"
+
+AUTO_YES=false
+MODE="canary"                 # canary|full
+STRICT_BACKEND_HTTPS=false
+
+# Reuse Unraid's existing Docker network.
+UNRAID_DOCKER_NETWORK_NAME="br0"
+
+usage() {
+  cat <<EOF
+Usage: $(basename "$0") [options]
+
+Options:
+  --mode=canary|full       Rollout scope (default: canary)
+  --strict-backend-https   Require all upstream backends to be https://
+  --yes, -y                Skip confirmation prompts
+  --help, -h               Show this help
+EOF
+}
+
+for arg in "$@"; do
+  case "$arg" in
+    --mode=*) MODE="${arg#*=}" ;;
+    --strict-backend-https) STRICT_BACKEND_HTTPS=true ;;
+    --yes|-y) AUTO_YES=true ;;
+    --help|-h) usage; exit 0 ;;
+    *)
+      log_error "Unknown argument: $arg"
+      usage
+      exit 1
+      ;;
+  esac
+done
+
+if [[ "$MODE" != "canary" && "$MODE" != "full" ]]; then
+  log_error "Invalid --mode '$MODE' (use: canary|full)"
+  exit 1
+fi
+
+confirm_action() {
+  local prompt="$1"
+  if [[ "$AUTO_YES" == "true" ]]; then
+    log_info "Auto-confirmed (--yes): ${prompt}"
+    return 0
+  fi
+  printf '%s' "$prompt"
+  read -r confirm
+  [[ "$confirm" =~ ^[Yy]$ ]]
+}
+
+load_env
+require_vars UNRAID_IP UNRAID_SSH_USER UNRAID_COMPOSE_DIR \
+             UNRAID_CADDY_IP UNRAID_GITEA_IP \
+             GITEA_DOMAIN CADDY_DATA_PATH TLS_MODE
+
+if [[ "$TLS_MODE" == "cloudflare" ]]; then
+  require_vars CLOUDFLARE_API_TOKEN
+elif [[ "$TLS_MODE" == "existing" ]]; then
+  require_vars SSL_CERT_PATH SSL_KEY_PATH
+else
+  log_error "Invalid TLS_MODE='${TLS_MODE}' — must be 'cloudflare' or 'existing'"
+  exit 1
+fi
+
+phase_header "8.5" "Nginx to Caddy Migration (Multi-domain)"
+
+# host|upstream|streaming(true/false)|body_limit|insecure_skip_verify(true/false)
+FULL_HOST_MAP=(
+  "ai.sintheus.com|http://192.168.1.82:8181|true|50MB|false"
+  "photos.sintheus.com|http://192.168.1.222:2283|false|50GB|false"
+  "fin.sintheus.com|http://192.168.1.233:8096|true||false"
+  "disk.sintheus.com|http://192.168.1.52:80|false|20GB|false"
+  "pi.sintheus.com|http://192.168.1.4:80|false||false"
+  "plex.sintheus.com|http://192.168.1.111:32400|true||false"
+  "sync.sintheus.com|http://192.168.1.119:8384|false||false"
+  "syno.sintheus.com|https://100.108.182.16:5001|false||true"
+  "tower.sintheus.com|https://192.168.1.82:443|false||true"
+)
+
+CANARY_HOST_MAP=(
+  "tower.sintheus.com|https://192.168.1.82:443|false||true"
+)
+
+GITEA_ENTRY="${GITEA_DOMAIN}|http://${UNRAID_GITEA_IP}:3000|false||false"
+CADDY_COMPOSE_DIR="${UNRAID_COMPOSE_DIR}/caddy"
+
+SELECTED_HOST_MAP=()
+if [[ "$MODE" == "canary" ]]; then
+  SELECTED_HOST_MAP=( "${CANARY_HOST_MAP[@]}" "$GITEA_ENTRY" )
+else
+  SELECTED_HOST_MAP=( "${FULL_HOST_MAP[@]}" "$GITEA_ENTRY" )
+fi
+
+validate_backend_tls_policy() {
+  local -a non_tls_entries=()
+  local entry host upstream
+  for entry in "${SELECTED_HOST_MAP[@]}"; do
+    IFS='|' read -r host upstream _ <<< "$entry"
+    if [[ "$upstream" != https://* ]]; then
+      non_tls_entries+=( "${host} -> ${upstream}" )
+    fi
+  done
+
+  if [[ "${#non_tls_entries[@]}" -eq 0 ]]; then
+    log_success "All selected backends are HTTPS"
+    return 0
+  fi
+
+  if [[ "$STRICT_BACKEND_HTTPS" == "true" ]]; then
+    log_error "Strict backend HTTPS is enabled, but these entries are not HTTPS:"
+    printf '%s\n' "${non_tls_entries[@]}" | sed 's/^/  - /' >&2
+    return 1
+  fi
+
+  log_warn "Using mixed backend schemes (allowed):"
+  printf '%s\n' "${non_tls_entries[@]}" | sed 's/^/  - /' >&2
+}
+
+emit_site_block() {
+  local outfile="$1" host="$2" upstream="$3" streaming="$4" body_limit="$5" skip_verify="$6"
+
+  {
+    echo "${host} {"
+    if [[ "$TLS_MODE" == "existing" ]]; then
+      echo "    tls ${SSL_CERT_PATH} ${SSL_KEY_PATH}"
+    fi
+    echo "    import common_security"
+    echo
+    if [[ -n "$body_limit" ]]; then
+      echo "    request_body {"
+      echo "        max_size ${body_limit}"
+      echo "    }"
+      echo
+    fi
+    echo "    reverse_proxy ${upstream} {"
+    if [[ "$streaming" == "true" ]]; then
+      echo "        import proxy_streaming"
+    else
+      echo "        import proxy_headers"
+    fi
+    if [[ "$skip_verify" == "true" && "$upstream" == https://* ]]; then
+      echo "        transport http {"
+      echo "            tls_insecure_skip_verify"
+      echo "        }"
+    fi
+    echo "    }"
+    echo "}"
+    echo
+  } >> "$outfile"
+}
+
+build_caddyfile() {
+  local outfile="$1"
+  local entry host upstream streaming body_limit skip_verify
+
+  : > "$outfile"
+  {
+    echo "# Generated by phase7_5_nginx_to_caddy.sh"
+    echo "# Mode: ${MODE}"
+    echo
+    echo "{"
+    if [[ "$TLS_MODE" == "cloudflare" ]]; then
+      echo "    acme_dns cloudflare {env.CF_API_TOKEN}"
+    fi
+    echo "    servers {"
+    echo "        trusted_proxies static private_ranges"
+    echo "        protocols h1 h2 h3"
+    echo "    }"
+    echo "}"
+    echo
+    echo "(common_security) {"
+    echo "    encode zstd gzip"
+    echo "    header {"
+    echo "        Strict-Transport-Security \"max-age=31536000; includeSubDomains; preload\""
+    echo "        X-Content-Type-Options \"nosniff\""
+    echo "        X-Frame-Options \"SAMEORIGIN\""
+    echo "        Referrer-Policy \"strict-origin-when-cross-origin\""
+    echo "        -Server"
+    echo "    }"
+    echo "}"
+    echo
+    echo "(proxy_headers) {"
+    echo "    header_up Host {host}"
+    echo "    header_up X-Real-IP {remote_host}"
+    echo "}"
+    echo
+    echo "(proxy_streaming) {"
+    echo "    import proxy_headers"
+    echo "    flush_interval -1"
+    echo "}"
+    echo
+  } >> "$outfile"
+
+  for entry in "${SELECTED_HOST_MAP[@]}"; do
+    IFS='|' read -r host upstream streaming body_limit skip_verify <<< "$entry"
+    emit_site_block "$outfile" "$host" "$upstream" "$streaming" "$body_limit" "$skip_verify"
+  done
+}
+
+if ! validate_backend_tls_policy; then
+  exit 1
+fi
+
+log_step 1 "Creating Caddy data directories on Unraid..."
+ssh_exec UNRAID "mkdir -p '${CADDY_DATA_PATH}/data' '${CADDY_DATA_PATH}/config'"
+log_success "Caddy data directories ensured"
+
+log_step 2 "Deploying Caddy docker-compose on Unraid..."
+if ! ssh_exec UNRAID "docker network inspect '${UNRAID_DOCKER_NETWORK_NAME}'" &>/dev/null; then
+  log_error "Required Docker network '${UNRAID_DOCKER_NETWORK_NAME}' not found on Unraid"
+  exit 1
+fi
+ssh_exec UNRAID "mkdir -p '${CADDY_COMPOSE_DIR}'"
+
+TMP_COMPOSE=$(mktemp)
+CADDY_CONTAINER_IP="${UNRAID_CADDY_IP}"
+GITEA_NETWORK_NAME="${UNRAID_DOCKER_NETWORK_NAME}"
+export CADDY_CONTAINER_IP CADDY_DATA_PATH GITEA_NETWORK_NAME
+
+if [[ "$TLS_MODE" == "cloudflare" ]]; then
+  CADDY_ENV_VARS="      - CF_API_TOKEN=${CLOUDFLARE_API_TOKEN}"
+  CADDY_EXTRA_VOLUMES=""
+else
+  CADDY_ENV_VARS=""
+  CADDY_EXTRA_VOLUMES="      - ${SSL_CERT_PATH}:${SSL_CERT_PATH}:ro
+      - ${SSL_KEY_PATH}:${SSL_KEY_PATH}:ro"
+fi
+export CADDY_ENV_VARS CADDY_EXTRA_VOLUMES
+
+render_template "${SCRIPT_DIR}/templates/docker-compose-caddy.yml.tpl" "$TMP_COMPOSE" \
+  "\${CADDY_DATA_PATH} \${CADDY_CONTAINER_IP} \${CADDY_ENV_VARS} \${CADDY_EXTRA_VOLUMES} \${GITEA_NETWORK_NAME}"
+
+if [[ -z "$CADDY_ENV_VARS" ]]; then
+  sed -i.bak '/^[[:space:]]*environment:$/d' "$TMP_COMPOSE"
+  rm -f "${TMP_COMPOSE}.bak"
+fi
+if [[ -z "$CADDY_EXTRA_VOLUMES" ]]; then
+  sed -i.bak -e :a -e '/^\n*$/{$d;N;ba' -e '}' "$TMP_COMPOSE"
+  rm -f "${TMP_COMPOSE}.bak"
+fi
+
+scp_to UNRAID "$TMP_COMPOSE" "${CADDY_COMPOSE_DIR}/docker-compose.yml"
+rm -f "$TMP_COMPOSE"
+log_success "Caddy compose deployed to ${CADDY_COMPOSE_DIR}"
+
+log_step 3 "Generating and deploying multi-domain Caddyfile..."
+TMP_CADDYFILE=$(mktemp)
+build_caddyfile "$TMP_CADDYFILE"
+
+if ssh_exec UNRAID "test -f '${CADDY_DATA_PATH}/Caddyfile'" 2>/dev/null; then
+  BACKUP_PATH="${CADDY_DATA_PATH}/Caddyfile.pre_phase7_5.$(date +%Y%m%d%H%M%S)"
+  ssh_exec UNRAID "cp '${CADDY_DATA_PATH}/Caddyfile' '${BACKUP_PATH}'"
+  log_info "Backed up previous Caddyfile to ${BACKUP_PATH}"
+fi
+
+scp_to UNRAID "$TMP_CADDYFILE" "${CADDY_DATA_PATH}/Caddyfile"
+rm -f "$TMP_CADDYFILE"
+log_success "Caddyfile deployed"
+
+log_step 4 "Starting/reloading Caddy container..."
+ssh_exec UNRAID "cd '${CADDY_COMPOSE_DIR}' && docker compose up -d 2>/dev/null || docker-compose up -d"
+if ! ssh_exec UNRAID "docker exec caddy caddy reload --config /etc/caddy/Caddyfile --adapter caddyfile" &>/dev/null; then
+  log_warn "Hot reload failed; restarting caddy container"
+  ssh_exec UNRAID "docker restart caddy" >/dev/null
+fi
+log_success "Caddy container is running with new config"
+
+if [[ "$MODE" == "canary" ]]; then
+  if confirm_action "Run canary HTTPS probe for tower.sintheus.com via Caddy IP now? [y/N] "; then
+    if curl -skf --resolve "tower.sintheus.com:443:${UNRAID_CADDY_IP}" \
+      "https://tower.sintheus.com/" >/dev/null; then
+      log_success "Canary probe passed: tower.sintheus.com via ${UNRAID_CADDY_IP}"
+    else
+      log_error "Canary probe failed for tower.sintheus.com via ${UNRAID_CADDY_IP}"
+      exit 1
+    fi
+  fi
+else
+  log_step 5 "Probing all configured hosts via Caddy IP..."
+  PROBE_FAILS=0
+  for entry in "${SELECTED_HOST_MAP[@]}"; do
+    IFS='|' read -r host _ <<< "$entry"
+    if curl -skf --resolve "${host}:443:${UNRAID_CADDY_IP}" "https://${host}/" >/dev/null; then
+      log_success "Probe passed: ${host}"
+    else
+      log_error "Probe failed: ${host}"
+      PROBE_FAILS=$((PROBE_FAILS + 1))
+    fi
+  done
+  if [[ "$PROBE_FAILS" -gt 0 ]]; then
+    log_error "One or more probes failed (${PROBE_FAILS})"
+    exit 1
+  fi
+fi
+
+printf '\n'
+log_success "Phase 7.5 complete (${MODE} mode)"
+log_info "Next (no DNS change required): verify via curl --resolve and browser checks"
+log_info "LAN-only routing option: split-DNS/hosts override to ${UNRAID_CADDY_IP}"
+log_info "Public routing option: point public DNS to WAN ingress (not 192.168.x.x) and forward 443 to Caddy"
+if [[ "$MODE" == "canary" ]]; then
+  log_info "Canary host is tower.sintheus.com (plus Gitea host entry present in config)"
+else
+  log_info "Full host map is now active in Caddy"
+fi