feat: extract .env validators to common.sh and add validate_env()
Move 10 validation functions from configure_env.sh to lib/common.sh as shared utilities. Define variable-to-validator mapping using parallel arrays (bash 3.2 compatible). validate_env() checks all ~50 .env variables against their expected format and reports all failures at once. Wired into preflight.sh (Check 6b) and bitwarden_to_env.sh (post-restore). configure_env.sh now sources validators from common.sh instead of defining its own copies. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -36,11 +36,14 @@ backup/ # Backup and restore scripts
|
||||
## Key Commands
|
||||
- `setup/configure_env.sh` — Interactive .env setup wizard
|
||||
- `setup/cleanup.sh` — Reverse everything setup scripts installed (reads .manifests/)
|
||||
- `preflight.sh` — Validate everything before running phases
|
||||
- `preflight.sh` — Validate everything before running phases (includes .env format validation)
|
||||
- `run_all.sh` — Execute all phases sequentially
|
||||
- `teardown_all.sh` — Reverse teardown (add `--cleanup` to also uninstall prerequisites)
|
||||
- `manage_runner.sh add|remove|list` — Dynamic runner management
|
||||
|
||||
## .env Validation
|
||||
`validate_env()` in `lib/common.sh` checks all ~50 .env variables against their expected format (IP, port, email, path, URL, bool, integer, password, ssl_mode). Uses parallel arrays for the variable-to-validator mapping (bash 3.2 compatible). Called by `preflight.sh` and `bitwarden_to_env.sh`. `configure_env.sh` uses the same individual validators interactively.
|
||||
|
||||
## Version Checking
|
||||
Setup scripts and preflight validate minimum versions for all tools:
|
||||
- Local: jq>=1.6, curl>=7.70, git>=2.30, shellcheck>=0.8, gh>=2.0
|
||||
|
||||
@@ -65,7 +65,7 @@ Each phase has three scripts: the main script, a `_post_check.sh` that independe
|
||||
gitea-migration/
|
||||
├── .env.example # Configuration template (copy to .env)
|
||||
├── runners.conf.example # Runner definitions template
|
||||
├── lib/common.sh # Shared functions (535 lines)
|
||||
├── lib/common.sh # Shared functions + .env validators
|
||||
├── setup/
|
||||
│ ├── configure_env.sh # Interactive .env wizard (~50 prompts)
|
||||
│ ├── macbook.sh # Local prerequisites (brew packages)
|
||||
@@ -87,7 +87,7 @@ gitea-migration/
|
||||
├── backup/
|
||||
│ ├── backup_primary.sh # Gitea dump, SCP to Fedora
|
||||
│ └── restore_to_primary.sh # Restore dump to Unraid
|
||||
├── preflight.sh # 24 pre-flight validation checks
|
||||
├── preflight.sh # 25 pre-flight validation checks
|
||||
├── run_all.sh # Full pipeline orchestration
|
||||
├── teardown_all.sh # Reverse teardown (9 to 1)
|
||||
├── manage_runner.sh # Dynamic runner add/remove/list
|
||||
|
||||
@@ -584,7 +584,7 @@ bw unlock # unlock your vault first
|
||||
./preflight.sh # validate the restored .env
|
||||
```
|
||||
|
||||
The script uses `.env.example` as a template to preserve section headers and comments. Run preflight afterward to confirm all variables are correct and connectivity works.
|
||||
The script uses `.env.example` as a template to preserve section headers and comments. It automatically validates all restored values (IP formats, port ranges, email addresses, booleans, etc.) and warns about any failures. Run preflight afterward to confirm connectivity works.
|
||||
|
||||
**Important**: Do not use `bw export` (full vault export) to get the data — it dumps your entire vault to a plaintext JSON file on disk. The `--bw` flag fetches only the `gitea-migration-env` item.
|
||||
|
||||
|
||||
161
lib/common.sh
161
lib/common.sh
@@ -123,6 +123,167 @@ require_vars() {
|
||||
done
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# .env value validators — shared by configure_env.sh, preflight.sh,
|
||||
# bitwarden_to_env.sh. Each returns 0 (valid) or 1 (invalid).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
validate_ip() {
|
||||
[[ "$1" =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]
|
||||
}
|
||||
|
||||
validate_port() {
|
||||
[[ "$1" =~ ^[0-9]+$ ]] && [[ "$1" -ge 1 ]] && [[ "$1" -le 65535 ]]
|
||||
}
|
||||
|
||||
validate_email() {
|
||||
[[ "$1" == *@* ]]
|
||||
}
|
||||
|
||||
validate_path() {
|
||||
[[ "$1" == /* ]]
|
||||
}
|
||||
|
||||
validate_url() {
|
||||
[[ "$1" =~ ^https?:// ]]
|
||||
}
|
||||
|
||||
validate_bool() {
|
||||
[[ "$1" == "true" ]] || [[ "$1" == "false" ]]
|
||||
}
|
||||
|
||||
validate_integer() {
|
||||
[[ "$1" =~ ^[0-9]+$ ]]
|
||||
}
|
||||
|
||||
validate_nonempty() {
|
||||
[[ -n "$1" ]]
|
||||
}
|
||||
|
||||
validate_password() {
|
||||
[[ ${#1} -ge 8 ]]
|
||||
}
|
||||
|
||||
validate_ssl_mode() {
|
||||
[[ "$1" == "letsencrypt" ]] || [[ "$1" == "existing" ]]
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# .env format validation — validate_env()
|
||||
# Checks every .env variable against its expected type.
|
||||
# Requires .env to be loaded first (variables in environment).
|
||||
# Returns 0 if all pass, 1 if any fail. Does NOT exit — caller decides.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Parallel arrays: variable name → validator type (bash 3.2 compatible).
|
||||
# Order matches configure_env.sh prompt_var calls.
|
||||
_ENV_VAR_NAMES=(
|
||||
UNRAID_IP UNRAID_SSH_USER UNRAID_SSH_PORT
|
||||
UNRAID_GITEA_PORT UNRAID_GITEA_SSH_PORT UNRAID_GITEA_DATA_PATH
|
||||
FEDORA_IP FEDORA_SSH_USER FEDORA_SSH_PORT
|
||||
FEDORA_GITEA_PORT FEDORA_GITEA_SSH_PORT FEDORA_GITEA_DATA_PATH
|
||||
GITEA_ADMIN_USER GITEA_ADMIN_PASSWORD GITEA_ADMIN_EMAIL
|
||||
GITEA_ORG_NAME GITEA_INSTANCE_NAME GITEA_DB_TYPE GITEA_VERSION ACT_RUNNER_VERSION
|
||||
GITEA_DOMAIN GITEA_INTERNAL_URL
|
||||
GITEA_BACKUP_INTERNAL_URL GITEA_BACKUP_MIRROR_INTERVAL
|
||||
BACKUP_STORAGE_PATH BACKUP_RETENTION_COUNT
|
||||
GITHUB_USERNAME GITHUB_TOKEN
|
||||
REPO_1_NAME REPO_2_NAME REPO_3_NAME
|
||||
MIGRATE_ISSUES MIGRATE_LABELS MIGRATE_MILESTONES MIGRATE_WIKI
|
||||
GITHUB_MIRROR_TOKEN GITHUB_MIRROR_INTERVAL
|
||||
NGINX_CONTAINER_NAME NGINX_CONF_PATH SSL_MODE
|
||||
PROTECTED_BRANCH REQUIRE_PR_REVIEW REQUIRED_APPROVALS
|
||||
SEMGREP_VERSION TRIVY_VERSION GITLEAKS_VERSION SECURITY_FAIL_ON_ERROR
|
||||
)
|
||||
|
||||
_ENV_VAR_TYPES=(
|
||||
ip nonempty port
|
||||
port port path
|
||||
ip nonempty port
|
||||
port port path
|
||||
nonempty password email
|
||||
nonempty nonempty nonempty nonempty nonempty
|
||||
nonempty url
|
||||
url nonempty
|
||||
path integer
|
||||
nonempty nonempty
|
||||
nonempty nonempty nonempty
|
||||
bool bool bool bool
|
||||
nonempty nonempty
|
||||
nonempty path ssl_mode
|
||||
nonempty bool integer
|
||||
nonempty nonempty nonempty bool
|
||||
)
|
||||
|
||||
# Conditional variables — validated only when SSL_MODE matches.
|
||||
_ENV_CONDITIONAL_NAMES=(SSL_EMAIL SSL_CERT_PATH SSL_KEY_PATH)
|
||||
_ENV_CONDITIONAL_TYPES=(email path path)
|
||||
_ENV_CONDITIONAL_WHEN=( letsencrypt existing existing)
|
||||
|
||||
# Human-readable format hints for error messages.
|
||||
_validator_hint() {
|
||||
case "$1" in
|
||||
ip) echo "expected: x.x.x.x" ;;
|
||||
port) echo "expected: 1-65535" ;;
|
||||
email) echo "must contain @" ;;
|
||||
path) echo "must start with /" ;;
|
||||
url) echo "must start with http:// or https://" ;;
|
||||
bool) echo "must be true or false" ;;
|
||||
integer) echo "must be a number" ;;
|
||||
nonempty) echo "cannot be empty" ;;
|
||||
password) echo "must be at least 8 characters" ;;
|
||||
ssl_mode) echo "must be letsencrypt or existing" ;;
|
||||
*) echo "invalid" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
validate_env() {
|
||||
local errors=0
|
||||
local i var_name var_type value
|
||||
|
||||
# Validate main variables
|
||||
for ((i = 0; i < ${#_ENV_VAR_NAMES[@]}; i++)); do
|
||||
var_name="${_ENV_VAR_NAMES[$i]}"
|
||||
var_type="${_ENV_VAR_TYPES[$i]}"
|
||||
value="${!var_name:-}"
|
||||
|
||||
if [[ -z "$value" ]]; then
|
||||
log_error " → $var_name is empty ($(_validator_hint "$var_type"))"
|
||||
errors=$((errors + 1))
|
||||
elif ! "validate_${var_type}" "$value"; then
|
||||
log_error " → $var_name='$value' ($(_validator_hint "$var_type"))"
|
||||
errors=$((errors + 1))
|
||||
fi
|
||||
done
|
||||
|
||||
# Validate conditional variables (SSL_MODE-dependent)
|
||||
local ssl_mode="${SSL_MODE:-}"
|
||||
for ((i = 0; i < ${#_ENV_CONDITIONAL_NAMES[@]}; i++)); do
|
||||
var_name="${_ENV_CONDITIONAL_NAMES[$i]}"
|
||||
var_type="${_ENV_CONDITIONAL_TYPES[$i]}"
|
||||
local required_when="${_ENV_CONDITIONAL_WHEN[$i]}"
|
||||
|
||||
if [[ "$ssl_mode" != "$required_when" ]]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
value="${!var_name:-}"
|
||||
if [[ -z "$value" ]]; then
|
||||
log_error " → $var_name is empty (required when SSL_MODE=$required_when, $(_validator_hint "$var_type"))"
|
||||
errors=$((errors + 1))
|
||||
elif ! "validate_${var_type}" "$value"; then
|
||||
log_error " → $var_name='$value' ($(_validator_hint "$var_type"))"
|
||||
errors=$((errors + 1))
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ $errors -gt 0 ]]; then
|
||||
log_error "$errors .env variable(s) failed format validation"
|
||||
return 1
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# OS compatibility checks
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -162,6 +162,11 @@ check_required_vars() {
|
||||
}
|
||||
check 6 "All required .env vars are set" check_required_vars
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Check 6b: .env value format validation (IPs, ports, emails, booleans, etc.)
|
||||
# ---------------------------------------------------------------------------
|
||||
check 6b ".env values pass format validation" validate_env
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Check 7: SSH to Unraid
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -156,4 +156,15 @@ else
|
||||
fi
|
||||
|
||||
log_success "Restored .env to $OUTPUT_FILE ($field_count variables)"
|
||||
log_warn "Review the file before use — check for placeholder values that may need updating"
|
||||
|
||||
# Validate restored values against expected formats
|
||||
log_info "Validating restored .env values..."
|
||||
set -a
|
||||
# shellcheck source=/dev/null
|
||||
source "$OUTPUT_FILE"
|
||||
set +a
|
||||
if validate_env; then
|
||||
log_success "All .env values pass format validation"
|
||||
else
|
||||
log_warn "Some .env values failed validation — review and fix before running preflight"
|
||||
fi
|
||||
|
||||
@@ -11,6 +11,10 @@ PROJECT_ROOT="${SCRIPT_DIR}/.."
|
||||
ENV_FILE="${PROJECT_ROOT}/.env"
|
||||
ENV_EXAMPLE="${PROJECT_ROOT}/.env.example"
|
||||
|
||||
# shellcheck source=../lib/common.sh
|
||||
# shellcheck disable=SC1091
|
||||
source "${PROJECT_ROOT}/lib/common.sh"
|
||||
|
||||
# Colors
|
||||
if [[ -t 1 ]]; then
|
||||
C_RESET='\033[0m'; C_BOLD='\033[1m'; C_GREEN='\033[0;32m'
|
||||
@@ -52,47 +56,10 @@ while IFS= read -r line; do
|
||||
done < "$ENV_FILE"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Validation functions
|
||||
# Validation functions — sourced from lib/common.sh (validate_ip, validate_port,
|
||||
# validate_email, validate_path, validate_url, validate_bool, validate_integer,
|
||||
# validate_nonempty, validate_password, validate_ssl_mode)
|
||||
# ---------------------------------------------------------------------------
|
||||
validate_ip() {
|
||||
[[ "$1" =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]
|
||||
}
|
||||
|
||||
validate_port() {
|
||||
[[ "$1" =~ ^[0-9]+$ ]] && [[ "$1" -ge 1 ]] && [[ "$1" -le 65535 ]]
|
||||
}
|
||||
|
||||
validate_email() {
|
||||
[[ "$1" == *@* ]]
|
||||
}
|
||||
|
||||
validate_path() {
|
||||
[[ "$1" == /* ]]
|
||||
}
|
||||
|
||||
validate_url() {
|
||||
[[ "$1" =~ ^https?:// ]]
|
||||
}
|
||||
|
||||
validate_bool() {
|
||||
[[ "$1" == "true" ]] || [[ "$1" == "false" ]]
|
||||
}
|
||||
|
||||
validate_integer() {
|
||||
[[ "$1" =~ ^[0-9]+$ ]]
|
||||
}
|
||||
|
||||
validate_nonempty() {
|
||||
[[ -n "$1" ]]
|
||||
}
|
||||
|
||||
validate_password() {
|
||||
[[ ${#1} -ge 8 ]]
|
||||
}
|
||||
|
||||
validate_ssl_mode() {
|
||||
[[ "$1" == "letsencrypt" ]] || [[ "$1" == "existing" ]]
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Prompt function
|
||||
|
||||
Reference in New Issue
Block a user