Three-tier defaults system | security improvements | error_handler | improved logging | improved container creation | improved architecture (#9540)

* Refactor Core

Refactored misc/alpine-install.func to improve error handling, network checks, and MOTD setup. Added misc/alpine-tools.func and misc/error_handler.func for modular tool installation and error management. Enhanced misc/api.func with detailed exit code explanations and telemetry functions. Updated misc/core.func for better initialization, validation, and execution helpers. Removed misc/create_lxc.sh as part of cleanup.

* Delete config-file.func

* Update install.func

* Refactor stop_all_services function and variable names

Refactor service stopping logic and improve variable handling

* Refactor installation script and update copyright

Updated copyright information and adjusted package installation commands. Enhanced IPv6 disabling logic and improved container customization process.

* Update install.func

* Update license comment format in install.func

* Refactor IPv6 handling and enhance MOTD and SSH

Refactor IPv6 handling and update OS function. Enhance MOTD with additional details and configure SSH settings.

* big core refactor

* Enhance IPv6 configuration menu options

Updated IPv6 Address Management menu options for clarity and added a new option for fully disabling IPv6.

* Update default Node.js version to 24 LTS

* Update misc/alpine-tools.func

Co-authored-by: Michel Roegl-Brunner <73236783+michelroegl-brunner@users.noreply.github.com>

* indention

* remove debugf and duplicate codes

* Update whiptail backtitles and error codes

Removed '[dev]' from whiptail --backtitle strings for consistency. Refactored custom exit codes in build.func and error_handler.func: updated Proxmox error codes, shifted MySQL/MariaDB codes to 260-263, and removed unused MongoDB code. Updated error descriptions to match new codes.

* comments

* Refactor error handling and clean up debug comments

Standardized bash variable checks, removed unnecessary debug and commented code, and clarified error handling logic in container build and setup scripts. These changes improve code readability and maintainability without altering functional behavior.

* Update build.func

* feat: Improve LXC network checks and LINSTOR storage handling

Enhanced LXC container network setup to check for both IPv4 and IPv6 addresses, added connectivity (ping) tests, and provided troubleshooting tips on failure. Updated storage validation to support LINSTOR, including cluster connectivity checks and special handling for LINSTOR template storage.

---------

Co-authored-by: Michel Roegl-Brunner <73236783+michelroegl-brunner@users.noreply.github.com>
This commit is contained in:
CanbiZ
2025-12-04 07:52:18 +01:00
committed by GitHub
parent a5e6810872
commit a826769899
10 changed files with 5275 additions and 2347 deletions
+229 -87
View File
@@ -72,23 +72,19 @@ stop_all_services() {
local service_patterns=("$@")
for pattern in "${service_patterns[@]}"; do
# Find all matching services (use || true to avoid pipeline failures)
local services
services=$(systemctl list-units --type=service --all 2>/dev/null |
grep -oE "${pattern}[^ ]*\.service" 2>/dev/null |
sort -u 2>/dev/null || true)
# Find all matching services
systemctl list-units --type=service --all 2>/dev/null |
grep -oE "${pattern}[^ ]*\.service" |
sort -u |
while read -r service; do
# Only process if we found any services
if [[ -n "$services" ]]; then
while IFS= read -r service; do
[[ -z "$service" ]] && continue
$STD systemctl stop "$service" 2>/dev/null || true
$STD systemctl disable "$service" 2>/dev/null || true
done <<<"$services"
fi
done
done
return 0
}
# ------------------------------------------------------------------------------
@@ -1214,7 +1210,7 @@ setup_deb822_repo() {
local gpg_url="$2"
local repo_url="$3"
local suite="$4"
local component="${5-main}"
local component="${5:-main}"
local architectures="${6-}" # optional
# Validate required parameters
@@ -3242,7 +3238,6 @@ function setup_mongodb() {
return 1
}
# Verify MongoDB was installed correctly
if ! command -v mongod >/dev/null 2>&1; then
msg_error "MongoDB binary not found after installation"
return 1
@@ -3418,12 +3413,12 @@ EOF
# - Optionally installs or updates global npm modules
#
# Variables:
# NODE_VERSION - Node.js version to install (default: 22)
# NODE_VERSION - Node.js version to install (default: 24 LTS)
# NODE_MODULE - Comma-separated list of global modules (e.g. "yarn,@vue/cli@5.0.0")
# ------------------------------------------------------------------------------
function setup_nodejs() {
local NODE_VERSION="${NODE_VERSION:-22}"
local NODE_VERSION="${NODE_VERSION:-24}"
local NODE_MODULE="${NODE_MODULE:-}"
# ALWAYS clean up legacy installations first (nvm, etc.) to prevent conflicts
@@ -3485,14 +3480,11 @@ function setup_nodejs() {
return 1
}
# CRITICAL: Force APT cache refresh AFTER repository setup
# This ensures NodeSource is the only nodejs source in APT cache
# Force APT cache refresh after repository setup
$STD apt update
# Install dependencies (NodeSource is now the only nodejs source)
ensure_dependencies curl ca-certificates gnupg
# Install Node.js from NodeSource
install_packages_with_retry "nodejs" || {
msg_error "Failed to install Node.js ${NODE_VERSION} from NodeSource"
return 1
@@ -3643,12 +3635,12 @@ function setup_php() {
local CURRENT_PHP=""
CURRENT_PHP=$(is_tool_installed "php" 2>/dev/null) || true
# CRITICAL: If wrong version is installed, remove it FIRST before any pinning
# Remove conflicting PHP version before pinning
if [[ -n "$CURRENT_PHP" && "$CURRENT_PHP" != "$PHP_VERSION" ]]; then
msg_info "Removing conflicting PHP ${CURRENT_PHP} (need ${PHP_VERSION})"
stop_all_services "php.*-fpm"
$STD apt-get purge -y "php*" 2>/dev/null || true
$STD apt-get autoremove -y 2>/dev/null || true
$STD apt purge -y "php*" 2>/dev/null || true
$STD apt autoremove -y 2>/dev/null || true
fi
# NOW create pinning for the desired version
@@ -3675,7 +3667,7 @@ EOF
}
ensure_apt_working || return 1
$STD apt-get update
$STD apt update
# Get available PHP version from repository
local AVAILABLE_PHP_VERSION=""
@@ -3790,7 +3782,6 @@ EOF
local INSTALLED_VERSION=$(php -v 2>/dev/null | awk '/^PHP/{print $2}' | cut -d. -f1,2)
# Critical: if major.minor doesn't match, fail and cleanup
if [[ "$INSTALLED_VERSION" != "$PHP_VERSION" ]]; then
msg_error "PHP version mismatch: requested ${PHP_VERSION} but got ${INSTALLED_VERSION}"
msg_error "This indicates a critical package installation issue"
@@ -3870,74 +3861,14 @@ function setup_postgresql() {
local SUITE
case "$DISTRO_CODENAME" in
trixie | forky | sid)
# For Debian Testing/Unstable, try PostgreSQL repo first, fallback to native packages
if verify_repo_available "https://apt.postgresql.org/pub/repos/apt" "trixie-pgdg"; then
SUITE="trixie-pgdg"
setup_deb822_repo \
"pgdg" \
"https://www.postgresql.org/media/keys/ACCC4CF8.asc" \
"https://apt.postgresql.org/pub/repos/apt" \
"$SUITE" \
"main"
if ! $STD apt update; then
msg_warn "Failed to update PostgreSQL repository, falling back to native packages"
SUITE=""
fi
else
SUITE=""
SUITE="bookworm-pgdg"
fi
# If no repo or packages not installable, use native Debian packages
if [[ -z "$SUITE" ]] || ! apt-cache show "postgresql-${PG_VERSION}" 2>/dev/null | grep -q "Version:"; then
msg_info "Using native Debian packages for $DISTRO_CODENAME"
# Install ssl-cert dependency if available
if apt-cache search "^ssl-cert$" 2>/dev/null | grep -q .; then
$STD apt install -y ssl-cert 2>/dev/null || true
fi
if ! $STD apt install -y postgresql postgresql-client 2>/dev/null; then
msg_error "Failed to install native PostgreSQL packages"
return 1
fi
if ! command -v psql >/dev/null 2>&1; then
msg_error "PostgreSQL installed but psql command not found"
return 1
fi
# Restore database backup if we upgraded from previous version
if [[ -n "$CURRENT_PG_VERSION" ]]; then
msg_info "Restoring PostgreSQL databases from backup..."
$STD runuser -u postgres -- psql </var/lib/postgresql/backup_$(date +%F)_v${CURRENT_PG_VERSION}.sql 2>/dev/null || {
msg_warn "Failed to restore database backup - this may be expected for major version upgrades"
}
fi
$STD systemctl enable --now postgresql 2>/dev/null || true
# Get actual installed version
INSTALLED_VERSION="$(psql -V 2>/dev/null | awk '{print $3}' | cut -d. -f1)"
# Add PostgreSQL binaries to PATH
if ! grep -q '/usr/lib/postgresql' /etc/environment 2>/dev/null; then
echo 'PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/postgresql/'"${INSTALLED_VERSION}"'/bin"' >/etc/environment
fi
cache_installed_version "postgresql" "$INSTALLED_VERSION"
msg_ok "Setup PostgreSQL $INSTALLED_VERSION (native)"
# Install optional modules if specified
if [[ -n "$PG_MODULES" ]]; then
IFS=',' read -ra MODULES <<<"$PG_MODULES"
for module in "${MODULES[@]}"; do
$STD apt install -y "postgresql-${INSTALLED_VERSION}-${module}" 2>/dev/null || true
done
fi
return 0
fi
;;
*)
SUITE=$(get_fallback_suite "$DISTRO_ID" "$DISTRO_CODENAME" "https://apt.postgresql.org/pub/repos/apt")
@@ -4831,3 +4762,214 @@ function setup_yq() {
cache_installed_version "yq" "$FINAL_VERSION"
msg_ok "Setup yq $FINAL_VERSION"
}
# ------------------------------------------------------------------------------
# Docker Engine Installation and Management (All-In-One)
#
# Description:
# - Detects and migrates old Docker installations
# - Installs/Updates Docker Engine via official repository
# - Optional: Installs/Updates Portainer CE
# - Updates running containers interactively
# - Cleans up legacy repository files
#
# Usage:
# setup_docker
# DOCKER_PORTAINER="true" setup_docker
# DOCKER_LOG_DRIVER="json-file" setup_docker
#
# Variables:
# DOCKER_PORTAINER - Install Portainer CE (optional, "true" to enable)
# DOCKER_LOG_DRIVER - Log driver (optional, default: "journald")
# DOCKER_SKIP_UPDATES - Skip container update check (optional, "true" to skip)
#
# Features:
# - Migrates from get.docker.com to repository-based installation
# - Updates Docker Engine if newer version available
# - Interactive container update with multi-select
# - Portainer installation and update support
# ------------------------------------------------------------------------------
function setup_docker() {
local docker_installed=false
local portainer_installed=false
# Check if Docker is already installed
if command -v docker &>/dev/null; then
docker_installed=true
DOCKER_CURRENT_VERSION=$(docker --version | grep -oP '\d+\.\d+\.\d+' | head -1)
msg_info "Docker $DOCKER_CURRENT_VERSION detected"
fi
# Check if Portainer is running
if docker ps --format '{{.Names}}' 2>/dev/null | grep -q '^portainer$'; then
portainer_installed=true
msg_info "Portainer container detected"
fi
# Cleanup old repository configurations
if [ -f /etc/apt/sources.list.d/docker.list ]; then
msg_info "Migrating from old Docker repository format"
rm -f /etc/apt/sources.list.d/docker.list
rm -f /etc/apt/keyrings/docker.asc
fi
# Setup/Update Docker repository
msg_info "Setting up Docker Repository"
setup_deb822_repo \
"docker" \
"https://download.docker.com/linux/$(get_os_info id)/gpg" \
"https://download.docker.com/linux/$(get_os_info id)" \
"$(get_os_info codename)" \
"stable" \
"$(dpkg --print-architecture)"
# Install or upgrade Docker
if [ "$docker_installed" = true ]; then
msg_info "Checking for Docker updates"
DOCKER_LATEST_VERSION=$(apt-cache policy docker-ce | grep Candidate | awk '{print $2}' | cut -d':' -f2 | cut -d'-' -f1)
if [ "$DOCKER_CURRENT_VERSION" != "$DOCKER_LATEST_VERSION" ]; then
msg_info "Updating Docker $DOCKER_CURRENT_VERSION$DOCKER_LATEST_VERSION"
$STD apt install -y --only-upgrade \
docker-ce \
docker-ce-cli \
containerd.io \
docker-buildx-plugin \
docker-compose-plugin
msg_ok "Updated Docker to $DOCKER_LATEST_VERSION"
else
msg_ok "Docker is up-to-date ($DOCKER_CURRENT_VERSION)"
fi
else
msg_info "Installing Docker"
$STD apt install -y \
docker-ce \
docker-ce-cli \
containerd.io \
docker-buildx-plugin \
docker-compose-plugin
DOCKER_CURRENT_VERSION=$(docker --version | grep -oP '\d+\.\d+\.\d+' | head -1)
msg_ok "Installed Docker $DOCKER_CURRENT_VERSION"
fi
# Configure daemon.json
local log_driver="${DOCKER_LOG_DRIVER:-journald}"
mkdir -p /etc/docker
if [ ! -f /etc/docker/daemon.json ]; then
cat <<EOF >/etc/docker/daemon.json
{
"log-driver": "$log_driver"
}
EOF
fi
# Enable and start Docker
systemctl enable -q --now docker
# Portainer Management
if [[ "${DOCKER_PORTAINER:-}" == "true" ]]; then
if [ "$portainer_installed" = true ]; then
msg_info "Checking for Portainer updates"
PORTAINER_CURRENT=$(docker inspect portainer --format='{{.Config.Image}}' 2>/dev/null | cut -d':' -f2)
PORTAINER_LATEST=$(curl -fsSL https://registry.hub.docker.com/v2/repositories/portainer/portainer-ce/tags?page_size=100 | grep -oP '"name":"\K[0-9]+\.[0-9]+\.[0-9]+"' | head -1 | tr -d '"')
if [ "$PORTAINER_CURRENT" != "$PORTAINER_LATEST" ]; then
read -r -p "${TAB3}Update Portainer $PORTAINER_CURRENT$PORTAINER_LATEST? <y/N> " prompt
if [[ ${prompt,,} =~ ^(y|yes)$ ]]; then
msg_info "Updating Portainer"
docker stop portainer
docker rm portainer
docker pull portainer/portainer-ce:latest
docker run -d \
-p 9000:9000 \
-p 9443:9443 \
--name=portainer \
--restart=always \
-v /var/run/docker.sock:/var/run/docker.sock \
-v portainer_data:/data \
portainer/portainer-ce:latest
msg_ok "Updated Portainer to $PORTAINER_LATEST"
fi
else
msg_ok "Portainer is up-to-date ($PORTAINER_CURRENT)"
fi
else
msg_info "Installing Portainer"
docker volume create portainer_data
docker run -d \
-p 9000:9000 \
-p 9443:9443 \
--name=portainer \
--restart=always \
-v /var/run/docker.sock:/var/run/docker.sock \
-v portainer_data:/data \
portainer/portainer-ce:latest
LOCAL_IP=$(hostname -I | awk '{print $1}')
msg_ok "Installed Portainer (http://${LOCAL_IP}:9000)"
fi
fi
# Interactive Container Update Check
if [[ "${DOCKER_SKIP_UPDATES:-}" != "true" ]] && [ "$docker_installed" = true ]; then
msg_info "Checking for container updates"
# Get list of running containers with update status
local containers_with_updates=()
local container_info=()
local index=1
while IFS= read -r container; do
local name=$(echo "$container" | awk '{print $1}')
local image=$(echo "$container" | awk '{print $2}')
local current_digest=$(docker inspect "$name" --format='{{.Image}}' 2>/dev/null | cut -d':' -f2 | cut -c1-12)
# Pull latest image digest
docker pull "$image" >/dev/null 2>&1
local latest_digest=$(docker inspect "$image" --format='{{.Id}}' 2>/dev/null | cut -d':' -f2 | cut -c1-12)
if [ "$current_digest" != "$latest_digest" ]; then
containers_with_updates+=("$name")
container_info+=("${index}) ${name} (${image})")
((index++))
fi
done < <(docker ps --format '{{.Names}} {{.Image}}')
if [ ${#containers_with_updates[@]} -gt 0 ]; then
echo ""
echo "${TAB3}Container updates available:"
for info in "${container_info[@]}"; do
echo "${TAB3} $info"
done
echo ""
read -r -p "${TAB3}Select containers to update (e.g., 1,3,5 or 'all' or 'none'): " selection
if [[ ${selection,,} == "all" ]]; then
for container in "${containers_with_updates[@]}"; do
msg_info "Updating container: $container"
docker stop "$container"
docker rm "$container"
# Note: This requires the original docker run command - best to recreate via compose
msg_ok "Stopped and removed $container (please recreate with updated image)"
done
elif [[ ${selection,,} != "none" ]]; then
IFS=',' read -ra SELECTED <<<"$selection"
for num in "${SELECTED[@]}"; do
num=$(echo "$num" | xargs) # trim whitespace
if [[ "$num" =~ ^[0-9]+$ ]] && [ "$num" -ge 1 ] && [ "$num" -le "${#containers_with_updates[@]}" ]; then
container="${containers_with_updates[$((num - 1))]}"
msg_info "Updating container: $container"
docker stop "$container"
docker rm "$container"
msg_ok "Stopped and removed $container (please recreate with updated image)"
fi
done
fi
else
msg_ok "All containers are up-to-date"
fi
fi
msg_ok "Docker setup completed"
}