Hands-on 06-configuration-backups-and-recovery: Lab

objective

Create and test a configuration backup lifecycle without writing outside /opt/lab-classroom/class39/.

safety_boundary

Every filesystem mutation in this lab is explicitly located beneath /opt/lab-classroom/class39/. The attempt1 directory is created without a forceful cleanup operation; if it already exists, stop and choose a new attempt directory under the same classroom root after reviewing the previous work.

steps

Create a fresh workspace, sample configuration, and semantic validator:
BASE=/opt/lab-classroom/class39/attempt1
mkdir -p /opt/lab-classroom/class39
mkdir "$BASE"
mkdir "$BASE/config" "$BASE/backups" "$BASE/staging" "$BASE/restore" "$BASE/rollback"
cat > "$BASE/config/app.json" <<'EOF'
{
"service_name": "inventory",
"listen_port": 8080,
"feature_enabled": true
}
EOF
cat > "$BASE/config/service.ini" <<'EOF'
[service]
workers = 4
log_level = INFO
EOF
cat > "$BASE/validate.py" <<'PY'
import configparser
import json
import pathlib
import sys

root = pathlib.Path(sys.argv[1])
with (root / "app.json").open(encoding="utf-8") as handle:
app = json.load(handle)
required = {"service_name", "listen_port", "feature_enabled"}
missing = required.difference(app)
if missing:
raise SystemExit(f"missing JSON keys: {sorted(missing)}")
if app["service_name"] != "inventory":
raise SystemExit("unexpected service_name")
if not isinstance(app["listen_port"], int) or isinstance(app["listen_port"], bool):
raise SystemExit("listen_port must be an integer")
if not 1024 <= app["listen_port"] <= 65535:
raise SystemExit("listen_port is outside the approved range")
if not isinstance(app["feature_enabled"], bool):
raise SystemExit("feature_enabled must be Boolean")
parser = configparser.ConfigParser()
loaded = parser.read(root / "service.ini", encoding="utf-8")
if len(loaded) != 1 or not parser.has_section("service"):
raise SystemExit("service.ini is missing the service section")
workers = parser.getint("service", "workers")
if not 1 <= workers <= 32:
raise SystemExit("workers is outside the approved range")
if parser.get("service", "log_level") not in {"DEBUG", "INFO", "WARNING", "ERROR"}:
raise SystemExit("unsupported log_level")
print(f"validated configuration at {root}")
PY
python3 "$BASE/validate.py" "$BASE/config"
Assemble the package, generate a manifest, record metadata, create the archive, and generate a checksum for the archive:
BASE=/opt/lab-classroom/class39/attempt1
mkdir "$BASE/staging/package"
cp -a "$BASE/config" "$BASE/staging/package/config"
printf '%s\n' 'class_id=39' 'backup_version=1' 'source=config' 'format=tar-gzip' > "$BASE/staging/package/BACKUP_INFO.txt"
(
cd "$BASE/staging/package"
find config -type f -print0 | sort -z | xargs -0 sha256sum > MANIFEST.sha256
)
tar -C "$BASE/staging/package" -czf "$BASE/backups/config-backup-v1.tar.gz" .
sha256sum "$BASE/backups/config-backup-v1.tar.gz" > "$BASE/backups/config-backup-v1.tar.gz.sha256"
Verify the backup before simulating any failure:
BASE=/opt/lab-classroom/class39/attempt1
sha256sum -c "$BASE/backups/config-backup-v1.tar.gz.sha256"
tar -tzf "$BASE/backups/config-backup-v1.tar.gz"
(
cd "$BASE/staging/package"
sha256sum -c MANIFEST.sha256
)
python3 "$BASE/validate.py" "$BASE/staging/package/config"
Introduce configuration drift that remains syntactically and semantically valid, then demonstrate that only comparison with the trusted backup manifest detects it:
BASE=/opt/lab-classroom/class39/attempt1
cat > "$BASE/config/app.json" <<'EOF'
{
"service_name": "inventory",
"listen_port": 9090,
"feature_enabled": false
}
EOF
python3 "$BASE/validate.py" "$BASE/config"
mkdir "$BASE/restore/candidate"
tar -C "$BASE/restore/candidate" -xzf "$BASE/backups/config-backup-v1.tar.gz"
if (
cd "$BASE"
sha256sum -c restore/candidate/MANIFEST.sha256
); then
echo 'ERROR: drift was not detected'
exit 1
else
echo 'Expected result: active configuration differs from the backup manifest'
fi
Validate the isolated recovery candidate before promotion:
BASE=/opt/lab-classroom/class39/attempt1
sha256sum -c "$BASE/backups/config-backup-v1.tar.gz.sha256"
(
cd "$BASE/restore/candidate"
sha256sum -c MANIFEST.sha256
)
python3 "$BASE/validate.py" "$BASE/restore/candidate/config"
cat "$BASE/restore/candidate/BACKUP_INFO.txt"
Preserve the pre-recovery state and promote the validated candidate using directory renames within the same filesystem:
BASE=/opt/lab-classroom/class39/attempt1
cp -a "$BASE/config" "$BASE/rollback/pre-recovery-config"
mv "$BASE/config" "$BASE/config.drifted"
mv "$BASE/restore/candidate/config" "$BASE/config"
python3 "$BASE/validate.py" "$BASE/config"
(
cd "$BASE/restore/candidate"
sha256sum -c MANIFEST.sha256
)
Compare the promoted configuration with the retained drifted state and inspect the final recovery artifacts:
BASE=/opt/lab-classroom/class39/attempt1
printf '%s\n' 'Promoted port:'
python3 -c 'import json,sys; print(json.load(open(sys.argv[1], encoding="utf-8"))["listen_port"])' "$BASE/config/app.json"
printf '%s\n' 'Retained pre-recovery port:'
python3 -c 'import json,sys; print(json.load(open(sys.argv[1], encoding="utf-8"))["listen_port"])' "$BASE/config.drifted/app.json"
find "$BASE" -maxdepth 3 -type f -print | sort