airflow

#Introduction

The Airflow Package Skill manages one VPS running Airflow with LocalExecutor. PostgreSQL runs on the host, containers run Airflow and Caddy, WAL-G continuously archives the metadata database, and a restricted deploy key pushes DAGs from GitHub Actions.

Green, Red, and Blue consume the same colors.yml, render byte-identical artifacts, and manage the same state. Never run them concurrently.

#Quick start

npx skills add getcolors/airflow --skill package-airflow-green
cp .agents/skills/package-airflow-green/green ./green
./green build
./green create --dry-run
# after explicit approval:
./green create

Equivalent Red and Blue skills ship red and blue launchers.

#Workflow architecture

create/build  start ─ compute ─ smtp ─ dns ─ smtp-post ─┬─ ansible-local
                                                         └─ ansible-remote ─ github

delete        start ─ github ─ cleanup ─ smtp-post ─ dns ─┬─ smtp
                                                          └─ compute

GitHub publication follows remote configuration so a workflow cannot use a key before its public half is installed. Delete withdraws that credential before destroying the host.

#Desired state

Images require explicit tags. Optional values should be omitted rather than set to REPLACE_ME. See the full configuration reference.

#Server design

Docker is configured before PostgreSQL so its bridge address is stable. WAL-G is installed before archiving is enabled. The deploy account is created last and is restricted to write-only rrsync access to the DAG directory.

Caddy is the only public route to Airflow’s API server. It computes and verifies a bcrypt password on the machine, obtains TLS with HTTP-01, and provides the single-operator authentication boundary required by this deployment.

#PostgreSQL and recovery

The PostgreSQL major version is pinned because WAL-G base backups do not restore across major versions. A base backup is taken during create; a persistent systemd timer takes later backups, and a separate hourly freshness check detects missing backups.

#DAG delivery

The package creates a private repository if needed, seeds it once over the operator’s SSH identity, and publishes a fresh deployment key to a GitHub Actions environment named after the profile. Existing repository contents are never overwritten.

The server-side key is constrained with restrict and write-only rrsync. It cannot open a shell, read DAGs back, or use sudo.

#Command reference

#State and generated files

.colors/<profile>/
├── airflow-compute/
├── tofu-smtp/
├── tofu-dns/
├── tofu-smtp-post/
├── airflow-ansible-local/
├── airflow-ansible-remote/
└── airflow-github/

Remote keys use <profile>/<stage>.tfstate. The three delegated ONCE stages retain their tofu-* names, making profile uniqueness essential.

#Credentials and safety

• Credentials use COLORS_PAR_* and never belong in desired state.
• Never export COLORS_PAR_PROFILE.
• Keep compute-prevent-destroy: true; deletion removes the metadata database.
• Use a dedicated, narrowly scoped backup bucket separate from state.
• A Cloudflare zone must not be co-managed by another Colors deployment.

#Troubleshooting

DAG import failures under load

Check memory first; DAG parsing is often the first component to fail on an undersized host.

No backup alert arrived

Inspect both the base-backup timer and freshness timer. A timer that never ran cannot trigger its own OnFailure.

Authentication has two login screens

The intended public boundary is Caddy. Verify the API server is not exposed directly and SimpleAuthManager’s all-admin setting is present.