GREEN · RED · BLUE PACKAGE SKILL · VULTR · LANGFUSE

Langfuse Package Skill

A reproducible Green, Red, and Blue workflow for self-hosted Langfuse v4 on six Vultr machines in one VPC: a self-hosted Neon storage tier, Redis, three ClickHouse replicas with their own Keeper quorum, and the application host behind Caddy and Cloudflare — with Cloudflare R2 holding events, media, Neon's layers and WAL, and the backups.

Separate machines, proven separate. Langfuse's own guidance is one Docker Compose host for testing or Kubernetes for production. This package is the shape in between: every component Langfuse says should be separate is on its own machine, reachable only from the peer that needs it — and the smoke gate proves a denial as well as the allows.

Quick start

npx skills add getcolors/langfuse
cp .agents/skills/package-langfuse-green/green ./green
chmod +x green
./green build
./green create --dry-run

Build renders reproducible output and dry-run walks the workflow without provider side effects. Real creation and deletion require explicit authorization.

Three implementations of one model — Clojure/Babashka, TypeScript/Bun, Python/uv — take the same verbs and the same colors.yml. Swap package-langfuse-green/green for package-langfuse-red/red or package-langfuse-blue/blue; scripts/parity.sh renders both fixtures through all three and diffs the trees byte for byte.

Architecture

Storage tier

The getcolors/neon templates rendered from a SHA pin — storage broker, pageserver, one safekeeper, Postgres 17 under compute_ctl — plus one Compose overlay publishing the compute port on the VPC.

Analytics tier

Three ClickHouse replicas, one shard, one Keeper voter each, cluster default so Langfuse migrates ON CLUSTER unaided. Secrets generated on node 0 and propagated; backups to R2 through a disk whose credentials never enter SQL.

Cache tier

Redis 7.2 with noeviction and an append-only file on a named volume, on the VPC address only.

Application tier

langfuse-web, langfuse-worker with the stuck-queue health flag, and Caddy behind Cloudflare. Headless initialization creates the organization, project, user and keys on first boot.

Network

Machine	Admits	From
every machine	22	`vultr-ssh-sources`, key-only
`<profile>-app`	80, 443	Cloudflare's published ranges, resolved at render time
`<profile>-neon`	55433	the app host's VPC address
`<profile>-redis`	6379	the app host's VPC address
`<profile>-clickhouse-{0,1,2}`	8123, 9000 · 9000, 9009, 9181, 9234	the app host · each other

One Vultr firewall group per role, because a group filters the private interface too; ufw mirrors the same rules on every host.

What convergence proves

a trace, a generation and a score in through the ingestion API, read back through the public API, present on node 0 and the last replica, and a new raw-event object in R2;
a media file up through a presigned URL and back with the same sha256;
200 traces queryable within the timeout, host memory under 85 %;
wrong API key, anonymous request, unauthenticated Redis PING, wrong ClickHouse and Postgres passwords refused;
raw TCP to every dependency the app needs, and a refusal on Keeper; UTC on both databases.

Recovery is rehearsed, not assumed

./green rehearse takes fresh backup sets, restores both stores, boots the pinned image in a second Compose project on loopback with the operator-held ENCRYPTION_KEY and SALT, reads the trace, the project and an encrypted LLM connection back through the API, stops a replica under ingestion, restarts Redis with a job queued — and only then writes .colors-recovery-verified, a marker distinct from .colors-ready.

Three secrets outlive the app host. ENCRYPTION_KEY, SALT and the initial user's password are operator-held on purpose: a Postgres backup restored onto a fresh host is readable only with the same two values, and the password is what logs you in when the generated project keys are gone. Keep them outside these machines.

Credentials

Every credential is a COLORS_PAR_* environment variable in a gitignored .envrc.private; the state pair reaches no host, the storage pair reaches the Neon and app hosts, the backup pair reaches the Neon host and ClickHouse node 0. The package refuses a create when pairs are shared unless colors.yml records that choice.

Source on GitHub · The langfuse-vultr deployment · Skills Catalog