green specification · 2026-07-06

A babashka-compatible Clojure library for building idempotent devops CLIs: desired state in YAML or EDN, workflows as data-threading step graphs, Selmer-scaffolded configuration, OpenTofu as the muscle, and Ansible for SSH provisioning.

Read the source tour (Docco-style).

1. Philosophy

• green is a library. Users define steps, wiring, and workflows in their own project; green supplies the engine, infrastructure helpers, and CLI plumbing.
• Everything is a map. The desired state is an EDN map loaded into opts. Every step reads and returns that map, possibly enriched with observed reality such as OpenTofu outputs or Ansible recaps.
• Idempotent apply. There is no plan phase in green itself. Re-running the same event should converge. create and delete are conventions; the event set is open.
• Unix exit-code semantics. Steps report success/failure with :green/exit; the workflow's final value becomes the process exit code.
• babashka-compatible. The library and tests run under babashka and the JVM.

2. Desired state & the opts contract

The CLI reads green.edn by default (override with -f/--file), stamps the lifecycle event, and passes the map to wf/run. green reserves :green/* keys:

3. Steps

A step is a plain function opts → opts, named by a qualified keyword such as :zk/node. The name anchors wiring, logging, CLI slices, and advice.

• A returned map without :green/exit is treated as success.
• Returning a non-map is an error.
• Throwing is allowed: the runner catches the exception and returns :green/exit 1 (or :green/exit from exception data), plus :green/err and :green/trace.
• Steps usually branch on :green/event. Library steps use the convention “any non-:delete event creates/applies; :delete destroys/deprovisions.”

4. Workflows: wire-fn, next-fn, and composition

(require '[green.workflow :as wf])

(def workflow
  (wf/workflow {:start   :zk/start      ;; required
                :end     nil            ;; optional inclusive slice boundary
                :wire-fn wire-fn        ;; required static graph
                :next-fn next-fn}))     ;; optional dynamic router

(wf/run workflow opts)                  ;; => final opts

wire-fn — the static graph

(wire-fn step run-opts) returns a vector containing the step function followed by its default successors for this run. run-opts is the initial opts passed to wf/run, so the graph may depend on stable run-level inputs such as :green/event.

(defn wire-fn [step run-opts]
  (case step
    :zk/start   [start-step :zk/node]
    :zk/node    [node-step  :zk/zoo-cfg]
    :zk/zoo-cfg [zoo-cfg-step]
    :ci/fork    [fork-step  :ci/a :ci/b]))

Multiple successors mean a static fork. The same graph is used for join scheduling, so keep the happy path visible even when next-fn adds dynamic behavior. Do not base wire-fn on values produced after the run starts; route those cases in next-fn.

next-fn — the dynamic router

When present, next-fn decides successors after each non-end step:

(next-fn step default-next opts) ;; => seq of [next-step opts] pairs

• Return no pairs to terminate a branch, one pair to continue, or many pairs to fan out in parallel.
• Because it sees the step result, including :green/exit, next-fn can route failures to cleanup, retry, or halt.
• Without next-fn, positive :green/exit stops that path; otherwise the default successors run.

Composition — a workflow is a step

(wf/step sub-wf)
(wf/step sub-wf
         {:in-fn  (fn [opts] sub-opts)
          :out-fn (fn [opts sub-result] parent-result)})

• The sub-workflow's result is just a step result, so failures propagate through parent routing naturally.
• With default :in-fn, ambient keys such as :green/event and :green/dry-run flow into the child. If custom :in-fn builds opts from scratch, carry the ambient keys the child needs.
• Advice inheritance is preserved even when :in-fn rebuilds opts from scratch. This is what lets a parent impose dry-run, backend, audit, or provider advice on embedded workflows.

Slices

The optional :end step is inclusive: it runs, then the workflow stops. The CLI's --start and --end override workflow boundaries for safe idempotent slice runs.

5. Parallelism, fan-out & joins

• Static fork: wire-fn lists several successors.
• Dynamic fan-out: next-fn returns several [step opts] pairs.
• Join: branches from different origins that converge on the same step run that step once, using the deepest shared fork's opts plus :green/branches.
• Failure: a failed branch inside a fork lets siblings finish their current step, collapses the fork, skips the join, and propagates the worst exit with all branch results under :green/branches.

Scheduler algorithm in plain English

1. Start with one live task: run :start with the initial opts.
2. Loop while work remains, tracking live and finished branch entries.
3. Group live entries by step.
4. Run only ready steps: if a different live branch can still reach a step, wait so it can join.
5. Run ready units concurrently with future.
6. Expand each result into zero, one, or many next entries.
7. When different origins arrive at one step, run a join unit once.
8. Collapse failed forks before scheduling more work.
9. Finalize to one opts map: first failure wins, otherwise the last success.

Independent pipelines inside a fan-out

When next-fn fans out N branches and each branch traverses multiple steps in sequence (e.g. :validate → :transform → :load), the engine joins at the second step. The N :validate entries share the same fan-out parent, so they run independently; but the N :transform entries arrive from N different :validate units, so same-origin? is false and the scheduler treats them as convergence — running :transform once as a join instead of N times independently.

The fix is wf/step: wrap the multi-step pipeline in a sub-workflow and wire the wf/step as a single step in the parent. The N branches all land on that one step name, sharing the same fan-out parent, so same-origin? is true and they run independently. Inside each branch the sub-workflow runs its own scheduler with no other branches to converge with. The parent's advice on the sub-workflow's step names is inherited by name — dry-run, backends, gates, progress all carry over without re-registration.

;; The per-item pipeline as a sub-workflow
(def pipeline-wf
  (wf/workflow {:start :validate
                :wire-fn (fn [step _]
                           (case step
                             :validate  [validate-step :transform]
                             :transform [transform-step :load]
                             :load      [load-step]))}))

;; Wire it as a single step in the parent
(defn wire-fn [step _run-opts]
  (case step
    :start   [start-step :process]
    :process [(wf/step pipeline-wf)]))

;; Fan out one :process per item
(defn next-fn [step _default-next opts]
  (when (= step :start)
    (mapv (fn [item] [:process (assoc opts :item item)])
          (:items opts))))

A join consumes the deepest fork shared by all incoming branches and preserves its enclosing forks. Failed joins use the highest exit code and that branch's error and trace. Equal codes select the first incoming branch.

YAML plain scientific notation, octal and hexadecimal values are numeric. Date-like and sexagesimal values remain strings. Quoted scalars remain strings. Unknown CLI options return exit 2 before workflow execution.

Process helpers return 127 when a command cannot start, 124 on timeout and 130 on interruption. The timeout covers process exit and output capture. Cleanup waits up to one additional second for pipes, plus a bounded group-kill command. Captured commands use POSIX process groups when setsid and kill are available. Timed commands also check for surviving session members; ps distinguishes running children from unreaped zombies when available. Other platforms and inherited-terminal execution use Java descendant handles, which cannot recover children already reparented after the leader exits. A child that starts its own session can escape group cleanup.

6. Advice

Advice wraps step functions without changing graph wiring. Registries are immutable workflow data; adding/removing advice returns a new workflow.

(-> workflow
    (wf/advice-add :zk/node :before ::backend write-backend!)
    (wf/advice-add-all :around ::progress progress)
    (wf/advice-remove :zk/node ::backend)
    (wf/advice-remove-all ::progress))

For :after-while and :after-until, a map is true when :green/exit is missing or zero and false when it is positive; non-map values keep Clojure truthiness for direct green.advice/compose use.

Stacking is Emacs-like: at equal :depth, newest advice is outermost. :depth (-100..100, default 0) overrides add order: lower depths run farther outside, higher depths farther inside. Re-adding the same id replaces the old entry and moves it according to the new sequence/depth.

Advice inheritance through wf/step. The enclosing run stamps its effective advice into opts under a private key, and the nested run merges that advice over the child's own registries. Ancestor entries are outermost at equal depth; same-id ancestor entries replace child entries. Use wf/advice-plan with a workflow or chain such as [parent child] to inspect the composed stack.

• green.dry-run/advise attaches :around advice id :green.dry-run/skip to named steps. With :green/dry-run, it prints dry-run: would run :step (event) and returns exit 0 without calling the step.
• green.progress/advise attaches all-step :around advice id :green.progress/progress. It prints >>> :step (event) before and <<< :step (Nms) after, reading :green/step.

7. The scaffolding DSL

green.scaffold/scaffold materializes or removes a flat sequence of file specs:

(sc/scaffold opts
  [{:template :zk/main.tf
    :target   "{{workdir}}/nodes/{{node.id}}/main.tf"
    :data     {:workdir (:zk/workdir opts) :node node}}])

On :green/event :delete, the same specs name targets to remove. The implementation prunes the immediate parent directory if it becomes empty. Return keys are :green.scaffold/written or :green.scaffold/deleted.

8. OpenTofu integration

(require '[green.tofu :as tofu])

(tofu/tofu-step opts {:dir "work/nodes/1"})
(tofu/tofu-step opts {:dir "work/nodes/1" :output-key :zk/node-outputs})

• Any non-:delete event: tofu init -input=false -no-color, then tofu apply -auto-approve -input=false -no-color, then tofu output -json.
• :delete: tofu init, then tofu destroy -auto-approve.
• Apply outputs are parsed to {keyword value} and assoc'd under :tofu/outputs by default. If you override :output-key, keep it namespaced.
• Non-zero process exits become :green/exit and :green/err.
• A launch failure — no such :dir, or no tofu binary on PATH — is a failed exit, never a raw exception: the runner reports exit 127 with the start error as :err, the same shape as red's and blue's runtimes and as green.process/run. tofu-step then returns the ordinary tofu init failed: outcome, and outputs throws its tofu output failed: ex-info carrying {:dir dir} — the one shape a caller reading tofu state may rely on.
• Backends are not hardwired. backend-advice writes backend.tf.json for any backend type and config map (or function of opts), preserving native JSON values and nested collections. Helpers ship for local, S3, and GCS.

9. Ansible integration

(require '[green.ansible :as ansible])

(ansible/ansible-step opts
  {:dir "ansible"
   :inventory "inventory.ini"
   :private-key "work/ssh/id_ed25519"
   :host-key-checking false})

• Any non-:delete event runs the create playbook (create.yml by default); :delete runs the delete playbook (delete.yml by default). Override with :playbooks.
• Options map to ansible-playbook: :inventory, :private-key, :user, :extra-vars (JSON -e), and :host-key-checking false (exports ANSIBLE_HOST_KEY_CHECKING=False).
• On success, PLAY RECAP is parsed into per-host counters under :ansible/recap by default; override with namespaced :recap-key.
• inventory-ini renders deterministic INI from {group {:hosts [{:name .. :vars {..}}] :vars {..}}}. Strings preserve whitespace, quotes, backslashes and literal-looking values such as "123". Host values include shell quoting around their Python literal; group values use the literal directly.
• inventory-advice is a :before advice that writes an inventory file from data or a function of opts, mirroring OpenTofu backend advice.

10. The CLI

./green <event> [-f|--file green.edn] [--start step] [--end step] [--dry-run]

./green create
./green delete -f prod.edn
./green create --start zk/node --end zk/node
./green create --dry-run

• The first positional argument is keywordized into :green/event.
• --start and --end coerce to keywords and override workflow boundaries.
• --dry-run stamps :green/dry-run true; it has an effect only on steps advised with green.dry-run.
• Missing event, missing state file, or parse/runtime CLI errors return exit 2 from run-cli. exec prints :green/err/:green/trace to stderr and exits.

A consumer launcher should use a git dependency with an explicit SHA until green is published to Clojars:

#!/usr/bin/env bb
(babashka.deps/add-deps
  '{:deps {io.github.getcolors/green {:git/sha "REPLACE_WITH_COMMIT_SHA"}}})
(babashka.classpath/add-classpath "resources")
(require '[green.cli :as cli] '[green.workflow :as wf])

(defn wire-fn [step _run-opts]
  (case step
    :app/start [identity]))

(def workflow (wf/workflow {:start :app/start :wire-fn wire-fn}))
(cli/exec workflow)

11. Examples

Every example is a self-contained babashka CLI that depends on this checkout with :local/root "../..". Four examples are mock-only OpenTofu demos; floci-zookeeper creates real local containers through floci.

cd examples/zookeeper
./green create --dry-run
./green create
./green delete

cd ../multi-zookeeper
./green create --dry-run
./green create
./green delete

cd ../once
./green create --dry-run
./green create
./green delete

cd ../multi-once
./green create --dry-run   # offline path; real create needs an S3 bucket
./green create
./green delete

cd ../floci-zookeeper
./green create --dry-run   # validates and prints; touches nothing
./green create             # real cluster on floci + Ansible
./green delete             # delete.yml first, then tofu destroys

12. Testing strategy

The suite runs under babashka (bb test) and the JVM (clojure -X:test). Under babashka, bb.edn explicitly requires and runs each test namespace; under the JVM, deps.edn's :test alias uses cognitect.test-runner discovery.

• test/green/workflow_test.clj covers linear runs, errors, slices, event-specific static graphs, dynamic routing, fork/join, failure collapse, concurrency, and composition.
• test/green/advice_test.clj covers every advice combinator, ordering/depth, all-step advice, removal/replacement, and inheritance through wf/step.
• test/green/scaffold_test.clj covers template resolution, create/delete symmetry, and idempotent rendering.
• test/green/tofu_test.clj covers backend advice rendering without invoking tofu.
• test/green/ansible_test.clj covers playbook selection, PLAY RECAP parsing, inventory rendering, and inventory advice without invoking ansible-playbook.
• test/green/zookeeper_test.clj is the e2e suite: if tofu is on PATH, it runs real render/apply/output/destroy cycles over HCL containing only locals and output blocks.

The real floci example is intentionally not in the automated suite because it requires Docker/floci, Ansible, provider downloads, and minutes of runtime; verify it manually with its ./green create/delete.

13. Project layout & API index

green/
├── deps.edn            JVM deps: selmer, babashka/cli, cheshire (+ :test, :build)
├── bb.edn              paths + `bb test` task
├── build.clj           tools.build: jar / install / deploy
├── index.html          this specification
├── docco.html          generated source tour
├── src/green/
│   ├── advice.clj      nadvice combinators and pure registry ops
│   ├── workflow.clj    workflow, run, step, advice add/remove/all, advice-plan
│   ├── scaffold.clj    flat Selmer file-spec DSL
│   ├── tofu.clj        tofu-step, outputs, backend advices (local/s3/gcs)
│   ├── ansible.clj     ansible-step, recap parsing, inventory advice
│   ├── dry_run.clj     dry-run advice + advise
│   ├── progress.clj    progress advice + advise
│   └── cli.clj         run-cli, exec, arg parsing
├── test/green/         advice, workflow, scaffold, cli, dry-run, progress, tofu, ansible, zookeeper
└── examples/           zookeeper, multi-zookeeper, once, multi-once, floci-zookeeper

Distribution

Library coordinate: io.github.getcolors/green. It has not been published to Clojars yet.

• Current rule: external consumers and launcher snippets must use a git dep with an explicit commit SHA: io.github.getcolors/green {:git/sha "REPLACE_WITH_COMMIT_SHA"}.
• Development: in-repo examples use {:local/root "../.."}.
• Future Clojars publishing: clojure -T:build jar, install, and deploy are available; deploy reads CLOJARS_USERNAME/CLOJARS_PASSWORD.