JoobQ.Workflow (joobq_ex v0.8.0)

Copy Markdown View Source

Builds a workflow blueprint: the steps an SDK worker runs and the engine conducts (it retries, waits, compensates and keeps the run's state). JoobQ.Workflow builds and registers workflows; JoobQ.Workflows starts runs and talks to them.

import JoobQ.Workflow

new("checkout", task_queue: "workflows")
|> step("reserve", &Shop.reserve/2, input: %{sku: input("sku")})
|> step("charge", &Shop.charge/2, input: output("reserve"))

A handler is a 2-arity function fn input, job -> ... end returning what perform/1 returns. Its input is the input: spec with each reference replaced by the value it names (the run's input when no input: is given). Building raises ArgumentError naming the workflow, the node and the option, never a value or a handler.

Summary

Functions

Declares a change id the workflow's code went through: runs started from now on get version max: (1), runs already past a gated node keep theirs; min: (0) is the oldest version still supported. Declare it before the nodes gated on it.

Runs the workflow workflow as a child and waits for it. input: is references only (in maps and lists): the engine resolves it, no handler runs. A child has no retry: or timeout_ms:: give its own steps those.

Undoes a node when the run fails after it completed: handler runs with the input: spec resolved (default: the node's output).

Ends the run as continued_as_new and starts a fresh one when it completes (and only_if: holds), with input: (references only, default: the same input), at most max_runs: (1..1000000) runs in the chain. max_events: (10..10000000) continues earlier, mid-run: its references may only read the input.

Runs a node only for runs whose version of the change change is within min:..max: (one is required).

A reference to the run's input, or to key in it (a name, a dotted path or a list index).

A workflow named name. Options: version: (1), task_queue: ("workflows") and timeout_ms: (the run's deadline, none by default).

A reference to the output of the earlier step step, or to key in it.

A group whose nodes run at the same time; fun adds them to the workflow it is given and returns it. It is named parallel_<n>, n being its position in the enclosing list.

A query name answered from the run's state: ref (a reference to the input, an earlier step, a signal or an update). Not state or steps, at most 100.

A reference given as a path, such as signals.approve or ${updates.rename}.

A step's retry policy, applied by the engine: attempts: (3, 1..100), initial_ms: (1000), backoff: (2.0, 1..10), maximum_ms: (30000) and non_retryable:, the exceptions (modules) or error class names not retried.

A cancellation scope id around the nodes fun adds: timeout_ms: cancels them when it passes, non_cancellable: true shields them from the run's cancellation (and takes no timeout_ms:). Scopes do not nest.

A group whose nodes run one after the other, named sequence_<n> (see parallel/2).

A step: handler runs with the input: spec resolved (default: the run's input). The engine enforces timeout_ms: (one attempt), heartbeat_ms: and schedule_to_start_ms: (at least 1000 each) and retry: (see retry/1).

Waits for_ms: milliseconds.

The blueprint the gateway registers, as a map ready for JSON (absent options are left out).

An update name callers send to a run (JoobQ.Workflows). validate: (a reference, e.g. update_input("street")) must be truthy or the update is rejected with message:, a static text: never put data in it. With a handler, the handler runs as a step of the run with input: (default: the update's input) within timeout_ms:, and its output is the result; without one, the result is result: (references only). At most 50 updates.

An update with a handler and options (see update/3).

A reference to the input of the update being handled, or to key in it.

Waits for the signal signal:, at most timeout_ms: (600000).

Types

t()

@type t() :: %JoobQ.Workflow{
  changes: term(),
  continue_as_new: term(),
  count: term(),
  depth: term(),
  in_scope: term(),
  leaves: term(),
  name: term(),
  names: term(),
  nodes: term(),
  queries: term(),
  task_queue: term(),
  timeout_ms: term(),
  updates: term(),
  version: term()
}

Functions

change(wf, id, opts \\ [])

@spec change(t(), String.t(), keyword()) :: t()

Declares a change id the workflow's code went through: runs started from now on get version max: (1), runs already past a gated node keep theirs; min: (0) is the oldest version still supported. Declare it before the nodes gated on it.

child(wf, name, workflow, opts \\ [])

@spec child(t(), String.t(), String.t(), keyword()) :: t()

Runs the workflow workflow as a child and waits for it. input: is references only (in maps and lists): the engine resolves it, no handler runs. A child has no retry: or timeout_ms:: give its own steps those.

compensation(handler, opts \\ [])

@spec compensation((term(), JoobQ.Job.t() -> term()), keyword()) ::
  {:compensation, function(), term()}

Undoes a node when the run fails after it completed: handler runs with the input: spec resolved (default: the node's output).

continue_as_new(wf, opts \\ [])

@spec continue_as_new(t(), keyword()) :: t()

Ends the run as continued_as_new and starts a fresh one when it completes (and only_if: holds), with input: (references only, default: the same input), at most max_runs: (1..1000000) runs in the chain. max_events: (10..10000000) continues earlier, mid-run: its references may only read the input.

gate(change, opts)

@spec gate(String.t(), keyword()) :: {:gate, map()}

Runs a node only for runs whose version of the change change is within min:..max: (one is required).

input(key \\ nil)

@spec input(term()) :: JoobQ.Workflow.Ref.t()

A reference to the run's input, or to key in it (a name, a dotted path or a list index).

new(name, opts \\ [])

@spec new(String.t(), keyword()) :: t()

A workflow named name. Options: version: (1), task_queue: ("workflows") and timeout_ms: (the run's deadline, none by default).

output(step, key \\ nil)

@spec output(String.t(), term()) :: JoobQ.Workflow.Ref.t()

A reference to the output of the earlier step step, or to key in it.

parallel(wf, fun)

@spec parallel(t(), (t() -> t())) :: t()

A group whose nodes run at the same time; fun adds them to the workflow it is given and returns it. It is named parallel_<n>, n being its position in the enclosing list.

query(wf, name, ref)

@spec query(t(), String.t(), JoobQ.Workflow.Ref.t()) :: t()

A query name answered from the run's state: ref (a reference to the input, an earlier step, a signal or an update). Not state or steps, at most 100.

ref(path)

@spec ref(String.t()) :: JoobQ.Workflow.Ref.t()

A reference given as a path, such as signals.approve or ${updates.rename}.

retry(opts \\ [])

@spec retry(keyword()) :: {:retry, map()}

A step's retry policy, applied by the engine: attempts: (3, 1..100), initial_ms: (1000), backoff: (2.0, 1..10), maximum_ms: (30000) and non_retryable:, the exceptions (modules) or error class names not retried.

scope(wf, id, opts \\ [], fun)

@spec scope(t(), String.t(), keyword(), (t() -> t())) :: t()

A cancellation scope id around the nodes fun adds: timeout_ms: cancels them when it passes, non_cancellable: true shields them from the run's cancellation (and takes no timeout_ms:). Scopes do not nest.

sequence(wf, fun)

@spec sequence(t(), (t() -> t())) :: t()

A group whose nodes run one after the other, named sequence_<n> (see parallel/2).

step(wf, name, handler, opts \\ [])

@spec step(t(), String.t(), (term(), JoobQ.Job.t() -> term()), keyword()) :: t()

A step: handler runs with the input: spec resolved (default: the run's input). The engine enforces timeout_ms: (one attempt), heartbeat_ms: and schedule_to_start_ms: (at least 1000 each) and retry: (see retry/1).

Every step, wait, timer and child also takes compensate: (see compensation/2), only_if: (a reference: the node runs only when it is truthy, else it is skipped), after: (earlier steps it also waits for) and gate: (see gate/2).

timer(wf, name, opts)

@spec timer(t(), String.t(), keyword()) :: t()

Waits for_ms: milliseconds.

to_blueprint(wf)

@spec to_blueprint(t()) :: map()

The blueprint the gateway registers, as a map ready for JSON (absent options are left out).

update(wf, name, opts)

@spec update(t(), String.t(), (term(), JoobQ.Job.t() -> term()) | keyword()) :: t()

An update name callers send to a run (JoobQ.Workflows). validate: (a reference, e.g. update_input("street")) must be truthy or the update is rejected with message:, a static text: never put data in it. With a handler, the handler runs as a step of the run with input: (default: the update's input) within timeout_ms:, and its output is the result; without one, the result is result: (references only). At most 50 updates.

update(wf, name, handler, opts)

@spec update(t(), String.t(), (term(), JoobQ.Job.t() -> term()), keyword()) :: t()

An update with a handler and options (see update/3).

update_input(key \\ nil)

@spec update_input(term()) :: JoobQ.Workflow.Ref.t()

A reference to the input of the update being handled, or to key in it.

wait_for(wf, name, opts)

@spec wait_for(t(), String.t(), keyword()) :: t()

Waits for the signal signal:, at most timeout_ms: (600000).