JoobQ SDK for Elixir

Copy Markdown View Source

A client for the JoobQ SDK gateway of a self-hosted JoobQUI server: enqueue jobs now, later or at a given time, with deadlines, idempotency keys and fields the gateway seals, one at a time (JoobQ.enqueue/4, enqueue_async/4) or up to 500 in one call (enqueue_many/2), and run them in the application with use JoobQ.Worker and the instance's :queues and :workers options (JoobQ.Config); register job schedules, use JoobQ Pro batches and rate limits, and start and talk to workflow runs, and build workflows whose steps the worker runs (JoobQ.Workflow, the instance's :workflows option). JoobQUI version 0.8.0; an SDK talks to a server of its own version or later.

  • Requirements: Elixir 1.16 or newer; depends on finch, jason and telemetry.
  • Install: from the repository until the first Hex release, {:joobq_ex, git: "https://github.com/azutoolkit/joobqui.git", sparse: "sdks/elixir", tag: "v0.8.0"} in mix.exs; the Hex package {:joobq_ex, "~> 0.8.0"} follows the first release (the name is not published yet).
  • Use: the module JoobQ.
  • Connect: the server's origin and its SDK token, read by the examples from JOOBQ_GATEWAY_URL and JOOBQ_SDK_TOKEN.
  • Documentation: https://www.joobq.io/docs/develop/ (workers, jobs, workflows, the gateway contract).

Workers

children = [
  {JoobQ,
   url: "https://joobqui.example.com",
   token: System.fetch_env!("JOOBQ_SDK_TOKEN"),
   queues: ["mail"],
   workers: [MyApp.Welcome],
   middleware: [MyApp.Timing]}
]
  • Jobs: a module that does use JoobQ.Worker, queue: "mail" and implements perform/1 (JoobQ.Worker); it answers :ok, {:ok, output} or {:error, reason}; JoobQ.NonRetryable skips the retries.
  • Context: inside perform/1, what the handler enqueues, batches or starts carries the job's context under its own (JoobQ.Context); a process the handler spawns does not.
  • Middleware (JoobQ.Middleware): call(job, next, opts) around every run, the first listed outermost; the job it gets has no lease token.
  • Stopping: when the instance stops (the application shuts down, on SIGTERM too, or Supervisor.stop/1) the worker stops polling, runs what it holds and sends the outcomes within :drain_timeout (30 s by default), then deregisters, after a registration under way and within 5 s. At most a stop takes :drain_timeout + 1 s, plus a registration under way (a sixth of :lease_ttl_ms + 5 s) and 5 s for a connection, plus 5 s: 52 s with the defaults. A job not settled by then is left to its lease: the gateway queues it again when the lease expires, with one retry used. A process of the worker that crashes restarts alone, without a drain. That is longer than Kubernetes' default termination grace period (30 s): raise terminationGracePeriodSeconds to 60, or lower :drain_timeout (with 8 000 ms and the default lease, a stop takes at most 30 s).
  • Error reports: a failed job sends its error's class, a cleaned message (valid UTF-8, no control characters, at most 4 000 code points; standard exceptions that quote the job's data send their class only) and, for a raise, an exit or a throw, up to 50 stack frames (module, function, arity, file and line, never the arguments). A message you raise yourself is sent as it is: keep secrets out of it.

Schedules

JoobQ.register_schedule("nightly", "mail", "digest", cron: "0 3 * * *") registers a job schedule, or updates it: it is idempotent by its name (:every takes "30 seconds", "5 minutes", ...; see JoobQ.Schedules). It fires on the server; editing, pausing and deleting schedules are done in the console.

Workflows (JoobQ Pro)

The joobq-pro engine conducts every run of a registered blueprint: it retries, waits, compensates and keeps the run's state; a worker only runs steps. JoobQ.Workflow builds and registers workflows, JoobQ.Workflows starts runs and talks to them.

import JoobQ.Workflow

checkout =
  new("checkout", task_queue: "workflows")
  |> step("reserve", &Shop.reserve/2,
    input: %{sku: input("sku")},
    retry: retry(attempts: 3, non_retryable: [Shop.OutOfStock]),
    compensate: compensation(&Shop.release/2))
  |> parallel(fn wf -> wf |> step("charge", &Shop.charge/2) |> step("receipt", &Shop.receipt/2, after: ["charge"]) end)
  |> wait_for("approval", signal: "approve", timeout_ms: 60_000)
  |> query("total", output("charge", "amount"))

children = [{JoobQ, url: url, token: token, workflows: [checkout]}]
  • Steps are jobs: a handler is fn input, job -> ... end (or &Mod.fun/2), answering what perform/1 answers; it runs once per attempt inside the instance's middleware, with the job's context and its [:joobq, :job, *] span (task_kind "activity", "compensation" or "update"). Its input is the input: spec with each reference (input/1, output/2, ref/1) replaced by the value it names, the run's input when there is none. A numeric segment indexes a list (input("items.0")); a negative one is refused, as is ref("steps.<step>") without .output (the engine reads that as the step's status, not its output). The worker sets no timer of its own: the engine enforces timeout_ms and heartbeat_ms (call JoobQ.Job.heartbeat/1 more often than that). A task for a step no workflow in :workflows has fails without retries (JoobQ.UnknownActivity). A step still running when a stop's :drain_timeout ends is left to its lease: the gateway fails that attempt as LeaseExpired, and the step runs again only if it declares retries.
  • Registration: when the worker starts, before it registers itself, it sends each blueprint; a refused one (4xx) is logged with the gateway's code and message and not sent again, one that met a 5xx is sent again with the next registration. The gateway versions a blueprint by its structure and references only: changing a literal or an input mapping of a step (what the handler gets around the references) changes neither its hash nor its version, so runs in flight pick the change up.
  • Building raises ArgumentError naming the workflow, the node and the option, never a value; a step may only read steps declared before it. inspect/1 of a workflow shows its name, version and task queue only. An update's message: is static text stored in the blueprint: never put data in it.
  • Not retried: a raised exception whose module is listed in the step's non_retryable: (as inspect/1 prints it, e.g. Shop.OutOfStock), also when the handler returns {:error, exception} (the error's class is the exception's module), and JoobQ.NonRetryable.
  • :workflows takes %JoobQ.Workflow{} values (what new/2 and the builder return), not modules; each is checked when the instance starts, and one whose blueprint cannot be made is refused with an ArgumentError.

From a producer: JoobQ.start_workflow(name, input, id: "order-1"), signal_workflow/4, update_workflow/4, query_workflow/3, upsert_workflow_attributes/2, and the reads describe_workflow/2, workflow_result/2 and workflow_history/2 (JoobQ.Workflows). Each is sent once; a run id is one percent-encoded path segment and stays out of telemetry.

Batches (JoobQ Pro)

JoobQ.create_batch(jobs, on_complete: {"mail", "report"}, context: %{"tenant" => "t1"}) creates, fills and seals a batch of 1 to 1000 jobs in one call; JoobQ.get_batch(id) reads its progress. The gateway enqueues the callbacks (:on_failure at the first dead member, :on_complete when all are counted, :on_success then too when none is dead), each exactly once; the SDK never does. See JoobQ.Batches for what a member and a callback may carry.

Rate limits (JoobQ Pro)

JoobQ.rate_limit("tenant-1", 10, 60_000) reserves one of 10 slots a minute for the key "tenant-1" on the gateway, in one step: {:ok, %{"allowed" => true}}, or "allowed" => false with "retry_after_ms". It paces your own code and never waits; without a Pro licence it answers {:error, %JoobQ.Error{reason: :pro_required}}.

Telemetry

Spans (:start, then :stop or :exception, with duration in native units; see JoobQ.Telemetry):

EventMetadata:stop adds
[:joobq, :enqueue, *]instance, queue, job_typeresult, status, jid, reason
[:joobq, :schedule, *]instance, schedule, queue, job_typeresult, reason
[:joobq, :rate_limit, *]instance, limit, period_ms (never the key)result, allowed, reason
[:joobq, :batch, *]instance, size (create_batch/2; get_batch/2 has only :request)result, reason, batch_id
[:joobq, :job, *]instance, queue, job_type, jid, attempt, task_kindoutcome, retryable, error_class
[:joobq, :workflow, *]instance, call (:start, :signal, ...), nameresult, reason
[:joobq, :blueprint, *]instance, workflowresult (:registered, :refused, :retry)
[:joobq, :request, *]name, method, pathstatus

A workflow step's job span is a job's: it adds only task_kind ("activity", "compensation", "update"; "job" for a job), and its job_type is the activity name ("<workflow>/<step>").

:exception adds kind and error_class only. None of JoobQ's own events carries a payload, an output, a lease token or the SDK token. Finch's own [:finch, ...] events carry the request (the token in its headers, the body): do not forward those to a sink you do not trust.

Raising variants

enqueue!/4 and enqueue_many!/2 raise the error instead of answering it; the other calls (schedules, rate limits, batches, workflows) answer {:ok, answer} or {:error, %JoobQ.Error{}} only.

Retries and limits

A producer call is sent once and never retried. An error with reason :transport (no answer) or :bad_response (a 2xx answer that is not JSON) MAY mean the job was stored: the gateway can store it before the connection breaks, and it does not deduplicate by jid. Give a job a unique_key to make sending it again safe.

  • :request_timeout (60 000 ms by default) bounds each call, until the whole answer is in.
  • Pooled connections idle for 5 seconds are closed, below the usual proxy keep-alive limits.
  • Finch emits its own [:finch, :request, :*] telemetry events whose metadata carries the request, headers included: do not attach a handler to them that logs the metadata, or the token is logged. Their request also carries the URL, and a worker's own heartbeat call (/sdk/tasks/<lease token>/heartbeat, the gateway's only route for it) has a lease token in it; JoobQ's [:joobq, :request] events name that call by its pattern, /tasks/:token/heartbeat.
  • A worker keeps its lease tokens in a public named ETS table (<instance>.Leases) that any process in the VM can read; inspect/1 of a JoobQ.Job leaves out its lease token, payload and context, but inspect(job, structs: false) and Erlang's ~p (:io_lib.format/2, some crash reports) print them: the language cannot prevent it, so do not log a job that way.
  • The size of an answer is not capped yet (a follow-up); the gateway's own answers are small.
  • :url must be https://; http:// only for a loopback host or with allow_insecure: true (development only).

More about JoobQ: https://www.joobq.io

License

MIT (see LICENSE).