# defworker Spawn a persistent worker — Deft's answer to Node's `worker_threads`. ## Syntax 
[defworker "path.dft"]            # → worker handle
[defworker "path.dft" 4]          # → pool of 4 workers (round-robin)
## How it differs from `deftask` `deftask { body }` snapshots the closure's bytecode and referenced globals into a fresh one-shot runtime — the body must be self-contained (literals, globals, and native stdlib functions only; captured locals arrive as `nil`). `defworker` loads a file into a long-lived child runtime on its own OS thread. The worker file gets its full module graph: user-defined functions, imports, top-level state — everything works, exactly like Node workers. The worker survives between jobs, so state set at load time (or mutated by handlers) persists. ## Marking handlers (`@worker`) A regular `def` marked with the `@worker` decorator becomes a sendable handler. Unmarked defs stay private to the worker file (callable internally, not reachable from the parent). 
# workers/image.dft — top-level code runs at spawn (entry point, state setup)
def config %{ quality 90 }
def resize-bytes {|bytes^string w^number h^number| ... }   # internal helper

def process-image {|bytes^string w^number h^number|
    @worker
    @doc "Resize an image blob on a worker thread."
    [resize-bytes $bytes $w $h]       # full code access
}
## Dispatch 
set w   [defworker "workers/image.dft"]
set fut [worker/send $w "process-image" $bytes 640 480]
set out [await $fut]                  # works in coroutines AND plain scripts
[worker/stop $w]
`worker/send` accepts a worker handle or a pool handle; pools round-robin. The result is a normal Future — `await` it (single or via `await-all`). ## Streaming (`worker/stream`) `worker/stream` is the streaming sibling of `worker/send`: `yield` in the handler pushes values incrementally onto a stream instead of returning one final result. Read them with the same `task/stream-*` family `deftask` { @stream ... }` uses:` 
def generate {|n^number|
    @worker
    set i 0
    while ($i < $n) {
        incr i
        yield $i
    }
}

set s   [worker/stream $w "generate" 100]
set row [task/stream-next $s]
while (not [nil? $row]) {
    # ... process $row as it arrives ...
    set row [task/stream-next $s]
}
# or just: def rows [task/stream-collect $s]
- The handler's `yield` values cross the heap boundary deep-copied, like job results — structured payloads (maps, vectors, binary strings) work. - The handler's final return value is dropped; a handler error ends the stream (earlier yields survive) and the worker keeps running — the same contract as `deftask { @stream ... }`. - Backpressure is end-to-end: the worker blocks on a full result mailbox until the consumer drains the stream. - Stopping a worker mid-stream finishes any open streams — consumers see `nil`/done rather than hanging. - Calling `worker/stream` on a handler that never yields returns an immediately-empty stream. ### Streams implement ISeq A stream is seqable: `map`/`filter`/`take`/`drop`/`reduce`/`for` treat it like a lazy sequence, pulling items incrementally (with the runtime's worker drains pumped while waiting). This composes with pipelines: 
# single item (plain call in pipeline position)
set one $s |> task/stream-next

# whole stream, transformed lazily
set evens $s |> filter {|x| ([mod $x 2] == 0)} |> take 10 |> collect

# or bracket form
set evens [collect [take 10 [filter $s {|x| ([mod $x 2] == 0)}]]]

# or a for loop over the stream directly
for x in $s { ... }
Note: pipelines (`|>`) parse at statement position only, not inside `[...]` — use the bracket forms there. `deftask { @stream ... }` streams are seqable the same way. ## Stdlib functions | Function | Purpose | |-----------|---------| | `[defworker "path.dft"]` | Spawn a worker; returns a handle | | `[defworker "path.dft" n]` | Spawn a pool of n workers; returns a pool handle | | `[worker/send $w "name" args...]` | Dispatch a job; returns a Future | | `[worker/stream $w "name" args...]` | Dispatch a streaming job; `yield` in the handler pushes values onto the returned stream | | `[worker/stop $w]` | Stop accepting, drain queued jobs, join, free the child runtime | | `[worker/jobs $w]` | Pending job count (queued + in-flight) | | `[worker/handlers $w]` | List of `@worker`-marked handler names | ## Value passing Job arguments and results are deep-copied across the heap boundary (structured values: strings, keywords, vectors, sets, maps, tagged maps). There is no serialization — copies are structural, so binary strings (image bytes) cross at memcpy cost. Return data only: functions, futures, and resources degrade to their display strings (as with `deftask`). ## Error semantics A handler throw resolves the job's future with `%Err{:message "..."}` — and the worker keeps running (persistent). Check for the error shape: 
set v [await $fut]
if [is? "result" $v] {           # Ok/Err tagged maps report type "result"
    echo "job failed:" $v~message
}
Unknown handler names are rejected at `worker/send` time with `%Err`. ## Backpressure Each worker has bounded job/result mailboxes (64 slots). `worker/send` blocks when a worker's queue is full — a natural backpressure valve. ## Shutdown `[worker/stop $w]` stops accepting jobs, drains queued ones (their futures resolve normally), joins the worker thread, and frees the child runtime. If the parent runtime is torn down with workers still alive, teardown cascades automatically. Stale handles fail cleanly (`worker/send` on a stopped worker returns `%Err`). ## Notes - `[await $f1 $f2]` (multi-arg form) blocks the runtime and must not be used with worker futures; use single `[await $f]` or `await-all`. - Worker→parent progress reporting: workers share the parent's pub/sub broker, so a worker can `pub/publish` on a per-worker topic and the parent `pub/subscribe`s it. - Job cancellation and timeouts are not yet implemented. ## See Also `deftask`, `await`, `await-all` (stdext/async), `defcoro`, `pub/publish`
← defunion
deref →