> ## Documentation Index
> Fetch the complete documentation index at: https://motiadev-add-real-system-tutorial-round-2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Queue
> Queue module for async job processing with named queues, retries, and dead-letter support.
A module for asynchronous job processing. It supports two modes: **topic-based queues** (register a consumer per topic, emit events) and **named queues** (enqueue function calls via `TriggerAction.Enqueue`, no trigger registration).
```
modules::queue::QueueModule
```
For step-by-step instructions on configuring and using queues, see [Use Queues](../how-to/use-queues). For DLQ management, see [Manage Failed Triggers](../how-to/dead-letter-queues).
## Queue Modes
### Topic-based queues
Register a consumer for a topic and emit events to it.
1. **Register a consumer** with `registerTrigger({ type: 'queue', function_id: 'my::handler', config: { topic: 'order.created' } })`. This subscribes the handler to that topic.
2. **Emit events** by calling `trigger({ function_id: 'enqueue', payload: { topic: 'order.created', data: payload } })` or `trigger({ function_id: 'enqueue', payload: { topic, data }, action: TriggerAction.Void() })` for fire-and-forget. The `enqueue` function routes the payload to all subscribers of that topic.
3. **Action on the trigger**: the handler receives the `data` as its input. Optional `queue_config` on the trigger controls per-subscriber retries and concurrency.
The producer knows the topic name; consumers register to receive it. Queue settings can live at the trigger registration site.
### Named queues
Define queues in `iii-config.yaml`, then enqueue function calls directly. No trigger registration needed.
1. **Define queues** in `queue_configs` (see [Configuration](#configuration)).
2. **Enqueue a function call** with `trigger({ function_id: 'orders::process', payload, action: TriggerAction.Enqueue({ queue: 'payment' }) })`. The engine routes the job to the named queue and invokes the function when a worker consumes it.
3. **Action on the trigger**: the target function receives `payload` as its input. Retries, concurrency, and FIFO are configured centrally in `iii-config.yaml`.
The producer targets the function and queue explicitly. Queue configuration is centralized. For a hands-on walkthrough, see [Use Queues](../how-to/use-queues).
### When to use which
| | Topic-based | Named queues |
| ------------ | --------------------------------------------------------------------- | ----------------------------------------------------------------------------------- |
| **Producer** | Calls `trigger({ function_id: 'enqueue', payload: { topic, data } })` | Calls `trigger({ function_id, payload, action: TriggerAction.Enqueue({ queue }) })` |
| **Consumer** | Registers `registerTrigger({ type: 'queue', config: { topic } })` | No registration — function is the target |
| **Config** | Optional `queue_config` on trigger | `queue_configs` in `iii-config.yaml` |
| **Use case** | Pub/sub, multiple subscribers per topic | Direct function invocation with retries, FIFO, DLQ |
Both modes are valid. Named queues offer config-driven retries, concurrency, and FIFO ordering.
Named queues use the `Enqueue` trigger action. For a full comparison of synchronous, Void, and Enqueue invocation modes, see [Trigger Actions](../how-to/trigger-actions).
## Sample Configuration
```yaml theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
- class: modules::queue::QueueModule
config:
queue_configs:
default:
max_retries: 5
concurrency: 5
type: standard
payment:
max_retries: 10
concurrency: 2
type: fifo
message_group_field: transaction_id
adapter:
class: modules::queue::BuiltinQueueAdapter
config:
store_method: file_based
file_path: ./data/queue_store
```
## Configuration
A map of named queue configurations. Each key is the queue name referenced in `TriggerAction.Enqueue({ queue: 'name' })`. Define a queue named `default` in config for the common case; reference it as `TriggerAction.Enqueue({ queue: 'default' })`.
The transport adapter for queue persistence and distribution. Defaults to `modules::queue::BuiltinQueueAdapter` when not specified.
## Queue Configuration
Each entry in `queue_configs` defines an independent named queue with its own retry, concurrency, and ordering settings.
Maximum delivery attempts before routing the job to the dead-letter queue. Defaults to `3`.
Maximum number of jobs processed simultaneously from this queue. Defaults to `10`. For FIFO queues, the engine overrides this to `prefetch=1` to guarantee ordering — see the note below.
Delivery mode: `standard` (concurrent, default) or `fifo` (ordered within a message group).
Required when `type` is `fifo`. The JSON field in the job payload whose value determines the ordering group. Jobs with the same group value are processed strictly in order. The field must be present and non-null in every enqueued payload.
Base retry backoff in milliseconds. Applied with exponential scaling: `backoff_ms × 2^(attempt − 1)`. Defaults to `1000`.
Worker poll interval in milliseconds. Defaults to `100`.
When `type` is `fifo`, the engine sets `prefetch=1` regardless of the `concurrency` value. This ensures only one job is in-flight at a time, which is required for strict ordering. FIFO queues also retry failed jobs inline (blocking the queue) rather than in parallel.
## Adapters
### modules::queue::BuiltinQueueAdapter
Built-in in-process queue. No external dependencies. Suitable for single-instance deployments — messages are not shared across engine instances.
```yaml theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
class: modules::queue::BuiltinQueueAdapter
config:
store_method: file_based # in_memory | file_based
file_path: ./data/queue_store # required when store_method is file_based
```
Persistence strategy: `in_memory` (lost on restart) or `file_based` (durable across restarts). Defaults to `in_memory`.
Path to the queue store directory. Required when `store_method` is `file_based`.
### modules::queue::RedisAdapter
Uses Redis as the queue backend for topic-based pub/sub. Enables message distribution across multiple engine instances.
The Redis adapter supports publishing to named queues but does not implement named queue consumption, retries, or dead-letter queues. It is suitable for topic-based pub/sub only. For full named queue support in multi-instance deployments, use the [RabbitMQ adapter](#modulesqueuerabbitmqadapter).
```yaml theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
class: modules::queue::RedisAdapter
config:
redis_url: ${REDIS_URL:redis://localhost:6379}
```
The URL of the Redis instance to connect to.
### modules::queue::RabbitMQAdapter
Uses RabbitMQ as the queue backend. Supports durable delivery, retries, and dead-letter queues across multiple engine instances.
The engine owns consumer loops, retry acknowledgement, and backoff logic — RabbitMQ is used as a transport only. Retry uses explicit ack + republish to a retry exchange with an `x-attempt` header, keeping compatibility with both classic and quorum queues.
```yaml theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
class: modules::queue::RabbitMQAdapter
config:
amqp_url: ${RABBITMQ_URL:amqp://localhost:5672}
```
The AMQP URL of the RabbitMQ instance to connect to.
#### Queue naming in RabbitMQ
For each named queue defined in `queue_configs`, iii creates the following RabbitMQ resources:
| Resource | Format | Example (`payment`) |
| -------------- | ------------------------------------- | -------------------------------------- |
| Main exchange | `iii.__fn_queue::` | `iii.__fn_queue::payment` |
| Main queue | `iii.__fn_queue::.queue` | `iii.__fn_queue::payment.queue` |
| Retry exchange | `iii.__fn_queue::::retry` | `iii.__fn_queue::payment::retry` |
| Retry queue | `iii.__fn_queue::::retry.queue` | `iii.__fn_queue::payment::retry.queue` |
| DLQ exchange | `iii.__fn_queue::::dlq` | `iii.__fn_queue::payment::dlq` |
| DLQ queue | `iii.__fn_queue::::dlq.queue` | `iii.__fn_queue::payment::dlq.queue` |
Each named queue creates six RabbitMQ objects to support delayed retry and dead-lettering. For the design rationale, see [Queue Architecture](./module-queue#queue-flow).
## Adapter Comparison
| | BuiltinQueueAdapter | RabbitMQAdapter | RedisAdapter |
| --------------------------- | ------------------- | --------------- | ----------------- |
| **Retries** | Yes | Yes | No |
| **Dead-letter queue** | Yes | Yes | No |
| **FIFO ordering** | Yes | Yes | No |
| **Named queue consumption** | Yes | Yes | No (publish only) |
| **Topic-based pub/sub** | Yes | Yes | Yes |
| **Multi-instance** | No | Yes | Yes |
| **External dependency** | None | RabbitMQ | Redis |
For practical guidance on which adapter to use in different deployment scenarios, see [Use Queues — Choosing an Adapter](../how-to/use-queues#choosing-an-adapter).
## Builtin Functions
The queue module registers the following functions automatically when it initializes. These are callable via `trigger()` from any SDK or via the `iii trigger` CLI command.
### enqueue
Publishes a message to a topic-based queue.
| Field | Type | Description |
| ------- | -------- | --------------------------------------------- |
| `topic` | `string` | The topic to publish to (required, non-empty) |
| `data` | `any` | The payload to deliver to subscribers |
Returns `null` on success.
### iii::queue::redrive
Moves all messages from a named queue's dead-letter queue back to the main queue. Each message gets its attempt counter reset to zero.
**Input:**
| Field | Type | Description |
| ------- | -------- | ------------------------------------------------------------------ |
| `queue` | `string` | The named queue whose DLQ should be redriven (required, non-empty) |
**Output:**
| Field | Type | Description |
| ---------- | -------- | --------------------------------------------------- |
| `queue` | `string` | The queue name that was redriven |
| `redriven` | `number` | The number of messages moved back to the main queue |
**CLI example:**
```bash theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
iii trigger \
--function-id='iii::queue::redrive' \
--payload='{"queue": "payment"}'
```
For a complete guide on inspecting DLQ messages before redriving, see [Manage Failed Triggers](../how-to/dead-letter-queues).
## Queue Flow
```mermaid theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}}
sequenceDiagram
participant C as Caller
participant E as Engine
participant Q as Queue Adapter
participant F as Target Function
C->>E: trigger({ function_id, payload, action: Enqueue({ queue }) })
E->>E: Validate queue config & FIFO field
E->>Q: Enqueue job
Q-->>E: Job accepted
E-->>C: { messageReceiptId }
loop Consumer
Q->>E: Job available
E->>F: Invoke with payload
alt Success
F-->>E: Return value
E->>Q: Ack
else Failure — attempt < max_retries
F-->>E: Error
E->>Q: Ack + republish with backoff delay
else Failure — attempts exhausted
F-->>E: Error
E->>Q: Route to DLQ
end
end
```