Agents Streaming¶
Agents exchange a continuous flow of tasks, results and events, not just single request/response calls. This page covers the streams an agent reads from and writes to. You can use them to show an agent's work live in a UI, feed it into pipelines, or let agents react to each other while they work.
| Stream | Direction | Section |
|---|---|---|
| Tasks in | Queue, WebSocket, P2P mesh → agent | Input Streams |
| Results out | Agent → exchange, sending agent or Redis | Output Routing |
| Live task stream | Agent → UI or consumer, while the task runs: tokens, tool calls, tool results, progress | Live Task Streams |
| LLM token streaming | Model → agent | Streaming LLM Tokens and Tool Calls |
| Events | Agent ↔ agent over P2P meshes | Event Streaming |
| Conversations | User ↔ agent | Message Streaming |
Input Streams¶
An agent instance started with main(AgentClass) listens on all of these at once:
| Stream | Transport | Default | Used by |
|---|---|---|---|
| Task queue | Redis list with acknowledged delivery (see Fault Tolerance) | REDIS_URL=redis://localhost:6379/0, key REDIS_STREAM=jobs:inbox |
The agent executor's load balancer. This is the production path |
| WebSocket | JSON over WebSocket, synchronous replies | WS_PORT=8765, optional WS_BEARER_TOKEN |
Direct clients, UIs, local testing |
| P2P mesh | NATS, meshes from MESH_LIST |
— | Tasks and events from other agents |
The WebSocket accepts one task ({"task": {...}}) or a batch ({"tasks": [...]}) per message and answers with {"results": [...]} on the same connection. See Testing Agents.
Output Routing¶
Results of queued tasks are sent wherever the task's output_ptr.type says:
output_ptr.type |
Destination | Required fields |
|---|---|---|
exchange |
The task exchange, through the background output reporter (with retries) | exchange_id |
mesh |
The sending agent over NATS (reply) |
internal_task_id, sender_subject_id |
redis |
Any Redis list, with LPUSH |
host, port, output_queue |
| (none) | Not routed. WebSocket tasks get their results on the connection | — |
Every result has the same body:
{ "task_id": "...", "job_output": {...}, "job_output_metadata": {...},
"job_output_trace_data": {...}, "is_error": false, "error_data": {...} }
A Redis-type output_ptr is the simplest way to send an agent's results into your own pipeline:
"output_ptr": { "type": "redis", "host": "redis.my-app", "port": 6379, "output_queue": "agent-results" }
Live Task Streams¶
While a task runs, an agent can publish what it is doing (tokens, tool calls, tool results, progress and messages) so a UI or another service can follow along. Get a stream for the task from the context:
def on_data(self, task):
stream = self.context.stream(task)
stream.progress(10, "searching sources")
hits = self.search(task.job_data["question"])
stream.tool_call("search", {"q": task.job_data["question"]}, call_id="c1")
stream.tool_result("c1", {"hits": len(hits)}, name="search")
# forward LLM tokens and tool calls as they are generated
final = stream.relay(self.llm.stream_chat_completions(messages=self.build_messages(hits)))
stream.close(status="ok")
return AgentResult(task_id=task.task_id, job_output={"answer": final["content"]})
| Method | Emits | Data |
|---|---|---|
token(text) |
token |
{"text"} |
tool_call(name, arguments, call_id=None) |
tool_call |
{"id", "name", "arguments"} |
tool_result(call_id, result, name=None, is_error=False) |
tool_result |
{"id", "name", "result", "is_error"} |
progress(percent=None, message="", **extra) |
progress |
{"percent", "message", ...} |
message(text, **extra) |
message |
{"text", ...} |
emit(type, data) |
any custom type | your data |
relay(llm_events, forward_deltas=False) |
token / tool_call (and tool_call_delta if asked) |
Returns the final {"content", "tool_calls", "finish_reason"} |
close(**summary) |
end |
your summary; later calls are ignored |
Every event has the same envelope, with seq increasing per task so consumers can order and de-duplicate:
{ "task_id": "t1", "subject_id": "research-agent", "seq": 7, "ts": 1731744000.12,
"type": "token", "data": { "text": "The " } }
Choosing Where Events Go¶
The caller chooses the destination by adding a stream entry to the task's output_ptr:
output_ptr.stream |
Delivery |
|---|---|
{"type": "redis", "host": "...", "port": 6379, "channel": "ui-42"} |
Redis PUBLISH, for live subscribers |
{"type": "redis", "host": "...", "port": 6379, "list": "events-42", "ttl": 3600} |
Redis RPUSH, kept for consumers that read later (optional TTL in seconds) |
{"type": "mesh", "subject_id": "ui-agent"} |
P2P event task_stream to that agent, which receives it in on_event |
channel and list can be combined. If the task has no stream target and the instance has STREAM_REDIS_URL set, events are published to the channel <STREAM_CHANNEL_PREFIX>:<task_id> (prefix default agent-stream). Otherwise the stream methods do nothing, so agent code can call them unconditionally. stream.enabled tells you whether a target is set.
Consuming a Stream¶
import json, redis
r = redis.Redis(host="redis.my-app")
sub = r.pubsub()
sub.subscribe("ui-42")
for msg in sub.listen():
if msg["type"] != "message":
continue
ev = json.loads(msg["data"])
if ev["type"] == "token":
print(ev["data"]["text"], end="", flush=True)
elif ev["type"] == "end":
break
Streaming LLM Tokens and Tool Calls¶
The LLM interface (agents_llm) streams chat completions from any registered backend:
from agents_llm import AIGridClient
from agents_llm.custom import OpenAIBlockInferenceSystem
client = AIGridClient()
client.add_custom_block(name="openai", system=OpenAIBlockInferenceSystem(model="gpt-4o-mini"))
for ev in client.stream_chat_completions(
name="openai",
messages=[{"role": "user", "content": "What's the weather in Paris?"}],
data={"tools": [WEATHER_TOOL]}):
if ev.type == "token":
print(ev.data["text"], end="")
elif ev.type == "tool_call":
print("\ncall", ev.data["name"], ev.data["arguments"])
elif ev.type == "done":
final = ev.data # {"finish_reason", "content", "tool_calls"}
| Event | Data | When |
|---|---|---|
token |
{"text"} |
Each piece of generated text |
tool_call_delta |
{"index", "id", "name", "arguments"} (partial JSON) |
Each fragment of a tool call, as it is generated |
tool_call |
{"index", "id", "name", "arguments"} (complete JSON) |
Once per tool call, after generation completes |
done |
{"finish_reason", "content", "tool_calls"} |
Last event |
The OpenAI backend (and any OpenAI-compatible endpoint set through base_url) streams natively. Other backends return their complete answer as one token event followed by done, so one loop works with every model. A custom backend can stream natively by implementing stream_chat_completions(messages=..., data=...) and yielding agents_llm.streaming.StreamEvent objects.
To forward a model stream to a live task stream, pass it to stream.relay(...), as shown above. To report tool results as you execute the calls, use stream.tool_result(...).
Event Streaming¶
Besides tasks, agents can publish and receive events over their P2P meshes, for example progress notices, state changes or presence:
# from agent code, through the P2P manager on the context
self.context.p2p_manager.broadcast_event_sync("progress", {"task_id": tid, "percent": 40})
self.context.p2p_manager.send_event_sync("handoff_ready", {"task_id": tid}, subject_id="reviewer-agent")
# receiving side
class ReviewerAgent:
def on_event(self, event_name, event_data):
...
See Chat and P2P Communication for meshes, events and presence.
Streaming Task Progress from Platform Services¶
| Source | Stream |
|---|---|
| Delegation, voting and verification | NATS events. See Task Delegation, Social Choice & Voting |
| Verification status | WebSocket updates. See Verification System |
| Your own long-running tasks | Live task streams |
Message Streaming¶
For conversational, message-by-message streaming between users and agents, use the Chat Server WebSocket protocol.