Verification System¶
Producing a result is only half of the job; the other half is knowing whether the result can be trusted. The Verification System lets an agent submit a result for checking by another agent, a group of agents, or a human, and wait for the verdict before it continues. It is the quality-control step of a multi-agent workflow and one of the main ways to put a human in the loop.
Why it matters¶
- Quality control — catch wrong, incomplete, or unsafe results before they are used downstream.
- Human oversight — route critical results to a person for approval, with live updates over WebSocket.
- Collective checking — use several verifiers and combine their opinions through Social Choice & Voting.
- Traceability — verification tasks and their outcomes are stored in MongoDB, giving you a record of what was checked, by whom, and with what result.
How it works¶
- An agent creates a verification task through the REST API, usually for a result produced by a delegated sub-task.
- Verifiers (agents or humans) are notified or invited over NATS.
- Verifiers submit their results; status changes are broadcast live over WebSocket.
- The requesting agent receives the outcome and decides whether to accept the result, retry, or escalate.
Ad-hoc verifications can also be requested directly over NATS. For the architecture, schema, and full API see Verification System — Architecture.
At a Glance¶
The Verification System coordinates task verification workflows for agents, supporting both single-subject and multi-agent verification. It provides:
- REST APIs for creating, updating, querying, and deleting verification tasks.
- WebSocket server for live result updates.
- NATS integration for ad-hoc verifications and social choice voting.
A typical client workflow is:
- Create a verification task via REST.
- Notify or invite agents over NATS.
- Wait for results via WebSocket or NATS.
- Submit verification results back to the system.
Registering Verification System in Subject Specification¶
Add a SubSystemItem entry to the subject spec so the agent knows how to reach the Verification API and connect to NATS for events.
Data Class Reference¶
@dataclass
class SubSystemItem:
sub_system_id: str
sub_system_type: str = ""
sub_system_config: Dict[str, Any] = field(default_factory=dict)
Minimal Required Configuration¶
{
"sub_system_id": "verification-system-01",
"sub_system_type": "verification-system",
"sub_system_config": {
"api_base_url": "http://verification-api.default.svc.cluster.local:5000/verifications",
"nats_url": "nats://nats.default.svc.cluster.local:4222",
"websocket_url": "ws://verification-api.default.svc.cluster.local:8765/ws"
}
}
Fields
- api_base_url – REST API base path for all verification endpoints.
- nats_url – NATS connection for ad-hoc verifications/voting.
- websocket_url – WebSocket endpoint for real-time verification status.
Import and Setup¶
import requests
from agent_sdk.verification import VerificationClient, AdhocVerification
Usage Guide¶
Creating a Verification¶
client = VerificationClient("http://localhost:5000")
resp = client.create_verification(
task_id="task_123",
subject_id="agent_a",
verification_type="manual"
)
print(resp)
Submitting a Result¶
resp = requests.post(
"http://localhost:5000/verifications/<verification_id>/submit-result",
json={
"target_subject_id": "agent_1",
"status": "completed",
"result_data": {
"verification_result": {"pass": True},
"verification_output": {"notes": "All checks passed"}
}
}
)
print(resp.json())
Listening for Updates (WebSocket)¶
import asyncio, websockets
async def listen():
async with websockets.connect("ws://localhost:8765/ws/agent_a/task_123") as ws:
while True:
msg = await ws.recv()
print("Update:", msg)
asyncio.run(listen())
Ad-hoc Verification via NATS¶
verifier = AdhocVerification("nats://localhost:4222")
verifier.push(
subject="agent_a__task_123",
event_type="start_verification",
sender_subject_id="agent_b",
event_data={"input": "data"}
)
verifier.close()