Skip to content

Verification System

Producing a result is only half of the job; the other half is knowing whether the result can be trusted. The Verification System lets an agent submit a result for checking by another agent, a group of agents, or a human, and wait for the verdict before it continues. It is the quality-control step of a multi-agent workflow and one of the main ways to put a human in the loop.

Why it matters

  • Quality control — catch wrong, incomplete, or unsafe results before they are used downstream.
  • Human oversight — route critical results to a person for approval, with live updates over WebSocket.
  • Collective checking — use several verifiers and combine their opinions through Social Choice & Voting.
  • Traceability — verification tasks and their outcomes are stored in MongoDB, giving you a record of what was checked, by whom, and with what result.

How it works

  1. An agent creates a verification task through the REST API, usually for a result produced by a delegated sub-task.
  2. Verifiers (agents or humans) are notified or invited over NATS.
  3. Verifiers submit their results; status changes are broadcast live over WebSocket.
  4. The requesting agent receives the outcome and decides whether to accept the result, retry, or escalate.

Ad-hoc verifications can also be requested directly over NATS. For the architecture, schema, and full API see Verification System — Architecture.


At a Glance

The Verification System coordinates task verification workflows for agents, supporting both single-subject and multi-agent verification. It provides:

  • REST APIs for creating, updating, querying, and deleting verification tasks.
  • WebSocket server for live result updates.
  • NATS integration for ad-hoc verifications and social choice voting.

A typical client workflow is:

  1. Create a verification task via REST.
  2. Notify or invite agents over NATS.
  3. Wait for results via WebSocket or NATS.
  4. Submit verification results back to the system.

Registering Verification System in Subject Specification

Add a SubSystemItem entry to the subject spec so the agent knows how to reach the Verification API and connect to NATS for events.

Data Class Reference

@dataclass
class SubSystemItem:
    sub_system_id: str
    sub_system_type: str = ""
    sub_system_config: Dict[str, Any] = field(default_factory=dict)

Minimal Required Configuration

{
  "sub_system_id": "verification-system-01",
  "sub_system_type": "verification-system",
  "sub_system_config": {
    "api_base_url": "http://verification-api.default.svc.cluster.local:5000/verifications",
    "nats_url": "nats://nats.default.svc.cluster.local:4222",
    "websocket_url": "ws://verification-api.default.svc.cluster.local:8765/ws"
  }
}

Fields

  • api_base_url – REST API base path for all verification endpoints.
  • nats_url – NATS connection for ad-hoc verifications/voting.
  • websocket_url – WebSocket endpoint for real-time verification status.

Import and Setup

import requests
from agent_sdk.verification import VerificationClient, AdhocVerification

Usage Guide

Creating a Verification

client = VerificationClient("http://localhost:5000")
resp = client.create_verification(
    task_id="task_123",
    subject_id="agent_a",
    verification_type="manual"
)
print(resp)

Submitting a Result

resp = requests.post(
    "http://localhost:5000/verifications/<verification_id>/submit-result",
    json={
        "target_subject_id": "agent_1",
        "status": "completed",
        "result_data": {
            "verification_result": {"pass": True},
            "verification_output": {"notes": "All checks passed"}
        }
    }
)
print(resp.json())

Listening for Updates (WebSocket)

import asyncio, websockets

async def listen():
    async with websockets.connect("ws://localhost:8765/ws/agent_a/task_123") as ws:
        while True:
            msg = await ws.recv()
            print("Update:", msg)

asyncio.run(listen())

Ad-hoc Verification via NATS

verifier = AdhocVerification("nats://localhost:4222")
verifier.push(
    subject="agent_a__task_123",
    event_type="start_verification",
    sender_subject_id="agent_b",
    event_data={"input": "data"}
)
verifier.close()