# Stop false consensus in multi-agent swarms

Majority vote is the default way to reconcile a multi-agent run. Three agents answer, two or three agree, and the orchestrator moves on. The trouble is that agents in the same swarm often share a prompt, a retrieved document, or each other's messages. When they agree, they may just be repeating the same mistake. That's **false consensus**: high agreement with little independent evidence behind it.

This post shows how to check for it with [Screwtape.ai](https://screwtape.ai), a judge-agent API with two endpoints:

| Endpoint | You send | You get |
| --- | --- | --- |
| `POST /v1/judge` | `task`, `agent_output` (required), `evidence[]`, `criteria[]`, `retain` (optional) | `verdict` (`pass` | `fail` | `retry`), `score` (0–1), `reasons[]`, `request_id`, `engine` |
| `POST /v1/consensus` | `task`, `answers[]` (min 2, each `{agent_id?, output}`), `retain` (optional) | `consensus_answer`, `agreement_score`, `false_consensus_risk` (`low` | `medium` | `high`), `reasons[]`, `request_id` |

Auth is `Authorization: Bearer <key>`. Free keys come from [screwtape.ai/signup](https://screwtape.ai/signup) and include 100 calls per UTC day. Without a key, each call can be paid with an x402 `PAYMENT-SIGNATURE` header: $0.005 USDC on Base (`eip155:8453`).

## 1\. The curl

This is the request from the OpenAPI `grounded` example, field for field:

```bash
curl -s https://screwtape.ai/v1/judge \
  -H "Authorization: Bearer $SCREWTAPE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"task":"State the crew size and launch site.","agent_output":"The crew size was 4. The mission launched from Cape Canaveral.","evidence":["The mission launched on 2024-03-01 from Cape Canaveral.","The crew size was 4."],"criteria":["crew size","Cape Canaveral"]}'
```

**Real response with no key** (I ran this on 2026-10-20 without an `Authorization` header). The API answers `402` and describes the x402 payment it accepts:

```json
{"error":"payment_required","message":"PAYMENT-SIGNATURE header is required. Send a Bearer API key, or pay per call with x402.","request_id":"97bf7d4e-4930-41eb-88ce-89c9e75dcd65","x402":{"x402Version":2,"error":"PAYMENT-SIGNATURE header is required. Send a Bearer API key, or pay per call with x402.","resource":{"url":"https://screwtape.ai/v1/judge","description":"One Screwtape.ai judge or consensus call","mimeType":"application/json","serviceName":"Screwtape.ai"},"accepts":[{"scheme":"exact","network":"eip155:8453","amount":"5000","asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","payTo":"0x028Bb9A57B8C27c22f6B39dd00e7eFF1FB60344B","maxTimeoutSeconds":300,"extra":{"name":"USD Coin","version":"2","assetTransferMethod":"eip3009"}}],"extensions":{}}}
```

**With a key, the** `200` **response has this shape.** This is the example published on the screwtape.ai homepage for the deterministic engine. It's an example, not a call I ran:

```json
{
  "verdict": "pass",
  "score": 0.9,
  "reasons": [
    "2 of 2 claims are grounded in the supplied evidence.",
    "2 of 2 criteria matched the output.",
    "Deterministic string checks only. No model was called."
  ],
  "engine": "deterministic",
  "retention": { "retained": false },
  "request_id": "8d0c2f1e-6b4a-4e3a-9c1d-2f6a0b7e5d44"
}
```

When the service has a model key configured, `engine` is `llm`, and the default model is `openai/gpt-oss-120b`. The wording of the reasons will differ.

## 2\. Catching copied agreement

```bash
curl -s https://screwtape.ai/v1/consensus \
  -H "Authorization: Bearer $SCREWTAPE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"task":"State the crew size.","answers":[{"agent_id":"alpha","output":"The crew size was 4."},{"agent_id":"beta","output":"The crew size was 4."},{"agent_id":"gamma","output":"The crew size was 4."}]}'
```

Example shape, from the homepage (not executed here):

```json
{
  "consensus_answer": "The crew size was 4.",
  "agreement_score": 1,
  "false_consensus_risk": "high",
  "reasons": [
    "Answers are nearly verbatim. High agreement may be copied rather than independent.",
    "Agreement is token overlap, not a model judgment."
  ],
  "request_id": "..."
}
```

`agreement_score: 1` together with `false_consensus_risk: "high"` is the point. A majority vote would accept this answer immediately. Screwtape tells you the agreement looks copied, so you can bring in an independent check before you trust it.

## 3\. A shared client (stdlib only)

All three framework examples import this file. Note the explicit `User-Agent`: the Cloudflare edge in front of screwtape.ai returned `403` to Python's default `Python-urllib/x.y` agent when I tested.

```python
# screwtape_client.py
"""Minimal Screwtape.ai HTTP client (stdlib only). Spec: https://screwtape.ai/openapi.json"""
import json
import os
import urllib.request

BASE = "https://screwtape.ai"


def _post(path: str, body: dict) -> dict:
    req = urllib.request.Request(
        BASE + path,
        data=json.dumps(body).encode(),
        headers={
            "Authorization": f"Bearer {os.environ['SCREWTAPE_API_KEY']}",
            "Content-Type": "application/json",
            # Cloudflare in front of screwtape.ai returns 403 to the default
            # "Python-urllib/x.y" User-Agent, so send an explicit one.
            "User-Agent": "screwtape-example/1.0",
        },
        method="POST",
    )
    with urllib.request.urlopen(req, timeout=30) as resp:
        return json.load(resp)


def judge(task: str, agent_output: str, evidence=None, criteria=None) -> dict:
    """POST /v1/judge -> {verdict: pass|fail|retry, score, reasons, request_id, ...}"""
    body = {"task": task, "agent_output": agent_output}
    if evidence:
        body["evidence"] = evidence
    if criteria:
        body["criteria"] = criteria
    return _post("/v1/judge", body)


def consensus(task: str, answers: list[dict]) -> dict:
    """POST /v1/consensus. answers: [{"agent_id": "...", "output": "..."}], at least 2.
    -> {consensus_answer, agreement_score, false_consensus_risk: low|medium|high, reasons, request_id}"""
    return _post("/v1/consensus", {"task": task, "answers": answers})
```

## 4\. CrewAI: a task guardrail

CrewAI tasks accept a `guardrail` callable that takes a `TaskOutput` and returns `(bool, Any)`. If it returns `False`, the second value goes back to the agent as feedback, and the task retries up to `guardrail_max_retries` times.

```python
# crewai_guardrail.py
# pip install crewai   (tested against crewai 1.15.x)
# Needs SCREWTAPE_API_KEY plus whatever LLM your crew uses (default: OPENAI_API_KEY).
from crewai import Agent, Crew, Task
from crewai.tasks.task_output import TaskOutput

from screwtape_client import judge

EVIDENCE = [
    "The mission launched on 2024-03-01 from Cape Canaveral.",
    "The crew size was 4.",
]


def screwtape_guardrail(output: TaskOutput) -> tuple[bool, str]:
    result = judge(
        task="State the crew size and launch site.",
        agent_output=output.raw,
        evidence=EVIDENCE,
        criteria=["crew size", "Cape Canaveral"],
    )
    if result["verdict"] == "pass":
        return True, output.raw
    # On fail/retry, CrewAI sends this feedback back to the agent and retries.
    return False, "Screwtape verdict: {}. {}".format(result["verdict"], " ".join(result["reasons"]))


writer = Agent(
    role="Mission summarizer",
    goal="Answer only from the supplied facts.",
    backstory="You summarize mission briefs without adding details.",
)

task = Task(
    description="Facts: " + " ".join(EVIDENCE) + " State the crew size and launch site.",
    expected_output="Two short sentences.",
    agent=writer,
    guardrail=screwtape_guardrail,
    guardrail_max_retries=2,
)

if __name__ == "__main__":
    print(Crew(agents=[writer], tasks=[task]).kickoff().raw)
```

## 5\. LangGraph: route on false-consensus risk

The consensus check is a node, and a conditional edge routes on `false_consensus_risk`. The agent node is a stub, so this graph runs with only a Screwtape key.

```python
# langgraph_consensus.py
# pip install langgraph   (tested against langgraph 1.2.x)
# Needs only SCREWTAPE_API_KEY. The three "agents" are stubs; swap in your LLM calls.
from typing import TypedDict

from langgraph.graph import END, START, StateGraph

from screwtape_client import consensus

TASK = "State the crew size."


class State(TypedDict, total=False):
    answers: list[dict]
    verdict: dict


def agents(state: State) -> State:
    # Replace with real agent calls. Identical answers are what false consensus looks like.
    outs = {"alpha": "The crew size was 4.", "beta": "The crew size was 4.", "gamma": "The crew size was 4."}
    return {"answers": [{"agent_id": k, "output": v} for k, v in outs.items()]}


def score(state: State) -> State:
    return {"verdict": consensus(TASK, state["answers"])}


def route(state: State) -> str:
    return "escalate" if state["verdict"]["false_consensus_risk"] == "high" else "accept"


def accept(state: State) -> State:
    print("accepted:", state["verdict"]["consensus_answer"])
    return {}


def escalate(state: State) -> State:
    print("needs independent check:", state["verdict"].get("reasons"))
    return {}


g = StateGraph(State)
g.add_node("agents", agents)
g.add_node("score", score)
g.add_node("accept", accept)
g.add_node("escalate", escalate)
g.add_edge(START, "agents")
g.add_edge("agents", "score")
g.add_conditional_edges("score", route, {"accept": "accept", "escalate": "escalate"})
g.add_edge("accept", END)
g.add_edge("escalate", END)
app = g.compile()

if __name__ == "__main__":
    app.invoke({})
```

## 6\. AutoGen: use Screwtape's MCP tools

Screwtape exposes `judge` and `consensus` over MCP Streamable HTTP at `https://screwtape.ai/mcp`, using the same auth and quota as REST. AutoGen's `mcp_server_tools` turns them into agent tools.

```python
# autogen_mcp.py
# pip install "autogen-agentchat" "autogen-ext[mcp,openai]" "mcp<2"
# (tested with autogen 0.7.5; autogen-ext 0.7.5 fails to import with mcp 2.x, hence the pin)
# Needs SCREWTAPE_API_KEY and OPENAI_API_KEY.
import asyncio
import os

from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient
from autogen_ext.tools.mcp import StreamableHttpServerParams, mcp_server_tools


async def main() -> None:
    server = StreamableHttpServerParams(
        url="https://screwtape.ai/mcp",
        headers={"Authorization": f"Bearer {os.environ['SCREWTAPE_API_KEY']}"},
    )
    tools = await mcp_server_tools(server)  # -> judge, consensus
    agent = AssistantAgent(
        name="reviewer",
        model_client=OpenAIChatCompletionClient(model="gpt-4o-mini"),
        tools=tools,
        system_message=(
            "Before you accept another agent's answer, call the judge tool with the task, "
            "the answer as agent_output, and any evidence. Report the verdict and reasons."
        ),
    )
    await Console(agent.run_stream(task=(
        "Task: State the crew size and launch site. "
        "Answer from worker agent: 'The crew size was 5. The mission launched from Vandenberg.' "
        "Evidence: 'The mission launched on 2024-03-01 from Cape Canaveral.', 'The crew size was 4.'"
    )))


if __name__ == "__main__":
    asyncio.run(main())
```

You can call `tools/list` without a key, so discovery works before you sign up. `tools/call` needs a Bearer key or an x402 payment.

## What it doesn't do

*   It doesn't run locally. It's a hosted API.
    
*   In deterministic mode, agreement is token overlap and grounding is claim-vs-evidence token matching. It isn't semantic reasoning. The `reasons` say so.
    
*   Judged content is kept in memory only, unless you send `retain: true` with an API key. Retained rows expire after 24 hours ([privacy](https://screwtape.ai/privacy)).
    

## Try it

Get a free key (100 calls/day): **https://screwtape.ai/signup**. Then read the [API reference](https://screwtape.ai/docs) or the raw [OpenAPI](https://screwtape.ai/openapi.json).

This article was created with the help of AI by the Screwtape.ai team.