# PyAgent > PyAgent: a spec-driven architecture for orchestrating multi-agent systems — YAML blueprints, 18 orchestration patterns, three-tier memory, and OTel tracing PyAgent is a spec-driven architecture for orchestrating multi-agent systems — 18 named design patterns (also published as structured JSON at /patterns.json), four independent architecture pillars (Blueprint, Execution & Routing, Context & Memory, Observability), every package's full capability surface as structured JSON at /capabilities.json, and a Cookbook of 37 complete, runnable multi-agent orchestration examples across all 21 domains. # Docs # Adapters A `RuntimeAdapter` decides **how** a blueprint runs. The same manifest compiles against five structurally different adapters today, unmodified — see [Why Blueprint?](https://pyagent.org/why-blueprint/index.md) for the case for declaring **what** a system does separately from **how** it runs. ``` api_version: pyagent/v1 metadata: name: customer-support version: 1.0.0 providers: primary: model: gpt-4.1-mini agents: classifier: prompt: "Classify into: billing, tech, general" provider: primary billing: prompt: "Handle billing inquiries" provider: primary tech: prompt: "Handle technical support" provider: primary workflows: support: pattern: supervisor agents: classifier: classifier routes: billing: billing tech: tech ``` ``` pyagent-blueprint validate customer-support.yaml pyagent-blueprint adapters # list every registered adapter pyagent-blueprint test customer-support.yaml # contract conformance vs. MockLLM, no live API calls ``` Choosing *which* adapter compiles and runs a workflow is a Python-API decision today (the CLI's `compile` command uses the bundled native runtime by default) — pick an adapter class from `AdapterRegistry.discover()` and call its `compile()`/`run()` directly: ``` from pyagent_blueprint.adapter import AdapterRegistry from pyagent_blueprint.ir import BlueprintIR from pyagent_blueprint.loader import load_blueprint spec = load_blueprint("customer-support.yaml") ir = BlueprintIR.from_spec(spec) for name in ("langgraph", "crewai", "pyagent"): adapter_cls = AdapterRegistry.discover()[name] adapter = adapter_cls() artifact = adapter.compile(ir) result = await adapter.run(artifact, workflow="support", input_="I was charged twice") ``` That's not a hypothetical — it's what the `RuntimeAdapter` conformance suite actually certifies. Every adapter below implements the same contract (`compile(ir) -> CompiledArtifact`, an always-async `run()`) and is tested against the same `AdapterConformanceSuite`, which checks compile/run correctness, diagnostic completeness (governance features are honored or reported via a stable diagnostic code — never silently dropped), and pattern-intent preservation: | Adapter | Execution model | What actually runs | | --------------------------------------------------------------------- | -------------------------------- | ------------------------------------------------------------------------ | | `pyagent` (native) | Full 18-pattern registry | `pyagent_patterns.orchestration.Supervisor` etc. | | `langgraph` | Declared node/edge graph | Real `StateGraph(...).add_node(...).add_edge(...)`, compiled and invoked | | `openai_agents` | Handoff/turn-based | Real `Agent` + `Runner.run()` | | `crewai` | Role-based crew | Real `Agent`/`Task`/`Crew.kickoff_async()` | | `semantic_kernel` | Event/service-oriented | Real `Kernel` + `ChatCompletionAgent.get_response()` | | `single_agent` / `sequential_chain` / `state_machine` / `simple_loop` | Zero-dependency reference shapes | Pure stdlib, ships in core | ## Governance diagnostics Budgets, SLAs, memory tiers, guardrails, recovery policies, and human-in-the-loop checkpoints are either honored by the adapter or surfaced as a stable `CompileDiagnostic` code (e.g. `BUDGET_UNSUPPORTED`, `MEMORY_TIER_UNSUPPORTED`, `CHECKPOINT_UNSUPPORTED`) — so you always know, deterministically, what a given runtime supports, rather than a feature silently doing nothing. ## See also - [Why Blueprint?](https://pyagent.org/why-blueprint/index.md) — the case for declaring what a system does separately from how it runs - [pyagent-blueprint API Reference](https://pyagent.org/api/blueprint/index.md) — `AdapterRegistry`, `RuntimeAdapter`, `CompiledArtifact` - [pyagent-blueprint vs. LangGraph](https://pyagent.org/compare/vs-langgraph/index.md) - [pyagent-blueprint vs. CrewAI](https://pyagent.org/compare/vs-crewai/index.md) # API & Hooks Bibliography **An at-a-glance map of every public class, method, hook, and protocol across all pyagent-\* packages.** This is the companion overview to the per-package API Reference pages in this section — use it as a quick-lookup cheat sheet, then open a package page (e.g. [pyagent-patterns](https://pyagent.org/api/patterns/base/index.md), [pyagent-trace](https://pyagent.org/api/trace/index.md)) for the full generated signatures and docstrings, or follow a **source** link below to read the real file. It also collects cross-cutting references you won't find on a single package page: the [Hook Event Reference](#hook-event-reference) and the [Cross-Package Integration Matrix](#cross-package-integration-matrix). Every row on this page is checked against the real `packages/*/src/` source, not hand-maintained from memory — if this page and the source ever disagree, the source wins and this page has drifted. ## pyagent-blueprint [source ↗](https://github.com/pyagent-core/pyagent/blob/main/packages/pyagent-blueprint/src/pyagent_blueprint/) Declarative YAML specs — validate, compile, test, diff, render, scaffold, and package. Compiles onto five real runtime engines via the [adapter layer](#runtime-adapters) — see [Adapters](https://pyagent.org/api/adapters/index.md) for the full comparison table and code. ### Core Classes | Class | Constructor | Key Methods | | -------------------- | ---------------------- | --------------------------------------------------------------------------- | | `BlueprintValidator` | `BlueprintValidator()` | `.validate(spec) → list[ValidationIssue]` | | `BlueprintRenderer` | `BlueprintRenderer()` | `.to_mermaid(spec)`, `.to_markdown(spec)` | | `BlueprintDiffer` | `BlueprintDiffer()` | `.diff(old, new) → list[Change]`, `.summary(changes)` | | `BlueprintTester` | `BlueprintTester()` | `.test(spec) → list[TestResult]`, `.summary(results)` | | `BlueprintGenerator` | `BlueprintGenerator()` | `.generate(pattern, agents, name?, version?, description?, adapter?) → str` | `BlueprintCompiler` is a deprecated shim `BlueprintCompiler().compile(spec)` still works but raises a `DeprecationWarning` — it now delegates internally to `PyAgentAdapter`. Use the [Runtime Adapters](#runtime-adapters) API (`AdapterRegistry.discover()["pyagent"]().compile(ir)`) directly in new code. ### Runtime Adapters The subsystem behind [Adapters](https://pyagent.org/api/adapters/index.md) — one `BlueprintIR` compiles unmodified onto any registered adapter. | Class / Function | Description | | ---------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | | `RuntimeAdapter` | Abstract base every adapter implements: `compile(ir) → CompiledArtifact`, always-async `run()` | | `AdapterRegistry` | `.discover() → dict[str, type[RuntimeAdapter]]`, `.get(name)` | | `AdapterResult` | Result of `adapter.run(...)` | | `CompiledArtifact` | What `adapter.compile(ir)` returns — the runtime-native graph/crew/agent | | `CompileDiagnostic` | Stable diagnostic code emitted when a governance feature isn't supported (e.g. `BUDGET_UNSUPPORTED`) | | `Capability` | `Flag` enum describing what an adapter supports | | `UnknownWorkflowError` | Raised when `run(workflow=...)` names a workflow the IR doesn't define | | `render_adapter_template` / `write_adapter_template` | Scaffold a starter third-party `RuntimeAdapter` package | ### RuntimeGraph | Method | Description | | ------------------------------- | ------------------------------- | | `graph.run(workflow, input)` | Execute a named workflow | | `graph.stream(workflow, input)` | Stream workflow execution | | `graph.workflow_names` | List of compiled workflow names | | `graph.agents` | Dict of all compiled agents | | `graph.describe()` | Returns metadata dict | ### RuntimeGraph Hook Wiring | Method | Targets | Hook | | ----------------------------------- | --------------------- | ----------------- | | `graph.wire_trace(bus)` | All patterns + agents | `_trace_bus` | | `graph.wire_context(ledger)` | All agents | `_context_ledger` | | `graph.wire_compressor(compressor)` | All agents | `_compressor` | | `graph.wire_cost_tracker(tracker)` | All agents | `_cost_tracker` | ### Schema Models (Pydantic) | Model | Key Fields | | --------------------- | ------------------------------------------------------------------------------------------------------ | | `BlueprintSpec` | `api_version`, `metadata`, `providers`, `agents`, `workflows`, `contracts`, `observability`, `context` | | `MetadataSpec` | Blueprint name, version, description | | `AgentSpec` | `prompt`, `provider`, `guardrails` | | `WorkflowSpec` | `pattern`, `agents` | | `ProviderBindingSpec` | Named provider → model binding used by `providers:` | | `ContractSpec` | `input`, `output`, `sla` | | `ObservabilitySpec` | `tracing`, `cost_budget` | | `ContextConfigSpec` | `memory`, `compression`, `redaction` | | `PackageSpec` | Fields consumed by `package_blueprint()` when building an Agent Unit archive | ### Packaging | Function / Class | Description | | ------------------------ | ---------------------------------------------------------------- | | `package_blueprint(...)` | Builds a distributable "Agent Unit" archive from a compiled spec | | `AgentUnitMetadata` | Metadata embedded in a packaged Agent Unit | | `PackagingError` | Raised on a packaging failure | ### Loader Functions | Function | Description | | ------------------------------------------- | ---------------------------- | | `load_blueprint(path)` | Load from a YAML file path | | `load_blueprint_from_str(text, fmt="yaml")` | Load from a YAML/JSON string | ### CLI ``` pyagent-blueprint validate blueprint.yaml pyagent-blueprint compile blueprint.yaml pyagent-blueprint render blueprint.yaml --format mermaid pyagent-blueprint test blueprint.yaml pyagent-blueprint diff v1.yaml v2.yaml pyagent-blueprint generate --pattern supervisor --agents a,b,c pyagent-blueprint package blueprint.yaml # build an Agent Unit archive pyagent-blueprint adapters # list every registered RuntimeAdapter pyagent-blueprint adapter-template my-adapter # scaffold a starter RuntimeAdapter package ``` [→ Full pyagent-blueprint API reference](https://pyagent.org/api/blueprint/index.md) ## pyagent-patterns [source ↗](https://github.com/pyagent-core/pyagent/blob/main/packages/pyagent-patterns/src/pyagent_patterns/) Core orchestration framework — agents, patterns, guardrails, recovery, and composition. ### Base Classes | Class | Key Methods / Attributes | Description | | ------------- | -------------------------------------------------------------------------------------------------------- | ----------------------------------------- | | `Message` | `.user(content)`, `.system(content)`, `.assistant(content)`, `.role`, `.content` | Typed message with role enum | | `Agent` | `.run(messages: list[Message]) → Message`, `.name`, `.system_prompt`, `.description` | LLM-backed callable agent | | `Pattern` | `.run(input, context?)`, `.stream(input, context?)`, `._execute(ctx: Context) → Result`, `.pattern_type` | Base class for all orchestration patterns | | `Context` | `.task`, `.metadata`, `.parent_id`, `.messages`, `.id`, `.child(task?)` | Shared execution context | | `Result` | `.output`, `.messages`, `.metadata`, `.duration_seconds`, `.token_estimate`, `.cost_estimate` | Pattern execution result | | `MockLLM` | `MockLLM(responses, delay?)` | Deterministic test double | | `LLMCallable` | `async __call__(messages) -> str` | Protocol any LLM adapter must satisfy | ### Agent Hook Methods | Method | Returns | Hook Attribute | Description | | ---------------------------------- | ------- | ----------------- | ------------------------------------------------------- | | `agent.set_trace_bus(bus)` | `self` | `_trace_bus` | Wire TraceEventBus for `agent_start`/`agent_end` events | | `agent.set_context(ledger)` | `self` | `_context_ledger` | Wire ContextLedger for read/write per call | | `agent.set_compressor(compressor)` | `self` | `_compressor` | Wire MessageCompressor for output compression | | `agent.set_cost_tracker(tracker)` | `self` | `_cost_tracker` | Wire CostTracker for token/cost recording | ### Pattern Hook Methods | Method | Returns | Hook Attribute | Description | | ---------------------------- | ------- | -------------- | ----------------------------------------------------------- | | `pattern.set_trace_bus(bus)` | `self` | `_trace_bus` | Wire TraceEventBus for `pattern_start`/`pattern_end` events | ### Registry & Streaming | Function / Class | Description | | ------------------------------- | --------------------------------------------------------------------------- | | `register_pattern(name, cls)` | Register a pattern class under a lookup name | | `get_pattern_class(name)` | Resolve a registered pattern class by name (used by the Blueprint compiler) | | `list_patterns()` | List every registered pattern name | | `stream_pattern(pattern, task)` | Async-iterate a pattern's execution as it runs | | `StreamChunk` | One chunk yielded by `stream_pattern` / `pattern.stream()` | ### Orchestration Patterns (Tier 1) | Pattern | Constructor Params | Metadata Keys | | --------------------- | ------------------------------------------------ | -------------------------------------- | | `Pipeline` | `stages: list[Agent]` | `stages`, `stage_names` | | `Supervisor` | `classifier, routes, formatter?, default_route?` | `route_key`, `classifier_output` | | `FanOutFanIn` | `agents: list[Agent], aggregator: Agent` | `parallel_agents`, `agent_names` | | `Hierarchical` | `manager: Agent, teams: list[Team]` | `teams`, `total_workers`, `team_names` | | `OrchestratorWorkers` | `orchestrator: Agent, workers: list[Agent]` | `assignments`, `workers_used` | ### Resolution Patterns (Tier 2) | Pattern | Constructor Params | Metadata Keys | | -------------------- | -------------------------------------------------------------- | ------------------------------------------- | | `SelfReflection` | `agent, critic?, max_rounds?, stop_phrase?` | `rounds`, `max_rounds`, `early_stop` | | `CrossReflection` | `generator, reviewer, max_rounds?, stop_phrase?` | `rounds`, `generator`, `reviewer` | | `Debate` | `debaters, judge, rounds?, positions?` | `rounds`, `positions`, `debate_log` | | `Voting` | `voters, strategy?, weights?, normalize?` | `strategy`, `votes`, `tally`, `winner` | | `EvaluatorOptimizer` | `generator, evaluator, criteria, max_rounds?, pass_threshold?` | `rounds`, `scores`, `final_score`, `passed` | ### Structural Patterns (Tier 3) | Pattern | Constructor Params | Metadata Keys | | ------------ | -------------------------------------------------------- | ------------------------------------------------ | | `RoleBased` | `agents, rounds?, shared_context?` | `rounds`, `roles`, `shared_context` | | `Layered` | `layers: list[Layer]` | `layer_count`, `layer_names`, `agents_per_layer` | | `Topology` | `agents, topology, hub_index?, rounds?` | `topology` | | `Blackboard` | `agents: list[BlackboardAgent], rounds?, initial_state?` | `rounds`, `final_state` | ### Advanced Patterns (Tier 4) | Pattern | Constructor Params | Metadata Keys | | ---------------- | ------------------------------------------------------ | ----------------------------------------- | | `TalkerReasoner` | `talker, reasoner, classifier?, complexity_threshold?` | `system`, `escalated` | | `Swarm` | `agents, rounds?, neighbor_count?, aggregation?` | `agents`, `rounds`, `final_states` | | `HumanInTheLoop` | `agent, review_fn, max_revisions?` | `approved`, `revisions`, `human_modified` | | `ReAct` | `agent, tools: dict, max_steps?, finish_token?` | `steps`, `tools_used`, `trace` | ### Supporting Types Types referenced by the constructors above, not full patterns on their own: `Team` (Hierarchical) · `Layer` (Layered) · `TopologyType` enum: `CHAIN`, `STAR`, `MESH` (Topology) · `VotingStrategy` enum: `MAJORITY`, `WEIGHTED` (Voting) · `CircuitState` enum: `CLOSED`, `OPEN`, `HALF_OPEN` (CircuitBreaker) · `BlackboardAgent` / `BlackboardState` / `BlackboardEntry` (Blackboard) · `Quality` / `Latency` / `Constraints` / `Recommendation` (PatternAdvisor) ### Guardrails | Class | Constructor | Key Method | | ---------------- | ------------------------------------------- | ---------------------------------------------------------- | | `GuardrailChain` | `GuardrailChain(guards)` | `.check(content) → GuardrailResult` | | `PIIGuard` | `PIIGuard(redact=True)` | `.check(content)` — detects email, phone, SSN, credit card | | `LengthGuard` | `LengthGuard(max_chars, truncate?)` | `.check(content)` — enforces length | | `ContentGuard` | `ContentGuard(deny_words?, deny_patterns?)` | `.check(content)` — blocks forbidden content | ### Recovery | Class | Constructor | Key Method | | ------------------ | ----------------------------------------------------------------------------------- | ------------------------------------------------------ | | `BoundedExecution` | `BoundedExecution(pattern, fallback?, max_retries?, timeout_seconds?, max_tokens?)` | `.run(input)` — retry → fallback → degrade | | `CircuitBreaker` | `CircuitBreaker(failure_threshold, reset_timeout_seconds, fallback_result?)` | `.execute(pattern, input)` — CLOSED → OPEN → HALF_OPEN | ### Composition & Advisor | Class | Constructor | Key Method | | ------------------ | ----------------------- | ------------------------------------------------ | | `CompositePattern` | `CompositePattern(...)` | Nest patterns as stages/agents in other patterns | | `PatternAdvisor` | `PatternAdvisor()` | `.recommend(task, constraints) → Recommendation` | [→ Full pyagent-patterns API reference](https://pyagent.org/api/patterns/base/index.md) ## pyagent-router [source ↗](https://github.com/pyagent-core/pyagent/blob/main/packages/pyagent-router/src/pyagent_router/) Difficulty-aware model selection, cost estimation, and routing middleware. | Class | Constructor | Key Methods | | ------------------ | ------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | | `ModelSelector` | `ModelSelector()` | `.select(task, required_capability?) → SelectionResult` | | `DifficultyScorer` | `DifficultyScorer(custom_signals?)` | `.score(task) → DifficultyScore` | | `CostEstimator` | `CostEstimator(pricing?)` | `.compare(text, models?) → list[CostEstimate]`, `.estimate_from_text(model, text)` | | `RouterMiddleware` | `RouterMiddleware(model_registry, required_capability?, selector?)` | `.wrap(agent) → RoutedAgent`, `.wrap_all(agents)` | | `Capability` | Enum | `CODE`, `MATH`, `REASONING`, `CREATIVE`, `GENERAL`, `VISION` | ### Return Types | Type | Description | | ----------------------------------- | ---------------------------------------------------------------- | | `SelectionResult` | What `ModelSelector.select()` returns — chosen model + reasoning | | `DifficultyScore` | What `.score()` returns — `.is_easy` / `.is_medium` / `.is_hard` | | `CostEstimate` | One model's cost estimate from `.compare()` | | `ModelSpec` / `DEFAULT_MODEL_SPECS` | Built-in model catalog `ModelSelector` scores against | | `RoutedAgent` | What `RouterMiddleware.wrap()` returns — exposes `.routing_log` | [→ Full pyagent-router API reference](https://pyagent.org/api/router/index.md) ## pyagent-providers [source ↗](https://github.com/pyagent-core/pyagent/blob/main/packages/pyagent-providers/src/pyagent_providers/) Multi-provider abstraction, registry, routing, fallback, capability negotiation, and cost optimization. | Class | Constructor | Key Methods | | ---------------------- | ----------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | | `ProviderProtocol` | (Protocol) | `.complete(messages, model?) → str`, `.__call__(messages) → str` | | `ProviderRegistry` | `ProviderRegistry()` | `.register(name, provider)`, `.get(name)`, `.list_providers() → list[ProviderInfo]` | | `ProviderRouter` | `ProviderRouter(registry, strategy)` | `.route(messages, required?) → (provider, model)` | | `FallbackChain` | `FallbackChain(providers, circuit_breakers?)` | `.complete(messages) → FallbackResult` | | `CapabilityNegotiator` | `CapabilityNegotiator(registry)` | `.negotiate(required?, min_context?, needs_streaming?, needs_tools?, needs_vision?) → NegotiationResult`, `.negotiate_all(...)` | | `CostOptimizer` | `CostOptimizer(registry)` | `.compare(task, healthy_only?, limit?)`, `.cheapest(task, ...)`, `.cheapest_provider(task, ...)` | | `TracedProvider` | `TracedProvider(provider, trace_bus)` | Wraps any provider; emits `provider_call_start/end/error` trace events | | `MockProvider` | `MockProvider(name="mock", responses?, models?, capabilities?, max_context?, health_status?, delay?)` | Test double implementing `ProviderProtocol` | ### Concrete Adapters | Class | Backend | | ------------------- | ------------------------------------------------------------------ | | `AnthropicProvider` | Anthropic API (degrades gracefully if `anthropic` isn't installed) | | `OpenAIProvider` | OpenAI API | | `LiteLLMProvider` | Any LiteLLM-supported backend | ### Routing Strategies | Strategy | Description | | ------------------ | --------------------------------------------- | | `capability_first` | First provider matching required capabilities | | `cost_first` | Cheapest provider | | `latency_first` | Lowest-latency provider | | `round_robin` | Rotate across providers | ### Supporting Types `HealthStatus` enum: `HEALTHY`, `DEGRADED`, `UNHEALTHY` · `ProviderCapabilities` / `ProviderInfo` · `FallbackResult` / `FallbackAttempt` · `NegotiationResult` · `ProviderCostEstimate` [→ Full pyagent-providers API reference](https://pyagent.org/api/providers/index.md) ## pyagent-compress [source ↗](https://github.com/pyagent-core/pyagent/blob/main/packages/pyagent-compress/src/pyagent_compress/) Inter-agent message compression, token budgets, and agent pruning. | Class | Constructor | Key Methods | | -------------------- | ----------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | | `MessageCompressor` | `MessageCompressor(target_ratio, min_sentence_length?, remove_filler?)` | `.compress(text) → CompressionResult` | | `CompressionResult` | (dataclass) | `.compressed`, `.original`, `.original_tokens`, `.compressed_tokens`, `.savings_pct` | | `CompressMiddleware` | `CompressMiddleware(compressor?, budget?, target_ratio?)` | `.wrap(agent) → CompressedAgent`, `.wrap_all(agents)` | | `TokenBudget` | `TokenBudget(workflow_limit, per_agent_limit?, strict?)` | `.consume(agent, tokens)`, `.remaining(agent?)`, `.summary()`, `.register_agent()`, `.total_used`, `.workflow_utilization` | | `AgentPruner` | `AgentPruner(min_contribution, window_size?)` | `.score_agents(messages, task) → list[ContributionScore]`, `.should_prune(scores)` | | `InteractionPruner` | `InteractionPruner(consensus_threshold, min_rounds?)` | `.has_consensus(outputs, current_round) → bool` | ### Supporting Types `BudgetExceededError` (raised in `strict` mode) · `AgentBudget` (per-agent allocation) · `ContributionScore` (return type of `.score_agents()`) · `CompressedAgent` (return type of `.wrap()`) [→ Full pyagent-compress API reference](https://pyagent.org/api/compress/index.md) ## pyagent-context [source ↗](https://github.com/pyagent-core/pyagent/blob/main/packages/pyagent-context/src/pyagent_context/) Structured context with trust metadata, three-tier memory, compression, retrieval, and redaction. | Class | Constructor | Key Methods | | ----------------------- | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `ContextItem` | `ContextItem(content, source, trust_level?, sensitivity?)` | `.content`, `.source`, `.trust_level`, `.sensitivity`, `.token_count`, `.id`, `.is_expired`, `.age_seconds`, `.to_dict()`/`.from_dict()` | | `ContextLedger` | `ContextLedger()` | `.append(item)`, `.add(content, source, trust_level?)`, `.to_messages(budget?)`, `.items`, `.total_tokens`, `.query(...)`, `.snapshot()`, `.from_snapshot()`, `.clear()` | | `WorkingMemory` | `WorkingMemory(max_items, max_tokens)` | `.add(item)`, `.items`, `.token_count`, `.utilization`, `.clear()` | | `SessionMemory` | `SessionMemory(session_id, backend="json", storage_path=".pyagent_sessions")` | `.save(items)`, `.load() → list[ContextItem]` | | `InMemorySemanticStore` | `InMemorySemanticStore()` | `.add(item)`, `.search(query, top_k) → list[ScoredItem]`, `.remove()`, `.clear()` | | `ContextCompressor` | `ContextCompressor(policy, threshold_tokens?, floor_tokens?, preserve_trust?)` | `.compress(items, target_tokens) → list[ContextItem]`, `.should_compress()`, `.policy` | | `TrustAwareRetriever` | `TrustAwareRetriever()` | `.retrieve(items, query, top_k) → list[ScoredItem]` | | `ContextRedactor` | `ContextRedactor(max_sensitivity)` | `.redact_item(item)`, `.redact_ledger(ledger)` | | `ContextLifecycle` | `ContextLifecycle()` | `.sweep_expired(ledger)`, `.apply_freshness_decay(ledger)`, `.consolidate(ledger, threshold)` | ### Enums | Enum | Values | | ------------- | --------------------------------------------------- | | `TrustLevel` | `VERIFIED`, `INFERRED`, `USER_PROVIDED`, `EXTERNAL` | | `Sensitivity` | `PUBLIC`, `INTERNAL`, `CONFIDENTIAL`, `RESTRICTED` | ### Compression Policies | Policy | Description | | ------------------- | -------------------------------------------- | | `none` | No compression | | `fifo` | Drop oldest items first | | `semantic_lossless` | Preserve high-trust, drop redundant | | `sawtooth` | Aggressive then gradual (preserves verified) | ### Supporting Types `SearchResult` / `ScoredItem` (return type of `.search()` / `.retrieve()`) · `SemanticMemoryProtocol` (protocol `InMemorySemanticStore` implements — bring your own vector backend) [→ Full pyagent-context API reference](https://pyagent.org/api/context/index.md) ## pyagent-trace [source ↗](https://github.com/pyagent-core/pyagent/blob/main/packages/pyagent-trace/src/pyagent_trace/) OpenTelemetry spans, TraceEventBus, cost tracking, record/replay, and exporters. ### Core Classes | Class | Constructor | Key Methods | | -------------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `TraceEventBus` | `TraceEventBus()` | `.emit(event)`, `.subscribe(callback)`, `.subscribe_filter(types, callback)`, `.unsubscribe()`, `.emit_async()` | | `TraceEvent` | (dataclass) | `.event_type`, `.agent_name`, `.pattern_type`, `.payload`, `.timestamp` | | `CostTracker` | `CostTracker(event_bus?)` | `.record(pattern, agent, model, input_tokens, output_tokens, cost)`, `.total_cost`, `.total_tokens`, `.by_agent()`, `.by_model()`, `.by_pattern()`, `.summary()` | | `CostEntry` | (dataclass) | `.pattern_name`, `.agent_name`, `.model`, `.input_tokens`, `.output_tokens`, `.cost_usd` | | `Recorder` | `Recorder(event_bus?)` | `.start(pattern)`, `.record_llm_call(...)`, `.end(output)`, `.save(path)`, `.load(path)`, `.entries`, `.llm_calls` | | `PatternSpanEmitter` | `PatternSpanEmitter()` | `.pattern_span(name, attrs?)`, `.agent_span(name, parent?)`, `.set_pattern_result(...)`, `.set_routing_info(...)`, `.set_compression_info(...)`, `.set_error(...)` | | `TraceExporter` | (Protocol) | Interface every exporter below implements: `.export_event(event)` | ### Decorators | Decorator | Usage | Description | | --------------------- | -------------------------------------------------- | ------------------------------------- | | `@traced_pattern` | `@traced_pattern class MyPipeline(Pipeline): pass` | Auto-emit OTel span on every `.run()` | | `traced_agent(agent)` | `agent = traced_agent(Agent(...))` | Wrap agent with OTel span emission | ### Exporters | Exporter | Backend | Key Method | | ------------------ | ------------------------------------ | ------------------------------------- | | `ConsoleExporter` | stdout | `.export_event(event)` | | `JsonlExporter` | JSONL file | `.export_event(event)`, `.flush()` | | `OTelExporter` | Jaeger / Tempo / Datadog / Honeycomb | `.export_event(event)`, `.shutdown()` | | `LangfuseExporter` | Langfuse | `.export_event(event)`, `.flush()` | ### OTel Attributes (`pyagent.*` namespace) `PyAgentAttributes` defines ~20 constants; the ones you'll reach for most: | Attribute | Type | Description | | ------------------------------- | ------ | ------------------------------------------------ | | `pyagent.pattern.type` | string | Pattern name | | `pyagent.pattern.rounds` | int | Rounds executed | | `pyagent.pattern.consensus` | bool | Whether a resolution pattern reached consensus | | `pyagent.pattern.escalated` | bool | Whether TalkerReasoner escalated to the reasoner | | `pyagent.agent.name` | string | Agent name | | `pyagent.agent.role` | string | Agent role (structural patterns) | | `pyagent.router.difficulty` | int | Difficulty 1–10 | | `pyagent.router.selected_model` | string | Routed model | | `pyagent.router.alternatives` | string | Other candidate models considered | | `pyagent.compress.savings_pct` | float | Compression savings 0–1 | | `pyagent.cost.total_usd` | float | Total cost USD | | `pyagent.exec.duration_ms` | float | Duration in ms | | `pyagent.exec.token_estimate` | int | Estimated tokens for the call | [→ Full pyagent-trace API reference](https://pyagent.org/api/trace/index.md) ## pyagent-studio [source ↗](https://github.com/pyagent-core/pyagent/blob/main/packages/pyagent-studio/src/pyagent_studio/) CLI + web control plane for agent blueprints. Installs as the `pyagent` command. ### CLI Commands | Command | Description | | --------------------------------------------------- | ---------------------------------------------------------- | | `pyagent apply ` | Load, validate, summarize | | `pyagent validate ` | Validate a blueprint | | `pyagent test ` | Contract conformance vs. MockLLM | | `pyagent simulate ` | Run a workflow with MockLLM | | `pyagent simulate --live` | Run a workflow with real LLMs | | `pyagent diff ` | Semantic diff between two blueprint versions | | `pyagent render [--format]` | Render to Mermaid or Markdown | | `pyagent generate --pattern

--agents ` | Scaffold a starter blueprint | | `pyagent get ` | Inspect `agents`, `workflows`, `providers`, or `contracts` | | `pyagent describe ` | Print blueprint metadata | | `pyagent providers list` | List registered providers | | `pyagent providers health [--model]` | Provider health check | | `pyagent dashboard` | Launch the web UI | ### Headless Services | Service | Key Methods | | ------------------- | ------------------------------------------------------------------------------------------------------------ | | `BlueprintService` | `.load(path)`, `.validate()`, `.compile()`, `.discover_blueprints()`, `.summary()`, `.spec`/`.graph`/`.path` | | `SimulationService` | `.run(spec, workflow, input)`, `.run_all()` | | `GovernanceService` | `.check_compliance(spec)`, `.format_report(report)`, `.diff()`, `.diff_summary()` | | `TraceService` | `.load(path)`, `.summary()`, `.query()` | ### Web Dashboard Pages | Page | Description | | ---------- | --------------------------------------------------------- | | Overview | Blueprint summary, validation status, quick actions | | Agents | Agent table with prompts, providers, guardrails | | Workflows | Workflow DAGs with Mermaid diagrams | | Simulate | Run workflows with MockLLM or live providers | | Traces | Live SSE trace stream + historical JSONL viewer | | Governance | Compliance score, validation issues, blueprint diff | | Providers | LLM model catalog, health checks | | Diff | Side-by-side semantic diff between two blueprint versions | | Docs | In-app rendering of this documentation site | [→ Full pyagent-studio API reference](https://pyagent.org/api/studio/index.md) ## Hook Event Reference Complete list of trace events emitted by all hooks: | Event Type | Emitter | When | Payload | | --------------------- | -------------------- | ------------------------ | --------------------------------------------------------------------- | | `agent_start` | `Agent` | Before LLM call | `agent_name`, `input` | | `agent_end` | `Agent` | After LLM call + hooks | `agent_name`, `output`, `duration_seconds`, `output_tokens` | | `pattern_start` | `Pattern` | Before `_execute()` | `pattern_type`, `input` | | `pattern_end` | `Pattern` | After `_execute()` | `pattern_type`, `output_length`, `duration_seconds`, `token_estimate` | | `compression` | `Agent` (compressor) | After output compression | `agent_name`, `original_tokens`, `compressed_tokens`, `savings_pct` | | `provider_call_start` | `TracedProvider` | Before provider call | `provider_name`, `model`, `message_count` | | `provider_call_end` | `TracedProvider` | After provider call | `provider_name`, `model`, `duration_seconds`, `output_length` | | `provider_call_error` | `TracedProvider` | On provider error | `provider_name`, `model`, `error` | | `cost_recorded` | `CostTracker` | After cost entry added | `pattern`, `agent`, `model`, `cost_usd`, `tokens` | | `llm_call` | `Recorder` | After recording a call | `agent_name`, `response`, `metadata` | ## Cross-Package Integration Matrix Shows which packages interact and how: | Producer | Consumer | Integration Point | | ------------------- | ------------------ | ---------------------------------------------------- | | `pyagent-patterns` | `pyagent-trace` | `Agent.set_trace_bus()`, `Pattern.set_trace_bus()` | | `pyagent-patterns` | `pyagent-context` | `Agent.set_context()` | | `pyagent-patterns` | `pyagent-compress` | `Agent.set_compressor()` | | `pyagent-patterns` | `pyagent-trace` | `Agent.set_cost_tracker()` | | `pyagent-providers` | `pyagent-trace` | `TracedProvider(provider, trace_bus)` | | `pyagent-router` | `pyagent-patterns` | `RouterMiddleware.wrap(agent)` | | `pyagent-compress` | `pyagent-patterns` | `CompressMiddleware.wrap(agent)` | | `pyagent-blueprint` | `pyagent-patterns` | `RuntimeAdapter.compile()` creates Agents + Patterns | | `pyagent-blueprint` | `pyagent-trace` | `RuntimeGraph.wire_trace()` | | `pyagent-blueprint` | `pyagent-context` | `RuntimeGraph.wire_context()` | | `pyagent-blueprint` | `pyagent-compress` | `RuntimeGraph.wire_compressor()` | | `pyagent-blueprint` | `pyagent-trace` | `RuntimeGraph.wire_cost_tracker()` | | `pyagent-trace` | `pyagent-studio` | `TraceEventBus` feeds Studio dashboard | # pyagent-blueprint API Reference ## `pyagent_blueprint` PyAgent Blueprint — declarative YAML specs for multi-agent LLM systems. ### `AdapterRegistry` Discovers adapters via Python entry points. Core never imports any adapter package directly — third parties can ship a backend without touching this repo at all by registering an entry point in the `pyagent_blueprint.adapters` group. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/adapter.py` ``` class AdapterRegistry: """Discovers adapters via Python entry points. Core never imports any adapter package directly — third parties can ship a backend without touching this repo at all by registering an entry point in the ``pyagent_blueprint.adapters`` group. """ GROUP = "pyagent_blueprint.adapters" @staticmethod def discover() -> dict[str, type[RuntimeAdapter]]: """Return all installed adapters, keyed by entry-point name. An entry point that fails to import (e.g. an adapter registered by `pyproject.toml` whose own optional dependency — like `pyagent-patterns` for the `pyagent` adapter — isn't installed) is skipped rather than raised: one adapter's missing dependency must never break discovery of every OTHER adapter. This is what lets `validate`/`generate` degrade gracefully with zero runtime packages installed, while the zero-dependency reference adapters remain fully discoverable. """ import logging from importlib.metadata import entry_points logger = logging.getLogger(__name__) found: dict[str, type[RuntimeAdapter]] = {} try: eps = entry_points(group=AdapterRegistry.GROUP) except TypeError: # pragma: no cover - py<3.10 signature fallback eps = entry_points().get(AdapterRegistry.GROUP, []) # type: ignore[attr-defined] for ep in eps: try: found[ep.name] = ep.load() except ImportError as exc: logger.debug( "Adapter '%s' could not be loaded (missing dependency?): %s", ep.name, exc ) continue return found @staticmethod def get(name: str) -> type[RuntimeAdapter]: """Look up a single adapter class by entry-point name. Raises: KeyError: If no adapter is registered under `name`. """ adapters = AdapterRegistry.discover() if name not in adapters: raise KeyError(f"No adapter registered as '{name}'. Installed: {sorted(adapters)}") return adapters[name] ``` #### `discover()` Return all installed adapters, keyed by entry-point name. An entry point that fails to import (e.g. an adapter registered by `pyproject.toml` whose own optional dependency — like `pyagent-patterns` for the `pyagent` adapter — isn't installed) is skipped rather than raised: one adapter's missing dependency must never break discovery of every OTHER adapter. This is what lets `validate`/`generate` degrade gracefully with zero runtime packages installed, while the zero-dependency reference adapters remain fully discoverable. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/adapter.py` ``` @staticmethod def discover() -> dict[str, type[RuntimeAdapter]]: """Return all installed adapters, keyed by entry-point name. An entry point that fails to import (e.g. an adapter registered by `pyproject.toml` whose own optional dependency — like `pyagent-patterns` for the `pyagent` adapter — isn't installed) is skipped rather than raised: one adapter's missing dependency must never break discovery of every OTHER adapter. This is what lets `validate`/`generate` degrade gracefully with zero runtime packages installed, while the zero-dependency reference adapters remain fully discoverable. """ import logging from importlib.metadata import entry_points logger = logging.getLogger(__name__) found: dict[str, type[RuntimeAdapter]] = {} try: eps = entry_points(group=AdapterRegistry.GROUP) except TypeError: # pragma: no cover - py<3.10 signature fallback eps = entry_points().get(AdapterRegistry.GROUP, []) # type: ignore[attr-defined] for ep in eps: try: found[ep.name] = ep.load() except ImportError as exc: logger.debug( "Adapter '%s' could not be loaded (missing dependency?): %s", ep.name, exc ) continue return found ``` #### `get(name)` Look up a single adapter class by entry-point name. Raises: | Type | Description | | ---------- | --------------------------------------- | | `KeyError` | If no adapter is registered under name. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/adapter.py` ``` @staticmethod def get(name: str) -> type[RuntimeAdapter]: """Look up a single adapter class by entry-point name. Raises: KeyError: If no adapter is registered under `name`. """ adapters = AdapterRegistry.discover() if name not in adapters: raise KeyError(f"No adapter registered as '{name}'. Installed: {sorted(adapters)}") return adapters[name] ``` ### `AdapterResult` Normalized result envelope. Every adapter must map its native return shape into this, so callers never branch on adapter identity (e.g. never special-case "if adapter is LangGraph, read `.raw['messages'][-1]`"). Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/adapter.py` ``` class AdapterResult: """Normalized result envelope. Every adapter must map its native return shape into this, so callers never branch on adapter identity (e.g. never special-case "if adapter is LangGraph, read `.raw['messages'][-1]`"). """ def __init__(self, output: Any, raw: Any = None, usage: dict[str, Any] | None = None) -> None: self.output = output # the primary answer, always present self.raw = raw # adapter-native object, for advanced users self.usage = usage or {} # tokens/cost if the adapter can report it def __repr__(self) -> str: # pragma: no cover - debugging aid only return f"AdapterResult(output={self.output!r}, usage={self.usage!r})" ``` ### `Capability` Bases: `Flag` Optional features an adapter may or may not support. Core only ever requires COMPILE + RUN (i.e. the two abstract methods below). Everything else is negotiated at runtime via these flags so the contract never assumes a graph, async streaming, or native tool-calling exists. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/adapter.py` ``` class Capability(Flag): """Optional features an adapter may or may not support. Core only ever requires COMPILE + RUN (i.e. the two abstract methods below). Everything else is negotiated at runtime via these flags so the contract never assumes a graph, async streaming, or native tool-calling exists. """ NONE = 0 STREAMING = auto() NATIVE_TOOL_CALLING = auto() SYNC_EXECUTION = auto() # some SDKs are sync-only PARTIAL_WORKFLOW_RUN = auto() # can run a subset of a workflow (debugging) ROUND_TRIP = auto() # can export back to a BlueprintIR losslessly for shared constructs ``` ### `CompiledArtifact` Result of `compile()`: the opaque native handle plus diagnostics. Attributes: | Name | Type | Description | | ------------- | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `handle` | `Any` | Opaque, framework-native compiled object. Core never inspects this — that's the whole point of the abstraction. | | `diagnostics` | `list[CompileDiagnostic]` | Every governance feature the blueprint declared that this adapter could NOT honor, as structured diagnostics. Empty means every declared feature was either honored or not applicable to this blueprint. | | `intent` | `dict[str, str]` | Optional map of workflow name -> original pattern name, preserved so pattern intent survives even when an adapter lowers a named pattern (e.g. "debate") to a generic graph. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/adapter.py` ``` @dataclass class CompiledArtifact: """Result of `compile()`: the opaque native handle plus diagnostics. Attributes: handle: Opaque, framework-native compiled object. Core never inspects this — that's the whole point of the abstraction. diagnostics: Every governance feature the blueprint declared that this adapter could NOT honor, as structured diagnostics. Empty means every declared feature was either honored or not applicable to this blueprint. intent: Optional map of workflow name -> original pattern name, preserved so pattern intent survives even when an adapter lowers a named pattern (e.g. "debate") to a generic graph. """ handle: Any diagnostics: list[CompileDiagnostic] = field(default_factory=list) intent: dict[str, str] = field(default_factory=dict) ``` ### `CompileDiagnostic` A single structured diagnostic emitted during `compile()`. Attributes: | Name | Type | Description | | -------- | ---------------- | ------------------------------------------------------------------------------------------------- | | `code` | `DiagnosticCode` | A stable DiagnosticCode from diagnostics.py. | | `path` | `str` | Dotted path into the blueprint that triggered this diagnostic, e.g. "workflows.support.recovery". | | `detail` | `str` | Adapter-specific human-readable detail. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/adapter.py` ``` @dataclass(frozen=True) class CompileDiagnostic: """A single structured diagnostic emitted during `compile()`. Attributes: code: A stable `DiagnosticCode` from `diagnostics.py`. path: Dotted path into the blueprint that triggered this diagnostic, e.g. ``"workflows.support.recovery"``. detail: Adapter-specific human-readable detail. """ code: DiagnosticCode path: str detail: str = "" ``` ### `RuntimeAdapter` Bases: `ABC` Compiles a framework-agnostic `BlueprintIR` into a runnable object native to a specific agent framework, and executes it. Deliberately minimal: only `compile` and `run` are required. This is the lowest common denominator across graph-based (LangGraph), turn-based (AutoGen), role-based (CrewAI), handoff-based (OpenAI Agents SDK), event-driven (Semantic Kernel), and hand-rolled loop runtimes. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/adapter.py` ``` class RuntimeAdapter(ABC): """Compiles a framework-agnostic `BlueprintIR` into a runnable object native to a specific agent framework, and executes it. Deliberately minimal: only `compile` and `run` are required. This is the lowest common denominator across graph-based (LangGraph), turn-based (AutoGen), role-based (CrewAI), handoff-based (OpenAI Agents SDK), event-driven (Semantic Kernel), and hand-rolled loop runtimes. """ name: str capabilities: Capability = Capability.NONE @abstractmethod def compile(self, ir: BlueprintIR) -> CompiledArtifact: """Compile a `BlueprintIR` into a `CompiledArtifact`. Must report every governance feature it cannot honor via `CompiledArtifact.diagnostics` — never drop one silently. """ @abstractmethod async def run( self, compiled: CompiledArtifact, workflow: str, input_: str, **kwargs: Any ) -> AdapterResult: """Execute a compiled workflow. Adapters that are natively sync (`Capability.SYNC_EXECUTION`) wrap their own sync call internally (e.g. via `asyncio.to_thread`) — callers of `RuntimeAdapter` always await, even for sync-native SDKs. Raises: UnknownWorkflowError: If `workflow` doesn't exist in `compiled`. """ # -- Optional capability-gated methods (default: not implemented) -- async def stream( self, compiled: CompiledArtifact, workflow: str, input_: str, **kwargs: Any ) -> AsyncIterator[Any]: raise NotImplementedError(f"{self.name} does not declare Capability.STREAMING") def supported_patterns(self) -> list[str]: """Pattern/topology vocabulary this adapter understands, for `validator.py`'s optional pattern-existence check. Adapters without a fixed pattern vocabulary (e.g. a loop-based adapter) return an empty list — validator treats that as "no constraint", not an error.""" return [] def export(self, compiled: CompiledArtifact) -> Any: """Export a compiled artifact back toward a portable form. Only meaningful if `Capability.ROUND_TRIP` is declared. Default raises — adapters that support round-tripping (e.g. a future Agent Spec bridge) override this. """ raise NotImplementedError(f"{self.name} does not declare Capability.ROUND_TRIP") ``` #### `compile(ir)` Compile a `BlueprintIR` into a `CompiledArtifact`. Must report every governance feature it cannot honor via `CompiledArtifact.diagnostics` — never drop one silently. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/adapter.py` ``` @abstractmethod def compile(self, ir: BlueprintIR) -> CompiledArtifact: """Compile a `BlueprintIR` into a `CompiledArtifact`. Must report every governance feature it cannot honor via `CompiledArtifact.diagnostics` — never drop one silently. """ ``` #### `run(compiled, workflow, input_, **kwargs)` Execute a compiled workflow. Adapters that are natively sync (`Capability.SYNC_EXECUTION`) wrap their own sync call internally (e.g. via `asyncio.to_thread`) — callers of `RuntimeAdapter` always await, even for sync-native SDKs. Raises: | Type | Description | | ---------------------- | -------------------------------------- | | `UnknownWorkflowError` | If workflow doesn't exist in compiled. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/adapter.py` ``` @abstractmethod async def run( self, compiled: CompiledArtifact, workflow: str, input_: str, **kwargs: Any ) -> AdapterResult: """Execute a compiled workflow. Adapters that are natively sync (`Capability.SYNC_EXECUTION`) wrap their own sync call internally (e.g. via `asyncio.to_thread`) — callers of `RuntimeAdapter` always await, even for sync-native SDKs. Raises: UnknownWorkflowError: If `workflow` doesn't exist in `compiled`. """ ``` #### `supported_patterns()` Pattern/topology vocabulary this adapter understands, for `validator.py`'s optional pattern-existence check. Adapters without a fixed pattern vocabulary (e.g. a loop-based adapter) return an empty list — validator treats that as "no constraint", not an error. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/adapter.py` ``` def supported_patterns(self) -> list[str]: """Pattern/topology vocabulary this adapter understands, for `validator.py`'s optional pattern-existence check. Adapters without a fixed pattern vocabulary (e.g. a loop-based adapter) return an empty list — validator treats that as "no constraint", not an error.""" return [] ``` #### `export(compiled)` Export a compiled artifact back toward a portable form. Only meaningful if `Capability.ROUND_TRIP` is declared. Default raises — adapters that support round-tripping (e.g. a future Agent Spec bridge) override this. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/adapter.py` ``` def export(self, compiled: CompiledArtifact) -> Any: """Export a compiled artifact back toward a portable form. Only meaningful if `Capability.ROUND_TRIP` is declared. Default raises — adapters that support round-tripping (e.g. a future Agent Spec bridge) override this. """ raise NotImplementedError(f"{self.name} does not declare Capability.ROUND_TRIP") ``` ### `UnknownWorkflowError` Bases: `Exception` Raised by `RuntimeAdapter.run()`/`stream()` for an unresolvable workflow name. Adapters MUST raise this (not let an internal AttributeError/KeyError leak through) so callers get one documented, catchable error type regardless of which adapter is installed. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/adapter.py` ``` class UnknownWorkflowError(Exception): """Raised by `RuntimeAdapter.run()`/`stream()` for an unresolvable workflow name. Adapters MUST raise this (not let an internal AttributeError/KeyError leak through) so callers get one documented, catchable error type regardless of which adapter is installed. """ ``` ### `BlueprintCompiler` DEPRECATED: use `PyAgentAdapter` (or any `RuntimeAdapter`) directly. Compile a BlueprintSpec into a RuntimeGraph, delegating to `pyagent_blueprint.adapters.pyagent_adapter.PyAgentAdapter`. Parameters: | Name | Type | Description | Default | | ------------------- | ----- | -------------------------------------------------------------------------------------- | ------- | | `provider_registry` | `Any` | Optional ProviderRegistry for real providers. If None, uses MockLLM for all providers. | `None` | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/compiler.py` ``` class BlueprintCompiler: """DEPRECATED: use `PyAgentAdapter` (or any `RuntimeAdapter`) directly. Compile a BlueprintSpec into a RuntimeGraph, delegating to `pyagent_blueprint.adapters.pyagent_adapter.PyAgentAdapter`. Args: provider_registry: Optional ``ProviderRegistry`` for real providers. If ``None``, uses ``MockLLM`` for all providers. """ def __init__(self, provider_registry: Any = None) -> None: warnings.warn( "BlueprintCompiler is deprecated; use " "pyagent_blueprint.adapters.pyagent_adapter.PyAgentAdapter " "(or any RuntimeAdapter) directly. This shim will be removed " "in a future major version.", DeprecationWarning, stacklevel=2, ) self._provider_registry = provider_registry def compile(self, spec: BlueprintSpec) -> RuntimeGraph: """Compile a blueprint spec into a runnable RuntimeGraph. Args: spec: Validated ``BlueprintSpec``. Returns: ``RuntimeGraph`` ready to execute. Raises: CompilationError: If the spec references unknown patterns or agents. """ from pyagent_blueprint.adapters.pyagent_adapter import ( PyAgentAdapter, PyAgentCompilationError, ) ir = BlueprintIR.from_spec(spec) adapter = PyAgentAdapter(provider_registry=self._provider_registry) try: compiled = adapter.compile(ir) except PyAgentCompilationError as exc: raise CompilationError(str(exc)) from exc return compiled.handle ``` #### `compile(spec)` Compile a blueprint spec into a runnable RuntimeGraph. Parameters: | Name | Type | Description | Default | | ------ | --------------- | ------------------------ | ---------- | | `spec` | `BlueprintSpec` | Validated BlueprintSpec. | *required* | Returns: | Type | Description | | -------------- | ------------------------------ | | `RuntimeGraph` | RuntimeGraph ready to execute. | Raises: | Type | Description | | ------------------ | -------------------------------------------------- | | `CompilationError` | If the spec references unknown patterns or agents. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/compiler.py` ``` def compile(self, spec: BlueprintSpec) -> RuntimeGraph: """Compile a blueprint spec into a runnable RuntimeGraph. Args: spec: Validated ``BlueprintSpec``. Returns: ``RuntimeGraph`` ready to execute. Raises: CompilationError: If the spec references unknown patterns or agents. """ from pyagent_blueprint.adapters.pyagent_adapter import ( PyAgentAdapter, PyAgentCompilationError, ) ir = BlueprintIR.from_spec(spec) adapter = PyAgentAdapter(provider_registry=self._provider_registry) try: compiled = adapter.compile(ir) except PyAgentCompilationError as exc: raise CompilationError(str(exc)) from exc return compiled.handle ``` ### `CompilationError` Bases: `Exception` Raised when a blueprint cannot be compiled. Preserved for backward compatibility — wraps `adapters.pyagent_adapter.PyAgentCompilationError`. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/compiler.py` ``` class CompilationError(Exception): """Raised when a blueprint cannot be compiled. Preserved for backward compatibility — wraps `adapters.pyagent_adapter.PyAgentCompilationError`. """ ``` ### `BlueprintDiffer` Compute semantic diff between two blueprint specs. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/differ.py` ``` class BlueprintDiffer: """Compute semantic diff between two blueprint specs.""" def diff(self, old: BlueprintSpec, new: BlueprintSpec) -> list[Change]: """Diff two blueprint specs. Args: old: Previous version. new: Updated version. Returns: List of ``Change`` objects. """ changes: list[Change] = [] old_dict = old.model_dump() new_dict = new.model_dump() self._diff_dicts(old_dict, new_dict, "", changes) return changes def summary(self, changes: list[Change]) -> str: """Human-readable summary of changes. Args: changes: List from ``diff()``. Returns: Multi-line summary string. """ if not changes: return "No changes detected." lines: list[str] = [] for severity in (ChangeSeverity.BREAKING, ChangeSeverity.WARNING, ChangeSeverity.INFO): filtered = [c for c in changes if c.severity == severity] if filtered: lines.append(f"\n{severity.upper()} ({len(filtered)}):") for c in filtered: lines.append(f" [{c.change_type}] {c.path}") return "\n".join(lines) def _diff_dicts( self, old: dict, new: dict, prefix: str, changes: list[Change], ) -> None: """Recursively diff two dictionaries.""" all_keys = set(old.keys()) | set(new.keys()) for key in sorted(all_keys): path = f"{prefix}.{key}" if prefix else key old_val = old.get(key) new_val = new.get(key) if key not in old: changes.append( Change( path=path, change_type=ChangeType.ADDED, old_value=None, new_value=new_val, severity=self._classify_severity(key, ChangeType.ADDED), ) ) elif key not in new: changes.append( Change( path=path, change_type=ChangeType.REMOVED, old_value=old_val, new_value=None, severity=self._classify_severity(key, ChangeType.REMOVED), ) ) elif isinstance(old_val, dict) and isinstance(new_val, dict): self._diff_dicts(old_val, new_val, path, changes) elif old_val != new_val: changes.append( Change( path=path, change_type=ChangeType.MODIFIED, old_value=old_val, new_value=new_val, severity=self._classify_severity(key, ChangeType.MODIFIED), ) ) @staticmethod def _classify_severity(field_name: str, change_type: ChangeType) -> ChangeSeverity: """Determine severity based on field name and change type.""" if field_name in _BREAKING_FIELDS: return ChangeSeverity.BREAKING if change_type == ChangeType.REMOVED: return ChangeSeverity.WARNING if field_name in _WARNING_FIELDS: return ChangeSeverity.WARNING return ChangeSeverity.INFO ``` #### `diff(old, new)` Diff two blueprint specs. Parameters: | Name | Type | Description | Default | | ----- | --------------- | ----------------- | ---------- | | `old` | `BlueprintSpec` | Previous version. | *required* | | `new` | `BlueprintSpec` | Updated version. | *required* | Returns: | Type | Description | | -------------- | ----------------------- | | `list[Change]` | List of Change objects. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/differ.py` ``` def diff(self, old: BlueprintSpec, new: BlueprintSpec) -> list[Change]: """Diff two blueprint specs. Args: old: Previous version. new: Updated version. Returns: List of ``Change`` objects. """ changes: list[Change] = [] old_dict = old.model_dump() new_dict = new.model_dump() self._diff_dicts(old_dict, new_dict, "", changes) return changes ``` #### `summary(changes)` Human-readable summary of changes. Parameters: | Name | Type | Description | Default | | --------- | -------------- | ----------------- | ---------- | | `changes` | `list[Change]` | List from diff(). | *required* | Returns: | Type | Description | | ----- | -------------------------- | | `str` | Multi-line summary string. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/differ.py` ``` def summary(self, changes: list[Change]) -> str: """Human-readable summary of changes. Args: changes: List from ``diff()``. Returns: Multi-line summary string. """ if not changes: return "No changes detected." lines: list[str] = [] for severity in (ChangeSeverity.BREAKING, ChangeSeverity.WARNING, ChangeSeverity.INFO): filtered = [c for c in changes if c.severity == severity] if filtered: lines.append(f"\n{severity.upper()} ({len(filtered)}):") for c in filtered: lines.append(f" [{c.change_type}] {c.path}") return "\n".join(lines) ``` ### `Change` A single semantic change between two blueprint versions. Attributes: | Name | Type | Description | | ------------- | ---------------- | --------------------------------- | | `path` | `str` | Dotted path to the changed field. | | `change_type` | `ChangeType` | Added, removed, or modified. | | `old_value` | `Any` | Previous value (None for added). | | `new_value` | `Any` | New value (None for removed). | | `severity` | `ChangeSeverity` | Impact level. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/differ.py` ``` @dataclass(frozen=True) class Change: """A single semantic change between two blueprint versions. Attributes: path: Dotted path to the changed field. change_type: Added, removed, or modified. old_value: Previous value (None for added). new_value: New value (None for removed). severity: Impact level. """ path: str change_type: ChangeType old_value: Any new_value: Any severity: ChangeSeverity ``` ### `BlueprintGenerator` Generate scaffold blueprint YAML from a pattern name and agent list. Parameters: | Name | Type | Description | Default | | ------------------------ | ----- | --------------------------------------------- | ---------------- | | `default_provider_model` | `str` | Model to use in the default provider binding. | `'gpt-4.1-mini'` | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/generator.py` ``` class BlueprintGenerator: """Generate scaffold blueprint YAML from a pattern name and agent list. Args: default_provider_model: Model to use in the default provider binding. """ def __init__(self, default_provider_model: str = "gpt-4.1-mini") -> None: self._default_model = default_provider_model def generate( self, pattern: str, agents: list[str], *, name: str = "my-blueprint", version: str = "0.1.0", description: str = "", adapter: str | None = None, ) -> str: """Generate a blueprint YAML string. Args: pattern: Pattern registry name (e.g., ``"supervisor"``, ``"pipeline"``). agents: List of agent names. name: Blueprint name. version: Blueprint version. description: Blueprint description. adapter: Optional adapter entry-point name to validate `pattern` against. If omitted, any installed adapter's vocabulary is used; if none is installed, the pattern name isn't checked at all — scaffolding still works with zero runtime packages installed. Returns: YAML string. Raises: ValueError: If a pattern vocabulary is available (from the resolved adapter, or any installed adapter) and `pattern` isn't in it. """ known, _resolved_adapter = _resolve_pattern_vocabulary(adapter) if known and pattern not in known: raise ValueError(f"Unknown pattern '{pattern}'. Available: {sorted(known)}") spec: dict = { "api_version": "pyagent/v1", "metadata": { "name": name, "version": version, "description": description or f"A {pattern} blueprint", }, "providers": { "primary": {"model": self._default_model}, }, "agents": {}, "workflows": {}, } # Generate agent specs for agent_name in agents: spec["agents"][agent_name] = { "prompt": f"You are the {agent_name} agent. TODO: add your prompt here.", "provider": "primary", } # Generate workflow spec wf_agents = self._wire_pattern_agents(pattern, agents) spec["workflows"]["main"] = { "pattern": pattern, "agents": wf_agents, } return yaml.dump(spec, default_flow_style=False, sort_keys=False) @staticmethod def _wire_pattern_agents(pattern: str, agents: list[str]) -> dict: """Create the agents mapping for a workflow based on pattern type.""" if pattern == "supervisor" and len(agents) >= 2: return { "classifier": agents[0], "routes": {name: name for name in agents[1:]}, } if pattern == "pipeline": return {"stages": {name: name for name in agents}} if pattern in ("fan_out_fan_in", "voting", "debate"): return {"agents": {name: name for name in agents}} if pattern in ("self_reflection", "evaluator_optimizer") and len(agents) >= 2: return { "generator": agents[0], "evaluator": agents[1], } if pattern == "cross_reflection" and len(agents) >= 2: return { "agent_a": agents[0], "agent_b": agents[1], } # Default: pass all agents return {name: name for name in agents} ``` #### `generate(pattern, agents, *, name='my-blueprint', version='0.1.0', description='', adapter=None)` Generate a blueprint YAML string. Parameters: | Name | Type | Description | Default | | ------------- | ----------- | ------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `pattern` | `str` | Pattern registry name (e.g., "supervisor", "pipeline"). | *required* | | `agents` | `list[str]` | List of agent names. | *required* | | `name` | `str` | Blueprint name. | `'my-blueprint'` | | `version` | `str` | Blueprint version. | `'0.1.0'` | | `description` | `str` | Blueprint description. | `''` | | `adapter` | \`str | None\` | Optional adapter entry-point name to validate pattern against. If omitted, any installed adapter's vocabulary is used; if none is installed, the pattern name isn't checked at all — scaffolding still works with zero runtime packages installed. | Returns: | Type | Description | | ----- | ------------ | | `str` | YAML string. | Raises: | Type | Description | | ------------ | ------------------------------------------------------------------------------------------------------------------- | | `ValueError` | If a pattern vocabulary is available (from the resolved adapter, or any installed adapter) and pattern isn't in it. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/generator.py` ``` def generate( self, pattern: str, agents: list[str], *, name: str = "my-blueprint", version: str = "0.1.0", description: str = "", adapter: str | None = None, ) -> str: """Generate a blueprint YAML string. Args: pattern: Pattern registry name (e.g., ``"supervisor"``, ``"pipeline"``). agents: List of agent names. name: Blueprint name. version: Blueprint version. description: Blueprint description. adapter: Optional adapter entry-point name to validate `pattern` against. If omitted, any installed adapter's vocabulary is used; if none is installed, the pattern name isn't checked at all — scaffolding still works with zero runtime packages installed. Returns: YAML string. Raises: ValueError: If a pattern vocabulary is available (from the resolved adapter, or any installed adapter) and `pattern` isn't in it. """ known, _resolved_adapter = _resolve_pattern_vocabulary(adapter) if known and pattern not in known: raise ValueError(f"Unknown pattern '{pattern}'. Available: {sorted(known)}") spec: dict = { "api_version": "pyagent/v1", "metadata": { "name": name, "version": version, "description": description or f"A {pattern} blueprint", }, "providers": { "primary": {"model": self._default_model}, }, "agents": {}, "workflows": {}, } # Generate agent specs for agent_name in agents: spec["agents"][agent_name] = { "prompt": f"You are the {agent_name} agent. TODO: add your prompt here.", "provider": "primary", } # Generate workflow spec wf_agents = self._wire_pattern_agents(pattern, agents) spec["workflows"]["main"] = { "pattern": pattern, "agents": wf_agents, } return yaml.dump(spec, default_flow_style=False, sort_keys=False) ``` ### `BlueprintIR` Root intermediate representation of a complete blueprint. This is the single normalized structure that `RuntimeAdapter.compile()`, `BlueprintDiffer`, `BlueprintRenderer`, and any future exporter (e.g. an Agent Spec backend) should consume — never the raw `BlueprintSpec`. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/ir.py` ``` @dataclass(frozen=True) class BlueprintIR: """Root intermediate representation of a complete blueprint. This is the single normalized structure that `RuntimeAdapter.compile()`, `BlueprintDiffer`, `BlueprintRenderer`, and any future exporter (e.g. an Agent Spec backend) should consume — never the raw `BlueprintSpec`. """ api_version: str name: str version: str description: str = "" tags: tuple[str, ...] = () owner: str = "" providers: dict[str, ProviderBindingIR] = field(default_factory=dict) agents: dict[str, AgentIR] = field(default_factory=dict) workflows: dict[str, WorkflowIR] = field(default_factory=dict) contracts: dict[str, ContractIR] = field(default_factory=dict) memory: MemoryPolicyIR | None = None observability: ObservabilityIR | None = None extensions: dict[str, Any] = field(default_factory=dict) @classmethod def from_spec(cls, spec: BlueprintSpec) -> BlueprintIR: """Build a `BlueprintIR` from a validated `BlueprintSpec`. This is the ONE place that knows how to read the Pydantic schema. Everything downstream (adapters, differ, renderer) should be written against this dataclass shape instead. """ providers = { name: ProviderBindingIR( name=name, model=binding.model, provider=binding.provider, fallback_ref=binding.fallback_ref, ) for name, binding in spec.providers.items() } agents = { name: AgentIR( name=name, prompt=agent_spec.prompt, provider=agent_spec.provider, tools=tuple(agent_spec.tools), description=agent_spec.description, guardrails=tuple(agent_spec.guardrails), ) for name, agent_spec in spec.agents.items() } workflows = { name: WorkflowIR( name=name, pattern=wf.pattern, agents=dict(wf.agents), config=dict(wf.config), recovery=( RecoveryIR( max_retries=wf.recovery.max_retries, timeout_seconds=wf.recovery.timeout_seconds, fallback_provider=wf.recovery.fallback_provider, ) if wf.recovery is not None else None ), guardrails=tuple(wf.guardrails), ) for name, wf in spec.workflows.items() } contracts = { name: ContractIR( workflow=name, input_schema=dict(contract.input), output_schema=dict(contract.output), sla=SLAIR( latency_p95_ms=contract.sla.latency_p95_ms, cost_max_usd=contract.sla.cost_max_usd, quality_min=contract.sla.quality_min, ), ) for name, contract in spec.contracts.items() } memory = None if spec.context is not None: redaction = spec.context.redaction memory = MemoryPolicyIR( working_max_tokens=spec.context.memory.working_max_tokens, session_backend=spec.context.memory.session_backend, semantic_enabled=spec.context.memory.semantic_enabled, compression_policy=spec.context.compression.policy, compression_target_ratio=spec.context.compression.target_ratio, redaction_max_sensitivity=(redaction.max_sensitivity if redaction else None), ) observability = None if spec.observability is not None: budget = spec.observability.cost_budget observability = ObservabilityIR( tracing_enabled=spec.observability.tracing.enabled, tracing_exporter=spec.observability.tracing.exporter, cost_budget_daily_usd=(budget.daily_usd if budget else None), cost_budget_alert_threshold=(budget.alert_threshold if budget else 0.8), ) return cls( api_version=spec.api_version, name=spec.metadata.name, version=spec.metadata.version, description=spec.metadata.description, tags=tuple(spec.metadata.tags), owner=spec.metadata.owner, providers=providers, agents=agents, workflows=workflows, contracts=contracts, memory=memory, observability=observability, extensions={}, ) def governance_features(self) -> dict[str, bool]: """Which governance features this blueprint actually declares. Used by adapters/conformance checks to know what MUST be honored or diagnosed — never silently dropped. Keys map 1:1 to the diagnostic codes in `diagnostics.py`. """ return { "routing": any(p.fallback_ref for p in self.providers.values()), "budget": any(c.sla.cost_max_usd for c in self.contracts.values()) or ( self.observability is not None and self.observability.cost_budget_daily_usd is not None ), "sla": bool(self.contracts), "memory_tier": self.memory is not None, "recovery": any(w.recovery is not None for w in self.workflows.values()), "guardrails": any(a.guardrails for a in self.agents.values()) or any(w.guardrails for w in self.workflows.values()), "checkpoint": any(w.config.get("human_in_the_loop") for w in self.workflows.values()), } ``` #### `from_spec(spec)` Build a `BlueprintIR` from a validated `BlueprintSpec`. This is the ONE place that knows how to read the Pydantic schema. Everything downstream (adapters, differ, renderer) should be written against this dataclass shape instead. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/ir.py` ``` @classmethod def from_spec(cls, spec: BlueprintSpec) -> BlueprintIR: """Build a `BlueprintIR` from a validated `BlueprintSpec`. This is the ONE place that knows how to read the Pydantic schema. Everything downstream (adapters, differ, renderer) should be written against this dataclass shape instead. """ providers = { name: ProviderBindingIR( name=name, model=binding.model, provider=binding.provider, fallback_ref=binding.fallback_ref, ) for name, binding in spec.providers.items() } agents = { name: AgentIR( name=name, prompt=agent_spec.prompt, provider=agent_spec.provider, tools=tuple(agent_spec.tools), description=agent_spec.description, guardrails=tuple(agent_spec.guardrails), ) for name, agent_spec in spec.agents.items() } workflows = { name: WorkflowIR( name=name, pattern=wf.pattern, agents=dict(wf.agents), config=dict(wf.config), recovery=( RecoveryIR( max_retries=wf.recovery.max_retries, timeout_seconds=wf.recovery.timeout_seconds, fallback_provider=wf.recovery.fallback_provider, ) if wf.recovery is not None else None ), guardrails=tuple(wf.guardrails), ) for name, wf in spec.workflows.items() } contracts = { name: ContractIR( workflow=name, input_schema=dict(contract.input), output_schema=dict(contract.output), sla=SLAIR( latency_p95_ms=contract.sla.latency_p95_ms, cost_max_usd=contract.sla.cost_max_usd, quality_min=contract.sla.quality_min, ), ) for name, contract in spec.contracts.items() } memory = None if spec.context is not None: redaction = spec.context.redaction memory = MemoryPolicyIR( working_max_tokens=spec.context.memory.working_max_tokens, session_backend=spec.context.memory.session_backend, semantic_enabled=spec.context.memory.semantic_enabled, compression_policy=spec.context.compression.policy, compression_target_ratio=spec.context.compression.target_ratio, redaction_max_sensitivity=(redaction.max_sensitivity if redaction else None), ) observability = None if spec.observability is not None: budget = spec.observability.cost_budget observability = ObservabilityIR( tracing_enabled=spec.observability.tracing.enabled, tracing_exporter=spec.observability.tracing.exporter, cost_budget_daily_usd=(budget.daily_usd if budget else None), cost_budget_alert_threshold=(budget.alert_threshold if budget else 0.8), ) return cls( api_version=spec.api_version, name=spec.metadata.name, version=spec.metadata.version, description=spec.metadata.description, tags=tuple(spec.metadata.tags), owner=spec.metadata.owner, providers=providers, agents=agents, workflows=workflows, contracts=contracts, memory=memory, observability=observability, extensions={}, ) ``` #### `governance_features()` Which governance features this blueprint actually declares. Used by adapters/conformance checks to know what MUST be honored or diagnosed — never silently dropped. Keys map 1:1 to the diagnostic codes in `diagnostics.py`. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/ir.py` ``` def governance_features(self) -> dict[str, bool]: """Which governance features this blueprint actually declares. Used by adapters/conformance checks to know what MUST be honored or diagnosed — never silently dropped. Keys map 1:1 to the diagnostic codes in `diagnostics.py`. """ return { "routing": any(p.fallback_ref for p in self.providers.values()), "budget": any(c.sla.cost_max_usd for c in self.contracts.values()) or ( self.observability is not None and self.observability.cost_budget_daily_usd is not None ), "sla": bool(self.contracts), "memory_tier": self.memory is not None, "recovery": any(w.recovery is not None for w in self.workflows.values()), "guardrails": any(a.guardrails for a in self.agents.values()) or any(w.guardrails for w in self.workflows.values()), "checkpoint": any(w.config.get("human_in_the_loop") for w in self.workflows.values()), } ``` ### `BlueprintLoadError` Bases: `Exception` Raised when a blueprint cannot be loaded or validated. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/loader.py` ``` class BlueprintLoadError(Exception): """Raised when a blueprint cannot be loaded or validated.""" ``` ### `AgentUnitMetadata` Resolved, packaging-ready metadata for an Agent Unit. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/packaging.py` ``` @dataclass(frozen=True) class AgentUnitMetadata: """Resolved, packaging-ready metadata for an Agent Unit.""" name: str version: str author: str runtime: str dependencies: tuple[str, ...] = field(default_factory=tuple) spec_sha256: str = "" def to_dict(self) -> dict[str, object]: return { "name": self.name, "version": self.version, "author": self.author, "runtime": self.runtime, "dependencies": list(self.dependencies), "spec_sha256": self.spec_sha256, "unit_schema_version": "1.0", } ``` ### `PackagingError` Bases: `Exception` Raised when a blueprint cannot be packaged into an Agent Unit. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/packaging.py` ``` class PackagingError(Exception): """Raised when a blueprint cannot be packaged into an Agent Unit.""" ``` ### `BlueprintRenderer` Render a BlueprintSpec as Mermaid diagrams or Markdown docs. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/renderer.py` ```` class BlueprintRenderer: """Render a BlueprintSpec as Mermaid diagrams or Markdown docs.""" def to_mermaid(self, spec: BlueprintSpec) -> str: """Render the blueprint as a Mermaid flowchart. Args: spec: Blueprint specification. Returns: Mermaid diagram string. """ lines = ["graph TD"] # Agent nodes for name, agent_spec in spec.agents.items(): label = agent_spec.description or name lines.append(f" {name}[{label}]") # Workflow edges for _, wf_spec in spec.workflows.items(): agent_refs = self._extract_agent_refs(wf_spec.agents) if len(agent_refs) > 1: for i in range(len(agent_refs) - 1): lines.append(f" {agent_refs[i]} -->|{wf_spec.pattern}| {agent_refs[i + 1]}") # For supervisor-like patterns, show classifier → routes if "classifier" in wf_spec.agents and "routes" in wf_spec.agents: classifier = wf_spec.agents["classifier"] routes = wf_spec.agents.get("routes", {}) if isinstance(routes, dict): for route_name, route_ref in routes.items(): lines.append(f" {classifier} -->|{route_name}| {route_ref}") return "\n".join(lines) def to_markdown(self, spec: BlueprintSpec) -> str: """Render the blueprint as Markdown documentation. Args: spec: Blueprint specification. Returns: Markdown string. """ sections: list[str] = [] # Title sections.append(f"# {spec.metadata.name}") if spec.metadata.description: sections.append(f"\n{spec.metadata.description}") sections.append(f"\n**Version:** {spec.metadata.version}") if spec.metadata.owner: sections.append(f"**Owner:** {spec.metadata.owner}") # Providers if spec.providers: sections.append("\n## Providers\n") for name, p in spec.providers.items(): sections.append(f"- **{name}**: `{p.model}` ({p.provider})") # Agents sections.append("\n## Agents\n") for name, a in spec.agents.items(): desc = a.description or "No description" sections.append(f"### {name}\n") sections.append(f"- **Description:** {desc}") sections.append(f"- **Prompt:** {a.prompt[:100]}{'...' if len(a.prompt) > 100 else ''}") if a.provider: sections.append(f"- **Provider:** {a.provider}") if a.guardrails: sections.append(f"- **Guardrails:** {', '.join(a.guardrails)}") # Workflows sections.append("\n## Workflows\n") for name, w in spec.workflows.items(): sections.append(f"### {name}\n") sections.append(f"- **Pattern:** {w.pattern}") if w.recovery: sections.append( f"- **Recovery:** max_retries={w.recovery.max_retries}, " f"timeout={w.recovery.timeout_seconds}s" ) # Contracts if spec.contracts: sections.append("\n## Contracts\n") for name, c in spec.contracts.items(): sections.append(f"### {name}\n") sections.append( f"- **SLA:** p95 latency ≤ {c.sla.latency_p95_ms}ms, " f"cost ≤ ${c.sla.cost_max_usd}" ) # Diagram sections.append("\n## Architecture Diagram\n") sections.append("```mermaid") sections.append(self.to_mermaid(spec)) sections.append("```") return "\n".join(sections) @staticmethod def _extract_agent_refs(agents: dict) -> list[str]: """Flatten agent refs from a workflow's agents dict.""" refs: list[str] = [] for _, ref in agents.items(): if isinstance(ref, str): refs.append(ref) elif isinstance(ref, dict): refs.extend(ref.values()) return refs ```` #### `to_mermaid(spec)` Render the blueprint as a Mermaid flowchart. Parameters: | Name | Type | Description | Default | | ------ | --------------- | ------------------------ | ---------- | | `spec` | `BlueprintSpec` | Blueprint specification. | *required* | Returns: | Type | Description | | ----- | ----------------------- | | `str` | Mermaid diagram string. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/renderer.py` ``` def to_mermaid(self, spec: BlueprintSpec) -> str: """Render the blueprint as a Mermaid flowchart. Args: spec: Blueprint specification. Returns: Mermaid diagram string. """ lines = ["graph TD"] # Agent nodes for name, agent_spec in spec.agents.items(): label = agent_spec.description or name lines.append(f" {name}[{label}]") # Workflow edges for _, wf_spec in spec.workflows.items(): agent_refs = self._extract_agent_refs(wf_spec.agents) if len(agent_refs) > 1: for i in range(len(agent_refs) - 1): lines.append(f" {agent_refs[i]} -->|{wf_spec.pattern}| {agent_refs[i + 1]}") # For supervisor-like patterns, show classifier → routes if "classifier" in wf_spec.agents and "routes" in wf_spec.agents: classifier = wf_spec.agents["classifier"] routes = wf_spec.agents.get("routes", {}) if isinstance(routes, dict): for route_name, route_ref in routes.items(): lines.append(f" {classifier} -->|{route_name}| {route_ref}") return "\n".join(lines) ``` #### `to_markdown(spec)` Render the blueprint as Markdown documentation. Parameters: | Name | Type | Description | Default | | ------ | --------------- | ------------------------ | ---------- | | `spec` | `BlueprintSpec` | Blueprint specification. | *required* | Returns: | Type | Description | | ----- | ---------------- | | `str` | Markdown string. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/renderer.py` ```` def to_markdown(self, spec: BlueprintSpec) -> str: """Render the blueprint as Markdown documentation. Args: spec: Blueprint specification. Returns: Markdown string. """ sections: list[str] = [] # Title sections.append(f"# {spec.metadata.name}") if spec.metadata.description: sections.append(f"\n{spec.metadata.description}") sections.append(f"\n**Version:** {spec.metadata.version}") if spec.metadata.owner: sections.append(f"**Owner:** {spec.metadata.owner}") # Providers if spec.providers: sections.append("\n## Providers\n") for name, p in spec.providers.items(): sections.append(f"- **{name}**: `{p.model}` ({p.provider})") # Agents sections.append("\n## Agents\n") for name, a in spec.agents.items(): desc = a.description or "No description" sections.append(f"### {name}\n") sections.append(f"- **Description:** {desc}") sections.append(f"- **Prompt:** {a.prompt[:100]}{'...' if len(a.prompt) > 100 else ''}") if a.provider: sections.append(f"- **Provider:** {a.provider}") if a.guardrails: sections.append(f"- **Guardrails:** {', '.join(a.guardrails)}") # Workflows sections.append("\n## Workflows\n") for name, w in spec.workflows.items(): sections.append(f"### {name}\n") sections.append(f"- **Pattern:** {w.pattern}") if w.recovery: sections.append( f"- **Recovery:** max_retries={w.recovery.max_retries}, " f"timeout={w.recovery.timeout_seconds}s" ) # Contracts if spec.contracts: sections.append("\n## Contracts\n") for name, c in spec.contracts.items(): sections.append(f"### {name}\n") sections.append( f"- **SLA:** p95 latency ≤ {c.sla.latency_p95_ms}ms, " f"cost ≤ ${c.sla.cost_max_usd}" ) # Diagram sections.append("\n## Architecture Diagram\n") sections.append("```mermaid") sections.append(self.to_mermaid(spec)) sections.append("```") return "\n".join(sections) ```` ### `RuntimeGraph` Executable graph of compiled workflows. Each workflow is a fully wired `Pattern` instance ready to run. Parameters: | Name | Type | Description | Default | | ----------- | -------------------- | -------------------------------------------- | --------------------------------------- | | `workflows` | `dict[str, Pattern]` | Mapping of workflow name → compiled Pattern. | *required* | | `agents` | \`dict[str, Agent] | None\` | Mapping of agent name → Agent instance. | | `metadata` | \`dict[str, Any] | None\` | Blueprint metadata dict. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/runtime.py` ``` class RuntimeGraph: """Executable graph of compiled workflows. Each workflow is a fully wired ``Pattern`` instance ready to run. Args: workflows: Mapping of workflow name → compiled Pattern. agents: Mapping of agent name → Agent instance. metadata: Blueprint metadata dict. """ def __init__( self, workflows: dict[str, Pattern], agents: dict[str, Agent] | None = None, metadata: dict[str, Any] | None = None, ) -> None: self._workflows = workflows self._agents = agents or {} self._metadata = metadata or {} # -- Hook-wiring convenience methods -- def wire_trace(self, bus: Any) -> None: """Set trace_bus on all patterns and their agents. Args: bus: A ``TraceEventBus`` instance. """ for pattern in self._workflows.values(): pattern.set_trace_bus(bus) for agent in self._agents.values(): agent.set_trace_bus(bus) def wire_context(self, ledger: Any) -> None: """Set context ledger on all agents in all workflows. Args: ledger: A ``ContextLedger`` instance. """ for agent in self._agents.values(): agent.set_context(ledger) def wire_compressor(self, compressor: Any) -> None: """Set compressor on all agents. Args: compressor: A ``MessageCompressor`` instance. """ for agent in self._agents.values(): agent.set_compressor(compressor) def wire_cost_tracker(self, tracker: Any) -> None: """Set cost tracker on all agents. Args: tracker: A ``CostTracker`` instance. """ for agent in self._agents.values(): agent.set_cost_tracker(tracker) # -- Execution -- async def run(self, workflow: str, task: str) -> Result: """Run a workflow by name. Args: workflow: Workflow name from the blueprint. task: Input task string. Returns: Pattern ``Result``. Raises: KeyError: If workflow name doesn't exist. """ if workflow not in self._workflows: available = list(self._workflows.keys()) raise KeyError(f"Unknown workflow '{workflow}'. Available: {available}") pattern = self._workflows[workflow] return await pattern.run(task) async def stream(self, workflow: str, task: str) -> AsyncIterator[str]: """Stream results from a workflow. Falls back to ``run()`` and yields the full output if the pattern doesn't support native streaming. """ result = await self.run(workflow, task) yield result.output def describe(self) -> dict[str, Any]: """Introspect the runtime graph. Returns: Dict with metadata and workflow descriptions. """ return { "metadata": self._metadata, "workflows": { name: { "pattern_type": type(pattern).__name__, } for name, pattern in self._workflows.items() }, "agents": list(self._agents.keys()), } @property def workflow_names(self) -> list[str]: return list(self._workflows.keys()) @property def agents(self) -> dict[str, Agent]: return dict(self._agents) def __contains__(self, workflow: str) -> bool: return workflow in self._workflows ``` #### `wire_trace(bus)` Set trace_bus on all patterns and their agents. Parameters: | Name | Type | Description | Default | | ----- | ----- | ------------------------- | ---------- | | `bus` | `Any` | A TraceEventBus instance. | *required* | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/runtime.py` ``` def wire_trace(self, bus: Any) -> None: """Set trace_bus on all patterns and their agents. Args: bus: A ``TraceEventBus`` instance. """ for pattern in self._workflows.values(): pattern.set_trace_bus(bus) for agent in self._agents.values(): agent.set_trace_bus(bus) ``` #### `wire_context(ledger)` Set context ledger on all agents in all workflows. Parameters: | Name | Type | Description | Default | | -------- | ----- | ------------------------- | ---------- | | `ledger` | `Any` | A ContextLedger instance. | *required* | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/runtime.py` ``` def wire_context(self, ledger: Any) -> None: """Set context ledger on all agents in all workflows. Args: ledger: A ``ContextLedger`` instance. """ for agent in self._agents.values(): agent.set_context(ledger) ``` #### `wire_compressor(compressor)` Set compressor on all agents. Parameters: | Name | Type | Description | Default | | ------------ | ----- | ----------------------------- | ---------- | | `compressor` | `Any` | A MessageCompressor instance. | *required* | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/runtime.py` ``` def wire_compressor(self, compressor: Any) -> None: """Set compressor on all agents. Args: compressor: A ``MessageCompressor`` instance. """ for agent in self._agents.values(): agent.set_compressor(compressor) ``` #### `wire_cost_tracker(tracker)` Set cost tracker on all agents. Parameters: | Name | Type | Description | Default | | --------- | ----- | ----------------------- | ---------- | | `tracker` | `Any` | A CostTracker instance. | *required* | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/runtime.py` ``` def wire_cost_tracker(self, tracker: Any) -> None: """Set cost tracker on all agents. Args: tracker: A ``CostTracker`` instance. """ for agent in self._agents.values(): agent.set_cost_tracker(tracker) ``` #### `run(workflow, task)` Run a workflow by name. Parameters: | Name | Type | Description | Default | | ---------- | ----- | --------------------------------- | ---------- | | `workflow` | `str` | Workflow name from the blueprint. | *required* | | `task` | `str` | Input task string. | *required* | Returns: | Type | Description | | -------- | --------------- | | `Result` | Pattern Result. | Raises: | Type | Description | | ---------- | ------------------------------- | | `KeyError` | If workflow name doesn't exist. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/runtime.py` ``` async def run(self, workflow: str, task: str) -> Result: """Run a workflow by name. Args: workflow: Workflow name from the blueprint. task: Input task string. Returns: Pattern ``Result``. Raises: KeyError: If workflow name doesn't exist. """ if workflow not in self._workflows: available = list(self._workflows.keys()) raise KeyError(f"Unknown workflow '{workflow}'. Available: {available}") pattern = self._workflows[workflow] return await pattern.run(task) ``` #### `stream(workflow, task)` Stream results from a workflow. Falls back to `run()` and yields the full output if the pattern doesn't support native streaming. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/runtime.py` ``` async def stream(self, workflow: str, task: str) -> AsyncIterator[str]: """Stream results from a workflow. Falls back to ``run()`` and yields the full output if the pattern doesn't support native streaming. """ result = await self.run(workflow, task) yield result.output ``` #### `describe()` Introspect the runtime graph. Returns: | Type | Description | | ---------------- | --------------------------------------------- | | `dict[str, Any]` | Dict with metadata and workflow descriptions. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/runtime.py` ``` def describe(self) -> dict[str, Any]: """Introspect the runtime graph. Returns: Dict with metadata and workflow descriptions. """ return { "metadata": self._metadata, "workflows": { name: { "pattern_type": type(pattern).__name__, } for name, pattern in self._workflows.items() }, "agents": list(self._agents.keys()), } ``` ### `AgentSpec` Bases: `BaseModel` Specification of a single agent. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/schema/agents.py` ``` class AgentSpec(BaseModel): """Specification of a single agent.""" prompt: str = Field(..., description="System prompt for this agent") provider: str = Field(default="", description="Provider ref from providers dict") tools: list[str] = Field(default_factory=list, description="Tool names available to this agent") description: str = Field(default="", description="What this agent does") guardrails: list[str] = Field(default_factory=list, description="Guardrail refs") ``` ### `BlueprintSpec` Bases: `BaseModel` Root specification for a declarative agent system. This is the top-level Pydantic model that represents a complete blueprint YAML/JSON document. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/schema/spec.py` ``` class BlueprintSpec(BaseModel): """Root specification for a declarative agent system. This is the top-level Pydantic model that represents a complete blueprint YAML/JSON document. """ api_version: str = Field(default="pyagent/v1", description="Schema version") metadata: MetadataSpec package: PackageSpec | None = Field( default=None, description="Optional 'Agent Unit' packaging metadata" ) providers: dict[str, ProviderBindingSpec] = Field(default_factory=dict) context: ContextConfigSpec | None = Field(default=None) agents: dict[str, AgentSpec] workflows: dict[str, WorkflowSpec] contracts: dict[str, ContractSpec] = Field(default_factory=dict) observability: ObservabilitySpec | None = Field(default=None) ``` ### `ContextConfigSpec` Bases: `BaseModel` Context configuration for a blueprint. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/schema/context.py` ``` class ContextConfigSpec(BaseModel): """Context configuration for a blueprint.""" memory: MemoryConfig = Field(default_factory=MemoryConfig) compression: CompressionConfig = Field(default_factory=CompressionConfig) redaction: RedactionConfig | None = Field(default=None) ``` ### `ContractSpec` Bases: `BaseModel` Input/output contract for a workflow. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/schema/contracts.py` ``` class ContractSpec(BaseModel): """Input/output contract for a workflow.""" input: dict[str, Any] = Field( default_factory=dict, description="Input schema (JSON Schema-like)" ) output: dict[str, Any] = Field( default_factory=dict, description="Output schema (JSON Schema-like)" ) sla: SLASpec = Field(default_factory=SLASpec, description="SLA constraints") ``` ### `MetadataSpec` Bases: `BaseModel` Blueprint metadata. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/schema/metadata.py` ``` class MetadataSpec(BaseModel): """Blueprint metadata.""" name: str = Field(..., description="Human-readable blueprint name") version: str = Field(default="0.1.0", description="Semantic version") description: str = Field(default="", description="What this blueprint does") tags: list[str] = Field(default_factory=list, description="Categorization tags") owner: str = Field(default="", description="Team or individual owner") ``` ### `ObservabilitySpec` Bases: `BaseModel` Observability configuration for a blueprint. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/schema/observability.py` ``` class ObservabilitySpec(BaseModel): """Observability configuration for a blueprint.""" tracing: TracingConfig = Field(default_factory=TracingConfig) cost_budget: CostBudgetConfig | None = Field(default=None) ``` ### `PackageSpec` Bases: `BaseModel` Optional packaging metadata for producing an 'Agent Unit' artifact. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/schema/package.py` ``` class PackageSpec(BaseModel): """Optional packaging metadata for producing an 'Agent Unit' artifact.""" name: str = Field(..., description="Distribution name for the packaged Agent Unit") version: str = Field(default="0.1.0", description="Semantic version of the Agent Unit") author: str = Field(default="", description="Author or team publishing this unit") runtime: str = Field( ..., description=( "Name of the RuntimeAdapter this unit targets (e.g. 'pyagent', " "'simple_loop', 'state_machine', 'sequential_chain', 'single_agent', " "or a third-party adapter name). Must match a discoverable adapter " "at packaging time." ), ) dependencies: list[str] = Field( default_factory=list, description="Extra distribution dependencies beyond the runtime adapter itself", ) ``` ### `ProviderBindingSpec` Bases: `BaseModel` Binding of a logical provider name to a model + backend. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/schema/providers.py` ``` class ProviderBindingSpec(BaseModel): """Binding of a logical provider name to a model + backend.""" model: str = Field(..., description="Model identifier (e.g., 'gpt-4.1-mini')") provider: str = Field( default="mock", description="Provider backend (mock, openai, anthropic, litellm)" ) fallback_ref: str = Field(default="", description="Fallback provider ref name") ``` ### `WorkflowSpec` Bases: `BaseModel` Specification of a workflow: pattern + agent wiring. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/schema/workflows.py` ``` class WorkflowSpec(BaseModel): """Specification of a workflow: pattern + agent wiring.""" # Agent/pattern wiring (routes, stages, teams, ...) must be nested under # `agents:`/`config:` — a misplaced sibling key (e.g. `classifier:` next # to `pattern:` instead of inside `agents:`) is silently dropped under # the default "ignore extra" behavior, producing an empty `agents={}` # that only fails at run time with a confusing AttributeError deep in # the pattern's `_execute`. Forbidding extras turns that into a clear, # immediate ValidationError at load time. model_config = ConfigDict(extra="forbid") pattern: str = Field(..., description="Pattern registry name (e.g., 'supervisor', 'pipeline')") agents: dict[str, Any] = Field(default_factory=dict, description="Role → agent ref mapping") config: dict[str, Any] = Field(default_factory=dict, description="Pattern-specific config") recovery: RecoverySpec | None = Field(default=None, description="Recovery configuration") guardrails: list[str] = Field(default_factory=list, description="Guardrail refs") ``` ### `BlueprintTester` Run conformance tests for blueprint contracts using MockLLM. Compiles the blueprint with mock providers and verifies that: - Workflows produce output - Output type matches contract expectations - SLA constraints are within bounds (basic checks) Parameters: | Name | Type | Description | Default | | ---------- | ------------------- | ----------- | -------------------------------------------------------- | | `compiler` | \`BlueprintCompiler | None\` | Optional compiler instance. Creates one if not provided. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/tester.py` ``` class BlueprintTester: """Run conformance tests for blueprint contracts using MockLLM. Compiles the blueprint with mock providers and verifies that: - Workflows produce output - Output type matches contract expectations - SLA constraints are within bounds (basic checks) Args: compiler: Optional compiler instance. Creates one if not provided. """ def __init__(self, compiler: BlueprintCompiler | None = None) -> None: self._compiler = compiler or BlueprintCompiler() async def test( self, spec: BlueprintSpec, test_inputs: dict[str, str] | None = None, ) -> list[TestResult]: """Run contract conformance tests for all workflows with contracts. Args: spec: Blueprint spec to test. test_inputs: Optional mapping of workflow name → test input. If not provided, uses a default test string. Returns: List of ``TestResult`` for each contract. """ graph = self._compiler.compile(spec) results: list[TestResult] = [] for contract_name, contract in spec.contracts.items(): if contract_name not in spec.workflows: results.append( TestResult( workflow=contract_name, passed=False, error=f"Contract references non-existent workflow '{contract_name}'", ) ) continue test_input = (test_inputs or {}).get( contract_name, "Test input for contract validation" ) try: result = await graph.run(contract_name, test_input) checks: dict[str, bool] = {} # Check: output is non-empty checks["output_non_empty"] = bool(result.output) # Check: output type matches expected_type = contract.output.get("type", "string") if expected_type == "string": checks["output_is_string"] = isinstance(result.output, str) # Check: output length within bounds max_tokens = contract.input.get("max_tokens") if max_tokens is not None: estimated_tokens = len(test_input) // 4 checks["input_within_token_limit"] = estimated_tokens <= max_tokens all_passed = all(checks.values()) results.append( TestResult( workflow=contract_name, passed=all_passed, output=result.output, checks=checks, ) ) except Exception as exc: results.append( TestResult( workflow=contract_name, passed=False, error=f"{type(exc).__name__}: {exc}", ) ) return results def summary(self, results: list[TestResult]) -> str: """Human-readable summary of test results.""" lines: list[str] = [] passed = sum(1 for r in results if r.passed) total = len(results) lines.append(f"\nContract Tests: {passed}/{total} passed\n") for r in results: status = "✓ PASS" if r.passed else "✗ FAIL" lines.append(f" {status} {r.workflow}") if r.error: lines.append(f" Error: {r.error}") for check, ok in r.checks.items(): mark = "✓" if ok else "✗" lines.append(f" {mark} {check}") return "\n".join(lines) ``` #### `test(spec, test_inputs=None)` Run contract conformance tests for all workflows with contracts. Parameters: | Name | Type | Description | Default | | ------------- | ---------------- | ----------------------- | -------------------------------------------------------------------------------------------- | | `spec` | `BlueprintSpec` | Blueprint spec to test. | *required* | | `test_inputs` | \`dict[str, str] | None\` | Optional mapping of workflow name → test input. If not provided, uses a default test string. | Returns: | Type | Description | | ------------------ | ------------------------------------- | | `list[TestResult]` | List of TestResult for each contract. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/tester.py` ``` async def test( self, spec: BlueprintSpec, test_inputs: dict[str, str] | None = None, ) -> list[TestResult]: """Run contract conformance tests for all workflows with contracts. Args: spec: Blueprint spec to test. test_inputs: Optional mapping of workflow name → test input. If not provided, uses a default test string. Returns: List of ``TestResult`` for each contract. """ graph = self._compiler.compile(spec) results: list[TestResult] = [] for contract_name, contract in spec.contracts.items(): if contract_name not in spec.workflows: results.append( TestResult( workflow=contract_name, passed=False, error=f"Contract references non-existent workflow '{contract_name}'", ) ) continue test_input = (test_inputs or {}).get( contract_name, "Test input for contract validation" ) try: result = await graph.run(contract_name, test_input) checks: dict[str, bool] = {} # Check: output is non-empty checks["output_non_empty"] = bool(result.output) # Check: output type matches expected_type = contract.output.get("type", "string") if expected_type == "string": checks["output_is_string"] = isinstance(result.output, str) # Check: output length within bounds max_tokens = contract.input.get("max_tokens") if max_tokens is not None: estimated_tokens = len(test_input) // 4 checks["input_within_token_limit"] = estimated_tokens <= max_tokens all_passed = all(checks.values()) results.append( TestResult( workflow=contract_name, passed=all_passed, output=result.output, checks=checks, ) ) except Exception as exc: results.append( TestResult( workflow=contract_name, passed=False, error=f"{type(exc).__name__}: {exc}", ) ) return results ``` #### `summary(results)` Human-readable summary of test results. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/tester.py` ``` def summary(self, results: list[TestResult]) -> str: """Human-readable summary of test results.""" lines: list[str] = [] passed = sum(1 for r in results if r.passed) total = len(results) lines.append(f"\nContract Tests: {passed}/{total} passed\n") for r in results: status = "✓ PASS" if r.passed else "✗ FAIL" lines.append(f" {status} {r.workflow}") if r.error: lines.append(f" Error: {r.error}") for check, ok in r.checks.items(): mark = "✓" if ok else "✗" lines.append(f" {mark} {check}") return "\n".join(lines) ``` ### `TestResult` Result of a single contract test. Attributes: | Name | Type | Description | | ---------- | ----------------- | ------------------------------------ | | `workflow` | `str` | Workflow name tested. | | `passed` | `bool` | Whether the test passed. | | `output` | `str` | The actual output from the workflow. | | `checks` | `dict[str, bool]` | Dict of check name → pass/fail. | | `error` | \`str | None\` | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/tester.py` ``` @dataclass class TestResult: """Result of a single contract test. Attributes: workflow: Workflow name tested. passed: Whether the test passed. output: The actual output from the workflow. checks: Dict of check name → pass/fail. error: Error message if the test failed with an exception. """ workflow: str passed: bool output: str = "" checks: dict[str, bool] = field(default_factory=dict) error: str | None = None ``` ### `BlueprintValidator` Run static checks on a BlueprintSpec. Checks: - All agent refs in workflows exist in agents dict - All provider refs in agents exist in providers dict - Pattern names are registered - No cyclic workflow dependencies - SLA values are realistic - Security: no hardcoded API keys in prompts Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/validator.py` ``` class BlueprintValidator: """Run static checks on a BlueprintSpec. Checks: - All agent refs in workflows exist in agents dict - All provider refs in agents exist in providers dict - Pattern names are registered - No cyclic workflow dependencies - SLA values are realistic - Security: no hardcoded API keys in prompts """ def validate(self, spec: BlueprintSpec) -> list[ValidationIssue]: """Run all validation checks. Args: spec: The blueprint spec to validate. Returns: List of issues found (may be empty). """ issues: list[ValidationIssue] = [] issues.extend(self._check_agent_refs(spec)) issues.extend(self._check_provider_refs(spec)) issues.extend(self._check_pattern_names(spec)) issues.extend(self._check_contract_refs(spec)) issues.extend(self._check_sla_values(spec)) issues.extend(self._check_security(spec)) return issues def _check_agent_refs(self, spec: BlueprintSpec) -> list[ValidationIssue]: """Ensure all agent refs in workflows point to defined agents.""" issues: list[ValidationIssue] = [] agent_names = set(spec.agents.keys()) for wf_name, wf_spec in spec.workflows.items(): for role, ref in wf_spec.agents.items(): if isinstance(ref, str) and ref not in agent_names: issues.append( ValidationIssue( path=f"workflows.{wf_name}.agents.{role}", message=f"Agent ref '{ref}' not found in agents. Available: {sorted(agent_names)}", severity=IssueSeverity.ERROR, ) ) elif isinstance(ref, dict): for sub_role, sub_ref in ref.items(): if isinstance(sub_ref, str) and sub_ref not in agent_names: issues.append( ValidationIssue( path=f"workflows.{wf_name}.agents.{role}.{sub_role}", message=f"Agent ref '{sub_ref}' not found in agents.", severity=IssueSeverity.ERROR, ) ) return issues def _check_provider_refs(self, spec: BlueprintSpec) -> list[ValidationIssue]: """Ensure all provider refs in agents point to defined providers.""" issues: list[ValidationIssue] = [] provider_names = set(spec.providers.keys()) for agent_name, agent_spec in spec.agents.items(): if agent_spec.provider and agent_spec.provider not in provider_names: issues.append( ValidationIssue( path=f"agents.{agent_name}.provider", message=f"Provider ref '{agent_spec.provider}' not found. Available: {sorted(provider_names)}", severity=IssueSeverity.ERROR, ) ) return issues def _check_pattern_names(self, spec: BlueprintSpec) -> list[ValidationIssue]: """Ensure all pattern names are registered, IF at least one installed adapter declares a pattern vocabulary. Degrades gracefully (no error) when no adapter is installed, or when the installed adapter(s) don't have a fixed pattern vocabulary at all (e.g. a loop-based adapter) — this is what lets `validate` keep working even with zero runtime packages installed.""" issues: list[ValidationIssue] = [] known = _known_pattern_names() if known is None: return issues for wf_name, wf_spec in spec.workflows.items(): if wf_spec.pattern not in known: issues.append( ValidationIssue( path=f"workflows.{wf_name}.pattern", message=f"Unknown pattern '{wf_spec.pattern}'. Known: {sorted(known)}", severity=IssueSeverity.ERROR, ) ) return issues def _check_contract_refs(self, spec: BlueprintSpec) -> list[ValidationIssue]: """Ensure contracts reference existing workflows.""" issues: list[ValidationIssue] = [] wf_names = set(spec.workflows.keys()) for contract_name in spec.contracts: if contract_name not in wf_names: issues.append( ValidationIssue( path=f"contracts.{contract_name}", message=f"Contract '{contract_name}' references non-existent workflow.", severity=IssueSeverity.WARNING, ) ) return issues def _check_sla_values(self, spec: BlueprintSpec) -> list[ValidationIssue]: """Warn about unrealistic SLA values.""" issues: list[ValidationIssue] = [] for name, contract in spec.contracts.items(): if contract.sla.latency_p95_ms < 100: issues.append( ValidationIssue( path=f"contracts.{name}.sla.latency_p95_ms", message=f"Latency SLA {contract.sla.latency_p95_ms}ms is unrealistically low for LLM calls.", severity=IssueSeverity.WARNING, ) ) return issues def _check_security(self, spec: BlueprintSpec) -> list[ValidationIssue]: """Check for hardcoded API keys in prompts.""" issues: list[ValidationIssue] = [] key_patterns = ["sk-", "sk-ant-", "api_key=", "API_KEY", "Bearer "] for agent_name, agent_spec in spec.agents.items(): for pattern in key_patterns: if pattern in agent_spec.prompt: issues.append( ValidationIssue( path=f"agents.{agent_name}.prompt", message=f"Possible hardcoded API key detected (contains '{pattern}').", severity=IssueSeverity.ERROR, ) ) break return issues ``` #### `validate(spec)` Run all validation checks. Parameters: | Name | Type | Description | Default | | ------ | --------------- | ------------------------------- | ---------- | | `spec` | `BlueprintSpec` | The blueprint spec to validate. | *required* | Returns: | Type | Description | | ----------------------- | ------------------------------------ | | `list[ValidationIssue]` | List of issues found (may be empty). | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/validator.py` ``` def validate(self, spec: BlueprintSpec) -> list[ValidationIssue]: """Run all validation checks. Args: spec: The blueprint spec to validate. Returns: List of issues found (may be empty). """ issues: list[ValidationIssue] = [] issues.extend(self._check_agent_refs(spec)) issues.extend(self._check_provider_refs(spec)) issues.extend(self._check_pattern_names(spec)) issues.extend(self._check_contract_refs(spec)) issues.extend(self._check_sla_values(spec)) issues.extend(self._check_security(spec)) return issues ``` ### `ValidationIssue` A single validation finding. Attributes: | Name | Type | Description | | ---------- | --------------- | ------------------------------------- | | `path` | `str` | Dotted path to the problematic field. | | `message` | `str` | Human-readable description. | | `severity` | `IssueSeverity` | Error, warning, or info. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/validator.py` ``` @dataclass(frozen=True) class ValidationIssue: """A single validation finding. Attributes: path: Dotted path to the problematic field. message: Human-readable description. severity: Error, warning, or info. """ path: str message: str severity: IssueSeverity ``` ### `render_adapter_template(framework_name, adapter_name=None, dist_name=None)` Render the file contents for a starter adapter package. Parameters: | Name | Type | Description | Default | | ---------------- | ----- | ------------------------------------------ | ----------------------------------------------------------------------------------------------- | | `framework_name` | `str` | Human-readable SDK name, e.g. "LangGraph". | *required* | | `adapter_name` | \`str | None\` | entry-point / RuntimeAdapter.name value. Defaults to a snake_case derivation of framework_name. | | `dist_name` | \`str | None\` | PyPI distribution name. Defaults to "pyagent-blueprint-adapter-". | Returns: | Type | Description | | ---------------- | ---------------------------------------------------------- | | `dict[str, str]` | Mapping of relative file path -> file content, ready to be | | `dict[str, str]` | written to disk. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/adapter_template.py` ``` def render_adapter_template( framework_name: str, adapter_name: str | None = None, dist_name: str | None = None, ) -> dict[str, str]: """Render the file contents for a starter adapter package. Args: framework_name: Human-readable SDK name, e.g. "LangGraph". adapter_name: entry-point / RuntimeAdapter.name value. Defaults to a snake_case derivation of framework_name. dist_name: PyPI distribution name. Defaults to "pyagent-blueprint-adapter-". Returns: Mapping of relative file path -> file content, ready to be written to disk. """ slug = "".join(c if c.isalnum() else "_" for c in framework_name.lower()).strip("_") while "__" in slug: slug = slug.replace("__", "_") adapter_name = adapter_name or slug module_name = f"pyagent_blueprint_adapter_{slug}" class_name = "".join(part.capitalize() for part in slug.split("_")) + "Adapter" dist_name = dist_name or f"pyagent-blueprint-adapter-{slug.replace('_', '-')}" fmt_kwargs = { "framework_name": framework_name, "adapter_name": adapter_name, "module_name": module_name, "class_name": class_name, "dist_name": dist_name, } return { "pyproject.toml": _PYPROJECT_TEMPLATE.format(**fmt_kwargs), f"src/{module_name}/__init__.py": "", f"src/{module_name}/adapter.py": _ADAPTER_MODULE_TEMPLATE.format(**fmt_kwargs), f"tests/test_{slug}_adapter_conformance.py": _TEST_MODULE_TEMPLATE.format(**fmt_kwargs), "README.md": _README_TEMPLATE.format(**fmt_kwargs), } ``` ### `write_adapter_template(framework_name, output_dir, adapter_name=None, dist_name=None)` Render and write a starter adapter package to `output_dir`. Returns the output directory path. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/adapter_template.py` ``` def write_adapter_template( framework_name: str, output_dir: str | Path, adapter_name: str | None = None, dist_name: str | None = None, ) -> Path: """Render and write a starter adapter package to `output_dir`. Returns the output directory path. """ files = render_adapter_template(framework_name, adapter_name, dist_name) out_dir = Path(output_dir) for rel_path, content in files.items(): full_path = out_dir / rel_path full_path.parent.mkdir(parents=True, exist_ok=True) full_path.write_text(content) return out_dir ``` ### `blueprint_contracts_json_schema(ir)` Render every declared contract in a `BlueprintIR` as JSON Schema documents, keyed by workflow name. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/contract.py` ``` def blueprint_contracts_json_schema(ir: BlueprintIR) -> dict[str, dict[str, Any]]: """Render every declared contract in a `BlueprintIR` as JSON Schema documents, keyed by workflow name.""" return {name: contract_json_schema(contract) for name, contract in ir.contracts.items()} ``` ### `contract_json_schema(contract)` Render a single workflow's `ContractIR` as a JSON Schema document. The document's `$id`-less shape is intentionally minimal: `input`/ `output` are the declared JSON Schemas verbatim (empty dict means "unconstrained"), and SLA constraints are surfaced under the `x-pyagent` extension namespace since they have no native JSON Schema representation — a non-Python consumer can still read them, it's just not part of core JSON Schema vocabulary. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/contract.py` ``` def contract_json_schema(contract: ContractIR) -> dict[str, Any]: """Render a single workflow's `ContractIR` as a JSON Schema document. The document's `$id`-less shape is intentionally minimal: `input`/ `output` are the declared JSON Schemas verbatim (empty dict means "unconstrained"), and SLA constraints are surfaced under the `x-pyagent` extension namespace since they have no native JSON Schema representation — a non-Python consumer can still read them, it's just not part of core JSON Schema vocabulary. """ return { "$schema": "https://json-schema.org/draft/2020-12/schema", "title": f"pyagent-blueprint contract: {contract.workflow}", "type": "object", "properties": { "input": contract.input_schema or {"description": "unconstrained"}, "output": contract.output_schema or {"description": "unconstrained"}, }, "x-pyagent": { "schema_version": CONTRACT_SCHEMA_VERSION, "sla": { "latency_p95_ms": contract.sla.latency_p95_ms, "cost_max_usd": contract.sla.cost_max_usd, "quality_min": contract.sla.quality_min, }, }, } ``` ### `load_blueprint(path)` Load a blueprint from a YAML or JSON file. Parameters: | Name | Type | Description | Default | | ------ | ----- | ----------- | ------------------------------------- | | `path` | \`str | Path\` | Path to a .yaml, .yml, or .json file. | Returns: | Type | Description | | --------------- | ------------------------ | | `BlueprintSpec` | Validated BlueprintSpec. | Raises: | Type | Description | | -------------------- | ----------------------------------------------- | | `BlueprintLoadError` | If the file is missing, unreadable, or invalid. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/loader.py` ``` def load_blueprint(path: str | Path) -> BlueprintSpec: """Load a blueprint from a YAML or JSON file. Args: path: Path to a ``.yaml``, ``.yml``, or ``.json`` file. Returns: Validated ``BlueprintSpec``. Raises: BlueprintLoadError: If the file is missing, unreadable, or invalid. """ path = Path(path) if not path.exists(): raise BlueprintLoadError(f"Blueprint file not found: {path}") text = path.read_text(encoding="utf-8") try: if path.suffix in (".yaml", ".yml"): data = yaml.safe_load(text) elif path.suffix == ".json": data = json.loads(text) else: raise BlueprintLoadError(f"Unsupported file extension: {path.suffix}") except (yaml.YAMLError, json.JSONDecodeError) as exc: raise BlueprintLoadError(f"Parse error in {path}: {exc}") from exc if not isinstance(data, dict): raise BlueprintLoadError(f"Blueprint must be a mapping, got {type(data).__name__}") try: return BlueprintSpec(**data) except ValidationError as exc: raise BlueprintLoadError(f"Schema validation failed for {path}:\n{exc}") from exc ``` ### `load_blueprint_from_str(text, fmt='yaml')` Load a blueprint from a string. Parameters: | Name | Type | Description | Default | | ------ | ----- | ------------------ | ---------- | | `text` | `str` | YAML or JSON text. | *required* | | `fmt` | `str` | "yaml" or "json". | `'yaml'` | Returns: | Type | Description | | --------------- | ------------------------ | | `BlueprintSpec` | Validated BlueprintSpec. | Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/loader.py` ``` def load_blueprint_from_str(text: str, fmt: str = "yaml") -> BlueprintSpec: """Load a blueprint from a string. Args: text: YAML or JSON text. fmt: ``"yaml"`` or ``"json"``. Returns: Validated ``BlueprintSpec``. """ data = json.loads(text) if fmt == "json" else yaml.safe_load(text) return BlueprintSpec(**data) ``` ### `package_blueprint(spec, raw_source, source_filename, output_dir)` Build a distributable Agent Unit archive. Produces `/-.agentunit.zip` containing: - `unit.json` — the AgentUnitMetadata manifest - the original blueprint source, preserved under its own filename Returns the path to the written archive. Source code in `packages/pyagent-blueprint/src/pyagent_blueprint/packaging.py` ``` def package_blueprint( spec: BlueprintSpec, raw_source: str, source_filename: str, output_dir: str | Path, ) -> Path: """Build a distributable Agent Unit archive. Produces ``/-.agentunit.zip`` containing: - ``unit.json`` — the AgentUnitMetadata manifest - the original blueprint source, preserved under its own filename Returns the path to the written archive. """ metadata = build_metadata(spec, raw_source) out_dir = Path(output_dir) out_dir.mkdir(parents=True, exist_ok=True) archive_path = out_dir / f"{metadata.name}-{metadata.version}.agentunit.zip" with zipfile.ZipFile(archive_path, "w", zipfile.ZIP_DEFLATED) as zf: zf.writestr("unit.json", json.dumps(metadata.to_dict(), indent=2)) zf.writestr(Path(source_filename).name, raw_source) return archive_path ``` # pyagent-compress API Reference ## `pyagent_compress.compressor.MessageCompressor` Compress inter-agent messages by removing filler and extracting key sentences. Parameters: | Name | Type | Description | Default | | --------------------- | ------- | ---------------------------------------------------- | ------- | | `target_ratio` | `float` | Target compression ratio (0.3 = keep 30% of tokens). | `0.5` | | `min_sentence_length` | `int` | Minimum characters for a sentence to be kept. | `20` | | `remove_filler` | `bool` | Whether to strip filler phrases. | `True` | ### `compress(text)` Compress a message text. Strategy: 1. Remove filler phrases 1. Split into sentences 1. Score sentences by information density 1. Keep top sentences until target ratio met ## `pyagent_compress.compressor.CompressionResult` Result of message compression. Attributes: | Name | Type | Description | | ------------------- | ------- | --------------------------------- | | `original` | `str` | The original text. | | `compressed` | `str` | The compressed text. | | `original_tokens` | `int` | Estimated original token count. | | `compressed_tokens` | `int` | Estimated compressed token count. | | `savings_pct` | `float` | Percentage of tokens saved (0-1). | ## `pyagent_compress.pruner.AgentPruner` Detect and flag non-contributing agents for removal. Scores each agent's contribution based on: - Unique information (not repeated from other agents) - Response diversity (different from own previous responses) - Task relevance (overlap with original task keywords) Parameters: | Name | Type | Description | Default | | ------------------ | ------- | -------------------------------------------------- | ------- | | `min_contribution` | `float` | Minimum contribution score (0-1) to keep an agent. | `0.3` | | `window_size` | `int` | Number of recent messages to analyze per agent. | `5` | ### `score_agents(messages, task)` Score each agent's contribution from message history. ### `should_prune(scores)` Return agent names that should be pruned (below min contribution). ## `pyagent_compress.pruner.InteractionPruner` Detect early consensus and skip remaining communication rounds. Analyzes agent outputs to determine if consensus has been reached, allowing patterns like Debate or Voting to terminate early. Parameters: | Name | Type | Description | Default | | --------------------- | ------- | ----------------------------------------------------------------------------- | ------- | | `consensus_threshold` | `float` | Minimum similarity between agent outputs to consider consensus reached (0-1). | `0.7` | | `min_rounds` | `int` | Minimum rounds before early termination is allowed. | `1` | ### `has_consensus(outputs, current_round)` Check if agents have reached consensus. Parameters: | Name | Type | Description | Default | | --------------- | ----------- | ---------------------------------------- | ---------- | | `outputs` | `list[str]` | Current round's outputs from all agents. | *required* | | `current_round` | `int` | The current round number (1-indexed). | *required* | Returns: | Type | Description | | ------ | --------------------------------------------------- | | `bool` | True if consensus is reached and we can stop early. | ## `pyagent_compress.pruner.ContributionScore` How much an agent is contributing to the conversation. ## `pyagent_compress.budget.TokenBudget` Manage token budgets across a multi-agent workflow. Parameters: | Name | Type | Description | Default | | ----------------- | ------ | -------------------------------------------------------- | -------- | | `workflow_limit` | `int` | Total token limit across all agents. | `100000` | | `per_agent_limit` | `int` | Default per-agent token limit. | `20000` | | `strict` | `bool` | If True, raises BudgetExceededError on limit violations. | `True` | ### `consume(agent_name, tokens)` Record token consumption for an agent. Parameters: | Name | Type | Description | Default | | ------------ | ----- | --------------------------- | ---------- | | `agent_name` | `str` | The consuming agent's name. | *required* | | `tokens` | `int` | Number of tokens consumed. | *required* | ### `register_agent(name, limit=None)` Register an agent with an optional custom limit. ### `remaining(agent_name=None)` Get remaining tokens for an agent or the whole workflow. ### `summary()` Return a summary of all agent budgets. ## `pyagent_compress.budget.AgentBudget` Budget tracking for a single agent. ### `consume(tokens, strict=True)` Consume tokens from budget. Parameters: | Name | Type | Description | Default | | -------- | ------ | ----------------------------------------------------------- | ---------- | | `tokens` | `int` | Number of tokens consumed. | *required* | | `strict` | `bool` | If True, raises BudgetExceededError. If False, just tracks. | `True` | ## `pyagent_compress.budget.BudgetExceededError` Bases: `Exception` Raised when a token budget is exceeded. ## `pyagent_compress.middleware.CompressMiddleware` Middleware that wraps agents with automatic output compression. Usage middleware = CompressMiddleware(target_ratio=0.5) compressed_agent = middleware.wrap(my_agent) Parameters: | Name | Type | Description | Default | | -------------- | ------------------- | ----------------------------------------------------------- | ---------------------------------------------------------- | | `compressor` | \`MessageCompressor | None\` | Optional MessageCompressor. Created with defaults if None. | | `budget` | \`TokenBudget | None\` | Optional TokenBudget for workflow-wide tracking. | | `target_ratio` | `float` | Target compression ratio (used if compressor not provided). | `0.5` | ### `wrap(agent)` Wrap an agent with compression. ### `wrap_all(agents)` Wrap multiple agents. ## `pyagent_compress.middleware.CompressedAgent` Bases: `Agent` Agent wrapper that compresses output messages. Parameters: | Name | Type | Description | Default | | ------------ | ------------------- | --------------------------- | ---------------------------------- | | `agent` | `Agent` | The original agent. | *required* | | `compressor` | `MessageCompressor` | MessageCompressor instance. | *required* | | `budget` | \`TokenBudget | None\` | Optional TokenBudget for tracking. | ### `run(messages)` Run the agent and compress the output. # pyagent-context API Reference ## `pyagent_context` PyAgent Context — three-tier memory with trust-aware context ledger. ### `CompressionPolicy` Bases: `StrEnum` Available compression strategies. Source code in `packages/pyagent-context/src/pyagent_context/compression.py` ``` class CompressionPolicy(StrEnum): """Available compression strategies.""" NONE = "none" FIFO = "fifo" # drop oldest items SEMANTIC_LOSSLESS = "semantic_lossless" # compress text, keep Knowledge block SAWTOOTH = "sawtooth" # compress to floor, grow again ``` ### `ContextCompressor` Apply compression policies to a ContextLedger. The compressor monitors token usage and compresses when a threshold is reached. Parameters: | Name | Type | Description | Default | | ------------------ | ------------------- | ----------------------------------------------------- | ---------- | | `policy` | `CompressionPolicy` | Compression strategy. | `FIFO` | | `threshold_tokens` | `int` | Token count that triggers compression. | `10000` | | `floor_tokens` | `int` | Token target after compression (for SAWTOOTH/FIFO). | `5000` | | `preserve_trust` | `TrustLevel` | Items at or above this trust level are never dropped. | `VERIFIED` | Source code in `packages/pyagent-context/src/pyagent_context/compression.py` ``` class ContextCompressor: """Apply compression policies to a ContextLedger. The compressor monitors token usage and compresses when a threshold is reached. Args: policy: Compression strategy. threshold_tokens: Token count that triggers compression. floor_tokens: Token target after compression (for SAWTOOTH/FIFO). preserve_trust: Items at or above this trust level are never dropped. """ def __init__( self, policy: CompressionPolicy = CompressionPolicy.FIFO, threshold_tokens: int = 10_000, floor_tokens: int = 5_000, preserve_trust: TrustLevel = TrustLevel.VERIFIED, ) -> None: self._policy = policy self._threshold = threshold_tokens self._floor = floor_tokens self._preserve_trust = preserve_trust @property def policy(self) -> CompressionPolicy: return self._policy def should_compress(self, ledger: ContextLedger) -> bool: """Check if the ledger's token count exceeds the threshold.""" if self._policy == CompressionPolicy.NONE: return False return ledger.total_tokens >= self._threshold def compress(self, ledger: ContextLedger) -> ContextLedger: """Apply the compression policy and return a new (compressed) ledger. The original ledger is not mutated. """ if self._policy == CompressionPolicy.NONE: return ledger if self._policy == CompressionPolicy.FIFO: return self._compress_fifo(ledger) if self._policy == CompressionPolicy.SEMANTIC_LOSSLESS: return self._compress_semantic(ledger) if self._policy == CompressionPolicy.SAWTOOTH: return self._compress_sawtooth(ledger) return ledger def _compress_fifo(self, ledger: ContextLedger) -> ContextLedger: """Drop oldest items until under floor, preserving high-trust items.""" items = list(ledger.items) tokens = ledger.total_tokens # Remove from front (oldest) until under floor new_items: list[ContextItem] = [] for item in items: if tokens <= self._floor: new_items.append(item) elif item.trust_level >= self._preserve_trust: new_items.append(item) # keep high-trust else: tokens -= item.token_estimate # drop return ContextLedger(items=new_items) def _compress_semantic(self, ledger: ContextLedger) -> ContextLedger: """Compress item content text while preserving verified items unchanged. Uses simple sentence extraction — for full compression, integrate ``MessageCompressor`` from ``pyagent-compress``. """ items = list(ledger.items) new_items: list[ContextItem] = [] for item in items: if item.trust_level >= self._preserve_trust: new_items.append(item) else: # Simple compression: keep first sentence only sentences = item.content.split(". ") compressed = sentences[0] + ("." if len(sentences) > 1 else "") new_item = ContextItem( content=compressed, source=item.source, timestamp=item.timestamp, trust_level=item.trust_level, sensitivity=item.sensitivity, expires_at=item.expires_at, derived_from=item.id, ) new_items.append(new_item) return ContextLedger(items=new_items) def _compress_sawtooth(self, ledger: ContextLedger) -> ContextLedger: """Compress to floor, then allow growth again. Combines FIFO eviction with semantic compression for remaining items. """ # First pass: FIFO eviction fifo_result = self._compress_fifo(ledger) # If still over floor, apply semantic compression if fifo_result.total_tokens > self._floor: return self._compress_semantic(fifo_result) return fifo_result ``` #### `should_compress(ledger)` Check if the ledger's token count exceeds the threshold. Source code in `packages/pyagent-context/src/pyagent_context/compression.py` ``` def should_compress(self, ledger: ContextLedger) -> bool: """Check if the ledger's token count exceeds the threshold.""" if self._policy == CompressionPolicy.NONE: return False return ledger.total_tokens >= self._threshold ``` #### `compress(ledger)` Apply the compression policy and return a new (compressed) ledger. The original ledger is not mutated. Source code in `packages/pyagent-context/src/pyagent_context/compression.py` ``` def compress(self, ledger: ContextLedger) -> ContextLedger: """Apply the compression policy and return a new (compressed) ledger. The original ledger is not mutated. """ if self._policy == CompressionPolicy.NONE: return ledger if self._policy == CompressionPolicy.FIFO: return self._compress_fifo(ledger) if self._policy == CompressionPolicy.SEMANTIC_LOSSLESS: return self._compress_semantic(ledger) if self._policy == CompressionPolicy.SAWTOOTH: return self._compress_sawtooth(ledger) return ledger ``` ### `ContextItem` A single piece of context with trust and lifecycle metadata. Attributes: | Name | Type | Description | | ---------------- | ------------- | -------------------------------------------- | | `content` | `str` | The text content. | | `source` | `str` | Origin — agent name, tool name, or "user". | | `timestamp` | `float` | Creation time (time.time()). | | `trust_level` | `TrustLevel` | How much to trust this item. | | `sensitivity` | `Sensitivity` | Data classification for redaction decisions. | | `expires_at` | \`float | None\` | | `derived_from` | \`str | None\` | | `token_estimate` | `int` | Rough token count (len(content) // 4). | Source code in `packages/pyagent-context/src/pyagent_context/item.py` ``` @dataclass class ContextItem: """A single piece of context with trust and lifecycle metadata. Attributes: content: The text content. source: Origin — agent name, tool name, or ``"user"``. timestamp: Creation time (``time.time()``). trust_level: How much to trust this item. sensitivity: Data classification for redaction decisions. expires_at: Expiration timestamp, or ``None`` for never. derived_from: Parent item ID if this was derived from another. token_estimate: Rough token count (``len(content) // 4``). """ content: str source: str timestamp: float = field(default_factory=time.time) trust_level: TrustLevel = TrustLevel.INFERRED sensitivity: Sensitivity = Sensitivity.INTERNAL expires_at: float | None = None derived_from: str | None = None token_estimate: int = 0 _id: str = field(default_factory=lambda: uuid.uuid4().hex[:12]) def __post_init__(self) -> None: if self.token_estimate == 0: self.token_estimate = max(1, len(self.content) // 4) @property def id(self) -> str: return self._id @property def is_expired(self) -> bool: """Check if this item has passed its expiry time.""" if self.expires_at is None: return False return time.time() > self.expires_at @property def age_seconds(self) -> float: """Time since creation in seconds.""" return time.time() - self.timestamp def to_dict(self) -> dict: """Serialize to a JSON-compatible dict.""" return { "id": self._id, "content": self.content, "source": self.source, "timestamp": self.timestamp, "trust_level": self.trust_level.value, "sensitivity": self.sensitivity.value, "expires_at": self.expires_at, "derived_from": self.derived_from, "token_estimate": self.token_estimate, } @classmethod def from_dict(cls, data: dict) -> ContextItem: """Deserialize from a dict.""" item = cls( content=data["content"], source=data["source"], timestamp=data["timestamp"], trust_level=TrustLevel(data["trust_level"]), sensitivity=Sensitivity(data["sensitivity"]), expires_at=data.get("expires_at"), derived_from=data.get("derived_from"), token_estimate=data.get("token_estimate", 0), ) item._id = data["id"] return item ``` #### `is_expired` Check if this item has passed its expiry time. #### `age_seconds` Time since creation in seconds. #### `to_dict()` Serialize to a JSON-compatible dict. Source code in `packages/pyagent-context/src/pyagent_context/item.py` ``` def to_dict(self) -> dict: """Serialize to a JSON-compatible dict.""" return { "id": self._id, "content": self.content, "source": self.source, "timestamp": self.timestamp, "trust_level": self.trust_level.value, "sensitivity": self.sensitivity.value, "expires_at": self.expires_at, "derived_from": self.derived_from, "token_estimate": self.token_estimate, } ``` #### `from_dict(data)` Deserialize from a dict. Source code in `packages/pyagent-context/src/pyagent_context/item.py` ``` @classmethod def from_dict(cls, data: dict) -> ContextItem: """Deserialize from a dict.""" item = cls( content=data["content"], source=data["source"], timestamp=data["timestamp"], trust_level=TrustLevel(data["trust_level"]), sensitivity=Sensitivity(data["sensitivity"]), expires_at=data.get("expires_at"), derived_from=data.get("derived_from"), token_estimate=data.get("token_estimate", 0), ) item._id = data["id"] return item ``` ### `Sensitivity` Bases: `StrEnum` Data sensitivity classification. Source code in `packages/pyagent-context/src/pyagent_context/item.py` ``` class Sensitivity(StrEnum): """Data sensitivity classification.""" PUBLIC = "public" INTERNAL = "internal" CONFIDENTIAL = "confidential" RESTRICTED = "restricted" ``` ### `TrustLevel` Bases: `StrEnum` Trust classification for context items. Source code in `packages/pyagent-context/src/pyagent_context/item.py` ``` class TrustLevel(StrEnum): """Trust classification for context items.""" VERIFIED = "verified" # from trusted source, validated INFERRED = "inferred" # LLM-generated, not validated USER_PROVIDED = "user" # direct user input EXTERNAL = "external" # from tool/API call def __ge__(self, other: TrustLevel) -> bool: order = { TrustLevel.INFERRED: 0, TrustLevel.EXTERNAL: 1, TrustLevel.USER_PROVIDED: 2, TrustLevel.VERIFIED: 3, } return order.get(self, 0) >= order.get(other, 0) def __gt__(self, other: TrustLevel) -> bool: return self != other and self.__ge__(other) def __le__(self, other: TrustLevel) -> bool: return other.__ge__(self) def __lt__(self, other: TrustLevel) -> bool: return self != other and self.__le__(other) ``` ### `ContextLedger` Append-only ledger of context items. Supports querying by trust level, age, and source. Converts items to `Message` lists for use with patterns. Parameters: | Name | Type | Description | Default | | ------- | ------------------- | ----------- | ----------------------- | | `items` | \`list[ContextItem] | None\` | Optional initial items. | Source code in `packages/pyagent-context/src/pyagent_context/ledger.py` ``` class ContextLedger: """Append-only ledger of context items. Supports querying by trust level, age, and source. Converts items to ``Message`` lists for use with patterns. Args: items: Optional initial items. """ def __init__(self, items: list[ContextItem] | None = None) -> None: self._items: list[ContextItem] = list(items) if items else [] def append(self, item: ContextItem) -> None: """Add an item to the ledger.""" self._items.append(item) def add( self, content: str, source: str, trust_level: TrustLevel = TrustLevel.INFERRED, **kwargs: Any, ) -> ContextItem: """Create and append a new item in one step. Returns: The created ``ContextItem``. """ item = ContextItem(content=content, source=source, trust_level=trust_level, **kwargs) self._items.append(item) return item def query( self, *, min_trust: TrustLevel | None = None, max_age_seconds: float | None = None, source: str | None = None, ) -> list[ContextItem]: """Filter items by trust level, age, and/or source. Args: min_trust: Only return items at or above this trust level. max_age_seconds: Only return items newer than this. source: Only return items from this source. Returns: Filtered list of items (chronological order). """ now = time.time() results: list[ContextItem] = [] for item in self._items: if min_trust is not None and TRUST_ORDER.get(item.trust_level, 0) < TRUST_ORDER.get( min_trust, 0 ): continue if max_age_seconds is not None and (now - item.timestamp) > max_age_seconds: continue if source is not None and item.source != source: continue results.append(item) return results @property def total_tokens(self) -> int: """Sum of token estimates across all items.""" return sum(item.token_estimate for item in self._items) def to_messages(self, max_tokens: int | None = None) -> list[Message]: """Convert ledger items to a list of Messages. If ``max_tokens`` is set, include items from most recent backward until the budget is exhausted. Args: max_tokens: Optional token budget. Returns: List of ``Message`` objects. """ if max_tokens is None: return [Message.assistant(item.content, name=item.source) for item in self._items] # Walk backward, accumulate until budget exhausted selected: list[ContextItem] = [] budget = max_tokens for item in reversed(self._items): if item.token_estimate <= budget: selected.append(item) budget -= item.token_estimate else: break selected.reverse() return [Message.assistant(item.content, name=item.source) for item in selected] def snapshot(self) -> dict: """Serialize the full ledger to a JSON-compatible dict.""" return { "items": [item.to_dict() for item in self._items], "total_tokens": self.total_tokens, "count": len(self._items), } @classmethod def from_snapshot(cls, data: dict) -> ContextLedger: """Restore a ledger from a snapshot dict.""" items = [ContextItem.from_dict(d) for d in data["items"]] return cls(items=items) @property def items(self) -> list[ContextItem]: return list(self._items) def __len__(self) -> int: return len(self._items) def __bool__(self) -> bool: return len(self._items) > 0 def clear(self) -> None: """Remove all items.""" self._items.clear() ``` #### `total_tokens` Sum of token estimates across all items. #### `append(item)` Add an item to the ledger. Source code in `packages/pyagent-context/src/pyagent_context/ledger.py` ``` def append(self, item: ContextItem) -> None: """Add an item to the ledger.""" self._items.append(item) ``` #### `add(content, source, trust_level=TrustLevel.INFERRED, **kwargs)` Create and append a new item in one step. Returns: | Type | Description | | ------------- | ------------------------ | | `ContextItem` | The created ContextItem. | Source code in `packages/pyagent-context/src/pyagent_context/ledger.py` ``` def add( self, content: str, source: str, trust_level: TrustLevel = TrustLevel.INFERRED, **kwargs: Any, ) -> ContextItem: """Create and append a new item in one step. Returns: The created ``ContextItem``. """ item = ContextItem(content=content, source=source, trust_level=trust_level, **kwargs) self._items.append(item) return item ``` #### `query(*, min_trust=None, max_age_seconds=None, source=None)` Filter items by trust level, age, and/or source. Parameters: | Name | Type | Description | Default | | ----------------- | ------------ | ----------- | ----------------------------------------------- | | `min_trust` | \`TrustLevel | None\` | Only return items at or above this trust level. | | `max_age_seconds` | \`float | None\` | Only return items newer than this. | | `source` | \`str | None\` | Only return items from this source. | Returns: | Type | Description | | ------------------- | --------------------------------------------- | | `list[ContextItem]` | Filtered list of items (chronological order). | Source code in `packages/pyagent-context/src/pyagent_context/ledger.py` ``` def query( self, *, min_trust: TrustLevel | None = None, max_age_seconds: float | None = None, source: str | None = None, ) -> list[ContextItem]: """Filter items by trust level, age, and/or source. Args: min_trust: Only return items at or above this trust level. max_age_seconds: Only return items newer than this. source: Only return items from this source. Returns: Filtered list of items (chronological order). """ now = time.time() results: list[ContextItem] = [] for item in self._items: if min_trust is not None and TRUST_ORDER.get(item.trust_level, 0) < TRUST_ORDER.get( min_trust, 0 ): continue if max_age_seconds is not None and (now - item.timestamp) > max_age_seconds: continue if source is not None and item.source != source: continue results.append(item) return results ``` #### `to_messages(max_tokens=None)` Convert ledger items to a list of Messages. If `max_tokens` is set, include items from most recent backward until the budget is exhausted. Parameters: | Name | Type | Description | Default | | ------------ | ----- | ----------- | ---------------------- | | `max_tokens` | \`int | None\` | Optional token budget. | Returns: | Type | Description | | --------------- | ------------------------ | | `list[Message]` | List of Message objects. | Source code in `packages/pyagent-context/src/pyagent_context/ledger.py` ``` def to_messages(self, max_tokens: int | None = None) -> list[Message]: """Convert ledger items to a list of Messages. If ``max_tokens`` is set, include items from most recent backward until the budget is exhausted. Args: max_tokens: Optional token budget. Returns: List of ``Message`` objects. """ if max_tokens is None: return [Message.assistant(item.content, name=item.source) for item in self._items] # Walk backward, accumulate until budget exhausted selected: list[ContextItem] = [] budget = max_tokens for item in reversed(self._items): if item.token_estimate <= budget: selected.append(item) budget -= item.token_estimate else: break selected.reverse() return [Message.assistant(item.content, name=item.source) for item in selected] ``` #### `snapshot()` Serialize the full ledger to a JSON-compatible dict. Source code in `packages/pyagent-context/src/pyagent_context/ledger.py` ``` def snapshot(self) -> dict: """Serialize the full ledger to a JSON-compatible dict.""" return { "items": [item.to_dict() for item in self._items], "total_tokens": self.total_tokens, "count": len(self._items), } ``` #### `from_snapshot(data)` Restore a ledger from a snapshot dict. Source code in `packages/pyagent-context/src/pyagent_context/ledger.py` ``` @classmethod def from_snapshot(cls, data: dict) -> ContextLedger: """Restore a ledger from a snapshot dict.""" items = [ContextItem.from_dict(d) for d in data["items"]] return cls(items=items) ``` #### `clear()` Remove all items. Source code in `packages/pyagent-context/src/pyagent_context/ledger.py` ``` def clear(self) -> None: """Remove all items.""" self._items.clear() ``` ### `ContextLifecycle` Manage the lifecycle of context items. Provides: - **Expiry sweep**: remove items past their `expires_at`. - **Freshness decay**: reduce token budgets for old items. - **Consolidation**: merge items from the same source with similar content. Parameters: | Name | Type | Description | Default | | ------------------------- | ------- | -------------------------------------------------------------------------------------- | ------- | | `consolidation_threshold` | `float` | Minimum keyword overlap ratio (0.0-1.0) to consider two items similar enough to merge. | `0.6` | Source code in `packages/pyagent-context/src/pyagent_context/lifecycle.py` ``` class ContextLifecycle: """Manage the lifecycle of context items. Provides: - **Expiry sweep**: remove items past their ``expires_at``. - **Freshness decay**: reduce token budgets for old items. - **Consolidation**: merge items from the same source with similar content. Args: consolidation_threshold: Minimum keyword overlap ratio (0.0-1.0) to consider two items similar enough to merge. """ def __init__(self, consolidation_threshold: float = 0.6) -> None: self._consolidation_threshold = consolidation_threshold def sweep_expired(self, ledger: ContextLedger) -> tuple[ContextLedger, list[ContextItem]]: """Remove expired items from the ledger. Args: ledger: The context ledger. Returns: Tuple of (new ledger without expired items, list of expired items). """ now = time.time() kept: list[ContextItem] = [] expired: list[ContextItem] = [] for item in ledger.items: if item.expires_at is not None and now > item.expires_at: expired.append(item) else: kept.append(item) return ContextLedger(items=kept), expired def apply_freshness_decay( self, ledger: ContextLedger, half_life_seconds: float = 3600.0, min_tokens: int = 1, ) -> ContextLedger: """Reduce token estimates based on age. Older items get smaller token budgets, making them less likely to survive compression. Items retain at least ``min_tokens``. Args: ledger: The context ledger. half_life_seconds: Seconds for token estimate to halve. min_tokens: Minimum token estimate after decay. Returns: New ledger with decayed token estimates. """ import math now = time.time() new_items: list[ContextItem] = [] for item in ledger.items: age = now - item.timestamp decay_factor = math.exp(-age / half_life_seconds) if half_life_seconds > 0 else 1.0 decayed_tokens = max(min_tokens, int(item.token_estimate * decay_factor)) new_item = ContextItem( content=item.content, source=item.source, timestamp=item.timestamp, trust_level=item.trust_level, sensitivity=item.sensitivity, expires_at=item.expires_at, derived_from=item.derived_from, token_estimate=decayed_tokens, ) new_item._id = item.id new_items.append(new_item) return ContextLedger(items=new_items) def consolidate(self, ledger: ContextLedger) -> ContextLedger: """Merge similar items from the same source. When two items from the same source have high keyword overlap, they are merged into one with combined content and the higher trust level. Args: ledger: The context ledger. Returns: New ledger with consolidated items. """ by_source: dict[str, list[ContextItem]] = defaultdict(list) for item in ledger.items: by_source[item.source].append(item) consolidated: list[ContextItem] = [] for source, items in by_source.items(): merged_indices: set[int] = set() for i, item_a in enumerate(items): if i in merged_indices: continue merged_content = item_a.content best_trust = item_a.trust_level latest_time = item_a.timestamp for j in range(i + 1, len(items)): if j in merged_indices: continue item_b = items[j] if ( self._similarity(item_a.content, item_b.content) >= self._consolidation_threshold ): merged_content = f"{merged_content}\n{item_b.content}" if item_b.trust_level > best_trust: best_trust = item_b.trust_level latest_time = max(latest_time, item_b.timestamp) merged_indices.add(j) new_item = ContextItem( content=merged_content, source=source, timestamp=latest_time, trust_level=best_trust, sensitivity=item_a.sensitivity, expires_at=item_a.expires_at, derived_from=item_a.id, ) consolidated.append(new_item) # Sort by timestamp to maintain chronological order consolidated.sort(key=lambda x: x.timestamp) return ContextLedger(items=consolidated) @staticmethod def _similarity(text_a: str, text_b: str) -> float: """Keyword overlap ratio (Jaccard-like).""" words_a = set(text_a.lower().split()) words_b = set(text_b.lower().split()) if not words_a or not words_b: return 0.0 intersection = words_a & words_b union = words_a | words_b return len(intersection) / len(union) ``` #### `sweep_expired(ledger)` Remove expired items from the ledger. Parameters: | Name | Type | Description | Default | | -------- | --------------- | ------------------- | ---------- | | `ledger` | `ContextLedger` | The context ledger. | *required* | Returns: | Type | Description | | ----------------------------------------- | ------------------------------------------------------------------- | | `tuple[ContextLedger, list[ContextItem]]` | Tuple of (new ledger without expired items, list of expired items). | Source code in `packages/pyagent-context/src/pyagent_context/lifecycle.py` ``` def sweep_expired(self, ledger: ContextLedger) -> tuple[ContextLedger, list[ContextItem]]: """Remove expired items from the ledger. Args: ledger: The context ledger. Returns: Tuple of (new ledger without expired items, list of expired items). """ now = time.time() kept: list[ContextItem] = [] expired: list[ContextItem] = [] for item in ledger.items: if item.expires_at is not None and now > item.expires_at: expired.append(item) else: kept.append(item) return ContextLedger(items=kept), expired ``` #### `apply_freshness_decay(ledger, half_life_seconds=3600.0, min_tokens=1)` Reduce token estimates based on age. Older items get smaller token budgets, making them less likely to survive compression. Items retain at least `min_tokens`. Parameters: | Name | Type | Description | Default | | ------------------- | --------------- | ------------------------------------ | ---------- | | `ledger` | `ContextLedger` | The context ledger. | *required* | | `half_life_seconds` | `float` | Seconds for token estimate to halve. | `3600.0` | | `min_tokens` | `int` | Minimum token estimate after decay. | `1` | Returns: | Type | Description | | --------------- | ---------------------------------------- | | `ContextLedger` | New ledger with decayed token estimates. | Source code in `packages/pyagent-context/src/pyagent_context/lifecycle.py` ``` def apply_freshness_decay( self, ledger: ContextLedger, half_life_seconds: float = 3600.0, min_tokens: int = 1, ) -> ContextLedger: """Reduce token estimates based on age. Older items get smaller token budgets, making them less likely to survive compression. Items retain at least ``min_tokens``. Args: ledger: The context ledger. half_life_seconds: Seconds for token estimate to halve. min_tokens: Minimum token estimate after decay. Returns: New ledger with decayed token estimates. """ import math now = time.time() new_items: list[ContextItem] = [] for item in ledger.items: age = now - item.timestamp decay_factor = math.exp(-age / half_life_seconds) if half_life_seconds > 0 else 1.0 decayed_tokens = max(min_tokens, int(item.token_estimate * decay_factor)) new_item = ContextItem( content=item.content, source=item.source, timestamp=item.timestamp, trust_level=item.trust_level, sensitivity=item.sensitivity, expires_at=item.expires_at, derived_from=item.derived_from, token_estimate=decayed_tokens, ) new_item._id = item.id new_items.append(new_item) return ContextLedger(items=new_items) ``` #### `consolidate(ledger)` Merge similar items from the same source. When two items from the same source have high keyword overlap, they are merged into one with combined content and the higher trust level. Parameters: | Name | Type | Description | Default | | -------- | --------------- | ------------------- | ---------- | | `ledger` | `ContextLedger` | The context ledger. | *required* | Returns: | Type | Description | | --------------- | ----------------------------------- | | `ContextLedger` | New ledger with consolidated items. | Source code in `packages/pyagent-context/src/pyagent_context/lifecycle.py` ``` def consolidate(self, ledger: ContextLedger) -> ContextLedger: """Merge similar items from the same source. When two items from the same source have high keyword overlap, they are merged into one with combined content and the higher trust level. Args: ledger: The context ledger. Returns: New ledger with consolidated items. """ by_source: dict[str, list[ContextItem]] = defaultdict(list) for item in ledger.items: by_source[item.source].append(item) consolidated: list[ContextItem] = [] for source, items in by_source.items(): merged_indices: set[int] = set() for i, item_a in enumerate(items): if i in merged_indices: continue merged_content = item_a.content best_trust = item_a.trust_level latest_time = item_a.timestamp for j in range(i + 1, len(items)): if j in merged_indices: continue item_b = items[j] if ( self._similarity(item_a.content, item_b.content) >= self._consolidation_threshold ): merged_content = f"{merged_content}\n{item_b.content}" if item_b.trust_level > best_trust: best_trust = item_b.trust_level latest_time = max(latest_time, item_b.timestamp) merged_indices.add(j) new_item = ContextItem( content=merged_content, source=source, timestamp=latest_time, trust_level=best_trust, sensitivity=item_a.sensitivity, expires_at=item_a.expires_at, derived_from=item_a.id, ) consolidated.append(new_item) # Sort by timestamp to maintain chronological order consolidated.sort(key=lambda x: x.timestamp) return ContextLedger(items=consolidated) ``` ### `InMemorySemanticStore` In-memory semantic store using TF-IDF cosine similarity. No external dependencies — suitable for testing and small datasets. For production, use a vector DB adapter (ChromaDB, Pinecone, etc.). Parameters: | Name | Type | Description | Default | | ------------ | ---------- | ----------- | ------------------------------------------- | | `stop_words` | \`set[str] | None\` | Optional set of words to ignore in scoring. | Source code in `packages/pyagent-context/src/pyagent_context/memory/semantic.py` ``` class InMemorySemanticStore: """In-memory semantic store using TF-IDF cosine similarity. No external dependencies — suitable for testing and small datasets. For production, use a vector DB adapter (ChromaDB, Pinecone, etc.). Args: stop_words: Optional set of words to ignore in scoring. """ def __init__(self, stop_words: set[str] | None = None) -> None: self._items: dict[str, ContextItem] = {} self._tf_cache: dict[str, dict[str, float]] = {} self._stop_words = stop_words or { "a", "an", "the", "is", "are", "was", "were", "be", "been", "being", "have", "has", "had", "do", "does", "did", "will", "would", "could", "should", "may", "might", "can", "shall", "to", "of", "in", "for", "on", "with", "at", "by", "from", "as", "into", "through", "during", "before", "after", "and", "but", "or", "nor", "not", "so", "yet", "both", "either", "neither", "each", "every", "all", "any", "few", "more", "most", "other", "some", "such", "no", "only", "own", "same", "than", "too", "very", "just", "because", "about", "between", "it", "its", "this", "that", "these", "those", "i", "me", "my", "we", "our", "you", "your", "he", "him", "his", "she", "her", "they", "them", "their", "what", "which", "who", } def add(self, item: ContextItem) -> None: self._items[item.id] = item self._tf_cache[item.id] = self._compute_tf(item.content) def search(self, query: str, top_k: int = 5) -> list[SearchResult]: if not self._items: return [] query_tf = self._compute_tf(query) idf = self._compute_idf() query_tfidf = {w: tf * idf.get(w, 0.0) for w, tf in query_tf.items()} results: list[SearchResult] = [] for item_id, item in self._items.items(): doc_tf = self._tf_cache.get(item_id, {}) doc_tfidf = {w: tf * idf.get(w, 0.0) for w, tf in doc_tf.items()} score = self._cosine_similarity(query_tfidf, doc_tfidf) if score > 0: results.append(SearchResult(item=item, score=score)) results.sort(key=lambda r: r.score, reverse=True) return results[:top_k] def remove(self, item_id: str) -> bool: if item_id in self._items: del self._items[item_id] self._tf_cache.pop(item_id, None) return True return False def clear(self) -> None: self._items.clear() self._tf_cache.clear() def __len__(self) -> int: return len(self._items) # ------------------------------------------------------------------ # TF-IDF helpers # ------------------------------------------------------------------ def _tokenize(self, text: str) -> list[str]: words = text.lower().split() return [ w.strip(".,!?;:\"'()[]{}") for w in words if w.strip(".,!?;:\"'()[]{}") not in self._stop_words ] def _compute_tf(self, text: str) -> dict[str, float]: tokens = self._tokenize(text) if not tokens: return {} counts = Counter(tokens) total = len(tokens) return {word: count / total for word, count in counts.items()} def _compute_idf(self) -> dict[str, float]: n_docs = len(self._items) if n_docs == 0: return {} doc_freq: Counter[str] = Counter() for tf in self._tf_cache.values(): for word in tf: doc_freq[word] += 1 return {word: math.log(n_docs / (1 + freq)) for word, freq in doc_freq.items()} @staticmethod def _cosine_similarity(a: dict[str, float], b: dict[str, float]) -> float: if not a or not b: return 0.0 common_keys = set(a) & set(b) if not common_keys: return 0.0 dot = sum(a[k] * b[k] for k in common_keys) mag_a = math.sqrt(sum(v**2 for v in a.values())) mag_b = math.sqrt(sum(v**2 for v in b.values())) if mag_a == 0 or mag_b == 0: return 0.0 return dot / (mag_a * mag_b) ``` ### `SemanticMemoryProtocol` Bases: `Protocol` Interface for any vector-backed semantic memory store. Source code in `packages/pyagent-context/src/pyagent_context/memory/semantic.py` ``` @runtime_checkable class SemanticMemoryProtocol(Protocol): """Interface for any vector-backed semantic memory store.""" def add(self, item: ContextItem) -> None: """Index an item for later retrieval.""" ... def search(self, query: str, top_k: int = 5) -> list[SearchResult]: """Find the most relevant items for a query.""" ... def remove(self, item_id: str) -> bool: """Remove an item by ID. Returns True if found.""" ... def clear(self) -> None: """Remove all items.""" ... def __len__(self) -> int: ... ``` #### `add(item)` Index an item for later retrieval. Source code in `packages/pyagent-context/src/pyagent_context/memory/semantic.py` ``` def add(self, item: ContextItem) -> None: """Index an item for later retrieval.""" ... ``` #### `search(query, top_k=5)` Find the most relevant items for a query. Source code in `packages/pyagent-context/src/pyagent_context/memory/semantic.py` ``` def search(self, query: str, top_k: int = 5) -> list[SearchResult]: """Find the most relevant items for a query.""" ... ``` #### `remove(item_id)` Remove an item by ID. Returns True if found. Source code in `packages/pyagent-context/src/pyagent_context/memory/semantic.py` ``` def remove(self, item_id: str) -> bool: """Remove an item by ID. Returns True if found.""" ... ``` #### `clear()` Remove all items. Source code in `packages/pyagent-context/src/pyagent_context/memory/semantic.py` ``` def clear(self) -> None: """Remove all items.""" ... ``` ### `SessionMemory` Session-scoped memory that persists across turns. Supports two backends: - `json`: simple JSON file per session (default) - `sqlite`: SQLite database for concurrent-safe access Parameters: | Name | Type | Description | Default | | -------------- | ----- | -------------------------- | ------------------------------------------------------------- | | `session_id` | `str` | Unique session identifier. | *required* | | `backend` | `str` | "json" or "sqlite". | `'json'` | | `storage_path` | \`str | Path\` | Directory for storage files. Defaults to ".pyagent_sessions". | Source code in `packages/pyagent-context/src/pyagent_context/memory/session.py` ``` class SessionMemory: """Session-scoped memory that persists across turns. Supports two backends: - ``json``: simple JSON file per session (default) - ``sqlite``: SQLite database for concurrent-safe access Args: session_id: Unique session identifier. backend: ``"json"`` or ``"sqlite"``. storage_path: Directory for storage files. Defaults to ``".pyagent_sessions"``. """ def __init__( self, session_id: str, backend: str = "json", storage_path: str | Path = ".pyagent_sessions", ) -> None: self._session_id = session_id self._backend = backend self._storage_path = Path(storage_path) self._storage_path.mkdir(parents=True, exist_ok=True) self._items: list[ContextItem] = [] self._loaded = False @property def session_id(self) -> str: return self._session_id def add(self, item: ContextItem) -> None: """Add an item to the session.""" self._ensure_loaded() self._items.append(item) def get_all(self) -> list[ContextItem]: """Return all items in this session.""" self._ensure_loaded() return list(self._items) def save(self) -> None: """Persist current items to storage.""" if self._backend == "sqlite": self._save_sqlite() else: self._save_json() def load(self) -> None: """Load items from storage.""" if self._backend == "sqlite": self._load_sqlite() else: self._load_json() self._loaded = True def clear(self) -> None: """Remove all items and delete persisted data.""" self._items.clear() path = self._file_path if path.exists(): path.unlink() # ------------------------------------------------------------------ # JSON backend # ------------------------------------------------------------------ def _save_json(self) -> None: path = self._file_path data = [item.to_dict() for item in self._items] path.write_text(json.dumps(data, indent=2)) def _load_json(self) -> None: path = self._file_path if path.exists(): data = json.loads(path.read_text()) self._items = [ContextItem.from_dict(d) for d in data] else: self._items = [] # ------------------------------------------------------------------ # SQLite backend # ------------------------------------------------------------------ def _save_sqlite(self) -> None: db_path = self._storage_path / "sessions.db" conn = sqlite3.connect(str(db_path)) try: conn.execute( "CREATE TABLE IF NOT EXISTS context_items " "(session_id TEXT, item_id TEXT PRIMARY KEY, data TEXT)" ) conn.execute( "DELETE FROM context_items WHERE session_id = ?", (self._session_id,), ) for item in self._items: conn.execute( "INSERT INTO context_items (session_id, item_id, data) VALUES (?, ?, ?)", (self._session_id, item.id, json.dumps(item.to_dict())), ) conn.commit() finally: conn.close() def _load_sqlite(self) -> None: db_path = self._storage_path / "sessions.db" if not db_path.exists(): self._items = [] return conn = sqlite3.connect(str(db_path)) try: conn.execute( "CREATE TABLE IF NOT EXISTS context_items " "(session_id TEXT, item_id TEXT PRIMARY KEY, data TEXT)" ) cursor = conn.execute( "SELECT data FROM context_items WHERE session_id = ? ORDER BY rowid", (self._session_id,), ) self._items = [ContextItem.from_dict(json.loads(row[0])) for row in cursor] finally: conn.close() # ------------------------------------------------------------------ # Helpers # ------------------------------------------------------------------ @property def _file_path(self) -> Path: suffix = ".db" if self._backend == "sqlite" else ".json" return self._storage_path / f"{self._session_id}{suffix}" def _ensure_loaded(self) -> None: if not self._loaded: self.load() def __len__(self) -> int: self._ensure_loaded() return len(self._items) ``` #### `add(item)` Add an item to the session. Source code in `packages/pyagent-context/src/pyagent_context/memory/session.py` ``` def add(self, item: ContextItem) -> None: """Add an item to the session.""" self._ensure_loaded() self._items.append(item) ``` #### `get_all()` Return all items in this session. Source code in `packages/pyagent-context/src/pyagent_context/memory/session.py` ``` def get_all(self) -> list[ContextItem]: """Return all items in this session.""" self._ensure_loaded() return list(self._items) ``` #### `save()` Persist current items to storage. Source code in `packages/pyagent-context/src/pyagent_context/memory/session.py` ``` def save(self) -> None: """Persist current items to storage.""" if self._backend == "sqlite": self._save_sqlite() else: self._save_json() ``` #### `load()` Load items from storage. Source code in `packages/pyagent-context/src/pyagent_context/memory/session.py` ``` def load(self) -> None: """Load items from storage.""" if self._backend == "sqlite": self._load_sqlite() else: self._load_json() self._loaded = True ``` #### `clear()` Remove all items and delete persisted data. Source code in `packages/pyagent-context/src/pyagent_context/memory/session.py` ``` def clear(self) -> None: """Remove all items and delete persisted data.""" self._items.clear() path = self._file_path if path.exists(): path.unlink() ``` ### `WorkingMemory` Bounded working memory for a single pattern run. Evicts the oldest items when capacity is exceeded (either by count or by total token estimate). Parameters: | Name | Type | Description | Default | | ------------ | ----- | --------------------------------------------- | ------- | | `max_items` | `int` | Maximum number of items to keep. | `100` | | `max_tokens` | `int` | Maximum total token estimate before eviction. | `50000` | Source code in `packages/pyagent-context/src/pyagent_context/memory/working.py` ``` class WorkingMemory: """Bounded working memory for a single pattern run. Evicts the oldest items when capacity is exceeded (either by count or by total token estimate). Args: max_items: Maximum number of items to keep. max_tokens: Maximum total token estimate before eviction. """ def __init__( self, max_items: int = 100, max_tokens: int = 50_000, ) -> None: self._max_items = max_items self._max_tokens = max_tokens self._items: deque[ContextItem] = deque() def add(self, item: ContextItem) -> list[ContextItem]: """Add an item, evicting oldest if necessary. Returns: List of evicted items (empty if none were evicted). """ evicted: list[ContextItem] = [] self._items.append(item) # Evict by count while len(self._items) > self._max_items: evicted.append(self._items.popleft()) # Evict by token budget while self.total_tokens > self._max_tokens and len(self._items) > 1: evicted.append(self._items.popleft()) return evicted @property def total_tokens(self) -> int: return sum(item.token_estimate for item in self._items) @property def items(self) -> list[ContextItem]: return list(self._items) @property def utilization(self) -> float: """Token utilization as a fraction of max_tokens.""" return self.total_tokens / self._max_tokens if self._max_tokens > 0 else 0.0 def clear(self) -> None: self._items.clear() def __len__(self) -> int: return len(self._items) def __bool__(self) -> bool: return len(self._items) > 0 ``` #### `utilization` Token utilization as a fraction of max_tokens. #### `add(item)` Add an item, evicting oldest if necessary. Returns: | Type | Description | | ------------------- | --------------------------------------------------- | | `list[ContextItem]` | List of evicted items (empty if none were evicted). | Source code in `packages/pyagent-context/src/pyagent_context/memory/working.py` ``` def add(self, item: ContextItem) -> list[ContextItem]: """Add an item, evicting oldest if necessary. Returns: List of evicted items (empty if none were evicted). """ evicted: list[ContextItem] = [] self._items.append(item) # Evict by count while len(self._items) > self._max_items: evicted.append(self._items.popleft()) # Evict by token budget while self.total_tokens > self._max_tokens and len(self._items) > 1: evicted.append(self._items.popleft()) return evicted ``` ### `ContextRedactor` Redact or filter context items based on sensitivity. Items above the allowed sensitivity threshold are either fully redacted (content replaced) or excluded entirely. Parameters: | Name | Type | Description | Default | | ----------------- | ------------- | ---------------------------------------------------------------------- | -------------- | | `max_sensitivity` | `Sensitivity` | Maximum allowed sensitivity level. Items above this are redacted. | `INTERNAL` | | `redaction_text` | `str` | Replacement text for redacted items. | `'[REDACTED]'` | | `exclude_above` | `bool` | If True, items above max_sensitivity are excluded instead of redacted. | `False` | Source code in `packages/pyagent-context/src/pyagent_context/redaction.py` ``` class ContextRedactor: """Redact or filter context items based on sensitivity. Items above the allowed sensitivity threshold are either fully redacted (content replaced) or excluded entirely. Args: max_sensitivity: Maximum allowed sensitivity level. Items above this are redacted. redaction_text: Replacement text for redacted items. exclude_above: If ``True``, items above max_sensitivity are excluded instead of redacted. """ def __init__( self, max_sensitivity: Sensitivity = Sensitivity.INTERNAL, redaction_text: str = "[REDACTED]", exclude_above: bool = False, ) -> None: self._max_sensitivity = max_sensitivity self._redaction_text = redaction_text self._exclude_above = exclude_above def redact_item(self, item: ContextItem) -> ContextItem | None: """Redact a single item if it exceeds the sensitivity threshold. Returns: The original item if within threshold, a redacted copy if above, or ``None`` if ``exclude_above`` is ``True`` and the item exceeds. """ item_level = SENSITIVITY_ORDER.get(item.sensitivity, 0) max_level = SENSITIVITY_ORDER.get(self._max_sensitivity, 0) if item_level <= max_level: return item if self._exclude_above: return None return ContextItem( content=self._redaction_text, source=item.source, timestamp=item.timestamp, trust_level=item.trust_level, sensitivity=item.sensitivity, expires_at=item.expires_at, derived_from=item.id, token_estimate=max(1, len(self._redaction_text) // 4), ) def redact_ledger(self, ledger: ContextLedger) -> ContextLedger: """Apply redaction to all items in a ledger. Returns: New ledger with redacted/filtered items. """ new_items: list[ContextItem] = [] for item in ledger.items: result = self.redact_item(item) if result is not None: new_items.append(result) return ContextLedger(items=new_items) ``` #### `redact_item(item)` Redact a single item if it exceeds the sensitivity threshold. Returns: | Type | Description | | ------------- | ----------- | | \`ContextItem | None\` | | \`ContextItem | None\` | Source code in `packages/pyagent-context/src/pyagent_context/redaction.py` ``` def redact_item(self, item: ContextItem) -> ContextItem | None: """Redact a single item if it exceeds the sensitivity threshold. Returns: The original item if within threshold, a redacted copy if above, or ``None`` if ``exclude_above`` is ``True`` and the item exceeds. """ item_level = SENSITIVITY_ORDER.get(item.sensitivity, 0) max_level = SENSITIVITY_ORDER.get(self._max_sensitivity, 0) if item_level <= max_level: return item if self._exclude_above: return None return ContextItem( content=self._redaction_text, source=item.source, timestamp=item.timestamp, trust_level=item.trust_level, sensitivity=item.sensitivity, expires_at=item.expires_at, derived_from=item.id, token_estimate=max(1, len(self._redaction_text) // 4), ) ``` #### `redact_ledger(ledger)` Apply redaction to all items in a ledger. Returns: | Type | Description | | --------------- | ---------------------------------------- | | `ContextLedger` | New ledger with redacted/filtered items. | Source code in `packages/pyagent-context/src/pyagent_context/redaction.py` ``` def redact_ledger(self, ledger: ContextLedger) -> ContextLedger: """Apply redaction to all items in a ledger. Returns: New ledger with redacted/filtered items. """ new_items: list[ContextItem] = [] for item in ledger.items: result = self.redact_item(item) if result is not None: new_items.append(result) return ContextLedger(items=new_items) ``` ### `ScoredItem` A context item with a composite retrieval score. Attributes: | Name | Type | Description | | ----------------- | ------------- | --------------------------------- | | `item` | `ContextItem` | The context item. | | `score` | `float` | Composite score (0.0-1.0). | | `trust_score` | `float` | Trust component of the score. | | `recency_score` | `float` | Recency component of the score. | | `relevance_score` | `float` | Relevance component of the score. | Source code in `packages/pyagent-context/src/pyagent_context/retrieval.py` ``` @dataclass(frozen=True) class ScoredItem: """A context item with a composite retrieval score. Attributes: item: The context item. score: Composite score (0.0-1.0). trust_score: Trust component of the score. recency_score: Recency component of the score. relevance_score: Relevance component of the score. """ item: ContextItem score: float trust_score: float recency_score: float relevance_score: float ``` ### `TrustAwareRetriever` Retrieve context items scored by trust, recency, and keyword relevance. Scoring formula `score = w_trust * trust + w_recency * recency + w_relevance * relevance` Where: - **trust** = `TRUST_ORDER[item.trust_level] / 3.0` - **recency** = `exp(-age_seconds / half_life)` - **relevance** = keyword overlap ratio with query Parameters: | Name | Type | Description | Default | | ------------------- | ------- | ----------------------------------- | -------- | | `weight_trust` | `float` | Weight for trust component. | `0.3` | | `weight_recency` | `float` | Weight for recency component. | `0.3` | | `weight_relevance` | `float` | Weight for relevance component. | `0.4` | | `recency_half_life` | `float` | Seconds for recency score to halve. | `3600.0` | Source code in `packages/pyagent-context/src/pyagent_context/retrieval.py` ``` class TrustAwareRetriever: """Retrieve context items scored by trust, recency, and keyword relevance. Scoring formula: ``score = w_trust * trust + w_recency * recency + w_relevance * relevance`` Where: - **trust** = ``TRUST_ORDER[item.trust_level] / 3.0`` - **recency** = ``exp(-age_seconds / half_life)`` - **relevance** = keyword overlap ratio with query Args: weight_trust: Weight for trust component. weight_recency: Weight for recency component. weight_relevance: Weight for relevance component. recency_half_life: Seconds for recency score to halve. """ def __init__( self, weight_trust: float = 0.3, weight_recency: float = 0.3, weight_relevance: float = 0.4, recency_half_life: float = 3600.0, ) -> None: self._w_trust = weight_trust self._w_recency = weight_recency self._w_relevance = weight_relevance self._half_life = recency_half_life def retrieve( self, ledger: ContextLedger, query: str, *, top_k: int = 10, min_score: float = 0.0, ) -> list[ScoredItem]: """Score and rank all items in the ledger against a query. Args: ledger: The context ledger to search. query: The query string for relevance scoring. top_k: Maximum results to return. min_score: Minimum composite score to include. Returns: Ranked list of ``ScoredItem`` objects. """ now = time.time() query_words = set(query.lower().split()) results: list[ScoredItem] = [] for item in ledger.items: if item.is_expired: continue trust = TRUST_ORDER.get(item.trust_level, 0) / 3.0 age = now - item.timestamp recency = math.exp(-age / self._half_life) if self._half_life > 0 else 0.0 relevance = self._keyword_relevance(item.content, query_words) score = ( self._w_trust * trust + self._w_recency * recency + self._w_relevance * relevance ) if score >= min_score: results.append( ScoredItem( item=item, score=score, trust_score=trust, recency_score=recency, relevance_score=relevance, ) ) results.sort(key=lambda r: r.score, reverse=True) return results[:top_k] @staticmethod def _keyword_relevance(content: str, query_words: set[str]) -> float: """Simple keyword overlap ratio.""" if not query_words: return 0.0 content_words = set(content.lower().split()) overlap = query_words & content_words return len(overlap) / len(query_words) if query_words else 0.0 ``` #### `retrieve(ledger, query, *, top_k=10, min_score=0.0)` Score and rank all items in the ledger against a query. Parameters: | Name | Type | Description | Default | | ----------- | --------------- | --------------------------------------- | ---------- | | `ledger` | `ContextLedger` | The context ledger to search. | *required* | | `query` | `str` | The query string for relevance scoring. | *required* | | `top_k` | `int` | Maximum results to return. | `10` | | `min_score` | `float` | Minimum composite score to include. | `0.0` | Returns: | Type | Description | | ------------------ | ---------------------------------- | | `list[ScoredItem]` | Ranked list of ScoredItem objects. | Source code in `packages/pyagent-context/src/pyagent_context/retrieval.py` ``` def retrieve( self, ledger: ContextLedger, query: str, *, top_k: int = 10, min_score: float = 0.0, ) -> list[ScoredItem]: """Score and rank all items in the ledger against a query. Args: ledger: The context ledger to search. query: The query string for relevance scoring. top_k: Maximum results to return. min_score: Minimum composite score to include. Returns: Ranked list of ``ScoredItem`` objects. """ now = time.time() query_words = set(query.lower().split()) results: list[ScoredItem] = [] for item in ledger.items: if item.is_expired: continue trust = TRUST_ORDER.get(item.trust_level, 0) / 3.0 age = now - item.timestamp recency = math.exp(-age / self._half_life) if self._half_life > 0 else 0.0 relevance = self._keyword_relevance(item.content, query_words) score = ( self._w_trust * trust + self._w_recency * recency + self._w_relevance * relevance ) if score >= min_score: results.append( ScoredItem( item=item, score=score, trust_score=trust, recency_score=recency, relevance_score=relevance, ) ) results.sort(key=lambda r: r.score, reverse=True) return results[:top_k] ``` # pyagent-providers API Reference ## `pyagent_providers` PyAgent Providers — multi-provider abstraction with capability negotiation and fallback chains. ### `HealthStatus` Bases: `StrEnum` Health status of a provider. Source code in `packages/pyagent-providers/src/pyagent_providers/base.py` ``` class HealthStatus(StrEnum): """Health status of a provider.""" HEALTHY = "healthy" DEGRADED = "degraded" UNHEALTHY = "unhealthy" ``` ### `ProviderCapabilities` Declares what a provider can do. Attributes: | Name | Type | Description | | -------------------- | ----------------- | ---------------------------------------------------------- | | `models` | `list[str]` | List of model identifiers this provider exposes. | | `capabilities` | `set[Capability]` | Set of capability tags (reuses pyagent_router.Capability). | | `max_context` | `int` | Maximum context window in tokens across all models. | | `supports_streaming` | `bool` | Whether the provider supports token-level streaming. | | `supports_tools` | `bool` | Whether the provider supports tool/function calling. | | `supports_vision` | `bool` | Whether the provider supports image inputs. | Source code in `packages/pyagent-providers/src/pyagent_providers/base.py` ``` @dataclass(frozen=True) class ProviderCapabilities: """Declares what a provider can do. Attributes: models: List of model identifiers this provider exposes. capabilities: Set of capability tags (reuses pyagent_router.Capability). max_context: Maximum context window in tokens across all models. supports_streaming: Whether the provider supports token-level streaming. supports_tools: Whether the provider supports tool/function calling. supports_vision: Whether the provider supports image inputs. """ models: list[str] capabilities: set[Capability] = field(default_factory=lambda: {Capability.GENERAL}) max_context: int = 128_000 supports_streaming: bool = True supports_tools: bool = False supports_vision: bool = False ``` ### `ProviderInfo` Snapshot of provider state returned by the registry. Attributes: | Name | Type | Description | | -------------- | ---------------------- | ------------------------------------------------ | | `name` | `str` | Provider name. | | `capabilities` | `ProviderCapabilities` | Provider capabilities. | | `health` | `HealthStatus` | Current health status. | | `metadata` | `dict[str, Any]` | Arbitrary extra metadata (e.g. region, version). | Source code in `packages/pyagent-providers/src/pyagent_providers/base.py` ``` @dataclass(frozen=True) class ProviderInfo: """Snapshot of provider state returned by the registry. Attributes: name: Provider name. capabilities: Provider capabilities. health: Current health status. metadata: Arbitrary extra metadata (e.g. region, version). """ name: str capabilities: ProviderCapabilities health: HealthStatus metadata: dict[str, Any] = field(default_factory=dict) ``` ### `ProviderProtocol` Bases: `Protocol` Interface that every LLM provider must implement. Also satisfies `LLMCallable` via `__call__` which delegates to `complete` with a default model. Source code in `packages/pyagent-providers/src/pyagent_providers/base.py` ``` @runtime_checkable class ProviderProtocol(Protocol): """Interface that every LLM provider must implement. Also satisfies ``LLMCallable`` via ``__call__`` which delegates to ``complete`` with a default model. """ @property def name(self) -> str: """Unique identifier for this provider.""" ... @property def capabilities(self) -> ProviderCapabilities: """Declared capabilities.""" ... async def health(self) -> HealthStatus: """Check current health of the provider endpoint.""" ... async def complete(self, messages: list[Message], model: str | None = None) -> str: """Generate a completion. Args: messages: Conversation history. model: Optional model override. Uses provider default if ``None``. Returns: The assistant response text. """ ... async def __call__(self, messages: list[Message]) -> str: """LLMCallable-compatible entry point — delegates to ``complete``.""" ... ``` #### `name` Unique identifier for this provider. #### `capabilities` Declared capabilities. #### `health()` Check current health of the provider endpoint. Source code in `packages/pyagent-providers/src/pyagent_providers/base.py` ``` async def health(self) -> HealthStatus: """Check current health of the provider endpoint.""" ... ``` #### `complete(messages, model=None)` Generate a completion. Parameters: | Name | Type | Description | Default | | ---------- | --------------- | --------------------- | ------------------------------------------------------- | | `messages` | `list[Message]` | Conversation history. | *required* | | `model` | \`str | None\` | Optional model override. Uses provider default if None. | Returns: | Type | Description | | ----- | ---------------------------- | | `str` | The assistant response text. | Source code in `packages/pyagent-providers/src/pyagent_providers/base.py` ``` async def complete(self, messages: list[Message], model: str | None = None) -> str: """Generate a completion. Args: messages: Conversation history. model: Optional model override. Uses provider default if ``None``. Returns: The assistant response text. """ ... ``` #### `__call__(messages)` LLMCallable-compatible entry point — delegates to `complete`. Source code in `packages/pyagent-providers/src/pyagent_providers/base.py` ``` async def __call__(self, messages: list[Message]) -> str: """LLMCallable-compatible entry point — delegates to ``complete``.""" ... ``` ### `CostOptimizer` Compare costs across all registered providers. Wraps `CostEstimator` from `pyagent-router` and iterates over every provider's model list to find the cheapest option. Parameters: | Name | Type | Description | Default | | ---------------- | ------------------ | -------------------------------------- | -------------------------------- | | `registry` | `ProviderRegistry` | Provider registry to draw models from. | *required* | | `cost_estimator` | \`CostEstimator | None\` | Optional CostEstimator instance. | Source code in `packages/pyagent-providers/src/pyagent_providers/cost.py` ``` class CostOptimizer: """Compare costs across all registered providers. Wraps ``CostEstimator`` from ``pyagent-router`` and iterates over every provider's model list to find the cheapest option. Args: registry: Provider registry to draw models from. cost_estimator: Optional ``CostEstimator`` instance. """ def __init__( self, registry: ProviderRegistry, cost_estimator: CostEstimator | None = None, ) -> None: self._registry = registry self._estimator = cost_estimator or CostEstimator() def compare( self, task: str, *, healthy_only: bool = True, limit: int = 10, ) -> list[ProviderCostEstimate]: """Estimate cost for every provider + model pair and rank cheapest first. Args: task: The task text to estimate. healthy_only: If ``True`` (default), only include healthy providers. limit: Maximum number of results to return. Returns: Sorted list of ``ProviderCostEstimate`` objects, cheapest first. """ candidates = self._registry.discover(healthy_only=healthy_only) results: list[ProviderCostEstimate] = [] for provider in candidates: for model_name in provider.capabilities.models: try: est = self._estimator.estimate_from_text(model_name, task) results.append( ProviderCostEstimate( provider_name=provider.name, model=model_name, estimate=est, ) ) except KeyError: continue results.sort(key=lambda x: x.estimate.total_cost) return results[:limit] def cheapest( self, task: str, *, healthy_only: bool = True, ) -> ProviderCostEstimate | None: """Return the single cheapest provider + model for a task. Returns: The cheapest option, or ``None`` if no estimates could be computed. """ estimates = self.compare(task, healthy_only=healthy_only, limit=1) return estimates[0] if estimates else None def cheapest_provider( self, task: str, *, healthy_only: bool = True, ) -> tuple[ProviderProtocol, str] | None: """Return the cheapest provider object + model name. Convenience method for passing directly to ``Agent`` or patterns. """ est = self.cheapest(task, healthy_only=healthy_only) if est is None: return None provider = self._registry.get(est.provider_name) if provider is None: return None return provider, est.model ``` #### `compare(task, *, healthy_only=True, limit=10)` Estimate cost for every provider + model pair and rank cheapest first. Parameters: | Name | Type | Description | Default | | -------------- | ------ | -------------------------------------------------- | ---------- | | `task` | `str` | The task text to estimate. | *required* | | `healthy_only` | `bool` | If True (default), only include healthy providers. | `True` | | `limit` | `int` | Maximum number of results to return. | `10` | Returns: | Type | Description | | ---------------------------- | ------------------------------------------------------------ | | `list[ProviderCostEstimate]` | Sorted list of ProviderCostEstimate objects, cheapest first. | Source code in `packages/pyagent-providers/src/pyagent_providers/cost.py` ``` def compare( self, task: str, *, healthy_only: bool = True, limit: int = 10, ) -> list[ProviderCostEstimate]: """Estimate cost for every provider + model pair and rank cheapest first. Args: task: The task text to estimate. healthy_only: If ``True`` (default), only include healthy providers. limit: Maximum number of results to return. Returns: Sorted list of ``ProviderCostEstimate`` objects, cheapest first. """ candidates = self._registry.discover(healthy_only=healthy_only) results: list[ProviderCostEstimate] = [] for provider in candidates: for model_name in provider.capabilities.models: try: est = self._estimator.estimate_from_text(model_name, task) results.append( ProviderCostEstimate( provider_name=provider.name, model=model_name, estimate=est, ) ) except KeyError: continue results.sort(key=lambda x: x.estimate.total_cost) return results[:limit] ``` #### `cheapest(task, *, healthy_only=True)` Return the single cheapest provider + model for a task. Returns: | Type | Description | | ---------------------- | ----------- | | \`ProviderCostEstimate | None\` | Source code in `packages/pyagent-providers/src/pyagent_providers/cost.py` ``` def cheapest( self, task: str, *, healthy_only: bool = True, ) -> ProviderCostEstimate | None: """Return the single cheapest provider + model for a task. Returns: The cheapest option, or ``None`` if no estimates could be computed. """ estimates = self.compare(task, healthy_only=healthy_only, limit=1) return estimates[0] if estimates else None ``` #### `cheapest_provider(task, *, healthy_only=True)` Return the cheapest provider object + model name. Convenience method for passing directly to `Agent` or patterns. Source code in `packages/pyagent-providers/src/pyagent_providers/cost.py` ``` def cheapest_provider( self, task: str, *, healthy_only: bool = True, ) -> tuple[ProviderProtocol, str] | None: """Return the cheapest provider object + model name. Convenience method for passing directly to ``Agent`` or patterns. """ est = self.cheapest(task, healthy_only=healthy_only) if est is None: return None provider = self._registry.get(est.provider_name) if provider is None: return None return provider, est.model ``` ### `ProviderCostEstimate` Cost estimate for a specific provider + model pair. Attributes: | Name | Type | Description | | --------------- | -------------- | --------------------------------------- | | `provider_name` | `str` | Provider that would serve this request. | | `model` | `str` | Model within that provider. | | `estimate` | `CostEstimate` | The underlying cost estimate. | Source code in `packages/pyagent-providers/src/pyagent_providers/cost.py` ``` @dataclass(frozen=True) class ProviderCostEstimate: """Cost estimate for a specific provider + model pair. Attributes: provider_name: Provider that would serve this request. model: Model within that provider. estimate: The underlying cost estimate. """ provider_name: str model: str estimate: CostEstimate ``` ### `FallbackChain` Try providers in order until one succeeds. Optionally integrates with `CircuitBreaker` from `pyagent-patterns` to skip providers whose circuits are open. Parameters: | Name | Type | Description | Default | | ------------------ | ------------------------ | --------------------------------- | --------------------------------------------------------------------------------------------------- | | `providers` | `list[ProviderProtocol]` | Ordered list of providers to try. | *required* | | `circuit_breakers` | \`dict[str, Any] | None\` | Optional mapping of provider name → CircuitBreaker. If a provider's circuit is open, it is skipped. | Source code in `packages/pyagent-providers/src/pyagent_providers/fallback.py` ``` class FallbackChain: """Try providers in order until one succeeds. Optionally integrates with ``CircuitBreaker`` from ``pyagent-patterns`` to skip providers whose circuits are open. Args: providers: Ordered list of providers to try. circuit_breakers: Optional mapping of provider name → CircuitBreaker. If a provider's circuit is open, it is skipped. """ def __init__( self, providers: list[ProviderProtocol], circuit_breakers: dict[str, Any] | None = None, ) -> None: if not providers: raise ValueError("FallbackChain requires at least one provider") self._providers = list(providers) self._circuit_breakers = circuit_breakers or {} async def complete( self, messages: list[Message], model: str | None = None, ) -> FallbackResult: """Try each provider in order. Args: messages: Conversation messages. model: Optional model override passed to each provider. Returns: ``FallbackResult`` with the first successful output. Raises: RuntimeError: If all providers fail. """ attempts: list[FallbackAttempt] = [] for provider in self._providers: # Check circuit breaker cb = self._circuit_breakers.get(provider.name) if cb is not None: from pyagent_patterns.recovery import CircuitState if hasattr(cb, "state") and cb.state == CircuitState.OPEN: attempts.append( FallbackAttempt(provider.name, success=False, error="circuit_open") ) logger.debug("Skipping %s — circuit open", provider.name) continue try: output = await provider.complete(messages, model) attempts.append(FallbackAttempt(provider.name, success=True)) # Record success on circuit breaker if cb is not None and hasattr(cb, "_on_success"): cb._on_success() return FallbackResult( output=output, provider_name=provider.name, attempts=attempts, ) except Exception as exc: error_msg = f"{type(exc).__name__}: {exc}" attempts.append(FallbackAttempt(provider.name, success=False, error=error_msg)) logger.warning("Provider %s failed: %s", provider.name, error_msg) # Record failure on circuit breaker if cb is not None and hasattr(cb, "_on_failure"): cb._on_failure() failed_names = [a.provider_name for a in attempts] raise RuntimeError( f"All providers failed in fallback chain: {failed_names}. " f"Errors: {[a.error for a in attempts if a.error]}" ) async def __call__(self, messages: list[Message]) -> str: """LLMCallable-compatible interface.""" result = await self.complete(messages) return result.output ``` #### `complete(messages, model=None)` Try each provider in order. Parameters: | Name | Type | Description | Default | | ---------- | --------------- | ---------------------- | ------------------------------------------------ | | `messages` | `list[Message]` | Conversation messages. | *required* | | `model` | \`str | None\` | Optional model override passed to each provider. | Returns: | Type | Description | | ---------------- | ------------------------------------------------ | | `FallbackResult` | FallbackResult with the first successful output. | Raises: | Type | Description | | -------------- | ---------------------- | | `RuntimeError` | If all providers fail. | Source code in `packages/pyagent-providers/src/pyagent_providers/fallback.py` ``` async def complete( self, messages: list[Message], model: str | None = None, ) -> FallbackResult: """Try each provider in order. Args: messages: Conversation messages. model: Optional model override passed to each provider. Returns: ``FallbackResult`` with the first successful output. Raises: RuntimeError: If all providers fail. """ attempts: list[FallbackAttempt] = [] for provider in self._providers: # Check circuit breaker cb = self._circuit_breakers.get(provider.name) if cb is not None: from pyagent_patterns.recovery import CircuitState if hasattr(cb, "state") and cb.state == CircuitState.OPEN: attempts.append( FallbackAttempt(provider.name, success=False, error="circuit_open") ) logger.debug("Skipping %s — circuit open", provider.name) continue try: output = await provider.complete(messages, model) attempts.append(FallbackAttempt(provider.name, success=True)) # Record success on circuit breaker if cb is not None and hasattr(cb, "_on_success"): cb._on_success() return FallbackResult( output=output, provider_name=provider.name, attempts=attempts, ) except Exception as exc: error_msg = f"{type(exc).__name__}: {exc}" attempts.append(FallbackAttempt(provider.name, success=False, error=error_msg)) logger.warning("Provider %s failed: %s", provider.name, error_msg) # Record failure on circuit breaker if cb is not None and hasattr(cb, "_on_failure"): cb._on_failure() failed_names = [a.provider_name for a in attempts] raise RuntimeError( f"All providers failed in fallback chain: {failed_names}. " f"Errors: {[a.error for a in attempts if a.error]}" ) ``` #### `__call__(messages)` LLMCallable-compatible interface. Source code in `packages/pyagent-providers/src/pyagent_providers/fallback.py` ``` async def __call__(self, messages: list[Message]) -> str: """LLMCallable-compatible interface.""" result = await self.complete(messages) return result.output ``` ### `FallbackResult` Result of a fallback chain execution. Attributes: | Name | Type | Description | | --------------- | ----------------------- | ------------------------------------------------------- | | `output` | `str` | The completion text from the first successful provider. | | `provider_name` | `str` | Name of the provider that succeeded. | | `attempts` | `list[FallbackAttempt]` | Ordered log of every attempt. | Source code in `packages/pyagent-providers/src/pyagent_providers/fallback.py` ``` @dataclass class FallbackResult: """Result of a fallback chain execution. Attributes: output: The completion text from the first successful provider. provider_name: Name of the provider that succeeded. attempts: Ordered log of every attempt. """ output: str provider_name: str attempts: list[FallbackAttempt] = field(default_factory=list) ``` ### `CapabilityNegotiator` Find the best provider for a given set of task requirements. Scores providers by capability overlap, context window adequacy, and feature support (streaming, tools, vision). Parameters: | Name | Type | Description | Default | | ---------- | ------------------ | ---------------------------- | ---------- | | `registry` | `ProviderRegistry` | Provider registry to search. | *required* | Source code in `packages/pyagent-providers/src/pyagent_providers/negotiation.py` ``` class CapabilityNegotiator: """Find the best provider for a given set of task requirements. Scores providers by capability overlap, context window adequacy, and feature support (streaming, tools, vision). Args: registry: Provider registry to search. """ def __init__(self, registry: ProviderRegistry) -> None: self._registry = registry def negotiate( self, required_capabilities: set[Capability] | None = None, *, min_context: int = 0, needs_streaming: bool = False, needs_tools: bool = False, needs_vision: bool = False, ) -> NegotiationResult | None: """Find the best provider matching the requirements. Args: required_capabilities: Must-have capabilities. min_context: Minimum context window size in tokens. needs_streaming: Whether streaming is required. needs_tools: Whether tool/function calling is required. needs_vision: Whether image input is required. Returns: Best ``NegotiationResult``, or ``None`` if no provider qualifies. """ required = required_capabilities or set() candidates = self._registry.discover(healthy_only=True) if not candidates: return None scored: list[tuple[ProviderProtocol, float, set[Capability], set[Capability]]] = [] for provider in candidates: caps = provider.capabilities # Hard filters if min_context > 0 and caps.max_context < min_context: continue if needs_streaming and not caps.supports_streaming: continue if needs_tools and not caps.supports_tools: continue if needs_vision and not caps.supports_vision: continue # Score: capability match ratio if required: matched = required & caps.capabilities missing = required - caps.capabilities cap_score = len(matched) / len(required) if required else 1.0 else: matched = caps.capabilities missing = set() cap_score = 1.0 # Bonus for extra capabilities bonus = min(len(caps.capabilities - required) * 0.05, 0.2) if required else 0.0 # Bonus for large context context_bonus = min(caps.max_context / 1_000_000, 0.1) total_score = min(cap_score + bonus + context_bonus, 1.0) scored.append((provider, total_score, matched, missing)) if not scored: return None scored.sort(key=lambda x: x[1], reverse=True) best_provider, best_score, best_matched, best_missing = scored[0] # Pick the most capable model from the provider models = best_provider.capabilities.models model = models[-1] if models else "" return NegotiationResult( provider=best_provider, model=model, match_score=best_score, matched_capabilities=best_matched, missing_capabilities=best_missing, ) def negotiate_all( self, required_capabilities: set[Capability] | None = None, *, min_context: int = 0, needs_streaming: bool = False, needs_tools: bool = False, needs_vision: bool = False, limit: int = 5, ) -> list[NegotiationResult]: """Return all matching providers ranked by score. Same args as ``negotiate`` plus ``limit`` to cap results. """ required = required_capabilities or set() candidates = self._registry.discover(healthy_only=True) results: list[NegotiationResult] = [] for provider in candidates: caps = provider.capabilities if min_context > 0 and caps.max_context < min_context: continue if needs_streaming and not caps.supports_streaming: continue if needs_tools and not caps.supports_tools: continue if needs_vision and not caps.supports_vision: continue if required: matched = required & caps.capabilities missing = required - caps.capabilities cap_score = len(matched) / len(required) else: matched = caps.capabilities missing = set() cap_score = 1.0 bonus = min(len(caps.capabilities - required) * 0.05, 0.2) if required else 0.0 context_bonus = min(caps.max_context / 1_000_000, 0.1) total_score = min(cap_score + bonus + context_bonus, 1.0) models = provider.capabilities.models model = models[-1] if models else "" results.append( NegotiationResult( provider=provider, model=model, match_score=total_score, matched_capabilities=matched, missing_capabilities=missing, ) ) results.sort(key=lambda r: r.match_score, reverse=True) return results[:limit] ``` #### `negotiate(required_capabilities=None, *, min_context=0, needs_streaming=False, needs_tools=False, needs_vision=False)` Find the best provider matching the requirements. Parameters: | Name | Type | Description | Default | | ----------------------- | ----------------- | ------------------------------------------ | ----------------------- | | `required_capabilities` | \`set[Capability] | None\` | Must-have capabilities. | | `min_context` | `int` | Minimum context window size in tokens. | `0` | | `needs_streaming` | `bool` | Whether streaming is required. | `False` | | `needs_tools` | `bool` | Whether tool/function calling is required. | `False` | | `needs_vision` | `bool` | Whether image input is required. | `False` | Returns: | Type | Description | | ------------------- | ----------- | | \`NegotiationResult | None\` | Source code in `packages/pyagent-providers/src/pyagent_providers/negotiation.py` ``` def negotiate( self, required_capabilities: set[Capability] | None = None, *, min_context: int = 0, needs_streaming: bool = False, needs_tools: bool = False, needs_vision: bool = False, ) -> NegotiationResult | None: """Find the best provider matching the requirements. Args: required_capabilities: Must-have capabilities. min_context: Minimum context window size in tokens. needs_streaming: Whether streaming is required. needs_tools: Whether tool/function calling is required. needs_vision: Whether image input is required. Returns: Best ``NegotiationResult``, or ``None`` if no provider qualifies. """ required = required_capabilities or set() candidates = self._registry.discover(healthy_only=True) if not candidates: return None scored: list[tuple[ProviderProtocol, float, set[Capability], set[Capability]]] = [] for provider in candidates: caps = provider.capabilities # Hard filters if min_context > 0 and caps.max_context < min_context: continue if needs_streaming and not caps.supports_streaming: continue if needs_tools and not caps.supports_tools: continue if needs_vision and not caps.supports_vision: continue # Score: capability match ratio if required: matched = required & caps.capabilities missing = required - caps.capabilities cap_score = len(matched) / len(required) if required else 1.0 else: matched = caps.capabilities missing = set() cap_score = 1.0 # Bonus for extra capabilities bonus = min(len(caps.capabilities - required) * 0.05, 0.2) if required else 0.0 # Bonus for large context context_bonus = min(caps.max_context / 1_000_000, 0.1) total_score = min(cap_score + bonus + context_bonus, 1.0) scored.append((provider, total_score, matched, missing)) if not scored: return None scored.sort(key=lambda x: x[1], reverse=True) best_provider, best_score, best_matched, best_missing = scored[0] # Pick the most capable model from the provider models = best_provider.capabilities.models model = models[-1] if models else "" return NegotiationResult( provider=best_provider, model=model, match_score=best_score, matched_capabilities=best_matched, missing_capabilities=best_missing, ) ``` #### `negotiate_all(required_capabilities=None, *, min_context=0, needs_streaming=False, needs_tools=False, needs_vision=False, limit=5)` Return all matching providers ranked by score. Same args as `negotiate` plus `limit` to cap results. Source code in `packages/pyagent-providers/src/pyagent_providers/negotiation.py` ``` def negotiate_all( self, required_capabilities: set[Capability] | None = None, *, min_context: int = 0, needs_streaming: bool = False, needs_tools: bool = False, needs_vision: bool = False, limit: int = 5, ) -> list[NegotiationResult]: """Return all matching providers ranked by score. Same args as ``negotiate`` plus ``limit`` to cap results. """ required = required_capabilities or set() candidates = self._registry.discover(healthy_only=True) results: list[NegotiationResult] = [] for provider in candidates: caps = provider.capabilities if min_context > 0 and caps.max_context < min_context: continue if needs_streaming and not caps.supports_streaming: continue if needs_tools and not caps.supports_tools: continue if needs_vision and not caps.supports_vision: continue if required: matched = required & caps.capabilities missing = required - caps.capabilities cap_score = len(matched) / len(required) else: matched = caps.capabilities missing = set() cap_score = 1.0 bonus = min(len(caps.capabilities - required) * 0.05, 0.2) if required else 0.0 context_bonus = min(caps.max_context / 1_000_000, 0.1) total_score = min(cap_score + bonus + context_bonus, 1.0) models = provider.capabilities.models model = models[-1] if models else "" results.append( NegotiationResult( provider=provider, model=model, match_score=total_score, matched_capabilities=matched, missing_capabilities=missing, ) ) results.sort(key=lambda r: r.match_score, reverse=True) return results[:limit] ``` ### `NegotiationResult` Result of capability negotiation. Attributes: | Name | Type | Description | | ---------------------- | ------------------ | ---------------------------------------------------- | | `provider` | `ProviderProtocol` | The best-matched provider. | | `model` | `str` | Recommended model from that provider. | | `match_score` | `float` | How well the provider matches (0.0-1.0). | | `matched_capabilities` | `set[Capability]` | Which requested capabilities the provider satisfies. | | `missing_capabilities` | `set[Capability]` | Which requested capabilities the provider lacks. | Source code in `packages/pyagent-providers/src/pyagent_providers/negotiation.py` ``` @dataclass(frozen=True) class NegotiationResult: """Result of capability negotiation. Attributes: provider: The best-matched provider. model: Recommended model from that provider. match_score: How well the provider matches (0.0-1.0). matched_capabilities: Which requested capabilities the provider satisfies. missing_capabilities: Which requested capabilities the provider lacks. """ provider: ProviderProtocol model: str match_score: float matched_capabilities: set[Capability] = field(default_factory=set) missing_capabilities: set[Capability] = field(default_factory=set) ``` ### `ProviderRegistry` Central registry of all available LLM providers. Supports registration, discovery by capability, and periodic health checks. Parameters: | Name | Type | Description | Default | | ------------------- | ------ | -------------------------------------------- | ------- | | `auto_health_check` | `bool` | If True, run a health check on registration. | `False` | Source code in `packages/pyagent-providers/src/pyagent_providers/registry.py` ``` class ProviderRegistry: """Central registry of all available LLM providers. Supports registration, discovery by capability, and periodic health checks. Args: auto_health_check: If ``True``, run a health check on registration. """ def __init__(self, *, auto_health_check: bool = False) -> None: self._providers: dict[str, ProviderProtocol] = {} self._health_cache: dict[str, HealthStatus] = {} self._metadata: dict[str, dict[str, Any]] = {} self._auto_health_check = auto_health_check # ------------------------------------------------------------------ # Registration # ------------------------------------------------------------------ async def register( self, provider: ProviderProtocol, metadata: dict[str, Any] | None = None, ) -> None: """Register a provider. Args: provider: Any object satisfying ``ProviderProtocol``. metadata: Optional extra metadata (region, version, etc.). """ self._providers[provider.name] = provider self._metadata[provider.name] = metadata or {} if self._auto_health_check: self._health_cache[provider.name] = await provider.health() else: self._health_cache[provider.name] = HealthStatus.HEALTHY def register_sync( self, provider: ProviderProtocol, metadata: dict[str, Any] | None = None, ) -> None: """Synchronous registration (skips health check).""" self._providers[provider.name] = provider self._metadata[provider.name] = metadata or {} self._health_cache[provider.name] = HealthStatus.HEALTHY def remove(self, name: str) -> None: """Remove a provider by name.""" self._providers.pop(name, None) self._health_cache.pop(name, None) self._metadata.pop(name, None) # ------------------------------------------------------------------ # Discovery # ------------------------------------------------------------------ def get(self, name: str) -> ProviderProtocol | None: """Look up a provider by name.""" return self._providers.get(name) def list_providers(self) -> list[ProviderInfo]: """Return info snapshots for all registered providers.""" return [ ProviderInfo( name=p.name, capabilities=p.capabilities, health=self._health_cache.get(p.name, HealthStatus.HEALTHY), metadata=self._metadata.get(p.name, {}), ) for p in self._providers.values() ] def discover( self, required_capabilities: set[Capability] | None = None, *, healthy_only: bool = True, ) -> list[ProviderProtocol]: """Find providers matching capability and health requirements. Args: required_capabilities: If given, only return providers whose capabilities are a superset of this set. healthy_only: If ``True`` (default), exclude unhealthy providers. Returns: List of matching providers. """ results: list[ProviderProtocol] = [] for p in self._providers.values(): if healthy_only and self._health_cache.get(p.name) == HealthStatus.UNHEALTHY: continue if required_capabilities and not required_capabilities.issubset( p.capabilities.capabilities ): continue results.append(p) return results # ------------------------------------------------------------------ # Health # ------------------------------------------------------------------ async def check_health(self, name: str | None = None) -> dict[str, HealthStatus]: """Run health checks. Args: name: If given, check only this provider. Otherwise check all. Returns: Mapping of provider name → health status. """ targets = ( {name: self._providers[name]} if name and name in self._providers else dict(self._providers) ) async def _check(n: str, p: ProviderProtocol) -> tuple[str, HealthStatus]: try: status = await p.health() except Exception: status = HealthStatus.UNHEALTHY return n, status results = await asyncio.gather(*[_check(n, p) for n, p in targets.items()]) for n, status in results: self._health_cache[n] = status return dict(results) async def remove_unhealthy(self) -> list[str]: """Run health checks and remove all unhealthy providers. Returns: List of removed provider names. """ statuses = await self.check_health() removed: list[str] = [] for name, status in statuses.items(): if status == HealthStatus.UNHEALTHY: self.remove(name) removed.append(name) return removed # ------------------------------------------------------------------ # Properties # ------------------------------------------------------------------ @property def count(self) -> int: return len(self._providers) def __contains__(self, name: str) -> bool: return name in self._providers def __len__(self) -> int: return len(self._providers) ``` #### `register(provider, metadata=None)` Register a provider. Parameters: | Name | Type | Description | Default | | ---------- | ------------------ | --------------------------------------- | ------------------------------------------------ | | `provider` | `ProviderProtocol` | Any object satisfying ProviderProtocol. | *required* | | `metadata` | \`dict[str, Any] | None\` | Optional extra metadata (region, version, etc.). | Source code in `packages/pyagent-providers/src/pyagent_providers/registry.py` ``` async def register( self, provider: ProviderProtocol, metadata: dict[str, Any] | None = None, ) -> None: """Register a provider. Args: provider: Any object satisfying ``ProviderProtocol``. metadata: Optional extra metadata (region, version, etc.). """ self._providers[provider.name] = provider self._metadata[provider.name] = metadata or {} if self._auto_health_check: self._health_cache[provider.name] = await provider.health() else: self._health_cache[provider.name] = HealthStatus.HEALTHY ``` #### `register_sync(provider, metadata=None)` Synchronous registration (skips health check). Source code in `packages/pyagent-providers/src/pyagent_providers/registry.py` ``` def register_sync( self, provider: ProviderProtocol, metadata: dict[str, Any] | None = None, ) -> None: """Synchronous registration (skips health check).""" self._providers[provider.name] = provider self._metadata[provider.name] = metadata or {} self._health_cache[provider.name] = HealthStatus.HEALTHY ``` #### `remove(name)` Remove a provider by name. Source code in `packages/pyagent-providers/src/pyagent_providers/registry.py` ``` def remove(self, name: str) -> None: """Remove a provider by name.""" self._providers.pop(name, None) self._health_cache.pop(name, None) self._metadata.pop(name, None) ``` #### `get(name)` Look up a provider by name. Source code in `packages/pyagent-providers/src/pyagent_providers/registry.py` ``` def get(self, name: str) -> ProviderProtocol | None: """Look up a provider by name.""" return self._providers.get(name) ``` #### `list_providers()` Return info snapshots for all registered providers. Source code in `packages/pyagent-providers/src/pyagent_providers/registry.py` ``` def list_providers(self) -> list[ProviderInfo]: """Return info snapshots for all registered providers.""" return [ ProviderInfo( name=p.name, capabilities=p.capabilities, health=self._health_cache.get(p.name, HealthStatus.HEALTHY), metadata=self._metadata.get(p.name, {}), ) for p in self._providers.values() ] ``` #### `discover(required_capabilities=None, *, healthy_only=True)` Find providers matching capability and health requirements. Parameters: | Name | Type | Description | Default | | ----------------------- | ----------------- | ----------------------------------------------- | ------------------------------------------------------------------------------ | | `required_capabilities` | \`set[Capability] | None\` | If given, only return providers whose capabilities are a superset of this set. | | `healthy_only` | `bool` | If True (default), exclude unhealthy providers. | `True` | Returns: | Type | Description | | ------------------------ | --------------------------- | | `list[ProviderProtocol]` | List of matching providers. | Source code in `packages/pyagent-providers/src/pyagent_providers/registry.py` ``` def discover( self, required_capabilities: set[Capability] | None = None, *, healthy_only: bool = True, ) -> list[ProviderProtocol]: """Find providers matching capability and health requirements. Args: required_capabilities: If given, only return providers whose capabilities are a superset of this set. healthy_only: If ``True`` (default), exclude unhealthy providers. Returns: List of matching providers. """ results: list[ProviderProtocol] = [] for p in self._providers.values(): if healthy_only and self._health_cache.get(p.name) == HealthStatus.UNHEALTHY: continue if required_capabilities and not required_capabilities.issubset( p.capabilities.capabilities ): continue results.append(p) return results ``` #### `check_health(name=None)` Run health checks. Parameters: | Name | Type | Description | Default | | ------ | ----- | ----------- | -------------------------------------------------------- | | `name` | \`str | None\` | If given, check only this provider. Otherwise check all. | Returns: | Type | Description | | ------------------------- | ----------------------------------------- | | `dict[str, HealthStatus]` | Mapping of provider name → health status. | Source code in `packages/pyagent-providers/src/pyagent_providers/registry.py` ``` async def check_health(self, name: str | None = None) -> dict[str, HealthStatus]: """Run health checks. Args: name: If given, check only this provider. Otherwise check all. Returns: Mapping of provider name → health status. """ targets = ( {name: self._providers[name]} if name and name in self._providers else dict(self._providers) ) async def _check(n: str, p: ProviderProtocol) -> tuple[str, HealthStatus]: try: status = await p.health() except Exception: status = HealthStatus.UNHEALTHY return n, status results = await asyncio.gather(*[_check(n, p) for n, p in targets.items()]) for n, status in results: self._health_cache[n] = status return dict(results) ``` #### `remove_unhealthy()` Run health checks and remove all unhealthy providers. Returns: | Type | Description | | ----------- | ------------------------------- | | `list[str]` | List of removed provider names. | Source code in `packages/pyagent-providers/src/pyagent_providers/registry.py` ``` async def remove_unhealthy(self) -> list[str]: """Run health checks and remove all unhealthy providers. Returns: List of removed provider names. """ statuses = await self.check_health() removed: list[str] = [] for name, status in statuses.items(): if status == HealthStatus.UNHEALTHY: self.remove(name) removed.append(name) return removed ``` ### `ProviderRouter` Route requests to the best provider + model pair. Parameters: | Name | Type | Description | Default | | ---------------- | ------------------ | ----------------------------------- | ----------------------------------------------------------- | | `registry` | `ProviderRegistry` | The provider registry to draw from. | *required* | | `strategy` | `RoutingStrategy` | Routing strategy to use. | `CAPABILITY_FIRST` | | `cost_estimator` | \`CostEstimator | None\` | Optional cost estimator for COST_FIRST strategy. | | `scorer` | \`DifficultyScorer | None\` | Optional difficulty scorer for capability-based strategies. | Source code in `packages/pyagent-providers/src/pyagent_providers/router.py` ``` class ProviderRouter: """Route requests to the best provider + model pair. Args: registry: The provider registry to draw from. strategy: Routing strategy to use. cost_estimator: Optional cost estimator for ``COST_FIRST`` strategy. scorer: Optional difficulty scorer for capability-based strategies. """ def __init__( self, registry: ProviderRegistry, strategy: RoutingStrategy = RoutingStrategy.CAPABILITY_FIRST, cost_estimator: CostEstimator | None = None, scorer: DifficultyScorer | None = None, ) -> None: self._registry = registry self._strategy = strategy self._estimator = cost_estimator or CostEstimator() self._scorer = scorer or DifficultyScorer() self._rr_cycle: itertools.cycle[str] | None = None async def route( self, messages: list[Message], required: set[Capability] | None = None, ) -> tuple[ProviderProtocol, str]: """Select the best provider and model for a request. Args: messages: Conversation messages (last user message used for scoring). required: Optional capability requirements. Returns: Tuple of (provider, model_name). Raises: RuntimeError: If no suitable provider is found. """ candidates = self._registry.discover(required, healthy_only=True) if not candidates: raise RuntimeError(f"No healthy provider found matching capabilities={required}") if self._strategy == RoutingStrategy.ROUND_ROBIN: return self._route_round_robin(candidates) if self._strategy == RoutingStrategy.COST_FIRST: return self._route_cost_first(candidates, messages) if self._strategy == RoutingStrategy.LATENCY_FIRST: return self._route_latency_first(candidates) # Default: CAPABILITY_FIRST return self._route_capability_first(candidates, messages) # ------------------------------------------------------------------ # Strategy implementations # ------------------------------------------------------------------ def _route_capability_first( self, candidates: list[ProviderProtocol], messages: list[Message], ) -> tuple[ProviderProtocol, str]: """Pick the provider with the broadest capability set, then its best model.""" task_text = self._extract_task(messages) difficulty = self._scorer.score(task_text) best: ProviderProtocol | None = None best_score = -1 for p in candidates: score = len(p.capabilities.capabilities) + p.capabilities.max_context // 100_000 if score > best_score: best_score = score best = p provider = best or candidates[0] model = self._pick_model(provider, difficulty.score) return provider, model def _route_cost_first( self, candidates: list[ProviderProtocol], messages: list[Message], ) -> tuple[ProviderProtocol, str]: """Pick the cheapest provider + model combo.""" task_text = self._extract_task(messages) cheapest_cost = float("inf") chosen_provider = candidates[0] chosen_model = ( candidates[0].capabilities.models[0] if candidates[0].capabilities.models else "" ) for p in candidates: for model_name in p.capabilities.models: try: est = self._estimator.estimate_from_text(model_name, task_text) if est.total_cost < cheapest_cost: cheapest_cost = est.total_cost chosen_provider = p chosen_model = model_name except KeyError: continue return chosen_provider, chosen_model def _route_latency_first( self, candidates: list[ProviderProtocol], ) -> tuple[ProviderProtocol, str]: """Pick the provider with the smallest model (proxy for low latency).""" best: ProviderProtocol | None = None best_ctx = float("inf") for p in candidates: if p.capabilities.max_context < best_ctx: best_ctx = p.capabilities.max_context best = p provider = best or candidates[0] model = provider.capabilities.models[0] if provider.capabilities.models else "" return provider, model def _route_round_robin( self, candidates: list[ProviderProtocol], ) -> tuple[ProviderProtocol, str]: """Cycle through providers in order.""" names = [p.name for p in candidates] if self._rr_cycle is None or set(names) != getattr(self, "_rr_names", set()): self._rr_cycle = itertools.cycle(names) self._rr_names = set(names) chosen_name = next(self._rr_cycle) provider = next(p for p in candidates if p.name == chosen_name) model = provider.capabilities.models[0] if provider.capabilities.models else "" return provider, model # ------------------------------------------------------------------ # Helpers # ------------------------------------------------------------------ @staticmethod def _extract_task(messages: list[Message]) -> str: """Get the last user message as a task string.""" for m in reversed(messages): if m.role.value == "user": return m.content return "" @staticmethod def _pick_model(provider: ProviderProtocol, difficulty: int) -> str: """Pick the first model from the provider's model list. A more sophisticated version could match difficulty ranges per model. """ models = provider.capabilities.models if not models: return "" # Simple heuristic: last model is most capable if difficulty >= 7 and len(models) > 1: return models[-1] return models[0] ``` #### `route(messages, required=None)` Select the best provider and model for a request. Parameters: | Name | Type | Description | Default | | ---------- | ----------------- | ----------------------------------------------------------- | --------------------------------- | | `messages` | `list[Message]` | Conversation messages (last user message used for scoring). | *required* | | `required` | \`set[Capability] | None\` | Optional capability requirements. | Returns: | Type | Description | | ------------------------------ | -------------------------------- | | `tuple[ProviderProtocol, str]` | Tuple of (provider, model_name). | Raises: | Type | Description | | -------------- | --------------------------------- | | `RuntimeError` | If no suitable provider is found. | Source code in `packages/pyagent-providers/src/pyagent_providers/router.py` ``` async def route( self, messages: list[Message], required: set[Capability] | None = None, ) -> tuple[ProviderProtocol, str]: """Select the best provider and model for a request. Args: messages: Conversation messages (last user message used for scoring). required: Optional capability requirements. Returns: Tuple of (provider, model_name). Raises: RuntimeError: If no suitable provider is found. """ candidates = self._registry.discover(required, healthy_only=True) if not candidates: raise RuntimeError(f"No healthy provider found matching capabilities={required}") if self._strategy == RoutingStrategy.ROUND_ROBIN: return self._route_round_robin(candidates) if self._strategy == RoutingStrategy.COST_FIRST: return self._route_cost_first(candidates, messages) if self._strategy == RoutingStrategy.LATENCY_FIRST: return self._route_latency_first(candidates) # Default: CAPABILITY_FIRST return self._route_capability_first(candidates, messages) ``` ### `RoutingStrategy` Bases: `StrEnum` How the router picks a provider + model pair. Source code in `packages/pyagent-providers/src/pyagent_providers/router.py` ``` class RoutingStrategy(StrEnum): """How the router picks a provider + model pair.""" CAPABILITY_FIRST = "capability_first" COST_FIRST = "cost_first" LATENCY_FIRST = "latency_first" ROUND_ROBIN = "round_robin" ``` ### `TracedProvider` Wrapper that emits trace events for every `complete()` / `__call__` invocation. Wraps any `ProviderProtocol` implementation without modifying the original. Parameters: | Name | Type | Description | Default | | ----------- | ------------------ | ---------------------------------------------- | ---------- | | `provider` | `ProviderProtocol` | The underlying provider to delegate calls to. | *required* | | `trace_bus` | `Any` | A TraceEventBus instance (from pyagent-trace). | *required* | Source code in `packages/pyagent-providers/src/pyagent_providers/traced.py` ``` class TracedProvider: """Wrapper that emits trace events for every ``complete()`` / ``__call__`` invocation. Wraps any ``ProviderProtocol`` implementation without modifying the original. Args: provider: The underlying provider to delegate calls to. trace_bus: A ``TraceEventBus`` instance (from pyagent-trace). """ def __init__(self, provider: ProviderProtocol, trace_bus: Any) -> None: self._provider = provider self._trace_bus = trace_bus @property def name(self) -> str: return self._provider.name @property def capabilities(self) -> ProviderCapabilities: return self._provider.capabilities async def health(self) -> HealthStatus: return await self._provider.health() async def complete(self, messages: list[Message], model: str | None = None) -> str: """Delegate to wrapped provider and emit trace events.""" start = time.time() self._emit( "provider_call_start", { "provider": self.name, "model": model or "default", "input_messages": len(messages), }, ) try: result = await self._provider.complete(messages, model) except Exception as exc: self._emit( "provider_call_error", { "provider": self.name, "error": str(exc), "duration_seconds": time.time() - start, }, ) raise duration = time.time() - start input_tokens = sum(len(m.content) for m in messages) // 4 output_tokens = len(result) // 4 self._emit( "provider_call_end", { "provider": self.name, "model": model or "default", "input_tokens": input_tokens, "output_tokens": output_tokens, "duration_seconds": duration, }, ) return result async def __call__(self, messages: list[Message]) -> str: """LLMCallable-compatible entry point — delegates to ``complete``.""" return await self.complete(messages) def _emit(self, event_type: str, payload: dict[str, Any]) -> None: """Emit a trace event to the bus.""" try: from pyagent_trace.events import TraceEvent self._trace_bus.emit( TraceEvent( timestamp=time.time(), event_type=event_type, agent_name=payload.get("provider", ""), payload=payload, ) ) except Exception: pass # trace must never break provider execution ``` #### `complete(messages, model=None)` Delegate to wrapped provider and emit trace events. Source code in `packages/pyagent-providers/src/pyagent_providers/traced.py` ``` async def complete(self, messages: list[Message], model: str | None = None) -> str: """Delegate to wrapped provider and emit trace events.""" start = time.time() self._emit( "provider_call_start", { "provider": self.name, "model": model or "default", "input_messages": len(messages), }, ) try: result = await self._provider.complete(messages, model) except Exception as exc: self._emit( "provider_call_error", { "provider": self.name, "error": str(exc), "duration_seconds": time.time() - start, }, ) raise duration = time.time() - start input_tokens = sum(len(m.content) for m in messages) // 4 output_tokens = len(result) // 4 self._emit( "provider_call_end", { "provider": self.name, "model": model or "default", "input_tokens": input_tokens, "output_tokens": output_tokens, "duration_seconds": duration, }, ) return result ``` #### `__call__(messages)` LLMCallable-compatible entry point — delegates to `complete`. Source code in `packages/pyagent-providers/src/pyagent_providers/traced.py` ``` async def __call__(self, messages: list[Message]) -> str: """LLMCallable-compatible entry point — delegates to ``complete``.""" return await self.complete(messages) ``` ## TracedProvider ## `pyagent_providers.traced.TracedProvider` Wrapper that emits trace events for every `complete()` / `__call__` invocation. Wraps any `ProviderProtocol` implementation without modifying the original. Parameters: | Name | Type | Description | Default | | ----------- | ------------------ | ---------------------------------------------- | ---------- | | `provider` | `ProviderProtocol` | The underlying provider to delegate calls to. | *required* | | `trace_bus` | `Any` | A TraceEventBus instance (from pyagent-trace). | *required* | ### `__call__(messages)` LLMCallable-compatible entry point — delegates to `complete`. ### `complete(messages, model=None)` Delegate to wrapped provider and emit trace events. # pyagent-router API Reference ## `pyagent_router.scorer.DifficultyScorer` Heuristic-based task difficulty scorer. Scores tasks on a 1-10 scale using multiple signals: - Token length - Keyword complexity - Question structure - Required reasoning depth Parameters: | Name | Type | Description | Default | | ---------------- | ---------------- | ----------- | ---------------------------------------------------------------------------------------------------- | | `custom_signals` | \`dict[str, Any] | None\` | Optional dict of custom signal functions. Each function takes a task string and returns a float 0-1. | ### `score(task)` Score the difficulty of a task string. ## `pyagent_router.scorer.DifficultyScore` Result of difficulty scoring. Attributes: | Name | Type | Description | | ---------- | ------------------ | --------------------------------------------------------- | | `score` | `int` | Difficulty score from 1 (trivial) to 10 (extremely hard). | | `signals` | `dict[str, float]` | Dictionary of individual signal scores that contributed. | | `category` | `str` | Human-readable difficulty category. | ## `pyagent_router.estimator.CostEstimator` Estimate LLM call costs based on model pricing registry. Parameters: | Name | Type | Description | Default | | ---------------------- | ------------------------- | -------------------------------------------------------------- | ----------------------------------------------------------------- | | `pricing` | \`dict[str, ModelPricing] | None\` | Optional custom pricing dict. Defaults to built-in pricing table. | | `default_output_ratio` | `float` | Estimated output/input token ratio when output length unknown. | `0.5` | ### `compare(text, models=None)` Compare costs across multiple models for the same input. Parameters: | Name | Type | Description | Default | | -------- | ----------- | --------------- | ----------------------------------------------------- | | `text` | `str` | The input text. | *required* | | `models` | \`list[str] | None\` | Models to compare. Defaults to all registered models. | Returns: | Type | Description | | -------------------- | ----------------------------------------------------- | | `list[CostEstimate]` | List of CostEstimates sorted by total_cost ascending. | ### `estimate(model, input_tokens, output_tokens=None)` Estimate cost for a single LLM call. Parameters: | Name | Type | Description | Default | | --------------- | ----- | ----------------------------------------- | ------------------------------------------------------- | | `model` | `str` | Model name (must be in pricing registry). | *required* | | `input_tokens` | `int` | Number of input tokens. | *required* | | `output_tokens` | \`int | None\` | Number of output tokens. If None, estimated from input. | Returns: | Type | Description | | -------------- | ---------------------------- | | `CostEstimate` | CostEstimate with breakdown. | Raises: | Type | Description | | ---------- | --------------------------------------- | | `KeyError` | If model not found in pricing registry. | ### `estimate_from_text(model, text)` Estimate cost from raw text (approximates 4 chars per token). ## `pyagent_router.estimator.CostEstimate` Estimated cost for a single LLM call. Attributes: | Name | Type | Description | | --------------- | ------- | ------------------------------ | | `model` | `str` | Model name. | | `input_tokens` | `int` | Estimated input tokens. | | `output_tokens` | `int` | Estimated output tokens. | | `input_cost` | `float` | Cost for input tokens in USD. | | `output_cost` | `float` | Cost for output tokens in USD. | | `total_cost` | `float` | Total estimated cost in USD. | ## `pyagent_router.estimator.ModelPricing` Pricing for a model per 1M tokens. ## `pyagent_router.selector.ModelSelector` Select the optimal model based on task difficulty and cost. Strategy: find the cheapest model whose difficulty range covers the task and whose capabilities match the required capability (if specified). Parameters: | Name | Type | Description | Default | | ---------------- | ------------------ | ----------- | ---------------------------------------------------------- | | `specs` | \`list[ModelSpec] | None\` | List of ModelSpec definitions. Defaults to built-in specs. | | `cost_estimator` | \`CostEstimator | None\` | CostEstimator instance. Created automatically if None. | | `scorer` | \`DifficultyScorer | None\` | DifficultyScorer instance. Created automatically if None. | ### `select(task, required_capability=None)` Select the best model for a given task. Parameters: | Name | Type | Description | Default | | --------------------- | ------------ | ------------------------- | --------------------------- | | `task` | `str` | The task text to analyze. | *required* | | `required_capability` | \`Capability | None\` | Optional capability filter. | Returns: | Type | Description | | ----------------- | ------------------------------------------------ | | `SelectionResult` | SelectionResult with chosen model and reasoning. | ## `pyagent_router.selector.ModelSpec` Specification of a model's capabilities and constraints. Attributes: | Name | Type | Description | | ---------------- | ----------------- | -------------------------------------------------- | | `name` | `str` | Model identifier (must match pricing registry). | | `min_difficulty` | `int` | Minimum difficulty score this model should handle. | | `max_difficulty` | `int` | Maximum difficulty score this model should handle. | | `capabilities` | `set[Capability]` | Set of capabilities this model excels at. | | `max_context` | `int` | Maximum context window in tokens. | ## `pyagent_router.selector.SelectionResult` Result of model selection. Attributes: | Name | Type | Description | | --------------- | ----------------- | -------------------------------------------------------- | | `model` | `str` | Selected model name. | | `difficulty` | `DifficultyScore` | The difficulty assessment. | | `cost_estimate` | `CostEstimate` | Estimated cost for this model. | | `reason` | `str` | Human-readable explanation of why this model was chosen. | | `alternatives` | `list[str]` | Other models that were considered. | ## `pyagent_router.selector.Capability` Bases: `StrEnum` Model capabilities for filtering. ## `pyagent_router.middleware.RouterMiddleware` Middleware that wraps agents with automatic model routing. Usage middleware = RouterMiddleware(model_registry={"gpt-4o": my_gpt4o, "gpt-4o-mini": my_mini}) routed_agent = middleware.wrap(my_agent) ### routed_agent now auto-selects model per call Parameters: | Name | Type | Description | Default | | --------------------- | ------------------------ | ---------------------------------------- | ------------------------------------------------------ | | `model_registry` | `dict[str, LLMCallable]` | Mapping of model names to LLM callables. | *required* | | `selector` | \`ModelSelector | None\` | Optional ModelSelector. Created with defaults if None. | | `required_capability` | \`Capability | None\` | Optional default capability filter. | ### `wrap(agent)` Wrap an agent with routing capabilities. ### `wrap_all(agents)` Wrap multiple agents. ## `pyagent_router.middleware.RoutedAgent` Bases: `Agent` An agent wrapper that routes each call through ModelSelector. The routing decision is recorded in metadata for tracing. Parameters: | Name | Type | Description | Default | | --------------------- | ------------------------ | ------------------------------------------- | ----------------------------------------------- | | `agent` | `Agent` | The original agent. | *required* | | `selector` | `ModelSelector` | ModelSelector to use for routing decisions. | *required* | | `model_registry` | `dict[str, LLMCallable]` | Mapping of model names to LLM callables. | *required* | | `required_capability` | \`Capability | None\` | Optional capability filter for model selection. | ### `run(messages)` Route the call to the optimal model, then execute. # pyagent-studio API Reference ## `pyagent_studio` PyAgent Studio — Kubernetes Dashboard for Agent Systems. ### `BlueprintService` Headless service for loading, validating, and compiling blueprints. Wraps `pyagent-blueprint` for use by TUI screens and tests. Source code in `packages/pyagent-studio/src/pyagent_studio/services/blueprint_service.py` ``` class BlueprintService: """Headless service for loading, validating, and compiling blueprints. Wraps ``pyagent-blueprint`` for use by TUI screens and tests. """ def __init__(self) -> None: self._compiler = BlueprintCompiler() self._validator = BlueprintValidator() self._spec: BlueprintSpec | None = None self._graph: RuntimeGraph | None = None self._path: Path | None = None def load(self, path: str | Path) -> BlueprintSpec: """Load a blueprint from file. Args: path: Path to YAML/JSON blueprint. Returns: Validated ``BlueprintSpec``. Raises: BlueprintLoadError: On load/validation failure. """ self._path = Path(path) self._spec = load_blueprint(self._path) self._graph = None return self._spec def validate(self) -> list[ValidationIssue]: """Run static validation on the loaded spec. Returns: List of validation issues. Raises: RuntimeError: If no spec loaded. """ if self._spec is None: raise RuntimeError("No blueprint loaded. Call load() first.") return self._validator.validate(self._spec) def compile(self) -> RuntimeGraph: """Compile the loaded spec into a RuntimeGraph. Returns: Executable ``RuntimeGraph``. Raises: RuntimeError: If no spec loaded. """ if self._spec is None: raise RuntimeError("No blueprint loaded. Call load() first.") self._graph = self._compiler.compile(self._spec) return self._graph def discover_blueprints(self, directory: str | Path = ".") -> list[Path]: """Find all YAML/JSON blueprint files in a directory. Args: directory: Root directory to search. Returns: List of file paths. """ root = Path(directory) files: list[Path] = [] for ext in ("*.yaml", "*.yml", "*.json"): files.extend(root.glob(f"**/{ext}")) return sorted(files) @property def spec(self) -> BlueprintSpec | None: return self._spec @property def graph(self) -> RuntimeGraph | None: return self._graph @property def path(self) -> Path | None: return self._path def summary(self) -> dict[str, Any]: """Quick summary of loaded blueprint.""" if self._spec is None: return {"loaded": False} return { "loaded": True, "name": self._spec.metadata.name, "version": self._spec.metadata.version, "agents": len(self._spec.agents), "workflows": len(self._spec.workflows), "providers": len(self._spec.providers), "contracts": len(self._spec.contracts), } ``` #### `load(path)` Load a blueprint from file. Parameters: | Name | Type | Description | Default | | ------ | ----- | ----------- | ---------------------------- | | `path` | \`str | Path\` | Path to YAML/JSON blueprint. | Returns: | Type | Description | | --------------- | ------------------------ | | `BlueprintSpec` | Validated BlueprintSpec. | Raises: | Type | Description | | -------------------- | --------------------------- | | `BlueprintLoadError` | On load/validation failure. | Source code in `packages/pyagent-studio/src/pyagent_studio/services/blueprint_service.py` ``` def load(self, path: str | Path) -> BlueprintSpec: """Load a blueprint from file. Args: path: Path to YAML/JSON blueprint. Returns: Validated ``BlueprintSpec``. Raises: BlueprintLoadError: On load/validation failure. """ self._path = Path(path) self._spec = load_blueprint(self._path) self._graph = None return self._spec ``` #### `validate()` Run static validation on the loaded spec. Returns: | Type | Description | | ----------------------- | -------------------------- | | `list[ValidationIssue]` | List of validation issues. | Raises: | Type | Description | | -------------- | ------------------ | | `RuntimeError` | If no spec loaded. | Source code in `packages/pyagent-studio/src/pyagent_studio/services/blueprint_service.py` ``` def validate(self) -> list[ValidationIssue]: """Run static validation on the loaded spec. Returns: List of validation issues. Raises: RuntimeError: If no spec loaded. """ if self._spec is None: raise RuntimeError("No blueprint loaded. Call load() first.") return self._validator.validate(self._spec) ``` #### `compile()` Compile the loaded spec into a RuntimeGraph. Returns: | Type | Description | | -------------- | ------------------------ | | `RuntimeGraph` | Executable RuntimeGraph. | Raises: | Type | Description | | -------------- | ------------------ | | `RuntimeError` | If no spec loaded. | Source code in `packages/pyagent-studio/src/pyagent_studio/services/blueprint_service.py` ``` def compile(self) -> RuntimeGraph: """Compile the loaded spec into a RuntimeGraph. Returns: Executable ``RuntimeGraph``. Raises: RuntimeError: If no spec loaded. """ if self._spec is None: raise RuntimeError("No blueprint loaded. Call load() first.") self._graph = self._compiler.compile(self._spec) return self._graph ``` #### `discover_blueprints(directory='.')` Find all YAML/JSON blueprint files in a directory. Parameters: | Name | Type | Description | Default | | ----------- | ----- | ----------- | ------------------------- | | `directory` | \`str | Path\` | Root directory to search. | Returns: | Type | Description | | ------------ | ------------------- | | `list[Path]` | List of file paths. | Source code in `packages/pyagent-studio/src/pyagent_studio/services/blueprint_service.py` ``` def discover_blueprints(self, directory: str | Path = ".") -> list[Path]: """Find all YAML/JSON blueprint files in a directory. Args: directory: Root directory to search. Returns: List of file paths. """ root = Path(directory) files: list[Path] = [] for ext in ("*.yaml", "*.yml", "*.json"): files.extend(root.glob(f"**/{ext}")) return sorted(files) ``` #### `summary()` Quick summary of loaded blueprint. Source code in `packages/pyagent-studio/src/pyagent_studio/services/blueprint_service.py` ``` def summary(self) -> dict[str, Any]: """Quick summary of loaded blueprint.""" if self._spec is None: return {"loaded": False} return { "loaded": True, "name": self._spec.metadata.name, "version": self._spec.metadata.version, "agents": len(self._spec.agents), "workflows": len(self._spec.workflows), "providers": len(self._spec.providers), "contracts": len(self._spec.contracts), } ``` ### `GovernanceService` Run governance checks on blueprints. Combines validation, compliance scoring, and diffing. Source code in `packages/pyagent-studio/src/pyagent_studio/services/governance_service.py` ``` class GovernanceService: """Run governance checks on blueprints. Combines validation, compliance scoring, and diffing. """ def __init__(self) -> None: self._validator = BlueprintValidator() self._differ = BlueprintDiffer() def check_compliance(self, spec: BlueprintSpec) -> ComplianceReport: """Run all validation checks and compute compliance score. Args: spec: Blueprint to check. Returns: ``ComplianceReport`` with score and issues. """ issues = self._validator.validate(spec) # Count checks as: 6 standard categories total_checks = 6 error_count = sum(1 for i in issues if i.severity == IssueSeverity.ERROR) passed = max(0, total_checks - error_count) score = passed / total_checks if total_checks > 0 else 1.0 return ComplianceReport( total_checks=total_checks, passed=passed, issues=issues, score=score, ) def diff(self, old: BlueprintSpec, new: BlueprintSpec) -> list[Change]: """Compute semantic diff between two specs. Args: old: Previous version. new: Updated version. Returns: List of ``Change`` objects. """ return self._differ.diff(old, new) def diff_summary(self, old: BlueprintSpec, new: BlueprintSpec) -> str: """Human-readable diff summary. Args: old: Previous version. new: Updated version. Returns: Summary string. """ changes = self.diff(old, new) return self._differ.summary(changes) def format_report(self, report: ComplianceReport) -> str: """Format a compliance report as text.""" lines: list[str] = [] lines.append( f"Compliance Score: {report.score:.0%} ({report.passed}/{report.total_checks})" ) if report.issues: lines.append("\nIssues:") for issue in report.issues: lines.append(f" [{issue.severity}] {issue.path}: {issue.message}") else: lines.append("No issues found.") return "\n".join(lines) ``` #### `check_compliance(spec)` Run all validation checks and compute compliance score. Parameters: | Name | Type | Description | Default | | ------ | --------------- | ------------------- | ---------- | | `spec` | `BlueprintSpec` | Blueprint to check. | *required* | Returns: | Type | Description | | ------------------ | --------------------------------------- | | `ComplianceReport` | ComplianceReport with score and issues. | Source code in `packages/pyagent-studio/src/pyagent_studio/services/governance_service.py` ``` def check_compliance(self, spec: BlueprintSpec) -> ComplianceReport: """Run all validation checks and compute compliance score. Args: spec: Blueprint to check. Returns: ``ComplianceReport`` with score and issues. """ issues = self._validator.validate(spec) # Count checks as: 6 standard categories total_checks = 6 error_count = sum(1 for i in issues if i.severity == IssueSeverity.ERROR) passed = max(0, total_checks - error_count) score = passed / total_checks if total_checks > 0 else 1.0 return ComplianceReport( total_checks=total_checks, passed=passed, issues=issues, score=score, ) ``` #### `diff(old, new)` Compute semantic diff between two specs. Parameters: | Name | Type | Description | Default | | ----- | --------------- | ----------------- | ---------- | | `old` | `BlueprintSpec` | Previous version. | *required* | | `new` | `BlueprintSpec` | Updated version. | *required* | Returns: | Type | Description | | -------------- | ----------------------- | | `list[Change]` | List of Change objects. | Source code in `packages/pyagent-studio/src/pyagent_studio/services/governance_service.py` ``` def diff(self, old: BlueprintSpec, new: BlueprintSpec) -> list[Change]: """Compute semantic diff between two specs. Args: old: Previous version. new: Updated version. Returns: List of ``Change`` objects. """ return self._differ.diff(old, new) ``` #### `diff_summary(old, new)` Human-readable diff summary. Parameters: | Name | Type | Description | Default | | ----- | --------------- | ----------------- | ---------- | | `old` | `BlueprintSpec` | Previous version. | *required* | | `new` | `BlueprintSpec` | Updated version. | *required* | Returns: | Type | Description | | ----- | --------------- | | `str` | Summary string. | Source code in `packages/pyagent-studio/src/pyagent_studio/services/governance_service.py` ``` def diff_summary(self, old: BlueprintSpec, new: BlueprintSpec) -> str: """Human-readable diff summary. Args: old: Previous version. new: Updated version. Returns: Summary string. """ changes = self.diff(old, new) return self._differ.summary(changes) ``` #### `format_report(report)` Format a compliance report as text. Source code in `packages/pyagent-studio/src/pyagent_studio/services/governance_service.py` ``` def format_report(self, report: ComplianceReport) -> str: """Format a compliance report as text.""" lines: list[str] = [] lines.append( f"Compliance Score: {report.score:.0%} ({report.passed}/{report.total_checks})" ) if report.issues: lines.append("\nIssues:") for issue in report.issues: lines.append(f" [{issue.severity}] {issue.path}: {issue.message}") else: lines.append("No issues found.") return "\n".join(lines) ``` ### `SimulationService` Run simulations against compiled blueprints. Uses `MockLLM` by default so no API keys are needed. Source code in `packages/pyagent-studio/src/pyagent_studio/services/simulation_service.py` ``` class SimulationService: """Run simulations against compiled blueprints. Uses ``MockLLM`` by default so no API keys are needed. """ def __init__( self, compiler: BlueprintCompiler | None = None, event_bus: TraceEventBus | None = None, use_live: bool = False, ) -> None: self._compiler = compiler or BlueprintCompiler() self._event_bus = event_bus self._use_live = use_live async def run( self, spec: BlueprintSpec, workflow: str, task: str, use_live: bool | None = None, ) -> SimulationResult: """Run a single simulation. Args: spec: Blueprint spec. workflow: Workflow name to run. task: Input task string. Returns: ``SimulationResult`` with output and timing. """ try: graph = self._compiler.compile(spec) except Exception as exc: return SimulationResult( workflow=workflow, input_task=task, success=False, error=f"Compilation failed: {exc}", ) if self._event_bus: from pyagent_trace.events import TraceEvent self._event_bus.emit( TraceEvent( timestamp=time.time(), event_type="pattern_start", pattern_type=workflow, payload={ "task": task, "live": use_live if use_live is not None else self._use_live, }, ) ) start = time.perf_counter() try: result = await graph.run(workflow, task) elapsed = (time.perf_counter() - start) * 1000 return SimulationResult( workflow=workflow, input_task=task, output=result.output, elapsed_ms=elapsed, success=True, ) except Exception as exc: elapsed = (time.perf_counter() - start) * 1000 return SimulationResult( workflow=workflow, input_task=task, elapsed_ms=elapsed, success=False, error=f"{type(exc).__name__}: {exc}", ) async def run_all( self, spec: BlueprintSpec, tasks: dict[str, str], use_live: bool | None = None, ) -> list[SimulationResult]: """Run simulations for multiple workflows. Args: spec: Blueprint spec. tasks: Mapping of workflow name → task string. Returns: List of simulation results. """ results: list[SimulationResult] = [] for workflow, task in tasks.items(): result = await self.run(spec, workflow, task, use_live=use_live) results.append(result) return results ``` #### `run(spec, workflow, task, use_live=None)` Run a single simulation. Parameters: | Name | Type | Description | Default | | ---------- | --------------- | --------------------- | ---------- | | `spec` | `BlueprintSpec` | Blueprint spec. | *required* | | `workflow` | `str` | Workflow name to run. | *required* | | `task` | `str` | Input task string. | *required* | Returns: | Type | Description | | ------------------ | ---------------------------------------- | | `SimulationResult` | SimulationResult with output and timing. | Source code in `packages/pyagent-studio/src/pyagent_studio/services/simulation_service.py` ``` async def run( self, spec: BlueprintSpec, workflow: str, task: str, use_live: bool | None = None, ) -> SimulationResult: """Run a single simulation. Args: spec: Blueprint spec. workflow: Workflow name to run. task: Input task string. Returns: ``SimulationResult`` with output and timing. """ try: graph = self._compiler.compile(spec) except Exception as exc: return SimulationResult( workflow=workflow, input_task=task, success=False, error=f"Compilation failed: {exc}", ) if self._event_bus: from pyagent_trace.events import TraceEvent self._event_bus.emit( TraceEvent( timestamp=time.time(), event_type="pattern_start", pattern_type=workflow, payload={ "task": task, "live": use_live if use_live is not None else self._use_live, }, ) ) start = time.perf_counter() try: result = await graph.run(workflow, task) elapsed = (time.perf_counter() - start) * 1000 return SimulationResult( workflow=workflow, input_task=task, output=result.output, elapsed_ms=elapsed, success=True, ) except Exception as exc: elapsed = (time.perf_counter() - start) * 1000 return SimulationResult( workflow=workflow, input_task=task, elapsed_ms=elapsed, success=False, error=f"{type(exc).__name__}: {exc}", ) ``` #### `run_all(spec, tasks, use_live=None)` Run simulations for multiple workflows. Parameters: | Name | Type | Description | Default | | ------- | ---------------- | --------------------------------------- | ---------- | | `spec` | `BlueprintSpec` | Blueprint spec. | *required* | | `tasks` | `dict[str, str]` | Mapping of workflow name → task string. | *required* | Returns: | Type | Description | | ------------------------ | --------------------------- | | `list[SimulationResult]` | List of simulation results. | Source code in `packages/pyagent-studio/src/pyagent_studio/services/simulation_service.py` ``` async def run_all( self, spec: BlueprintSpec, tasks: dict[str, str], use_live: bool | None = None, ) -> list[SimulationResult]: """Run simulations for multiple workflows. Args: spec: Blueprint spec. tasks: Mapping of workflow name → task string. Returns: List of simulation results. """ results: list[SimulationResult] = [] for workflow, task in tasks.items(): result = await self.run(spec, workflow, task, use_live=use_live) results.append(result) return results ``` ### `TraceService` Load and query trace JSONL files from pyagent-trace Recorder. Each line in a JSONL file is a JSON object representing a trace event. Source code in `packages/pyagent-studio/src/pyagent_studio/services/trace_service.py` ``` class TraceService: """Load and query trace JSONL files from pyagent-trace Recorder. Each line in a JSONL file is a JSON object representing a trace event. """ def __init__(self) -> None: self._spans: list[TraceSpan] = [] self._path: Path | None = None def load(self, path: str | Path) -> list[TraceSpan]: """Load spans from a JSONL file. Args: path: Path to ``.jsonl`` trace file. Returns: List of parsed ``TraceSpan`` objects. Raises: FileNotFoundError: If file doesn't exist. """ self._path = Path(path) if not self._path.exists(): raise FileNotFoundError(f"Trace file not found: {self._path}") self._spans = [] for line in self._path.read_text().splitlines(): line = line.strip() if not line: continue try: data = json.loads(line) span = TraceSpan( event_type=data.get("event_type", data.get("type", "unknown")), agent_name=data.get("agent_name", data.get("agent", "")), timestamp=data.get("timestamp", ""), duration_ms=data.get("duration_ms", 0.0), tokens=data.get("tokens", data.get("token_count", 0)), data=data, ) self._spans.append(span) except json.JSONDecodeError: continue return self._spans def query( self, event_type: str | None = None, agent_name: str | None = None, ) -> list[TraceSpan]: """Filter loaded spans. Args: event_type: Filter by event type. agent_name: Filter by agent name. Returns: Filtered list of spans. """ results = self._spans if event_type: results = [s for s in results if s.event_type == event_type] if agent_name: results = [s for s in results if s.agent_name == agent_name] return results @property def spans(self) -> list[TraceSpan]: return list(self._spans) def summary(self) -> dict[str, Any]: """Summary statistics for loaded traces.""" if not self._spans: return {"loaded": False, "count": 0} return { "loaded": True, "count": len(self._spans), "event_types": sorted({s.event_type for s in self._spans}), "agents": sorted({s.agent_name for s in self._spans if s.agent_name}), "total_tokens": sum(s.tokens for s in self._spans), "total_duration_ms": sum(s.duration_ms for s in self._spans), } ``` #### `load(path)` Load spans from a JSONL file. Parameters: | Name | Type | Description | Default | | ------ | ----- | ----------- | -------------------------- | | `path` | \`str | Path\` | Path to .jsonl trace file. | Returns: | Type | Description | | ----------------- | --------------------------------- | | `list[TraceSpan]` | List of parsed TraceSpan objects. | Raises: | Type | Description | | ------------------- | ---------------------- | | `FileNotFoundError` | If file doesn't exist. | Source code in `packages/pyagent-studio/src/pyagent_studio/services/trace_service.py` ``` def load(self, path: str | Path) -> list[TraceSpan]: """Load spans from a JSONL file. Args: path: Path to ``.jsonl`` trace file. Returns: List of parsed ``TraceSpan`` objects. Raises: FileNotFoundError: If file doesn't exist. """ self._path = Path(path) if not self._path.exists(): raise FileNotFoundError(f"Trace file not found: {self._path}") self._spans = [] for line in self._path.read_text().splitlines(): line = line.strip() if not line: continue try: data = json.loads(line) span = TraceSpan( event_type=data.get("event_type", data.get("type", "unknown")), agent_name=data.get("agent_name", data.get("agent", "")), timestamp=data.get("timestamp", ""), duration_ms=data.get("duration_ms", 0.0), tokens=data.get("tokens", data.get("token_count", 0)), data=data, ) self._spans.append(span) except json.JSONDecodeError: continue return self._spans ``` #### `query(event_type=None, agent_name=None)` Filter loaded spans. Parameters: | Name | Type | Description | Default | | ------------ | ----- | ----------- | --------------------- | | `event_type` | \`str | None\` | Filter by event type. | | `agent_name` | \`str | None\` | Filter by agent name. | Returns: | Type | Description | | ----------------- | ----------------------- | | `list[TraceSpan]` | Filtered list of spans. | Source code in `packages/pyagent-studio/src/pyagent_studio/services/trace_service.py` ``` def query( self, event_type: str | None = None, agent_name: str | None = None, ) -> list[TraceSpan]: """Filter loaded spans. Args: event_type: Filter by event type. agent_name: Filter by agent name. Returns: Filtered list of spans. """ results = self._spans if event_type: results = [s for s in results if s.event_type == event_type] if agent_name: results = [s for s in results if s.agent_name == agent_name] return results ``` #### `summary()` Summary statistics for loaded traces. Source code in `packages/pyagent-studio/src/pyagent_studio/services/trace_service.py` ``` def summary(self) -> dict[str, Any]: """Summary statistics for loaded traces.""" if not self._spans: return {"loaded": False, "count": 0} return { "loaded": True, "count": len(self._spans), "event_types": sorted({s.event_type for s in self._spans}), "agents": sorted({s.agent_name for s in self._spans if s.agent_name}), "total_tokens": sum(s.tokens for s in self._spans), "total_duration_ms": sum(s.duration_ms for s in self._spans), } ``` # pyagent-trace API Reference ## Event Bus ## `pyagent_trace.events.TraceEvent` Contract: every trace event pyagent emits. Attributes: | Name | Type | Description | | -------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | `timestamp` | `float` | Unix timestamp of the event. | | `event_type` | `str` | One of pattern_start, pattern_end, agent_start, agent_end, llm_call, llm_response, routing_decision, compression, error, cost_record. | | `agent_name` | `str` | Name of the agent involved (empty string if N/A). | | `pattern_type` | `str` | Name of the pattern involved (empty string if N/A). | | `payload` | `dict[str, Any]` | Event-specific data (tokens, cost, duration, model, messages, etc.). | ## `pyagent_trace.events.TraceEventBus` Pub/sub event bus — the contract between trace producers and studio consumers. Usage bus = TraceEventBus() sub_id = bus.subscribe(lambda event: print(event)) bus.emit(TraceEvent(timestamp=time.time(), event_type="llm_call", ...)) bus.unsubscribe(sub_id) ### `emit(event)` Emit event to all matching subscribers (sync). ### `emit_async(event)` Emit event to all matching subscribers, awaiting any coroutine callbacks. ### `subscribe(callback)` Subscribe to all trace events. Returns subscription ID. ### `subscribe_filter(event_types, callback)` Subscribe to specific event types only. Returns subscription ID. ### `unsubscribe(subscription_id)` Remove a subscription by ID. ## Exporters ## `pyagent_trace.exporters.base.TraceExporter` Bases: `Protocol` Portal-agnostic exporter contract. Any class implementing these three methods can receive pyagent trace events. Built-in: ConsoleExporter, JsonlExporter, OTelExporter, LangfuseExporter. Users can implement custom exporters for any backend. ### `export_event(event)` Export a single trace event to the backend. ### `flush()` Flush any buffered events to the backend. ### `shutdown()` Shutdown the exporter, flushing remaining events. ## `pyagent_trace.exporters.console.ConsoleExporter` Export trace events to stdout (or any file-like object). Usage exporter = ConsoleExporter() bus.subscribe(exporter.export_event) ### `export_event(event)` Print a formatted trace event. ### `flush()` Flush the output stream. ### `shutdown()` Flush remaining output. ## `pyagent_trace.exporters.jsonl.JsonlExporter` Export trace events to a JSONL file. Usage exporter = JsonlExporter("traces/run_001.jsonl") bus.subscribe(exporter.export_event) ### `export_event(event)` Append a trace event as a JSON line. ### `flush()` Flush buffered writes to disk. ### `shutdown()` Flush and close the file. ## `pyagent_trace.exporters.langfuse.LangfuseExporter` Export trace events to Langfuse. Usage exporter = LangfuseExporter( public_key="pk-...", secret_key="sk-...", host="https://cloud.langfuse.com", ) bus.subscribe(exporter.export_event) Requires: pip install pyagent-trace[langfuse] ### `export_event(event)` Export a trace event to Langfuse. ### `flush()` Flush pending events to Langfuse. ### `shutdown()` Flush and shutdown the Langfuse client. ## Spans ## `pyagent_trace.spans.PatternSpanEmitter` Emit OTel spans for pattern executions. Usage emitter = PatternSpanEmitter() with emitter.pattern_span("debate", {"rounds": 3}) as span: # ... pattern logic ... emitter.set_result(span, result) ### `agent_span(agent_name, parent_span=None, attributes=None)` Start a span for an individual agent call. ### `pattern_span(pattern_type, attributes=None)` Start a span for a pattern execution. ### `set_compression_info(span, input_tokens, output_tokens, savings_pct)` Set compression attributes on a span. ### `set_error(span, error)` Record an error on a span. ### `set_pattern_result(span, output_length, rounds=None, consensus=None, escalated=False, duration_ms=0.0, token_estimate=0, cost_estimate=0.0)` Set result attributes on a pattern span. ### `set_routing_info(span, difficulty, selected_model, cost_estimate, category='')` Set routing attributes on a span. ## `pyagent_trace.attributes.PyAgentAttributes` Attribute key constants for pyagent OTel spans. ## Decorators ## `pyagent_trace.decorators.traced_pattern(cls)` Class decorator: auto-emit OTel spans for every pattern.run() call. Usage @traced_pattern class MyPattern(Pattern): ... ## `pyagent_trace.decorators.traced_agent(agent)` Wrap an Agent instance to emit OTel spans on every run() call. Usage agent = traced_agent(Agent("my_agent", llm)) ## Cost Tracking ## `pyagent_trace.cost.CostTracker` Track costs across an entire workflow. Usage tracker = CostTracker() tracker.record("debate", "bull_agent", "gpt-4o", 500, 200, 0.003) tracker.record("debate", "bear_agent", "gpt-4o-mini", 500, 200, 0.0004) print(tracker.summary()) ### `by_agent()` Cost breakdown by agent name. ### `by_model()` Cost breakdown by model. ### `by_pattern()` Cost breakdown by pattern type. ### `record(pattern_type, agent_name, model, input_tokens, output_tokens, cost_usd)` Record a cost entry. ### `summary()` Full cost summary. ## `pyagent_trace.cost.CostEntry` A single cost record. ## Record & Replay ## `pyagent_trace.recorder.Recorder` Record pattern executions for debugging and replay. Usage recorder = Recorder() recorder.start("debate") recorder.record_llm_call("bull", messages, response) recorder.save("debug_trace.jsonl") ### `end(result_output)` Mark the end of a pattern execution. ### `load(path)` Load recorded entries from a JSONL file. ### `record_llm_call(agent_name, messages, response, metadata=None)` Record an LLM call and its response. ### `save(path)` Save recorded entries to a JSONL file. ### `start(pattern_type)` Mark the start of a pattern execution. ## `pyagent_trace.recorder.RecordEntry` A single recorded event. # pyagent-patterns — Advanced (Tier 4) ## `pyagent_patterns.advanced.talker_reasoner.TalkerReasoner` Bases: `Pattern` Dual-process: fast intuition (System 1) + slow deliberation (System 2). Parameters: | Name | Type | Description | Default | | ---------------------- | ----------- | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | | `talker` | `Agent` | Fast, cheap agent for routine queries (System 1). | *required* | | `reasoner` | `Agent` | Slow, expensive agent for complex queries (System 2). | *required* | | `classifier` | \`Agent | None\` | Optional agent that decides talker vs reasoner. If None, always starts with talker and escalates on uncertainty keywords. | | `complexity_threshold` | \`list[str] | None\` | Keywords in talker output that trigger escalation to reasoner. | ## `pyagent_patterns.advanced.swarm.Swarm` Bases: `Pattern` Decentralized swarm: agents interact locally, behavior emerges globally. Parameters: | Name | Type | Description | Default | | ---------------- | ------------- | ---------------------------------------------------------------------------------------------------- | ---------- | | `agents` | `list[Agent]` | Pool of swarm agents (all follow same local rules). | *required* | | `rounds` | `int` | Number of interaction rounds. | `3` | | `neighbor_count` | `int` | How many random peers each agent interacts with per round. | `2` | | `aggregation` | `str` | How to produce final output from swarm state. "last" = last round's outputs, "vote" = majority vote. | `'last'` | ## `pyagent_patterns.advanced.human_in_the_loop.HumanInTheLoop` Bases: `Pattern` Agent with human approval gate. Parameters: | Name | Type | Description | Default | | --------------- | --------------- | ---------------------------------------------------------------------------------------------------------------------- | -------------- | | `agent` | `Agent` | The LLM agent that processes the task. | *required* | | `review_fn` | `HumanReviewFn` | Callable that presents output to human and returns decision. Signature: (output: str, metadata: dict) -> HumanDecision | `auto_approve` | | `max_revisions` | `int` | Maximum number of revision attempts after rejection. | `3` | ## `pyagent_patterns.advanced.react.ReAct` Bases: `Pattern` Reasoning + Acting loop with tool use. Parameters: | Name | Type | Description | Default | | -------------- | ------------------- | ----------------------------------------------------- | -------------------------------------------- | | `agent` | `Agent` | The reasoning agent. | *required* | | `tools` | \`dict[str, ToolFn] | None\` | Mapping of tool names to callable functions. | | `max_steps` | `int` | Maximum number of Thought→Action→Observation cycles. | `5` | | `finish_token` | `str` | Token in agent response that signals task completion. | `'FINISH'` | # pyagent-patterns — Base Classes Core abstractions shared by every pattern. ## `pyagent_patterns.base.Message` A single message in an agent conversation. Attributes: | Name | Type | Description | | ---------- | ---------------- | ----------------------------------------------------- | | `role` | `Role` | The sender role (system, user, assistant, tool). | | `content` | `str` | The text content of the message. | | `name` | \`str | None\` | | `metadata` | `dict[str, Any]` | Arbitrary key-value metadata attached to the message. | ## `pyagent_patterns.base.Agent` An LLM-backed agent with a name, system prompt, and callable. Parameters: | Name | Type | Description | Default | | --------------- | ------------- | ----------------------------------------------------------- | ---------- | | `name` | `str` | Human-readable agent name. | *required* | | `llm` | `LLMCallable` | The LLM callable to use for this agent. | *required* | | `system_prompt` | `str` | Optional system prompt prepended to every call. | `''` | | `description` | `str` | Description of the agent's purpose (for routing/selection). | `''` | Optional hooks (set via setter methods — no constructor change): \_trace_bus: Emit agent_start/agent_end trace events. \_context_ledger: Read context before LLM call; write output after. \_compressor: Compress agent output before returning. \_cost_tracker: Record cost per LLM call. ### `run(messages)` Send messages to the LLM and return an assistant message. When hooks are wired the execution order is: 1. Emit `agent_start` trace event 1. Prepend context from ledger as messages 1. Call LLM 1. Record cost 1. Compress output 1. Write result to context ledger 1. Emit `agent_end` trace event with timing/tokens ### `set_compressor(compressor)` Attach a MessageCompressor — compresses output before returning. ### `set_context(ledger)` Attach a ContextLedger — reads context before LLM, writes output after. ### `set_cost_tracker(tracker)` Attach a CostTracker — records cost after each LLM call. ### `set_trace_bus(bus)` Attach a TraceEventBus — emits agent_start/agent_end events on run(). ## `pyagent_patterns.base.Pattern` Bases: `ABC` Abstract base class for all multi-agent patterns. Subclasses must implement `_execute`. The `run` method handles timing, context creation, and metadata collection. Optional hooks \_trace_bus: Emit pattern_start/pattern_end trace events. ### `pattern_type` Return the pattern type name (e.g., 'supervisor', 'debate'). ### `run(task, context=None)` Execute the pattern on the given task. Parameters: | Name | Type | Description | Default | | --------- | --------- | ------------------------ | --------------------------------------------------------- | | `task` | `str` | The user task or prompt. | *required* | | `context` | \`Context | None\` | Optional existing context. Created automatically if None. | Returns: | Type | Description | | -------- | ------------------------------------------------------------------- | | `Result` | Result with output, messages, metadata, timing, and cost estimates. | ### `set_trace_bus(bus)` Attach a TraceEventBus — emits pattern_start/pattern_end on run(). ### `stream(task, context=None)` Stream partial results as they become available. Default implementation runs the full pattern and yields the result. Subclasses can override for true streaming. ## `pyagent_patterns.base.Context` Shared execution context for a pattern run. Attributes: | Name | Type | Description | | ----------- | ---------------- | ------------------------------------- | | `task` | `str` | The original user task/prompt. | | `messages` | `list[Message]` | Accumulated message history. | | `metadata` | `dict[str, Any]` | Arbitrary shared state across agents. | | `parent_id` | \`str | None\` | ### `child(task=None)` Create a child context for nested pattern execution. ## `pyagent_patterns.base.Result` Outcome of a pattern execution. Attributes: | Name | Type | Description | | ------------------ | ---------------- | ----------------------------------------------------------- | | `output` | `str` | The final output text. | | `messages` | `list[Message]` | All messages generated during execution. | | `metadata` | `dict[str, Any]` | Pattern-specific metadata (rounds, consensus, votes, etc.). | | `duration_seconds` | `float` | Wall-clock execution time. | | `token_estimate` | `int` | Rough estimate of total tokens consumed. | | `cost_estimate` | `float` | Rough estimate of total cost in USD. | # pyagent-patterns — Composition, Guardrails & Recovery Cross-cutting building blocks: compose patterns together, guard their inputs and outputs, recover from failures, and get pattern recommendations. ## Composite ## `pyagent_patterns.composite.CompositePattern` Bases: `Pattern` Chain multiple patterns with escalation on quality failure. Parameters: | Name | Type | Description | Default | | ----------------- | ---------------- | ----------------------------------------------------------------------------------------------------------------- | ------------- | | `patterns` | `list[Pattern]` | Ordered list of patterns to try. | *required* | | `quality_check` | `QualityCheckFn` | Function that evaluates whether a pattern's result is acceptable. If it returns False, the next pattern is tried. | `always_pass` | | `combine_results` | `bool` | If True, passes previous output as context to next pattern. | `True` | ## Recovery ## `pyagent_patterns.recovery.BoundedExecution` Wrap a pattern with resource limits. Three-level recovery: 1. Retry with same pattern 1. Fallback to a cheaper/simpler pattern 1. Graceful degradation (return partial result) Parameters: | Name | Type | Description | Default | | ----------------- | --------- | ------------------------------------------------- | ------------------------------------------- | | `pattern` | `Pattern` | The primary pattern to execute. | *required* | | `fallback` | \`Pattern | None\` | Optional simpler pattern to use on failure. | | `max_retries` | `int` | Maximum retry attempts before escalating. | `2` | | `timeout_seconds` | `float` | Maximum wall-clock time for the entire execution. | `300.0` | | `max_tokens` | `int` | Maximum total tokens before stopping. | `100000` | ### `run(task, context=None)` Execute with bounded resources and three-level recovery. ## `pyagent_patterns.recovery.CircuitBreaker` Prevent cascading failures in multi-agent systems. When a pattern fails repeatedly, the circuit opens and rejects requests immediately. After a reset timeout, it enters half-open state and allows one test request through. Parameters: | Name | Type | Description | Default | | ----------------------- | ------- | ---------------------------------------------------- | --------------------------------------------------- | | `failure_threshold` | `int` | Number of consecutive failures before opening. | `3` | | `reset_timeout_seconds` | `float` | Seconds before transitioning from open to half-open. | `60.0` | | `fallback_result` | `str` | Result to return when circuit is open. | `'[Circuit Open] Service temporarily unavailable.'` | ### `execute(pattern, task, context=None)` Execute a pattern through the circuit breaker. ## Guardrails ## `pyagent_patterns.guardrails.GuardrailChain` Chain multiple guardrails together. All must pass. Parameters: | Name | Type | Description | Default | | ------------ | ----------------- | ------------------------------------- | ---------- | | `guardrails` | `list[Guardrail]` | List of guardrails to apply in order. | *required* | ## `pyagent_patterns.guardrails.LengthGuard` Bases: `Guardrail` Reject messages exceeding a maximum length. Parameters: | Name | Type | Description | Default | | ----------- | ------ | --------------------------------------- | ------- | | `max_chars` | `int` | Maximum allowed characters. | `10000` | | `truncate` | `bool` | If True, truncate instead of rejecting. | `False` | ## `pyagent_patterns.guardrails.PIIGuard` Bases: `Guardrail` Detect and optionally redact personally identifiable information. Detects: email addresses, phone numbers, SSNs, credit card numbers. Parameters: | Name | Type | Description | Default | | -------- | ------ | ------------------------------------------------------------ | ------- | | `redact` | `bool` | If True, redact PII and pass. If False, reject on detection. | `True` | ## `pyagent_patterns.guardrails.ContentGuard` Bases: `Guardrail` Block content matching configurable deny patterns. Parameters: | Name | Type | Description | Default | | --------------- | ----------- | ----------- | ---------------------------------------------- | | `deny_patterns` | \`list[str] | None\` | List of regex patterns that should be blocked. | | `deny_words` | \`list[str] | None\` | List of exact words/phrases to block. | ## Advisor ## `pyagent_patterns.advisor.PatternAdvisor` Recommend patterns based on task description and constraints. Usage advisor = PatternAdvisor() rec = advisor.recommend("Write and review code", Constraints(quality=Quality.HIGH)) print(rec.pattern, rec.reason) ### `recommend(task, constraints=None)` Recommend the best pattern for the given task and constraints. # pyagent-patterns — Orchestration (Tier 1) ## `pyagent_patterns.orchestration.supervisor.Supervisor` Bases: `Pattern` Classify → route → collect orchestration pattern. Parameters: | Name | Type | Description | Default | | --------------- | ------------------ | ------------------------------------------------ | ------------------------------------------------------- | | `classifier` | `Agent` | Agent that classifies the task into a route key. | *required* | | `routes` | `dict[str, Agent]` | Mapping of route keys to specialist agents. | *required* | | `formatter` | \`Agent | None\` | Optional agent that formats the final response. | | `default_route` | \`str | None\` | Key to use when classification doesn't match any route. | ## `pyagent_patterns.orchestration.pipeline.Pipeline` Bases: `Pattern` Sequential stage chain — output of stage N becomes input of stage N+1. Parameters: | Name | Type | Description | Default | | -------- | ------------- | ------------------------------------------------------------ | ---------- | | `stages` | `list[Agent]` | Ordered list of agents, each processing the previous output. | *required* | ### `stream(task, context=None)` Stream stage completions as they finish. ## `pyagent_patterns.orchestration.fan_out_fan_in.FanOutFanIn` Bases: `Pattern` Parallel execution with result aggregation. Parameters: | Name | Type | Description | Default | | ------------ | ------------- | --------------------------------------------------- | ---------- | | `agents` | `list[Agent]` | List of agents to run in parallel on the same task. | *required* | | `aggregator` | `Agent` | Agent that combines all parallel outputs into one. | *required* | ### `stream(task, context=None)` Stream individual agent results as they complete. ## `pyagent_patterns.orchestration.hierarchical.Hierarchical` Bases: `Pattern` Manager → Team Leads → Workers hierarchical coordination. Parameters: | Name | Type | Description | Default | | --------- | ------------ | --------------------------------------------------------------- | ---------- | | `manager` | `Agent` | Top-level manager that decomposes work and synthesizes results. | *required* | | `teams` | `list[Team]` | List of teams, each with a lead and workers. | *required* | ## `pyagent_patterns.orchestration.orchestrator_workers.OrchestratorWorkers` Bases: `Pattern` Dynamic task delegation: orchestrator plans → workers execute → synthesize. Parameters: | Name | Type | Description | Default | | -------------- | ------------- | --------------------------------------------------------------------- | ---------- | | `orchestrator` | `Agent` | Agent that plans the work and synthesizes results. | *required* | | `workers` | `list[Agent]` | Pool of available worker agents. The orchestrator selects from these. | *required* | # pyagent-patterns — Resolution (Tier 2) ## `pyagent_patterns.resolution.self_reflection.SelfReflection` Bases: `Pattern` Generate → critique → refine iterative loop. Parameters: | Name | Type | Description | Default | | ------------- | ------- | ---------------------------------------------------------- | ----------------------------------------------------------------------- | | `agent` | `Agent` | The agent that generates and refines output. | *required* | | `critic` | \`Agent | None\` | Optional separate critic agent. If None, the same agent self-critiques. | | `max_rounds` | `int` | Maximum number of generate-critique rounds. | `3` | | `stop_phrase` | `str` | If the critic's response contains this phrase, stop early. | `'APPROVED'` | ## `pyagent_patterns.resolution.cross_reflection.CrossReflection` Bases: `Pattern` Peer review: one agent generates, another reviews, generator revises. Parameters: | Name | Type | Description | Default | | ------------- | ------- | ----------------------------------------------------- | ------------ | | `generator` | `Agent` | Agent that produces the initial output and revisions. | *required* | | `reviewer` | `Agent` | Agent that reviews and provides feedback. | *required* | | `max_rounds` | `int` | Maximum number of generate-review-revise cycles. | `2` | | `stop_phrase` | `str` | If reviewer says this, stop early. | `'APPROVED'` | ## `pyagent_patterns.resolution.debate.Debate` Bases: `Pattern` Structured adversarial debate with judge resolution. Parameters: | Name | Type | Description | Default | | ----------- | ------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | `debaters` | `list[Agent]` | List of agents, each arguing a different position. | *required* | | `judge` | `Agent` | Agent that evaluates arguments and renders final decision. | *required* | | `rounds` | `int` | Number of argumentation rounds. | `3` | | `positions` | \`list[str] | None\` | Optional list of position labels (e.g., ["BUY", "SELL"]). If not provided, positions are assigned as "Position 1", "Position 2", etc. | ## `pyagent_patterns.resolution.voting.Voting` Bases: `Pattern` Independent voting with configurable aggregation strategy. Parameters: | Name | Type | Description | Default | | ----------- | ---------------- | --------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | | `voters` | `list[Agent]` | List of agents that each cast a vote. | *required* | | `strategy` | `VotingStrategy` | Voting strategy (majority or weighted). | `MAJORITY` | | `weights` | \`list[float] | None\` | Optional per-agent weights for weighted voting. Must match length of voters. Defaults to equal weights. | | `normalize` | `bool` | If True, ask each voter to respond with a concise answer suitable for comparison. | `True` | ## `pyagent_patterns.resolution.evaluator_optimizer.EvaluatorOptimizer` Bases: `Pattern` Generate → evaluate → revise loop with explicit evaluation criteria. Parameters: | Name | Type | Description | Default | | ---------------- | ----------- | ---------------------------------------------------------- | ------------------------------------------------- | | `generator` | `Agent` | Agent that produces and revises output. | *required* | | `evaluator` | `Agent` | Agent that scores output against criteria. | *required* | | `criteria` | \`list[str] | None\` | List of evaluation criteria the evaluator checks. | | `max_rounds` | `int` | Maximum optimization rounds. | `3` | | `pass_threshold` | `int` | Score (1-10) at which the output is considered acceptable. | `7` | # pyagent-patterns — Structural (Tier 3) ## `pyagent_patterns.structural.role_based.RoleBased` Bases: `Pattern` Agents with distinct roles collaborate in structured rounds. Parameters: | Name | Type | Description | Default | | ---------------- | ------------- | ------------------------------------------------------------------------------------------------------------- | ---------- | | `agents` | `list[Agent]` | List of role-specialized agents (order matters for turn-taking). | *required* | | `rounds` | `int` | Number of communication rounds. | `1` | | `shared_context` | `bool` | If True, all agents see all prior messages. If False, each agent only sees the immediately preceding message. | `True` | ## `pyagent_patterns.structural.layered.Layered` Bases: `Pattern` Hierarchical layers of agents with increasing abstraction. Parameters: | Name | Type | Description | Default | | -------- | ------------- | ------------------------------------------------------------- | ---------- | | `layers` | `list[Layer]` | Ordered list of layers from bottom (data) to top (synthesis). | *required* | ## `pyagent_patterns.structural.topology.Topology` Bases: `Pattern` Configurable agent communication topology. Parameters: | Name | Type | Description | Default | | ----------- | -------------- | ------------------------------------------------------------- | ---------- | | `agents` | `list[Agent]` | List of agents participating in the topology. | *required* | | `topology` | `TopologyType` | The communication structure (chain, star, mesh). | `CHAIN` | | `hub_index` | `int` | For star topology, the index of the hub agent. Defaults to 0. | `0` | | `rounds` | `int` | For mesh topology, number of communication rounds. | `1` | ## `pyagent_patterns.structural.blackboard.Blackboard` Bases: `Pattern` Shared-state communication via a blackboard store. Parameters: | Name | Type | Description | Default | | --------------- | ----------------------- | ------------------------------------------------------------ | ----------------------------------- | | `agents` | `list[BlackboardAgent]` | List of BlackboardAgent wrappers specifying read/write keys. | *required* | | `rounds` | `int` | Number of rounds agents process the blackboard. | `1` | | `initial_state` | \`dict[str, Any] | None\` | Optional initial blackboard values. | # Hooks Guide **Opt-in, zero-overhead integration points** for tracing, context memory, compression, and cost tracking on every Agent and Pattern. ______________________________________________________________________ ## Overview PyAgent's hook system lets you wire cross-cutting concerns into agents and patterns without modifying their core logic. Hooks are: - **Opt-in** — no constructor changes, no breaking API changes - **Zero-overhead** — `None`-checked; when not wired, hooks add zero cost - **Fault-tolerant** — all hook calls are wrapped in `try/except`; a broken exporter never crashes your agent - **Chainable** — setter methods return `self` for fluent wiring ``` flowchart LR A[Agent.run] -->|1| TE[trace: agent_start] TE -->|2| CL[context: read ledger] CL -->|3| LLM[LLM call] LLM -->|4| CM[compress output] CM -->|5| CT[cost: record tokens] CT -->|6| CW[context: write output] CW -->|7| TE2[trace: agent_end] ``` ## Available Hooks ### On Agent | Hook | Setter | What It Does | | ----------------- | ---------------------------- | --------------------------------------------------------------- | | `_trace_bus` | `set_trace_bus(bus)` | Emits `agent_start` and `agent_end` trace events | | `_context_ledger` | `set_context(ledger)` | Prepends context messages before LLM call; appends output after | | `_compressor` | `set_compressor(compressor)` | Compresses agent output; emits `compression` trace event | | `_cost_tracker` | `set_cost_tracker(tracker)` | Records input/output token counts and estimated cost | ### On Pattern | Hook | Setter | What It Does | | ------------ | -------------------- | ---------------------------------------------------- | | `_trace_bus` | `set_trace_bus(bus)` | Emits `pattern_start` and `pattern_end` trace events | ### On Provider (TracedProvider) | Hook | What It Does | | ------------------------------- | ----------------------------------------------------------------------------------------------------- | | `TracedProvider(provider, bus)` | Wraps any `ProviderProtocol`; emits `provider_call_start`, `provider_call_end`, `provider_call_error` | ## Wiring Hooks Manually ### Single Agent ``` from pyagent_patterns.base import Agent, MockLLM from pyagent_trace.events import TraceEventBus from pyagent_trace import CostTracker from pyagent_context import ContextLedger from pyagent_compress import MessageCompressor bus = TraceEventBus() ledger = ContextLedger() compressor = MessageCompressor(target_ratio=0.5) tracker = CostTracker(event_bus=bus) # Fluent chaining agent = ( Agent("analyst", MockLLM(responses=["Revenue grew 25%"]), system_prompt="Analyze data.") .set_trace_bus(bus) .set_context(ledger) .set_compressor(compressor) .set_cost_tracker(tracker) ) result = await agent.run("What are the revenue trends?") ``` ### Single Pattern ``` from pyagent_patterns.orchestration import Pipeline pipeline = Pipeline(stages=[agent_a, agent_b]) pipeline.set_trace_bus(bus) # emits pattern_start / pattern_end ``` ### TracedProvider ``` from pyagent_providers import TracedProvider # Wrap any provider to emit trace events on every LLM call traced = TracedProvider(original_provider, event_bus=bus) agent = Agent("analyst", llm=traced) ``` ## Wiring via RuntimeGraph When working with blueprints, `RuntimeGraph` provides convenience methods that wire hooks to **all** compiled patterns and agents at once: ``` from pyagent_blueprint import load_blueprint, BlueprintCompiler from pyagent_trace.events import TraceEventBus from pyagent_trace import CostTracker from pyagent_context import ContextLedger from pyagent_compress import MessageCompressor spec = load_blueprint("blueprint.yaml") graph = BlueprintCompiler().compile(spec) bus = TraceEventBus() graph.wire_trace(bus) # all patterns + agents graph.wire_context(ContextLedger()) # all agents graph.wire_compressor(MessageCompressor(0.5)) # all agents graph.wire_cost_tracker(CostTracker(event_bus=bus)) # all agents result = await graph.run("support", "I need help with billing") ``` ### RuntimeGraph.wire\_\* Methods | Method | Targets | Hook Attribute | | ----------------------------- | ------------------------- | ----------------- | | `wire_trace(bus)` | All patterns + all agents | `_trace_bus` | | `wire_context(ledger)` | All agents | `_context_ledger` | | `wire_compressor(compressor)` | All agents | `_compressor` | | `wire_cost_tracker(tracker)` | All agents | `_cost_tracker` | ## Blueprint Compiler Warnings When your blueprint YAML declares features that require hooks, the compiler emits warnings reminding you to wire them: ``` observability: tracing: enabled: true # ⚠ warns: "call graph.wire_trace(bus)" cost_budget: daily_usd: 100.0 # ⚠ warns: "call graph.wire_cost_tracker(tracker)" context: compression: policy: semantic_lossless # ⚠ warns: "call graph.wire_compressor(compressor)" memory: semantic_enabled: true # ⚠ warns: "call graph.wire_context(ledger)" ``` These are **warnings only** — the graph compiles and runs without hooks, but the declared features won't be active until wired. ## Agent.run() Hook Lifecycle When all four hooks are wired, `Agent.run()` executes this sequence: ``` 1. TRACE → emit agent_start (agent_name, input) 2. CONTEXT → read ledger → prepend context messages 3. LLM → call self._llm(messages) → raw output 4. COMPRESS→ compress output → emit compression trace event 5. COST → record input_tokens, output_tokens, cost 6. CONTEXT → append output to ledger (source=agent_name, trust=INFERRED) 7. TRACE → emit agent_end (agent_name, output, duration, tokens) ``` Each step is `None`-guarded and `try/except`-wrapped — if any hook is missing or throws, the remaining steps proceed normally. ## Pattern.run() Hook Lifecycle ``` 1. TRACE → emit pattern_start (pattern_type, input) 2. EXECUTE → call self._execute(input, context) # core pattern logic 3. TRACE → emit pattern_end (pattern_type, output, duration, tokens) ``` ## Trace Event Types | Event Type | Emitter | Payload Keys | | --------------------- | ----------------------- | --------------------------------------------------------------------- | | `agent_start` | Agent | `agent_name`, `input` | | `agent_end` | Agent | `agent_name`, `output`, `duration_seconds`, `output_tokens` | | `pattern_start` | Pattern | `pattern_type`, `input` | | `pattern_end` | Pattern | `pattern_type`, `output_length`, `duration_seconds`, `token_estimate` | | `compression` | Agent (compressor hook) | `agent_name`, `original_tokens`, `compressed_tokens`, `savings_pct` | | `provider_call_start` | TracedProvider | `provider_name`, `model`, `message_count` | | `provider_call_end` | TracedProvider | `provider_name`, `model`, `duration_seconds`, `output_length` | | `provider_call_error` | TracedProvider | `provider_name`, `model`, `error` | ## Subscribing to Trace Events ``` from pyagent_trace.events import TraceEventBus from pyagent_trace.exporters.console import ConsoleExporter from pyagent_trace.exporters.jsonl import JsonlExporter bus = TraceEventBus() # Fan-out to multiple backends bus.subscribe(ConsoleExporter().export_event) bus.subscribe(JsonlExporter("traces/run.jsonl").export_event) # Filter by event type bus.subscribe_filter({"agent_end", "compression"}, lambda e: print(e)) # Custom handler bus.subscribe(lambda event: my_metrics.record(event)) ``` ## Design Principles 1. **No auto-wiring** — hooks must be explicitly set. Blueprint config declares intent; you wire the runtime. 1. **No persistence changes** — `ContextLedger` is in-memory only; persistence is your concern. 1. **No ProviderProtocol changes** — `TracedProvider` wraps existing providers; the interface is unchanged. 1. **No pattern logic changes** — `_execute()` implementations are untouched; hooks wrap around them. 1. **Backward compatible** — all hooks default to `None`; existing code works identically. ## Full Example See [`examples/25_full_stack.py`](https://github.com/pyagent-core/pyagent/blob/main/examples/25_full_stack.py) for a complete blueprint → compile → wire → run → observe workflow. # Routing Guide Automatically route each agent call to the cheapest LLM model that can handle the task. `pyagent-router` scores task difficulty 1-10, filters by capability, and picks the lowest-cost model that meets both requirements — transparently, without changing your agent code. ``` pip install pyagent-router ``` ______________________________________________________________________ ## The Problem Running every agent call through `gpt-4o` or `claude-sonnet` is the path of least resistance. It's also the most expensive one. In a real workflow, most tasks are easy: ``` "What is 2+2?" → difficulty 1 → gpt-4.1-nano ($0.000001) "Summarize this paragraph" → difficulty 3 → gpt-4o-mini ($0.000015) "Design a consensus protocol" → difficulty 9 → claude-sonnet ($0.000180) ``` Routing the first two to cheaper models saves 90-99% on those calls. At 10,000 calls per day that's a significant monthly saving. ______________________________________________________________________ ## Architecture ``` flowchart LR T[Task text] --> DS[DifficultyScorer\n1–10 score] DS --> MS[ModelSelector\nfilter by capability + cost] MS --> R[RoutedAgent\ncall optimal LLM] R --> L[routing_log] subgraph CostEstimator P[Pricing registry\n$/1M tokens] end MS --> P ``` ______________________________________________________________________ ## Quick Start ``` from pyagent_router.scorer import DifficultyScorer from pyagent_router.selector import ModelSelector, Capability # Score task difficulty scorer = DifficultyScorer() score = scorer.score("What is the capital of France?") print(f"{score.score}/10 — {score.category}") # 2/10 — easy # Auto-select cheapest viable model selector = ModelSelector() result = selector.select("What is the capital of France?") print(result.model) # "gpt-4.1-nano" # Require a capability result = selector.select( "Write a Python async HTTP client", required_capability=Capability.CODE, ) print(result.model) # "gpt-4o-mini" (cheapest model with CODE capability) ``` ______________________________________________________________________ ## DifficultyScorer Scores any text 1-10 using heuristics — no LLM call required, sub-millisecond. ``` from pyagent_router.scorer import DifficultyScorer scorer = DifficultyScorer() # Easy e = scorer.score("What year was Python created?") print(e.score, e.is_easy) # 2, True # Medium m = scorer.score("Explain the difference between TCP and UDP with examples") print(m.score, m.is_medium) # 5, True # Hard h = scorer.score( "Design a Byzantine fault-tolerant consensus protocol achieving " "sub-second finality under 33% adversarial nodes. Prove safety " "and liveness properties formally." ) print(h.score, h.is_hard) # 9, True print(h.signals) # {"length": 0.4, "complexity_keywords": 0.9, "multi_part": 0.7, "technical": 0.95} ``` ### Score ranges | Range | Category | Typical tasks | | ----- | -------- | ---------------------------------------------------------------------------- | | 1–3 | easy | Factual lookups, arithmetic, translations, single-sentence rewrites | | 4–6 | medium | Summaries, code explanations, comparisons, multi-paragraph drafts | | 7–10 | hard | System design, formal proofs, multi-step reasoning, synthesis across sources | ______________________________________________________________________ ## CostEstimator Compare costs across models before committing to one. ``` from pyagent_router.estimator import CostEstimator estimator = CostEstimator() # Single estimate e = estimator.estimate("gpt-4o-mini", input_tokens=1_000, output_tokens=500) print(f"${e.total_cost:.6f}") # $0.000225 # Compare across models — sorted cheapest first estimates = estimator.compare( "Explain async/await in Python", models=["gpt-4.1-nano", "gpt-4o-mini", "gpt-4.1-mini", "gpt-4o"], ) for e in estimates: print(f"{e.model:22} ${e.total_cost:.6f}") # gpt-4.1-nano $0.000003 # gpt-4o-mini $0.000023 # gpt-4.1-mini $0.000089 # gpt-4o $0.000187 ``` ### Built-in pricing table | Model | Input ($/1M) | Output ($/1M) | | ---------------- | ------------ | ------------- | | gpt-4.1-nano | $0.10 | $0.40 | | gpt-4o-mini | $0.15 | $0.60 | | gpt-4.1-mini | $0.40 | $1.60 | | gpt-4.1 | $2.00 | $8.00 | | gpt-4o | $2.50 | $10.00 | | claude-haiku-3-5 | $0.80 | $4.00 | | claude-sonnet-4 | $3.00 | $15.00 | | o3-mini | $1.10 | $4.40 | | o3 | $10.00 | $40.00 | ______________________________________________________________________ ## ModelSelector Combines `DifficultyScorer` and `CostEstimator` — selects the cheapest model that meets both difficulty and capability requirements. ``` from pyagent_router.selector import ModelSelector, Capability selector = ModelSelector() # Basic — cheapest model for the difficulty result = selector.select("Translate 'hello' to French") print(result.model) # "gpt-4.1-nano" print(result.difficulty) # DifficultyScore(score=1, ...) print(result.estimate) # CostEstimate(total_cost=0.0000003...) # Capability filter — must support CODE result = selector.select( "Implement Dijkstra's algorithm in Python", required_capability=Capability.CODE, ) print(result.model) # "gpt-4o-mini" — cheapest with CODE # Vision task — must support image understanding result = selector.select( "Describe what's in this chart", required_capability=Capability.VISION, ) print(result.model) # "gpt-4o" — only model with VISION in default specs ``` ### Capabilities ``` from pyagent_router.selector import Capability Capability.GENERAL # all models — factual, summaries, rewrites Capability.CODE # code generation, review, debugging Capability.MATH # numerical reasoning, calculations Capability.REASONING # multi-step logical reasoning, planning Capability.CREATIVE # creative writing, brainstorming, storytelling Capability.VISION # image understanding (multimodal models only) ``` ### Custom model specs Override the default registry with your own models and pricing. ``` from pyagent_router.selector import ModelSelector, ModelSpec, Capability selector = ModelSelector(specs=[ ModelSpec( "my-fast-model", min_difficulty=1, max_difficulty=4, capabilities={Capability.GENERAL}, max_context=32_000, ), ModelSpec( "my-smart-model", min_difficulty=3, max_difficulty=10, capabilities={Capability.GENERAL, Capability.CODE, Capability.REASONING}, max_context=200_000, ), ]) ``` ______________________________________________________________________ ## RouterMiddleware Wrap any agent to auto-route each call — zero changes to your patterns or callers. ``` import asyncio from pyagent_patterns.base import Agent, Message from pyagent_patterns.orchestration import Pipeline from pyagent_providers import AnthropicLLM, OpenAILLM from pyagent_router.middleware import RouterMiddleware from pyagent_router.selector import Capability # Map model names to LLM callables model_registry = { "gpt-4.1-nano": OpenAILLM("gpt-4.1-nano"), "gpt-4o-mini": OpenAILLM("gpt-4o-mini"), "gpt-4o": OpenAILLM("gpt-4o"), "claude-sonnet-4-20250514": AnthropicLLM("claude-sonnet-4-20250514"), } middleware = RouterMiddleware(model_registry=model_registry) # Wrap a single agent routed = middleware.wrap( Agent("analyst", OpenAILLM("gpt-4o"), system_prompt="Analyze the data."), ) result = asyncio.run(routed.run([Message.user("What is 2+2?")])) print(result.metadata["routed_model"]) # "gpt-4.1-nano" print(routed.routing_log[-1].difficulty.score) # 1 ``` ### Wrap with required capability ``` # This agent always gets a CODE-capable model code_agent = middleware.wrap( Agent("coder", OpenAILLM("gpt-4o"), system_prompt="Write production Python."), required_capability=Capability.CODE, ) # This agent always gets a VISION-capable model vision_agent = middleware.wrap( Agent("vision", OpenAILLM("gpt-4o"), system_prompt="Describe images."), required_capability=Capability.VISION, ) ``` ### Wrap an entire pipeline ``` pipeline = Pipeline(stages=[ middleware.wrap(extractor_agent), middleware.wrap(analyst_agent), middleware.wrap(writer_agent), ]) result = asyncio.run(pipeline.run(document)) # Each stage independently routes to the cheapest model for its task ``` ______________________________________________________________________ ## Routing Log and Cost Analysis Every routing decision is recorded — use it to analyze cost savings and tune thresholds. ``` import asyncio from pyagent_router.estimator import CostEstimator routed = middleware.wrap(agent) estimator = CostEstimator() tasks = [ "What is 1+1?", "Summarize the French Revolution in 3 bullet points.", "Design a distributed rate-limiting system. Cover CAP theorem trade-offs.", ] for task in tasks: asyncio.run(routed.run([Message.user(task)])) # Inspect decisions print(f"{'Task':50} {'Model':22} {'Diff':5} {'Cost':>10}") print("-" * 92) for entry in routed.routing_log: print(f"{entry.task_text[:48]:50} {entry.model:22} {entry.difficulty.score:5} ${entry.estimate.total_cost:.6f}") # Calculate savings vs always using gpt-4o routed_cost = sum(e.estimate.total_cost for e in routed.routing_log) premium_cost = sum( estimator.estimate("gpt-4o", e.estimate.input_tokens, e.estimate.output_tokens).total_cost for e in routed.routing_log ) print(f"\nRouted cost: ${routed_cost:.6f}") print(f"Premium cost: ${premium_cost:.6f}") print(f"Saved: ${premium_cost - routed_cost:.6f} ({(1 - routed_cost/premium_cost):.0%} reduction)") ``` ______________________________________________________________________ ## Integrating with ProviderRouter `pyagent-router` selects **which model** to use. `pyagent-providers`' `ProviderRouter` selects **which API provider** to use. They compose naturally: ``` from pyagent_providers.router import ProviderRouter, RoutingStrategy from pyagent_providers.registry import ProviderRegistry from pyagent_router.middleware import RouterMiddleware # Provider routing: pick cheapest healthy provider provider_router = ProviderRouter(registry, strategy=RoutingStrategy.COST_FIRST) # Model routing: pick cheapest viable model model_middleware = RouterMiddleware(model_registry=registry_from_provider(provider_router)) # Combined: cheapest provider AND cheapest model for each call agent = model_middleware.wrap( Agent("analyst", llm_from_provider(provider_router)) ) ``` ______________________________________________________________________ ## See Also - [Router Package](https://pyagent.org/packages/router/index.md) — `DifficultyScorer`, `CostEstimator`, `ModelSelector` API - [Providers Package](https://pyagent.org/packages/providers/index.md) — `ProviderRouter` for provider-level routing - [Hooks Guide](https://pyagent.org/guides/hooks/index.md) — wiring routing via `set_trace_bus()` for cost tracking - [API Reference](https://pyagent.org/api/router/index.md) # Providers Guide Multi-provider abstraction layer for LLM applications — register, route, negotiate, and optimize across providers. ## Overview `pyagent-providers` gives you a unified interface for working with multiple LLM providers (OpenAI, Anthropic, local models, etc.) through a single `ProviderProtocol`. ## Key Concepts ### ProviderProtocol Every provider implements this protocol: ``` from pyagent_providers import ProviderProtocol class MyProvider(ProviderProtocol): async def complete(self, messages, **kwargs) -> CompletionResult: ... async def health_check(self) -> HealthStatus: ... def capabilities(self) -> ProviderCapabilities: ... ``` ### ProviderRegistry Central registry for named providers: ``` from pyagent_providers import ProviderRegistry, MockProvider registry = ProviderRegistry() registry.register("primary", MockProvider(name="gpt-4o", model="gpt-4o")) registry.register("fallback", MockProvider(name="gpt-mini", model="gpt-4o-mini")) provider = registry.get("primary") ``` ### ProviderRouter Route requests based on strategies: | Strategy | Description | | ------------------ | -------------------------------------------------- | | `capability_first` | Pick first provider matching required capabilities | | `cost_first` | Pick cheapest provider | | `latency_first` | Pick lowest-latency provider | | `round_robin` | Rotate across providers | ``` from pyagent_providers import ProviderRouter router = ProviderRouter(registry, strategy="cost_first") result = await router.route(messages, required_capabilities=["streaming"]) ``` ### FallbackChain Resilient completion with automatic failover: ``` from pyagent_providers import FallbackChain chain = FallbackChain(providers=[primary, secondary, tertiary]) result = await chain.complete(messages) # Automatically tries next provider on failure ``` ### CapabilityNegotiator Find providers matching specific capabilities: ``` from pyagent_providers import CapabilityNegotiator negotiator = CapabilityNegotiator(registry) matches = negotiator.find(required=["function_calling", "streaming"]) ``` ### CostOptimizer Compare costs across providers for a given workload: ``` from pyagent_providers import CostOptimizer optimizer = CostOptimizer(registry) ranked = optimizer.rank_by_cost(prompt_tokens=1000, completion_tokens=500) ``` ## TracedProvider Wrap any provider to emit trace events for every LLM call — useful for observability without changing agent code: ``` from pyagent_providers import TracedProvider from pyagent_trace.events import TraceEventBus bus = TraceEventBus() traced = TracedProvider(registry.get("primary"), event_bus=bus) # Use as a drop-in replacement agent = Agent("analyst", llm=traced) # → emits provider_call_start, provider_call_end (or provider_call_error) ``` `TracedProvider` implements `ProviderProtocol`, so it works anywhere a provider is expected — as an `Agent`'s `llm` parameter, in a `FallbackChain`, or registered in a `ProviderRegistry`. → See the full [Hooks Guide](https://pyagent.org/guides/hooks/index.md) for all hook types including agent-level tracing. ## Integration with Blueprint In a blueprint YAML, providers are declared and referenced by agents: ``` providers: primary: model: gpt-4o fallback: model: gpt-4o-mini agents: classifier: prompt: "Classify the input" provider: primary ``` The `BlueprintCompiler` resolves these references through the `ProviderRegistry`. ## API Reference → See the full [Providers API Reference](https://pyagent.org/api/providers/index.md). # Compression Guide Reduce the tokens passed between agents in pipelines and fan-outs. `pyagent-compress` intercepts inter-agent messages, extracts the highest-density content, and passes a compressed version to the next stage — saving cost without losing the signal. ``` pip install pyagent-compress ``` ______________________________________________________________________ ## The Problem In a 5-agent pipeline, each stage's verbose output becomes the next stage's full input. LLMs tend to produce filler: ``` "Let me think about this carefully. Based on my analysis, I believe that revenue increased by approximately 15% on a year-over-year basis. It's worth noting, and I think this is quite significant, that the profit margin also expanded to around 23%..." ``` The signal here is: **revenue +15% YoY, margin 23%**. The rest is padding — and you're paying for every token of it at every stage. Without compression: ``` Stage 1 output: 1,200 tokens → passes 1,200 to Stage 2 Stage 2 output: 1,800 tokens → passes 1,800 to Stage 3 Stage 3 output: 2,100 tokens → passes 2,100 to Stage 4 ───────────────────── Total input tokens (stages 2-4): 7,100 tokens ``` With `CompressMiddleware(target_ratio=0.5)`: ~3,550 input tokens across stages 2-4. ______________________________________________________________________ ## Architecture ``` flowchart LR A1[Agent 1\noutput: 1200 tok] --> MC[MessageCompressor\ntarget_ratio: 0.5] MC -->|600 tok| A2[Agent 2] A2 --> MC2[MessageCompressor] MC2 --> A3[Agent 3] subgraph TokenBudget WL[workflow_limit: 50k] PL[per_agent_limit: 10k] end MC --> WL MC2 --> WL ``` ______________________________________________________________________ ## Quick Start ### Compress a single message ``` from pyagent_compress import MessageCompressor compressor = MessageCompressor(target_ratio=0.5) result = compressor.compress( "Let me think carefully. Revenue increased 15% YoY. " "It's worth noting that margins expanded to 23%, which is significant." ) print(result.compressed_text) # "Revenue increased 15% YoY. Margins expanded to 23%." print(f"{result.original_tokens} → {result.compressed_tokens} tokens ({result.savings_pct:.0%} saved)") ``` ### Wrap a Pipeline ``` import asyncio from pyagent_compress import CompressMiddleware, TokenBudget from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Pipeline from pyagent_providers import AnthropicLLM, OpenAILLM budget = TokenBudget(workflow_limit=30_000, per_agent_limit=8_000) middleware = CompressMiddleware(target_ratio=0.5, budget=budget) pipeline = Pipeline(stages=[ middleware.wrap(Agent( "extractor", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Extract all facts, figures, and entities.", )), middleware.wrap(Agent( "analyst", OpenAILLM("gpt-4o-mini"), system_prompt="Analyze the extracted data.", )), # Last stage — output goes to the user, no compression needed Agent( "writer", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Write the final brief.", ), ]) result = asyncio.run(pipeline.run(open("earnings.txt").read())) print(budget.summary()) ``` ______________________________________________________________________ ## Compression Strategies Three strategies, each with different quality/speed trade-offs: ``` from pyagent_compress import MessageCompressor # Extractive (default for long text): keeps highest information-density sentences extractive = MessageCompressor(target_ratio=0.5, strategy="extractive") # Truncate: keeps the first N tokens — fastest, works well for structured data truncate = MessageCompressor(target_ratio=0.5, strategy="truncate") # Auto: extractive for inputs > 200 tokens, truncate for shorter ones auto = MessageCompressor(target_ratio=0.5) # default ``` | Strategy | Speed | Quality | Best for | | ------------ | ------ | ------- | --------------------- | | `extractive` | medium | high | Narrative LLM outputs | | `truncate` | fast | medium | JSON, code, lists | | `auto` | fast | high | General use | ______________________________________________________________________ ## TokenBudget Track total token consumption across the workflow and enforce limits per agent. ``` from pyagent_compress import TokenBudget, BudgetExceededError budget = TokenBudget( workflow_limit=50_000, # total tokens for the whole workflow per_agent_limit=10_000, # max tokens any single agent can consume ) # Manual tracking budget.consume("extractor", 3_200) budget.consume("analyst", 4_100) print(budget.summary()) # Total consumed: 7,300 / 50,000 (14.6%) # Remaining: 42,700 # By agent: {extractor: 3200, analyst: 4100} # Check before a call if budget.remaining("writer") < 2_000: print("Skipping writer — budget tight") # Strict mode raises on exceed strict = TokenBudget(workflow_limit=5_000, strict=True) try: strict.consume("big_agent", 6_000) except BudgetExceededError as e: print(f"Budget exceeded: {e}") ``` ______________________________________________________________________ ## CompressMiddleware `CompressMiddleware` wraps agents so output compression is automatic — no changes to the agent or the caller. ``` from pyagent_compress import CompressMiddleware, TokenBudget budget = TokenBudget(workflow_limit=40_000, per_agent_limit=8_000) middleware = CompressMiddleware(target_ratio=0.6, budget=budget) # Wrap individually agent = middleware.wrap(my_agent) # Wrap a list at once compressed_agents = middleware.wrap_all([stage1, stage2, stage3]) ``` ### Different ratios per agent High-verbosity agents (analysts, researchers) benefit from more aggressive compression. Output agents (writers) should never be compressed — their output goes to the user. ``` tight = CompressMiddleware(target_ratio=0.3, budget=budget) # aggressive moderate = CompressMiddleware(target_ratio=0.6, budget=budget) # light pipeline = Pipeline(stages=[ tight.wrap(verbose_researcher), tight.wrap(data_extractor), moderate.wrap(analyst), writer_agent, # no compression — output goes to user ]) ``` ______________________________________________________________________ ## Fan-Out Integration Fan-outs produce N independent verbose outputs. Compress each before the aggregator to stay within its context window. ``` import asyncio from pyagent_compress import CompressMiddleware, TokenBudget from pyagent_patterns.orchestration import FanOutFanIn from pyagent_patterns.base import Agent from pyagent_providers import GeminiLLM, AnthropicLLM budget = TokenBudget(workflow_limit=60_000, per_agent_limit=8_000) middleware = CompressMiddleware(target_ratio=0.4, budget=budget) fanout = FanOutFanIn( agents=[ middleware.wrap(Agent("bull", GeminiLLM("gemini-2.5-flash"), system_prompt="Strongest bullish case.")), middleware.wrap(Agent("bear", GeminiLLM("gemini-2.5-flash"), system_prompt="Strongest bearish case.")), middleware.wrap(Agent("macro", GeminiLLM("gemini-2.5-flash"), system_prompt="Macroeconomic risk factors.")), ], aggregator=Agent( "synthesis", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Synthesize all three perspectives into an investment memo.", ), ) result = asyncio.run(fanout.run("Nvidia at $3.2T — buy or pass?")) print(f"Budget used: {budget.summary()}") ``` ______________________________________________________________________ ## Agent Hook Integration Instead of wrapping with middleware, you can attach a compressor directly to an agent via the hook system. The agent compresses its own output automatically. ``` from pyagent_compress import MessageCompressor from pyagent_trace.events import TraceEventBus from pyagent_trace.exporters import ConsoleExporter from pyagent_patterns.base import Agent from pyagent_providers import AnthropicLLM bus = TraceEventBus() bus.subscribe(ConsoleExporter().export_event) agent = ( Agent("analyst", AnthropicLLM("claude-sonnet-4-20250514")) .set_compressor(MessageCompressor(target_ratio=0.5)) .set_trace_bus(bus) # compression event is emitted to the bus ) ``` When both `set_compressor()` and `set_trace_bus()` are wired, every compression emits a trace event with `original_tokens`, `compressed_tokens`, and `savings_pct`. **Hook vs. middleware:** | | `set_compressor()` hook | `CompressMiddleware.wrap()` | | ----------------- | ----------------------- | ---------------------------- | | API | fluent, per-agent | wraps agent externally | | Trace integration | automatic via bus | manual | | Best for | single agent, new code | existing pipelines, wrap_all | ______________________________________________________________________ ## AgentPruner Detect and remove agents that add no unique information — useful for optimizing fan-outs after a few runs. ``` from pyagent_compress import AgentPruner pruner = AgentPruner(min_contribution=0.3) message_history = [ {"agent": "bull", "content": "Strong earnings justify premium valuation."}, {"agent": "bear", "content": "Strong earnings justify premium valuation."}, # duplicate {"agent": "neutral", "content": "Key risk: AMD competition, $40B TAM affected."}, ] scores = pruner.score_agents(message_history, task="Evaluate Nvidia investment") # {"bull": 0.82, "bear": 0.18, "neutral": 0.91} to_prune = pruner.should_prune(scores) # ["bear"] — score 0.18 is below threshold 0.3 ``` ______________________________________________________________________ ## Cost Savings Reference | Workflow | Without Compression | With 50% Compression | Monthly saving (1k runs/day) | | ------------------------- | ------------------- | -------------------- | ---------------------------- | | 5-stage Pipeline (gpt-4o) | ~25k tokens | ~13k tokens | ~$900/mo | | 5-agent Fan-Out (gpt-4o) | ~30k tokens | ~16k tokens | ~$1,100/mo | | 3-round Debate (gpt-4o) | ~18k tokens | ~10k tokens | ~$600/mo | ______________________________________________________________________ ## See Also - [Compress Package](https://pyagent.org/packages/compress/index.md) — full API reference with all classes - [Tracing Guide](https://pyagent.org/guides/tracing/index.md) — viewing compression events in trace spans - [Hooks Guide](https://pyagent.org/guides/hooks/index.md) — `set_compressor()` and the other three hooks - [API Reference](https://pyagent.org/api/compress/index.md) # Context Guide Structured context memory with trust metadata, three-tier storage, compression, and retrieval for LLM agent systems. ## Overview `pyagent-context` provides a `ContextLedger` — an append-only log of `ContextItem` objects, each annotated with trust level and sensitivity metadata. Items flow through three memory tiers and can be compressed, retrieved by relevance, and redacted for safety. ## ContextItem The atomic unit of context: ``` from pyagent_context import ContextItem, TrustLevel, Sensitivity item = ContextItem( content="The customer's account was created on 2024-01-15.", source="database", trust=TrustLevel.VERIFIED, sensitivity=Sensitivity.INTERNAL, ) ``` ### TrustLevel | Level | Description | | --------------- | ----------------------------------- | | `VERIFIED` | Confirmed from authoritative source | | `INFERRED` | Derived by agent reasoning | | `USER_PROVIDED` | Supplied by end user | | `UNVERIFIED` | Unknown provenance | ### Sensitivity | Level | Description | | -------------- | ----------------------------------- | | `PUBLIC` | Safe to expose | | `INTERNAL` | Internal use only | | `CONFIDENTIAL` | Restricted access | | `PII` | Personally identifiable information | ## ContextLedger Append-only log with querying: ``` from pyagent_context import ContextLedger ledger = ContextLedger() ledger.append(item) ledger.total_tokens() messages = ledger.to_messages(budget=4000) ``` ## Three-Tier Memory ### WorkingMemory Bounded deque for current conversation turn: ``` from pyagent_context import WorkingMemory wm = WorkingMemory(max_items=20, max_tokens=8000) ``` ### SessionMemory Persisted across turns (JSON or SQLite backend): ``` from pyagent_context import SessionMemory sm = SessionMemory(backend="json", path="session.json") ``` ### SemanticMemory Vector-indexed for similarity search: ``` from pyagent_context import InMemorySemanticStore store = InMemorySemanticStore() store.add(item) results = store.search("billing question", top_k=3) ``` ## Compression Four policies for managing context window: | Policy | Description | | ------------------- | -------------------------------------------- | | `none` | No compression | | `fifo` | Drop oldest items first | | `semantic_lossless` | Preserve high-trust items, drop redundant | | `sawtooth` | Aggressive then gradual (preserves verified) | ``` from pyagent_context import ContextCompressor compressor = ContextCompressor(policy="semantic_lossless") compressed = compressor.compress(items, target_tokens=4000) ``` ## Trust-Aware Retrieval Score and rank items by trust, recency, and keyword relevance: ``` from pyagent_context import TrustAwareRetriever retriever = TrustAwareRetriever() results = retriever.retrieve(items, query="billing", top_k=5) ``` ## Lifecycle Management Expiry sweeps, freshness decay, and consolidation: ``` from pyagent_context import ContextLifecycle lifecycle = ContextLifecycle() lifecycle.sweep_expired(ledger) lifecycle.apply_freshness_decay(ledger) lifecycle.consolidate(ledger, similarity_threshold=0.8) ``` ## Redaction Filter items by sensitivity before sending to LLM: ``` from pyagent_context import ContextRedactor redactor = ContextRedactor(max_sensitivity=Sensitivity.INTERNAL) safe_items = redactor.redact(items) # PII items excluded ``` ## Agent Hook Integration Wire a `ContextLedger` into any agent using the built-in hook — the agent automatically reads context before each LLM call and writes its output back: ``` from pyagent_patterns.base import Agent, MockLLM from pyagent_context import ContextLedger, ContextItem, TrustLevel ledger = ContextLedger() ledger.append(ContextItem( content="Customer account created 2024-01-15", source="database", trust=TrustLevel.VERIFIED, )) agent = Agent("support", MockLLM(responses=["Your account was created on Jan 15, 2024."])) agent.set_context(ledger) result = await agent.run("When was my account created?") # → ledger now contains the agent's output as an INFERRED item print(len(ledger.items)) # 2 (original + agent output) ``` ### With RuntimeGraph ``` from pyagent_blueprint import load_blueprint, BlueprintCompiler graph = BlueprintCompiler().compile(load_blueprint("blueprint.yaml")) graph.wire_context(ledger) # sets context on ALL agents ``` → See the full [Hooks Guide](https://pyagent.org/guides/hooks/index.md) for all four hook types. ## API Reference → See the full [Context API Reference](https://pyagent.org/api/context/index.md). # pyagent-router **Automatic model routing based on task difficulty and cost** — score each task 1-10, pick the cheapest model that can handle it, and route calls transparently without changing your agent code. ``` pip install pyagent-router ``` ______________________________________________________________________ ## Why Routing Matters Running every task through `gpt-4o` or `claude-sonnet` is the easiest default — and the most expensive one. Most tasks in a real workflow are easy. ``` "What is 2+2?" → difficulty 1 → gpt-4.1-nano ($0.000001) "Summarize this email" → difficulty 3 → gpt-4o-mini ($0.000015) "Design a distributed → difficulty 8 → claude-sonnet ($0.000180) consensus algorithm" ``` Routing the first two tasks to cheaper models saves 90–99% of cost on those calls. Over 10,000 calls per day that compounds fast. ______________________________________________________________________ ## Architecture ``` flowchart LR T[Task text] --> DS[DifficultyScorer\n1–10 score] DS --> MS[ModelSelector\nfilter by capability + cost] MS --> R[RoutedAgent\ncall optimal LLM] R --> L[routing_log] subgraph CostEstimator P[Pricing registry\n$/1M tokens] end MS --> P ``` ______________________________________________________________________ ## DifficultyScorer Score any text 1-10 using heuristics — no LLM call required. ``` from pyagent_router.scorer import DifficultyScorer scorer = DifficultyScorer() easy = scorer.score("What is the capital of France?") print(easy.score) # 2 print(easy.category) # "easy" print(easy.is_easy) # True hard = scorer.score( "Design a Byzantine fault-tolerant consensus protocol that " "achieves sub-second finality under 33% adversarial nodes. " "Prove safety and liveness properties formally." ) print(hard.score) # 9 print(hard.is_hard) # True print(hard.signals) # {"length": 0.4, "complexity_keywords": 0.9, ...} ``` ### Score ranges | Range | Category | Examples | | ----- | -------- | ------------------------------------------------------ | | 1–3 | easy | Factual lookups, simple arithmetic, translations | | 4–6 | medium | Summaries, code explanations, comparisons | | 7–10 | hard | System design, proofs, multi-step reasoning, synthesis | ______________________________________________________________________ ## CostEstimator Estimate cost before a call and compare across models. ``` from pyagent_router.estimator import CostEstimator estimator = CostEstimator() # Estimate cost for a specific model estimate = estimator.estimate("gpt-4o-mini", input_tokens=1_000, output_tokens=500) print(f"${estimate.total_cost:.6f}") # $0.000225 # Compare across models to find the cheapest fit estimates = estimator.compare("Explain async/await in Python", models=["gpt-4.1-nano", "gpt-4o-mini", "gpt-4o"]) for e in estimates: print(f"{e.model:20} ${e.total_cost:.6f}") # gpt-4.1-nano $0.000003 # gpt-4o-mini $0.000023 # gpt-4o $0.000187 ``` ______________________________________________________________________ ## ModelSelector Combines `DifficultyScorer` and `CostEstimator` — pick the cheapest model that meets difficulty and capability requirements. ``` from pyagent_router.selector import ModelSelector, Capability selector = ModelSelector() # Auto-select based on difficulty result = selector.select("What is 2+2?") print(result.model) # "gpt-4.1-nano" print(result.difficulty) # DifficultyScore(score=1, ...) print(result.estimate) # CostEstimate(total_cost=...) # Require a specific capability result = selector.select( "Write a Python function to implement Dijkstra's algorithm", required_capability=Capability.CODE, ) print(result.model) # "gpt-4o-mini" or better (has CODE capability) ``` ### Capabilities ``` from pyagent_router.selector import Capability Capability.GENERAL # default, all models Capability.CODE # code generation and review Capability.MATH # numerical and symbolic reasoning Capability.REASONING # multi-step logical reasoning Capability.CREATIVE # creative writing and ideation Capability.VISION # image understanding (multimodal) ``` ### Custom model specs ``` from pyagent_router.selector import ModelSelector, ModelSpec, Capability selector = ModelSelector(specs=[ ModelSpec("my-cheap-model", min_difficulty=1, max_difficulty=4, capabilities={Capability.GENERAL}, max_context=32_000), ModelSpec("my-smart-model", min_difficulty=3, max_difficulty=10, capabilities={Capability.GENERAL, Capability.CODE, Capability.REASONING}, max_context=200_000), ]) ``` ______________________________________________________________________ ## RouterMiddleware Wrap any agent to route each call automatically — zero changes to the agent or the caller. ``` import asyncio from pyagent_patterns.base import Agent, Message from pyagent_patterns.orchestration import Pipeline from pyagent_providers import AnthropicLLM, OpenAILLM from pyagent_router.middleware import RouterMiddleware from pyagent_router.selector import Capability # Build a model registry mapping name → LLM callable model_registry = { "gpt-4.1-nano": OpenAILLM("gpt-4.1-nano"), "gpt-4o-mini": OpenAILLM("gpt-4o-mini"), "gpt-4o": OpenAILLM("gpt-4o"), "claude-sonnet-4-20250514": AnthropicLLM("claude-sonnet-4-20250514"), } middleware = RouterMiddleware(model_registry=model_registry) # Wrap individual agents agent = Agent("analyst", OpenAILLM("gpt-4o"), system_prompt="Analyze the given data.") routed = middleware.wrap(agent) result = asyncio.run(routed.run([Message.user("What is revenue growth?")])) print(result.metadata["routed_model"]) # "gpt-4o-mini" (easy task) print(routed.routing_log[-1]) # SelectionResult(model=..., difficulty=...) ``` ### Require a capability per agent ``` # Wrap agents with different capability requirements code_agent = middleware.wrap( Agent("coder", OpenAILLM("gpt-4o"), system_prompt="Write production Python code."), required_capability=Capability.CODE, ) vision_agent = middleware.wrap( Agent("vision", OpenAILLM("gpt-4o"), system_prompt="Describe this image."), required_capability=Capability.VISION, ) ``` ### Wrap a full pipeline ``` pipeline = Pipeline(stages=[ middleware.wrap(extractor_agent), middleware.wrap(analyst_agent), middleware.wrap(writer_agent), ]) result = asyncio.run(pipeline.run(document)) ``` ______________________________________________________________________ ## Routing Log Every routed call is recorded — useful for cost analysis and debugging. ``` routed = middleware.wrap(agent) # Run several tasks for task in tasks: await routed.run([Message.user(task)]) # Inspect routing decisions for entry in routed.routing_log: print(f"{entry.model:25} difficulty={entry.difficulty.score} " f"cost=${entry.estimate.total_cost:.6f}") # Aggregate cost savings actual_cost = sum(e.estimate.total_cost for e in routed.routing_log) premium_cost = sum( CostEstimator().estimate("gpt-4o", e.estimate.input_tokens, e.estimate.output_tokens).total_cost for e in routed.routing_log ) print(f"Saved: ${premium_cost - actual_cost:.4f} " f"({(1 - actual_cost/premium_cost):.0%} reduction)") ``` ______________________________________________________________________ ## Integration with ProviderRouter `pyagent-router` handles model selection (which model to use). `pyagent-providers`' `ProviderRouter` handles provider selection (which API endpoint to use). They compose naturally: ``` from pyagent_router.middleware import RouterMiddleware from pyagent_providers.router import ProviderRouter, RoutingStrategy from pyagent_providers.registry import ProviderRegistry # 1. Route to the right provider (lowest cost, best health) provider_router = ProviderRouter(registry, strategy=RoutingStrategy.COST_FIRST) # 2. Route to the right model (based on task difficulty) model_middleware = RouterMiddleware(model_registry=model_registry) agent = model_middleware.wrap( Agent("analyst", llm_from_provider(provider_router)) ) ``` ______________________________________________________________________ ## See Also - [Patterns Package](https://pyagent.org/packages/patterns/index.md) — `Agent`, `Pipeline`, `FanOutFanIn` and all orchestration patterns - [Providers Package](https://pyagent.org/packages/providers/index.md) — `ProviderRouter` for provider-level routing and fallback - [API Reference](https://pyagent.org/api/router/index.md) ______________________________________________________________________ ## Cookbook examples Complete, runnable recipes that use this package — [all examples using this package](https://pyagent.org/cookbook/index.md): - [Multi-agent code review system](https://pyagent.org/cookbook/software-engineering/code-review/index.md) - [Multi-agent SQL analytics assistant](https://pyagent.org/cookbook/data-analytics/sql-analyst/index.md) [Browse the full Cookbook →](https://pyagent.org/cookbook/index.md) # pyagent-providers **Multi-provider abstraction layer** — unified interface across OpenAI, Anthropic, Gemini, LangChain, and LiteLLM. Capability negotiation, health checks, fallback chains, and cost-optimized routing built in. ``` pip install pyagent-providers # Core (MockLLM only) pip install pyagent-providers[openai] # + OpenAI adapter pip install pyagent-providers[anthropic] # + Anthropic adapter pip install pyagent-providers[gemini] # + Google Gemini adapter pip install pyagent-providers[litellm] # + LiteLLM (100+ models via one interface) pip install pyagent-providers[langchain] # + LangChain adapter pip install pyagent-providers[all] # All adapters ``` ______________________________________________________________________ ## Architecture ``` flowchart LR A[Agent] --> PR[ProviderRegistry] PR --> CAP[CapabilityNegotiator\nselect by capability] PR --> HC[HealthChecker\ncircuit breaker] PR --> FC[FallbackChain\nretry next on failure] FC --> OAI[OpenAILLM\ngpt-4o, gpt-4o-mini] FC --> ANT[AnthropicLLM\nsonnet, haiku, opus] FC --> GEM[GeminiLLM\ngemini-2.5-pro/flash] FC --> LIT[LiteLLM\n100+ models] FC --> LCH[LangChainLLM\nany LangChain model] FC --> MOCK[MockLLM\ntesting] ``` ______________________________________________________________________ ## Quick Start — Direct Adapters Use any provider directly — no registry required. ``` import asyncio from pyagent_patterns.base import Agent, Message from pyagent_providers import AnthropicLLM, OpenAILLM, GeminiLLM # Anthropic agent_a = Agent("analyst", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="You are a financial analyst.") # OpenAI agent_b = Agent("researcher", OpenAILLM("gpt-4o-mini"), system_prompt="You are a research assistant.") # Gemini agent_c = Agent("fast_agent", GeminiLLM("gemini-2.5-flash"), system_prompt="Answer concisely.") result = asyncio.run(agent_a.run("Analyze Tesla's Q3 2025 results")) print(result.output) print(f"Model: {result.metadata.get('model')}") print(f"Tokens: {result.metadata.get('input_tokens')} in, {result.metadata.get('output_tokens')} out") print(f"Cost: ${result.cost_estimate:.5f}") ``` ______________________________________________________________________ ## LiteLLM — 100+ Models via One Interface LiteLLM lets you use any model from any provider with a single consistent API, including cost tracking and fallback. ``` pip install pyagent-providers[litellm] ``` ``` from pyagent_providers import LiteLLM from pyagent_patterns.base import Agent import asyncio # OpenAI via LiteLLM agent_openai = Agent("a", LiteLLM("gpt-4o"), system_prompt="Analyze this.") # Anthropic via LiteLLM agent_anthropic = Agent("b", LiteLLM("anthropic/claude-sonnet-4-20250514"), system_prompt="Analyze this.") # Gemini via LiteLLM agent_gemini = Agent("c", LiteLLM("gemini/gemini-2.5-pro"), system_prompt="Analyze this.") # Mistral, Cohere, Together, Groq — same interface agent_groq = Agent("d", LiteLLM("groq/llama-3.1-70b-versatile"), system_prompt="Analyze this.") agent_together = Agent("e", LiteLLM("together_ai/meta-llama/Llama-3-70b-chat-hf"), system_prompt="Analyze this.") # Cost tracking built-in result = asyncio.run(agent_anthropic.run("Summarize Q3 earnings")) print(f"Cost: ${result.cost_estimate:.5f}") ``` ______________________________________________________________________ ## LangChain Adapter — Wrap Any LangChain Model ``` pip install pyagent-providers[langchain] ``` ``` from langchain_openai import ChatOpenAI from langchain_anthropic import ChatAnthropic from langchain_google_genai import ChatGoogleGenerativeAI from pyagent_providers import LangChainLLM from pyagent_patterns.base import Agent import asyncio # Wrap any LangChain chat model agent_oai = Agent("writer", LangChainLLM(ChatOpenAI(model="gpt-4o", temperature=0.7)), system_prompt="Write clear technical content.") agent_ant = Agent("reviewer", LangChainLLM(ChatAnthropic(model="claude-sonnet-4-20250514", temperature=0)), system_prompt="Review for accuracy and clarity.") agent_gem = Agent("summarizer", LangChainLLM(ChatGoogleGenerativeAI(model="gemini-2.5-flash")), system_prompt="Summarize concisely.") # LangChain streaming passes through result = asyncio.run(agent_oai.run("Write a technical post about RAG")) ``` ______________________________________________________________________ ## ProviderRegistry — Centralized Provider Management Register providers once; look them up by name throughout your codebase. ``` from pyagent_providers import ProviderRegistry, AnthropicLLM, OpenAILLM, GeminiLLM registry = ProviderRegistry() # Register with alias names registry.register("fast", AnthropicLLM("claude-haiku-3-5-20241022")) registry.register("balanced", OpenAILLM("gpt-4o-mini")) registry.register("expert", AnthropicLLM("claude-sonnet-4-20250514")) registry.register("vision", GeminiLLM("gemini-2.5-pro")) # Retrieve anywhere in your app from pyagent_patterns.base import Agent extractor = Agent("extractor", registry.get("fast"), system_prompt="Extract facts.") analyst = Agent("analyst", registry.get("expert"), system_prompt="Analyze deeply.") ``` ### Registry from Blueprint providers block ``` from pyagent_blueprint import load_blueprint from pyagent_providers import ProviderRegistry, AnthropicLLM, OpenAILLM spec = load_blueprint("customer-support.yaml") # spec.providers: {fast: {provider: anthropic, model: claude-haiku-...}, ...} registry = ProviderRegistry.from_blueprint(spec, adapters={ "anthropic": AnthropicLLM, "openai": OpenAILLM, }) # All providers from the YAML are now wired and ready ``` ______________________________________________________________________ ## Capability Negotiation Some patterns need specific capabilities — streaming, function calling, vision. Negotiate automatically. ``` from pyagent_providers import ProviderRegistry, Capability registry = ProviderRegistry() registry.register("vision_capable", GeminiLLM("gemini-2.5-pro"), capabilities=[Capability.VISION, Capability.STREAMING, Capability.FUNCTION_CALLING]) registry.register("text_only", AnthropicLLM("claude-haiku-3-5-20241022"), capabilities=[Capability.STREAMING]) # Find all providers with streaming + function calling matches = registry.find(capabilities=[Capability.STREAMING, Capability.FUNCTION_CALLING]) print(f"Capable providers: {[m.name for m in matches]}") # → ["vision_capable"] # Select best provider for a specific task best = registry.select_for(task_type="image_analysis") ``` ______________________________________________________________________ ## Fallback Chains Automatically fall back to the next provider if one fails — handle rate limits, outages, and quota exhaustion gracefully. ``` from pyagent_providers import FallbackChain, AnthropicLLM, OpenAILLM, GeminiLLM # Try in order: primary → secondary → emergency chain = FallbackChain([ AnthropicLLM("claude-sonnet-4-20250514"), # primary OpenAILLM("gpt-4o"), # fallback if Anthropic is down GeminiLLM("gemini-2.5-pro"), # last resort ]) from pyagent_patterns.base import Agent import asyncio agent = Agent("resilient_agent", chain, system_prompt="Analyze this document.") result = asyncio.run(agent.run("Analyze Tesla Q3 earnings")) print(f"Provider used: {result.metadata.get('provider_used')}") print(f"Fallback triggered: {result.metadata.get('fallback_triggered', False)}") ``` ### Conditional fallback by error type ``` from pyagent_providers import FallbackChain chain = FallbackChain( providers=[primary, secondary, tertiary], fallback_on=["rate_limit_error", "service_unavailable", "timeout"], max_retries_per_provider=2, retry_delay_seconds=1.0, ) ``` ______________________________________________________________________ ## Health Checks Monitor provider availability and latency in real time. ``` from pyagent_providers import ProviderHealthChecker, AnthropicLLM, OpenAILLM, GeminiLLM import asyncio checker = ProviderHealthChecker() checker.add("anthropic-haiku", AnthropicLLM("claude-haiku-3-5-20241022")) checker.add("openai-mini", OpenAILLM("gpt-4o-mini")) checker.add("gemini-flash", GeminiLLM("gemini-2.5-flash")) async def check_all(): results = await checker.check_all() for name, status in results.items(): icon = "✓" if status.healthy else "✗" print(f"{icon} {name} latency: {status.latency_ms:.0f}ms status: {status.status}") asyncio.run(check_all()) # ✓ anthropic-haiku latency: 340ms status: healthy # ✓ openai-mini latency: 480ms status: healthy # ✓ gemini-flash latency: 290ms status: healthy ``` ### Health-aware routing ``` from pyagent_providers import HealthAwareRouter router = HealthAwareRouter( providers={"fast": haiku_llm, "balanced": mini_llm, "expert": sonnet_llm}, health_checker=checker, ) # Route to fastest healthy provider for this task llm = asyncio.run(router.select(preferred="expert", min_health_score=0.9)) ``` ______________________________________________________________________ ## Streaming All real providers support async streaming: ``` from pyagent_providers import AnthropicLLM, OpenAILLM from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Pipeline import asyncio pipeline = Pipeline(stages=[ Agent("analyst", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Write a detailed analysis."), ]) async def stream(): async for chunk in pipeline.stream("Analyze the impact of AI on software engineering"): print(chunk, end="", flush=True) print() asyncio.run(stream()) ``` ______________________________________________________________________ ## TracedProvider — Observability Without Code Changes `TracedProvider` wraps any `ProviderProtocol` to emit trace events for every LLM call — without modifying agent code or the underlying provider. ``` import asyncio from pyagent_providers import AnthropicLLM from pyagent_providers.traced import TracedProvider from pyagent_trace.events import TraceEventBus from pyagent_trace.exporters import ConsoleExporter, JsonlExporter bus = TraceEventBus() bus.subscribe(ConsoleExporter().export_event) bus.subscribe(JsonlExporter("traces/run.jsonl").export_event) # Wrap any existing provider — no changes to agent code traced = TracedProvider(AnthropicLLM("claude-sonnet-4-20250514"), trace_bus=bus) from pyagent_patterns.base import Agent agent = Agent("analyst", llm=traced, system_prompt="Analyze this document.") result = asyncio.run(agent.run("Tesla Q3 2025 earnings")) # Bus receives: provider_call_start → provider_call_end (or provider_call_error) # Console: [provider_call_start] agent=analyst model=claude-sonnet-4-20250514 # [provider_call_end] agent=analyst latency_ms=342 tokens=... ``` `TracedProvider` implements `ProviderProtocol` and works anywhere a provider is expected: ``` # In a FallbackChain from pyagent_providers import FallbackChain chain = FallbackChain([ TracedProvider(AnthropicLLM("claude-sonnet-4-20250514"), trace_bus=bus), TracedProvider(OpenAILLM("gpt-4o"), trace_bus=bus), ]) # In a ProviderRegistry registry = ProviderRegistry() registry.register("primary", TracedProvider(AnthropicLLM("claude-haiku-3-5-20241022"), trace_bus=bus)) ``` → See the full [Hooks Guide](https://pyagent.org/guides/hooks/index.md) for `agent.set_trace_bus()` and other hook types. ______________________________________________________________________ ## Provider Comparison | Provider | Best For | Relative Cost | Context Window | | ------------------------------------------- | ---------------------------- | ------------- | -------------- | | `AnthropicLLM("claude-haiku-3-5-20241022")` | Fast classification, routing | Low | 200k | | `AnthropicLLM("claude-sonnet-4-20250514")` | Complex reasoning, writing | Medium | 200k | | `OpenAILLM("gpt-4o-mini")` | Balanced tasks, JSON output | Low | 128k | | `OpenAILLM("gpt-4o")` | Complex reasoning, vision | Medium-High | 128k | | `GeminiLLM("gemini-2.5-flash")` | Very fast, large docs | Low | 1M | | `GeminiLLM("gemini-2.5-pro")` | Deep reasoning | Medium | 1M | | `LiteLLM("groq/llama-3.1-70b-versatile")` | Fast inference, low cost | Very Low | 128k | ______________________________________________________________________ ## Environment Variables ``` # Set API keys — providers auto-discover from environment export ANTHROPIC_API_KEY="sk-ant-..." export OPENAI_API_KEY="sk-..." export GOOGLE_API_KEY="..." # LiteLLM also reads these automatically # For other providers, see LiteLLM docs ``` ______________________________________________________________________ ## See Also - [Blueprint Package](https://pyagent.org/packages/blueprint/index.md) — declare providers in YAML `providers:` block - [Studio Package](https://pyagent.org/packages/studio/index.md) — `/providers` health dashboard in web UI - [Routing Guide](https://pyagent.org/guides/router/index.md) — difficulty-based model routing with pyagent-router - [API Reference](https://pyagent.org/api/providers/index.md) ______________________________________________________________________ ## Cookbook examples Complete, runnable recipes that use this package — [browse the Cookbook](https://pyagent.org/cookbook/index.md): - [Multi-agent portfolio review](https://pyagent.org/cookbook/finance-trading/portfolio-review/index.md) - [Multi-agent customer support router](https://pyagent.org/cookbook/customer-support/support-router/index.md) [Browse the full Cookbook →](https://pyagent.org/cookbook/index.md) # pyagent-compress **Token budget enforcement and inter-agent message compression** — reduce the tokens passed between agents in pipelines and fan-outs, saving cost without losing critical information. ``` pip install pyagent-compress ``` ______________________________________________________________________ ## Why Compression Matters In a 5-agent pipeline, each stage's verbose LLM output becomes the next stage's input. Without compression, token costs compound linearly. ``` Stage 1 output: 1,200 tokens → passes 1,200 to Stage 2 Stage 2 output: 1,800 tokens → passes 1,800 to Stage 3 Stage 3 output: 2,100 tokens → passes 2,100 to Stage 4 ... Total tokens consumed by stages 2-5: ~12,000 tokens (just on inputs) ``` With `CompressMiddleware(target_ratio=0.5)`, that becomes ~6,000 tokens — half the cost, same signal. ______________________________________________________________________ ## Architecture ``` flowchart LR A1[Agent 1\noutput: 1200 tok] --> MC[MessageCompressor\ntarget_ratio: 0.5] MC -->|600 tok| A2[Agent 2] A2 --> MC2[MessageCompressor] MC2 --> A3[Agent 3] subgraph TokenBudget WL[workflow_limit: 50k] PL[per_agent_limit: 10k] end MC --> WL MC2 --> WL ``` ______________________________________________________________________ ## MessageCompressor Core compression primitive — reduce a single message or string to a target token ratio. ``` from pyagent_compress import MessageCompressor compressor = MessageCompressor(target_ratio=0.5) verbose_output = """ Let me think about this carefully. The analysis I've conducted shows that revenue increased by approximately 15% on a year-over-year basis. It's worth noting, and I think this is quite significant, that the profit margin also expanded to around 23%, which is a meaningful improvement from the prior period's 19% margin. Additionally, there are several other factors to consider here... """ result = compressor.compress(verbose_output) print(result.compressed_text) print(f"Tokens: {result.original_tokens} → {result.compressed_tokens}") print(f"Savings: {result.savings_pct:.0%}") ``` ### Compression strategies ``` from pyagent_compress import MessageCompressor # Extractive: keep highest information-density sentences extractive = MessageCompressor(target_ratio=0.5, strategy="extractive") # Truncate: keep first N tokens (fastest, least intelligent) truncate = MessageCompressor(target_ratio=0.5, strategy="truncate") # Default (auto): extractive for long outputs, truncate for short auto = MessageCompressor(target_ratio=0.5) ``` ______________________________________________________________________ ## TokenBudget Track token consumption across a workflow and enforce per-agent and workflow-level limits. ``` from pyagent_compress import TokenBudget budget = TokenBudget( workflow_limit=50_000, # total tokens for the whole workflow per_agent_limit=10_000, # max tokens any single agent can consume ) # Record usage per agent budget.consume("extractor", 3_200) budget.consume("fact_checker", 4_100) budget.consume("writer", 2_800) print(budget.summary()) # Total consumed: 10,100 / 50,000 (20.2%) # Remaining: 39,900 # By agent: {extractor: 3200, fact_checker: 4100, writer: 2800} # Check before a call if budget.remaining("analyst") < 5000: print("Compressing input before calling analyst") # Raise on budget exceeded (strict mode) strict_budget = TokenBudget(workflow_limit=10_000, strict=True) try: strict_budget.consume("agent_x", 11_000) except BudgetExceededError: print("Budget exceeded — skipping this stage") ``` ______________________________________________________________________ ## CompressMiddleware Wrap agents so their outputs are automatically compressed before reaching the next stage. ``` import asyncio from pyagent_compress import CompressMiddleware, TokenBudget from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Pipeline from pyagent_providers import AnthropicLLM, OpenAILLM budget = TokenBudget(workflow_limit=30_000, per_agent_limit=8_000) middleware = CompressMiddleware(target_ratio=0.5, budget=budget) pipeline = Pipeline(stages=[ middleware.wrap(Agent( "extractor", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Extract all facts, figures, and entities.", )), middleware.wrap(Agent( "analyst", OpenAILLM("gpt-4o-mini"), system_prompt="Analyze the extracted data.", )), # Last stage doesn't need compression — its output goes to the user Agent( "writer", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Write the final brief.", ), ]) result = asyncio.run(pipeline.run(open("earnings_transcript.txt").read())) print(result.output) print(f"Budget used: {budget.summary()}") ``` ### Wrap all agents in one call ``` agents = [extractor_agent, analyst_agent, risk_agent, writer_agent] compressed_agents = middleware.wrap_all(agents) pipeline = Pipeline(stages=compressed_agents) ``` ______________________________________________________________________ ## Hook Integration — Agent-Level Compression The hook system lets you attach a compressor directly to an agent instead of wrapping it in middleware. The agent automatically compresses its output before returning, and emits a `compression` trace event when a `TraceEventBus` is also wired. ``` import asyncio from pyagent_compress import MessageCompressor from pyagent_trace.events import TraceEventBus from pyagent_trace.exporters import ConsoleExporter from pyagent_patterns.base import Agent from pyagent_providers import AnthropicLLM bus = TraceEventBus() bus.subscribe(ConsoleExporter().export_event) compressor = MessageCompressor(target_ratio=0.5) agent = ( Agent("analyst", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Provide a detailed financial analysis.") .set_compressor(compressor) .set_trace_bus(bus) # optional: emits compression event to bus ) result = asyncio.run(agent.run("Tesla Q3 2025 earnings — summarize key risks")) # Output is compressed before being returned; trace event includes savings_pct ``` Hook vs. middleware — when to use each: | Approach | When | API | | --------------------------- | ------------------------------------------------------ | ---------------------------------- | | `agent.set_compressor()` | Single agent, fluent wiring, hook pipeline | `Agent.set_compressor(compressor)` | | `CompressMiddleware.wrap()` | Existing Pipeline/FanOut, wrap multiple agents at once | `middleware.wrap(agent)` | Both are compatible — you can mix them in the same workflow. → See the full [Hooks Guide](https://pyagent.org/guides/hooks/index.md) for all four hook types. ______________________________________________________________________ ## AgentPruner Detect low-contribution agents in a multi-agent workflow and remove them to save cost in future runs. ``` from pyagent_compress import AgentPruner pruner = AgentPruner(min_contribution=0.3) # agents below 30% contribution score # Score agents based on how much unique information they added message_history = [ {"agent": "bull", "content": "Strong earnings growth justifies premium valuation"}, {"agent": "bear", "content": "Strong earnings growth justifies premium valuation"}, # duplicate {"agent": "neutral", "content": "Key risk: competition from AMD custom silicon, $40B TAM"}, ] scores = pruner.score_agents(message_history, task="Evaluate Nvidia investment") print(scores) # {"bull": 0.82, "bear": 0.18, "neutral": 0.91} to_prune = pruner.should_prune(scores) print(f"Prune these agents: {to_prune}") # → ["bear"] (score 0.18 < threshold 0.3) ``` ______________________________________________________________________ ## InteractionPruner Thin a long conversation history by removing low-value exchanges before injecting into an agent's context. ``` from pyagent_compress import InteractionPruner pruner = InteractionPruner(max_interactions=10, min_relevance=0.4) long_history = [ {"role": "user", "content": "Hi"}, {"role": "assistant", "content": "Hello! How can I help?"}, {"role": "user", "content": "I need help with my billing"}, # ... 30 more turns ... ] pruned = pruner.prune(long_history, current_query="I want to upgrade my plan") print(f"Pruned: {len(long_history)} → {len(pruned)} interactions") ``` ______________________________________________________________________ ## Integration with Fan-Out / Fan-In Parallel agents produce verbose independent analyses. Compress each before the aggregator to stay within the aggregator's context window. ``` import asyncio from pyagent_compress import CompressMiddleware, TokenBudget from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import FanOutFanIn from pyagent_providers import GeminiLLM, AnthropicLLM budget = TokenBudget(workflow_limit=60_000, per_agent_limit=8_000) middleware = CompressMiddleware(target_ratio=0.4, budget=budget) fanout = FanOutFanIn( agents=[ middleware.wrap(Agent("bull", GeminiLLM("gemini-2.5-flash"), system_prompt="Argue the strongest bullish investment case.")), middleware.wrap(Agent("bear", GeminiLLM("gemini-2.5-flash"), system_prompt="Argue the strongest bearish case.")), middleware.wrap(Agent("macro", GeminiLLM("gemini-2.5-flash"), system_prompt="Provide macroeconomic context and risk factors.")), ], aggregator=Agent( "analyst", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Synthesize all three perspectives into an investment memo.", ), ) result = asyncio.run(fanout.run("Nvidia at $3.2T market cap — invest or pass?")) print(f"Budget used: {budget.summary()}") print(f"Cost savings vs uncompressed: ~{budget.savings_estimate()}") ``` ______________________________________________________________________ ## Cost Savings Reference | Workflow | Without Compression | With 50% Compression | Saving | | -------------------------------- | ------------------- | -------------------- | -------------------- | | 5-stage Pipeline (gpt-4o) | ~25k tokens | ~13k tokens | ~$0.06 | | 5-agent Fan-Out (gemini-2.5-pro) | ~30k tokens | ~16k tokens | ~$0.05 | | 3-round Debate (gpt-4o) | ~18k tokens | ~10k tokens | ~$0.04 | | Typical monthly (1000 runs/day) | — | — | **$1,200–$1,800/mo** | ______________________________________________________________________ ## See Also - [Context Package](https://pyagent.org/packages/context/index.md) — `ContextLedger.to_messages(max_tokens=)` for token-budgeted injection - [Tracing Guide](https://pyagent.org/guides/tracing/index.md) — `pyagent.compress.savings_pct` attribute in OTel spans - [Compression Guide](https://pyagent.org/guides/compression/index.md) — full integration walkthrough - [API Reference](https://pyagent.org/api/compress/index.md) ______________________________________________________________________ ## Cookbook examples Complete, runnable recipes that use this package — [all examples using this package](https://pyagent.org/cookbook/index.md): - [Multi-agent research assistant](https://pyagent.org/cookbook/research-analysis/research-assistant/index.md) [Browse the full Cookbook →](https://pyagent.org/cookbook/index.md) # pyagent-context **Three-tier memory with trust-aware context ledger** — working memory for a single run, session memory across turns, and semantic memory for long-term recall. Agents share context without overloading LLM context windows. ``` pip install pyagent-context # Core (working + session memory) pip install pyagent-context[compress] # + ContextCompressor pip install pyagent-context[chromadb] # + ChromaDB semantic backend ``` ______________________________________________________________________ ## Architecture ``` flowchart TD A[Agent] --> WM[WorkingMemory\nbounded deque, evicts oldest] A --> SM[SessionMemory\nJSON or SQLite, persists across turns] A --> SEM[SemanticMemory\nChromaDB, vector similarity search] WM --> CL[ContextLedger\nappend-only log, query by trust] SM --> CL SEM --> CL CL --> MSG[to_messages(max_tokens=)\nconvert to LLM message list] ``` **Three tiers:** | Tier | Class | Scope | Backend | | -------- | ---------------- | ----------------------- | ------------------- | | Working | `WorkingMemory` | Single pattern run | In-memory deque | | Session | `SessionMemory` | Multi-turn conversation | JSON file or SQLite | | Semantic | `SemanticMemory` | Long-term recall | ChromaDB (vector) | ______________________________________________________________________ ## Core Concept — ContextItem Every piece of memory is a `ContextItem` — content plus trust metadata. ``` from pyagent_context.item import ContextItem, TrustLevel item = ContextItem( content="Customer prefers email communication, timezone UTC+5:30", source="crm_agent", trust_level=TrustLevel.VERIFIED, # VERIFIED > INFERRED > SPECULATIVE ) print(item.id) # UUID print(item.token_estimate) # rough token count print(item.timestamp) # creation time (float) print(item.to_dict()) # serialize to JSON-compatible dict ``` `TrustLevel` controls what agents can rely on: | Level | Meaning | Example | | ------------- | ------------------------------------ | ------------------------- | | `VERIFIED` | Confirmed by a reliable source | DB lookup, user-provided | | `INFERRED` | Derived by an agent — probably right | LLM classification result | | `SPECULATIVE` | Agent's guess — use with caution | Predicted intent | ______________________________________________________________________ ## Tier 1 — WorkingMemory Bounded deque for a single run. Automatically evicts the oldest items when capacity (count or tokens) is exceeded. ``` from pyagent_context.memory.working import WorkingMemory from pyagent_context.item import ContextItem, TrustLevel wm = WorkingMemory(max_items=50, max_tokens=20_000) # Add items — eviction happens automatically wm.add(ContextItem(content="User asked about Q3 revenue", source="input")) wm.add(ContextItem(content="Extractor found: $25.2B revenue", source="extractor_agent")) wm.add(ContextItem(content="Fact check: VERIFIED", source="fact_checker", trust_level=TrustLevel.VERIFIED)) print(f"Items: {len(wm)}") print(f"Tokens: {wm.total_tokens}") print(f"Utilization: {wm.utilization:.1%}") # Eviction returns what was dropped evicted = wm.add(ContextItem(content="new item", source="agent_x")) if evicted: print(f"Evicted {len(evicted)} item(s) to stay within budget") ``` ### Use in a Pipeline stage ``` import asyncio from pyagent_patterns.base import Agent, Message from pyagent_patterns.orchestration import Pipeline from pyagent_context.memory.working import WorkingMemory from pyagent_context.item import ContextItem from pyagent_providers import AnthropicLLM wm = WorkingMemory(max_tokens=10_000) async def run_with_memory(task: str) -> str: # Seed working memory with prior context wm.add(ContextItem(content=f"User goal: {task}", source="user")) pipeline = Pipeline(stages=[ Agent("extractor", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Extract key claims and figures."), Agent("analyst", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Analyze the extracted data and give a recommendation."), ]) result = await pipeline.run(task) # Store the result back into working memory wm.add(ContextItem(content=result.output, source="pipeline", trust_level=ContextItem.INFERRED)) print(f"Memory: {len(wm)} items, {wm.total_tokens} tokens") return result.output asyncio.run(run_with_memory("Analyze Tesla Q3 2025 earnings")) ``` ______________________________________________________________________ ## Tier 2 — SessionMemory Persists across turns — store conversation history so agents remember prior interactions. ``` from pyagent_context.memory.session import SessionMemory from pyagent_context.item import ContextItem, TrustLevel # JSON backend (simple, default) session = SessionMemory(session_id="user-42-support-ticket-991", backend="json") # Add items during turn 1 session.add(ContextItem(content="User reported: API returns 429 on /v2/search", source="user_turn_1")) session.add(ContextItem(content="Agent diagnosed: rate limit exceeded, quota 1000/min", source="agent_turn_1", trust_level=TrustLevel.INFERRED)) session.save() # persist to disk # ---- next turn / new process ---- session2 = SessionMemory(session_id="user-42-support-ticket-991", backend="json") history = session2.get_all() # load all prior items print(f"Prior turns: {len(history)} items") for item in history: print(f" [{item.trust_level}] {item.source}: {item.content[:60]}") ``` ### SQLite backend for concurrent multi-agent access ``` # Multiple agents writing to the same session safely session = SessionMemory( session_id="team-analysis-run-001", backend="sqlite", storage_path=".sessions", ) session.add(ContextItem(content="bull_agent: Strong revenue growth supports premium", source="bull_agent")) session.add(ContextItem(content="bear_agent: P/E 45x unsustainable", source="bear_agent")) session.save() ``` ______________________________________________________________________ ## Tier 3 — ContextLedger Append-only log with trust-aware querying and token-budget-aware message conversion. ``` from pyagent_context.ledger import ContextLedger from pyagent_context.item import ContextItem, TrustLevel ledger = ContextLedger() # Add items with different trust levels ledger.add("User is a senior developer", source="profile_agent", trust_level=TrustLevel.VERIFIED) ledger.add("User prefers concise answers", source="style_inference", trust_level=TrustLevel.INFERRED) ledger.add("User may be frustrated", source="sentiment_agent", trust_level=TrustLevel.SPECULATIVE) # Query by trust — only act on verified and inferred context reliable = ledger.query(min_trust=TrustLevel.INFERRED) print(f"Reliable context items: {len(reliable)}") # Query by recency recent = ledger.query(max_age_seconds=300) # last 5 minutes # Query by source from_profile = ledger.query(source="profile_agent") # Convert to messages with token budget for LLM injection messages = ledger.to_messages(max_tokens=2000) # most recent first, within budget print(f"Injecting {len(messages)} context messages ({ledger.total_tokens} total tokens)") # Snapshot for persistence snapshot = ledger.snapshot() restored = ContextLedger.from_snapshot(snapshot) ``` ### Inject ledger context into an agent call ``` from pyagent_patterns.base import Agent, Message from pyagent_providers import AnthropicLLM ledger = ContextLedger() ledger.add("User is CFO at a 500-person SaaS company", source="crm", trust_level=TrustLevel.VERIFIED) ledger.add("User asked about churn forecasting last week", source="history", trust_level=TrustLevel.VERIFIED) agent = Agent("advisor", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="You are a financial advisor. Use the provided context.") # Build messages: context first, then the actual query context_messages = ledger.to_messages(max_tokens=1500) query = Message.user("What's the best way to reduce churn in our enterprise tier?") import asyncio result = asyncio.run(agent.run_messages([*context_messages, query])) print(result.output) ``` ### Hook Integration — Automatic Context Wiring The hook system wires a `ContextLedger` directly into an agent: the agent reads context before each LLM call and writes its output back to the ledger automatically — no manual `to_messages()` needed. ``` import asyncio from pyagent_context.ledger import ContextLedger from pyagent_context.item import ContextItem, TrustLevel from pyagent_patterns.base import Agent from pyagent_providers import AnthropicLLM ledger = ContextLedger() ledger.add("User is CFO at a 500-person SaaS company", source="crm", trust_level=TrustLevel.VERIFIED) ledger.add("User asked about churn forecasting last week", source="history", trust_level=TrustLevel.VERIFIED) agent = ( Agent("advisor", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="You are a financial advisor. Use the provided context.") .set_context(ledger) # reads ledger before LLM call; writes output back after ) result = asyncio.run(agent.run("What's the best way to reduce churn in our enterprise tier?")) print(result.output) print(f"Ledger now has {len(ledger)} items") # original 2 + agent output = 3 ``` To wire context on all agents in a blueprint at once: ``` from pyagent_blueprint import load_blueprint, BlueprintCompiler graph = BlueprintCompiler().compile(load_blueprint("blueprint.yaml")) graph.wire_context(ledger) # sets context hook on every agent in the graph ``` → See the full [Hooks Guide](https://pyagent.org/guides/hooks/index.md) for all four hook types. ______________________________________________________________________ ## SemanticMemory (ChromaDB) Long-term recall via vector similarity — retrieve the most relevant past context, not just the most recent. ``` pip install pyagent-context[chromadb] ``` ``` from pyagent_context.memory.semantic import SemanticMemory from pyagent_context.item import ContextItem, TrustLevel # Persistent collection backed by ChromaDB sem = SemanticMemory(collection_name="agent_knowledge_base", persist_directory=".chromadb") # Add domain knowledge sem.add(ContextItem(content="PCI-DSS v4 requires tokenization of all cardholder data at rest", source="compliance_docs", trust_level=TrustLevel.VERIFIED)) sem.add(ContextItem(content="SOC2 Type II audit completed 2024-11, no critical findings", source="audit_report", trust_level=TrustLevel.VERIFIED)) sem.add(ContextItem(content="Customer XYZ has a data residency requirement: EU only", source="crm", trust_level=TrustLevel.VERIFIED)) # Retrieve by semantic similarity results = sem.search("What are our data compliance requirements?", top_k=3) for item, score in results: print(f"[{score:.2f}] {item.content[:80]}") # Combine with ledger ledger = ContextLedger(items=results[0]) # inject top semantic results ``` ______________________________________________________________________ ## ContextCompressor When the ledger grows beyond the LLM's context window, compress older items while preserving high-trust content. ``` pip install pyagent-context[compress] ``` ``` from pyagent_context.compression import ContextCompressor from pyagent_context.ledger import ContextLedger from pyagent_context.item import TrustLevel compressor = ContextCompressor(target_tokens=4000) # Ledger with 12k tokens of history large_ledger = ContextLedger(items=long_history) # Compress: summarize SPECULATIVE items, keep VERIFIED intact compressed = compressor.compress(large_ledger) print(f"Compressed: {large_ledger.total_tokens} → {compressed.total_tokens} tokens") print(f"Verified items preserved: all") ``` ______________________________________________________________________ ## Redaction Strip PII before items leave the trust boundary. ``` from pyagent_context.redaction import Redactor redactor = Redactor(patterns=["email", "phone", "credit_card", "ssn"]) safe = redactor.redact("Contact john.doe@acme.com or call +1-555-867-5309") print(safe) # → "Contact [EMAIL] or call [PHONE]" ``` ______________________________________________________________________ ## Full Example — Multi-Turn Customer Support Agent ``` import asyncio from pyagent_context.ledger import ContextLedger from pyagent_context.memory.session import SessionMemory from pyagent_context.item import ContextItem, TrustLevel from pyagent_patterns.base import Agent, Message from pyagent_providers import AnthropicLLM SESSION_ID = "support-ticket-4421" async def handle_turn(user_message: str) -> str: # Load prior session session = SessionMemory(session_id=SESSION_ID, backend="sqlite") ledger = ContextLedger(items=session.get_all()) # Add verified facts about the user (from CRM, set once) if len(ledger) == 0: ledger.add("User: Enterprise customer, plan: Pro, billing monthly", source="crm", trust_level=TrustLevel.VERIFIED) # Add this turn's input ledger.add(user_message, source="user", trust_level=TrustLevel.VERIFIED) agent = Agent( "support_agent", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="You are a helpful support agent. Use prior context to avoid repetition.", ) context_msgs = ledger.to_messages(max_tokens=3000) result = await agent.run_messages([*context_msgs, Message.user(user_message)]) # Persist agent response back to session ledger.add(result.output, source="agent", trust_level=TrustLevel.INFERRED) session._items = ledger.items session.save() return result.output # Simulated multi-turn conversation for msg in ["I can't access my dashboard", "I already tried reloading", "Same issue on mobile too"]: reply = asyncio.run(handle_turn(msg)) print(f"User: {msg}\nAgent: {reply}\n") ``` ______________________________________________________________________ ## When to Use Which Tier | Situation | Use | | -------------------------------------- | ---------------------------------------- | | Single pattern run, ephemeral context | `WorkingMemory` | | Multi-turn chat / conversation history | `SessionMemory` | | Domain knowledge, long-term recall | `SemanticMemory` (ChromaDB) | | Trust-filtered context injection | `ContextLedger` | | Token-budget enforcement | `ContextLedger.to_messages(max_tokens=)` | | PII protection before LLM injection | `Redactor` | ______________________________________________________________________ ## See Also - [Compress Package](https://pyagent.org/packages/compress/index.md) — token budget enforcement between pattern stages - [Trace Package](https://pyagent.org/packages/trace/index.md) — record agent interactions for replay and debugging - [Blueprint Package](https://pyagent.org/packages/blueprint/index.md) — declare context config in YAML with `context:` block - [API Reference](https://pyagent.org/api/context/index.md) ______________________________________________________________________ ## Cookbook examples Complete, runnable recipes that use this package — [browse the Cookbook](https://pyagent.org/cookbook/index.md): - [Multi-agent clinical note summarizer](https://pyagent.org/cookbook/healthcare/clinical-summary/index.md) [Browse the full Cookbook →](https://pyagent.org/cookbook/index.md) # Blueprint (Spec-Driven) # Blueprint **Verb: Declare.** The Blueprint pillar is `pyagent-blueprint` — a single package, independently installable, with zero mandatory dependency on the other three pillars. ## What it solves Hand-written multi-agent orchestration code conflates two different concerns: *design* (which agents exist, how they're wired, what pattern connects them) and *execution* (the actual Python that runs it). That conflation means every design review is a full code review, every diff is a line-level Python diff instead of a semantic one, and there's no way to statically validate a system's shape before spending a token on it. Blueprint separates the two. The design becomes a typed YAML document — validated, diffable, testable against a `MockLLM` with no live API calls, and compiled onto whichever `RuntimeAdapter` actually executes it. ## When to use - More than one person needs to review changes to the system's design, and a Python diff isn't legible enough for that review. - You want to validate a system's structure (dangling references, schema violations) before an LLM is ever called. - You need to swap which framework executes the system (LangGraph, CrewAI, the native `pyagent` registry, ...) without rewriting the design. - Governance requirements (budgets, SLAs, memory tiers, human-in-the-loop checkpoints) need to be declared once and either honored or surfaced as a diagnostic — never silently dropped. ## When not to use - Your orchestration logic depends on dynamic, runtime-computed control flow that can't be expressed as a static graph (e.g. an agent that decides to spin up an arbitrary number of sub-agents from a live computation) — hand-written code in your chosen framework is still the right tool. - It's a single agent, single call, with no second agent to wire in — there's no design to declare. See [why-blueprint.md](https://pyagent.org/why-blueprint/#when-you-dont-need-pyagent-at-all). - You're prototyping and the design changes every few minutes — the validate/compile cycle is overhead until the shape stabilizes. ## Tradeoffs Declarative-by-design is a real constraint, not just a feature: anything Blueprint's IR can't express (agents, typed workflows, named patterns), you can't declare — you drop to Python via a custom `RuntimeAdapter` instead. That's the tradeoff for getting diff/review/validate for free on everything the IR *can* express. ## Package `pip install pyagent-blueprint` — zero mandatory runtime dependency beyond `pydantic`, `pyyaml`, and `click`. Runtime execution requires installing an adapter (bundled zero-dependency reference adapters, or an extra like `pyagent-blueprint[langgraph]`). ## Example ``` api_version: pyagent/v1 metadata: name: customer-support agents: classifier: { prompt: "Classify into: billing, tech, general" } billing: { prompt: "Handle billing inquiries" } workflows: support: pattern: supervisor agents: { classifier: classifier, routes: { billing: billing } } ``` ``` pyagent-blueprint validate customer-support.yaml pyagent-blueprint test customer-support.yaml ``` See the full [Blueprint guide](https://pyagent.org/guides/blueprint/index.md) and [Why Blueprint?](https://pyagent.org/why-blueprint/index.md) for the complete spec format, CLI, and cross-framework comparison. ## Related pillars Blueprint's `agents`/`workflows` compile onto [Execution & Routing](https://pyagent.org/architecture/#execution-routing); its `providers` block feeds routing; its `context`/`observability` blocks wire [Context & Memory](https://pyagent.org/architecture/#context-memory) and [Observability](https://pyagent.org/architecture/observability/index.md) — but none of that wiring is required to use Blueprint's validate/diff/test capabilities on their own. # The PyAgent Blueprint Specification This page is the formal reference for what a `pyagent-blueprint` YAML document *means* — not a tutorial (see the [Blueprint guide](https://pyagent.org/guides/blueprint/index.md) for that), but the semantics each operation (`validate`, `compile`, `diff`, `test`) actually guarantees. ## Versioning Every blueprint declares `api_version: pyagent/v1`. Schema versions are tracked in `pyagent_blueprint.extensions.SCHEMA_VERSIONS` with an explicit frozen/unfrozen flag: - **`pyagent/v1` is frozen.** Once shipped, no breaking changes to it — only non-breaking documentation or bugfix corrections. Any structural addition or removal requires a new version. A `pyagent/v1` blueprint written today keeps working indefinitely. - Unknown `api_version` values are treated as **not frozen** (i.e. still in development) by `pyagent_blueprint.extensions.is_frozen()`. ## The authoritative schema [**pyagent.org/blueprint-schema.json**](https://pyagent.org/blueprint-schema.json) is generated directly from the real `pydantic.BaseModel` (`BlueprintSpec`) that `pyagent-blueprint` validates every YAML document against — via `scripts/gen_docs.py`, checked in CI (`gen_docs.py --check`) so it can never drift from the actual validator. It is not a hand-maintained approximation. ## Compilation semantics "Compiling" a blueprint means passing its `BlueprintIR` (parsed from `BlueprintSpec`) to a `RuntimeAdapter.compile()` — there is no runtime-independent compiled form; compilation is always *onto* a specific adapter (`pyagent`, `langgraph`, `crewai`, ...). Two guarantees hold across every adapter, enforced by the shared `AdapterConformanceSuite`: 1. **Governance is never silently dropped.** A declared budget, SLA, memory tier, guardrail, recovery policy, or human-in-the-loop checkpoint is either honored by the target adapter, or the compile step surfaces a stable diagnostic code (`DiagnosticSeverity.WARNING` or `.ERROR`) — `BUDGET_UNSUPPORTED`, `SLA_UNSUPPORTED`, `MEMORY_TIER_UNSUPPORTED`, and others in `pyagent_blueprint.diagnostics`, one per identified gap (`G1`–`G10`). `DiagnosticSeverity.INFO` codes (e.g. `PATTERN_LOWERED`) are expected, not failures — they note a lossless representation change, not a dropped feature. 1. **Pattern intent is preserved.** A `supervisor` pattern compiles to *something* that actually routes between the declared agents on the target runtime — not an approximation the conformance suite can't tell apart from a different pattern. ## Validation semantics `pyagent-blueprint validate` runs static checks *before* any adapter sees the spec: schema conformance (via the pydantic model), dangling references (an agent referenced by a workflow that doesn't exist in `agents:`), and, when a `RuntimeAdapter` is installed, pattern-name validity against that adapter's declared vocabulary — best-effort and adapter-dependent, since not every adapter has a fixed pattern vocabulary (e.g. a bare loop-based adapter). No LLM call happens during validation. ## Diff semantics `pyagent-blueprint diff old.yaml new.yaml` produces a list of `Change` records — `path` (dotted field path), `change_type` (`ADDED` / `REMOVED` / `MODIFIED`), and `severity`: - `INFO` — a change with no behavioral impact. - `WARNING` — a change worth a reviewer's attention but not breaking. - `BREAKING` — a change that alters the system's observable behavior (e.g. removing an agent a workflow still references, changing a workflow's pattern). This is a semantic diff over the IR, not a text diff of YAML — reordering keys produces no changes; changing a provider binding does. ## Contract-testing semantics `pyagent-blueprint test` runs `BlueprintTester` — contract-conformance checks executed against a deterministic `MockLLM`, never a live model call. It validates the system's *shape* (every declared workflow runs end-to-end, contracts' input/output constraints are satisfiable) without spending a token or requiring API credentials, which is what makes it safe to run in CI on every commit. ## Extension and customization rules Constructs a blueprint needs that have no native field in `BlueprintSpec` go under the reserved `x-pyagent` key prefix (`pyagent_blueprint.extensions.EXTENSION_NAMESPACE`), mirroring the pattern used by JSON Schema's own vendor-extension convention. Any key under this namespace is: - **Additive** — never required for a document to validate. - **Round-trip-safe by construction** — a consumer that doesn't recognize an `x-pyagent:*` key must ignore it gracefully, not fail. `contract.py`'s SLA metadata (which has no native JSON Schema equivalent) is the current real example: it's surfaced under `x-pyagent` in the published per-workflow contract schema rather than forcing SLA into core JSON Schema vocabulary. ## See also - [Blueprint pillar overview](https://pyagent.org/architecture/blueprint/index.md) — what it solves, when to use it - [Blueprint guide](https://pyagent.org/guides/blueprint/index.md) — tutorial and full YAML field reference - [Why Blueprint?](https://pyagent.org/why-blueprint/index.md) — the case against hand-written orchestration code - [pyagent.org/blueprint-schema.json](https://pyagent.org/blueprint-schema.json) — the authoritative schema # Blueprint Guide Declarative YAML specs for multi-agent LLM systems — validate, compile, test, diff, render, and generate. ## Overview `pyagent-blueprint` lets you define your entire agent system as a versioned YAML document. The blueprint is validated against a Pydantic schema, compiled into an executable `RuntimeGraph`, and can be tested, diffed, and rendered without writing code. ## Blueprint YAML Schema ``` api_version: pyagent/v1 metadata: name: customer-support version: 1.0.0 description: Support routing system providers: primary: model: gpt-4.1-mini agents: classifier: prompt: "Classify into: billing, tech, general" provider: primary billing: prompt: "Handle billing inquiries" provider: primary workflows: support: pattern: supervisor agents: classifier: classifier routes: billing: billing contracts: support: input: { type: string, max_tokens: 2000 } output: { type: string } sla: latency_p95_ms: 5000 cost_max_usd: 0.05 observability: tracing: { enabled: true } cost_budget: { daily_usd: 100.0 } ``` ## Loading ``` from pyagent_blueprint import load_blueprint, load_blueprint_from_str spec = load_blueprint("blueprint.yaml") # File spec = load_blueprint_from_str(yaml_string) # String ``` ## Compilation Transform YAML into a runnable graph: ``` from pyagent_blueprint import BlueprintCompiler compiler = BlueprintCompiler() graph = compiler.compile(spec) result = await graph.run("support", "I can't see my invoice") ``` ## Validation Static analysis catches issues before runtime: ``` from pyagent_blueprint import BlueprintValidator validator = BlueprintValidator() issues = validator.validate(spec) ``` **Checks performed:** - Dangling agent references - Dangling provider references - Unknown pattern names - Contract → workflow mismatches - Unrealistic SLA values - Hardcoded API keys in prompts ## Rendering Generate documentation automatically: ``` from pyagent_blueprint import BlueprintRenderer renderer = BlueprintRenderer() mermaid = renderer.to_mermaid(spec) # Mermaid flowchart markdown = renderer.to_markdown(spec) # Full Markdown doc ``` ## Diffing Semantic diff between blueprint versions: ``` from pyagent_blueprint import BlueprintDiffer differ = BlueprintDiffer() changes = differ.diff(old_spec, new_spec) print(differ.summary(changes)) ``` Severity levels: **BREAKING** (pattern/API changes), **WARNING** (prompt/provider changes), **INFO** (metadata). ## Contract Testing Test blueprints with MockLLM: ``` from pyagent_blueprint import BlueprintTester tester = BlueprintTester() results = await tester.test(spec) print(tester.summary(results)) ``` ## Scaffolding Generate a blueprint from a pattern name: ``` from pyagent_blueprint import BlueprintGenerator gen = BlueprintGenerator() yaml_str = gen.generate( pattern="supervisor", agents=["classifier", "billing", "tech"], ) ``` ## CLI ``` pyagent-blueprint validate blueprint.yaml pyagent-blueprint compile blueprint.yaml pyagent-blueprint render blueprint.yaml --format markdown pyagent-blueprint test blueprint.yaml pyagent-blueprint diff v1.yaml v2.yaml pyagent-blueprint generate --pattern supervisor --agents "a,b,c" ``` ## API Reference → See the full [Blueprint API Reference](https://pyagent.org/api/blueprint/index.md). # pyagent-blueprint **Spec-driven development for multi-agent systems** — declare your entire agent system in YAML, validate it statically, compile it to a live `RuntimeGraph`, diff versions semantically, and run it through Studio. Like Kubernetes for agent systems: infrastructure as code, but for LLM workflows. ``` pip install pyagent-blueprint pip install pyagent-blueprint[cli] # + blueprint CLI commands ``` ______________________________________________________________________ ## Why Blueprint? Writing multi-agent systems in pure Python works — until you need to version them, review them in a PR, test them without hitting real APIs, or hand them off to a team member who doesn't know where the agents are wired together. Blueprint separates **what** your system does (the YAML spec) from **how** it runs (the compiler and runtime). The same file that deploys to production can be statically validated in CI, simulated with MockLLM in tests, and diffed against the previous version in a PR description. ______________________________________________________________________ ## Architecture ``` flowchart LR Y[blueprint.yaml] --> L[BlueprintLoader\nparse + validate schema] L --> V[BlueprintValidator\nstatic checks] L --> C[BlueprintCompiler\nspec → RuntimeGraph] C --> RG[RuntimeGraph\nrunnable pattern DAG] RG --> E[Execute\npattern.run] L --> R[BlueprintRenderer\nMermaid / Markdown] L --> D[BlueprintDiffer\nsemantic diff old→new] L --> T[BlueprintTester\ncontract conformance] V -.->|issues| CLI[blueprint CLI] R -.->|diagram| CLI D -.->|changes| CLI ``` ______________________________________________________________________ ## Quick Start ``` # hello.yaml api_version: pyagent/v1 metadata: name: hello-world version: "1.0.0" providers: fast: provider: anthropic model: claude-haiku-3-5-20241022 agents: answerer: provider: fast prompt: "Answer questions concisely and helpfully." workflows: main: pattern: pipeline agents: stages: [answerer] ``` ``` import asyncio from pyagent_blueprint import load_blueprint, BlueprintCompiler spec = load_blueprint("hello.yaml") graph = BlueprintCompiler().compile(spec) result = asyncio.run(graph.run("main", "What is pyagent?")) print(result.output) ``` ______________________________________________________________________ ## YAML Sections at a Glance | Section | Purpose | | ---------------------------------------------------------------------------------- | ------------------------------------------------------- | | [`metadata`](https://pyagent.org/packages/blueprint/spec-format/#metadata) | Name, version, owner, tags | | [`providers`](https://pyagent.org/packages/blueprint/providers/index.md) | LLM provider bindings (model, tokens, timeout) | | [`agents`](https://pyagent.org/packages/blueprint/agents/index.md) | Agent definitions (prompt, provider, tools, guardrails) | | [`workflows`](https://pyagent.org/packages/blueprint/workflows/index.md) | Orchestration patterns + agent wiring + recovery | | [`context`](https://pyagent.org/packages/blueprint/context/index.md) | Memory tiers, compression, redaction | | [`contracts`](https://pyagent.org/packages/blueprint/contracts/index.md) | Input/output schemas, SLA constraints | | [`observability`](https://pyagent.org/packages/blueprint/contracts/#observability) | Tracing flags, cost budgets | ______________________________________________________________________ ## Sub-pages - [Spec Format](https://pyagent.org/packages/blueprint/spec-format/index.md) — annotated full YAML reference - [Agents](https://pyagent.org/packages/blueprint/agents/index.md) — prompts, providers, tools, guardrails - [Workflows](https://pyagent.org/packages/blueprint/workflows/index.md) — patterns, agent mapping, recovery - [Providers](https://pyagent.org/packages/blueprint/providers/index.md) — model tiers, multi-provider setup - [Context](https://pyagent.org/packages/blueprint/context/index.md) — memory, compression, redaction - [Contracts & Observability](https://pyagent.org/packages/blueprint/contracts/index.md) — SLA, cost budgets, tracing - [CLI](https://pyagent.org/packages/blueprint/cli/index.md) — validate, compile, render, test, diff, generate - [Python API](https://pyagent.org/packages/blueprint/compiler/index.md) — compiler, validator, differ, renderer, tester ______________________________________________________________________ ## See Also - [Studio Package](https://pyagent.org/packages/studio/index.md) — visual editor, simulation, governance dashboard - [Blueprint Guide](https://pyagent.org/guides/blueprint/index.md) — narrative walkthrough - [API Reference](https://pyagent.org/api/blueprint/index.md) # Blueprint — Agents Each entry under `agents:` defines a named agent that can be referenced in workflows. ______________________________________________________________________ ## AgentSpec Fields ``` agents: my_agent: provider: fast # Required if providers: block exists description: "..." # Human-readable, shown in Studio /agents view prompt: | # System prompt injected before every call ... tools: [] # Tool names — must be registered at compile time guardrails: [] # Guardrail refs — applied before output is returned ``` | Field | Type | Default | Description | | ------------- | ------------ | -------- | ------------------------------- | | `provider` | string | `""` | Key from the `providers:` block | | `description` | string | `""` | Shown in Studio /agents panel | | `prompt` | string | required | System prompt for this agent | | `tools` | list[string] | `[]` | Registered tool names | | `guardrails` | list[string] | `[]` | Registered guardrail names | ______________________________________________________________________ ## Prompts Prompts are system-level instructions prepended to every call. Keep them role-focused: ``` agents: classifier: prompt: | Classify the customer message into exactly one of: billing, technical, returns, general. Respond with ONLY the category name. billing_agent: prompt: | You are a billing specialist. Handle disputes, refunds, and subscription questions. Rules: 1. Acknowledge the customer's frustration first. 2. Confirm the charge details before offering a resolution. 3. Always provide a concrete next step with a timeline. formatter: prompt: | Format the input as a professional customer service response. - Remove any internal notes or agent reasoning. - Keep the response under 200 words. - End with a friendly closing line. ``` ### Prompt templating Prompts are plain strings — no special templating syntax. Inject dynamic data at runtime by passing it in the task string or via a context ledger wired through the compiler. ______________________________________________________________________ ## Providers Each agent references a named provider from the `providers:` block. A fast cheap model for simple tasks, an expert model for complex ones: ``` providers: fast: provider: anthropic model: claude-haiku-3-5-20241022 max_tokens: 512 expert: provider: anthropic model: claude-sonnet-4-20250514 max_tokens: 2048 agents: classifier: provider: fast # Cheap — just classifies billing_agent: provider: expert # Expensive — handles real edge cases formatter: provider: fast # Cheap — just reformats ``` If no `providers:` block is defined, all agents use `MockLLM` — ideal for unit tests. ______________________________________________________________________ ## Tools Tools listed here must be registered in the `ProviderRegistry` (or passed to `BlueprintCompiler`) at runtime. The blueprint declares intent; the compiler wires the actual callables. ``` agents: billing_agent: tools: [lookup_invoice, issue_refund, send_email] ``` ``` from pyagent_blueprint import BlueprintCompiler compiler = BlueprintCompiler( tools={ "lookup_invoice": my_lookup_fn, "issue_refund": my_refund_fn, "send_email": my_email_fn, } ) graph = compiler.compile(spec) ``` The `BlueprintValidator` will raise an error if a tool name is listed in an agent spec but not registered with the compiler. ______________________________________________________________________ ## Guardrails Like tools, guardrail names are resolved at compile time. Guardrails run after the agent produces output and before it is returned to the caller. ``` agents: public_facing_agent: guardrails: [no_pii, max_length_400, tone_check] ``` ``` compiler = BlueprintCompiler( guardrails={ "no_pii": pii_redaction_guard, "max_length_400": length_guard(400), "tone_check": tone_guard, } ) ``` ______________________________________________________________________ ## Agent Descriptions in Studio The `description` field powers the Studio `/agents` panel — it shows each agent's role alongside its full prompt, provider, and tool list. Use it to describe the agent's responsibility in one sentence. ``` agents: classifier: description: "Route incoming messages to billing, technical, returns, or general agents" billing_agent: description: "Handle billing disputes, refund requests, and subscription changes" ``` ______________________________________________________________________ ## See Also - [Workflows](https://pyagent.org/packages/blueprint/workflows/index.md) — how agents are wired into patterns - [Providers](https://pyagent.org/packages/blueprint/providers/index.md) — model and token configuration - [Contracts](https://pyagent.org/packages/blueprint/contracts/index.md) — per-agent input/output validation - [Full Spec Reference](https://pyagent.org/packages/blueprint/spec-format/index.md) # Blueprint — CLI ``` pip install pyagent-blueprint[cli] ``` Installing with `[cli]` adds the `blueprint` command. All commands accept a path to a blueprint YAML file and exit with code `1` on errors. ______________________________________________________________________ ## validate Static analysis — catch errors before deployment. ``` blueprint validate customer-support.yaml # ✓ customer-support.yaml is valid — no issues found. ``` With errors: ``` [error] workflows.main.agents.routes.billing: Agent ref 'billing_agnt' not found. Available: [billing_agent, technical_agent, returns_agent, general_agent, formatter] [warning] agents.legacy_agent: Defined but not referenced in any workflow. ``` Exit code `0` = valid, `1` = errors found.\ Add to CI: ``` - run: blueprint validate blueprints/production.yaml ``` ______________________________________________________________________ ## compile Compile the spec and print the resulting graph description as JSON. ``` blueprint compile customer-support.yaml ``` ``` { "workflows": { "main": { "pattern": "supervisor", "agents": ["classifier", "billing_agent", "technical_agent", "returns_agent", "general_agent", "formatter"], "config": {"default_route": "general"} } }, "metadata": { "name": "customer-support-system", "version": "1.2.0" } } ``` Useful for debugging agent wiring without running any LLM calls. ______________________________________________________________________ ## render Generate a Mermaid diagram or full Markdown documentation. ``` # Mermaid diagram (default) — printed to stdout blueprint render customer-support.yaml # Save to file blueprint render customer-support.yaml -o docs/architecture.md # Full Markdown documentation blueprint render customer-support.yaml --format markdown -o docs/system.md ``` Example Mermaid output: ``` flowchart TD IN[Input] --> classifier classifier -->|billing| billing_agent classifier -->|technical| technical_agent classifier -->|returns| returns_agent classifier -->|general| general_agent billing_agent --> formatter technical_agent --> formatter returns_agent --> formatter general_agent --> formatter formatter --> OUT[Output] ``` ______________________________________________________________________ ## test Run contract conformance checks with MockLLM — no API costs. ``` blueprint test customer-support.yaml # Testing 'customer-support-system'... # All contract checks passed. ``` With failures: ``` Results: 1 error(s), 1 warning(s) [error] contracts.response_length: Output exceeded max_length 400 (got 612) [warning] contracts.no_pii_in_response: Check skipped (no output generated) ``` ______________________________________________________________________ ## diff Semantic diff between two blueprint versions — shows what changed and the severity of each change. ``` blueprint diff customer-support-v1.yaml customer-support-v2.yaml ``` ``` + providers.premium: {provider: anthropic, model: claude-opus-4-20250514, max_tokens: 4096} ~ agents.billing_agent.provider: expert → premium [WARNING: model upgrade — cost increase] ~ workflows.main.config.default_route: general → billing [WARNING: routing change] - agents.legacy_agent [INFO: agent removed] ``` Severity levels: | Level | Examples | | ---------- | -------------------------------------------------------------- | | `BREAKING` | Pattern type changed, required field removed, workflow deleted | | `WARNING` | Prompt changed, provider swapped, config value changed | | `INFO` | Metadata update, description change, agent added/removed | ______________________________________________________________________ ## generate Scaffold a new blueprint YAML from a pattern and agent list. ``` blueprint generate --pattern pipeline --agents extractor,analyst,writer # Generated pipeline.yaml blueprint generate --pattern supervisor --agents classifier,billing,technical,general # Generated supervisor.yaml ``` The generated file includes the correct `api_version`, a `metadata` stub, provider placeholders, agent stubs with empty prompts, and the workflow wired to the chosen pattern. ______________________________________________________________________ ## All Commands ``` blueprint validate Static analysis blueprint compile Compile and print graph JSON blueprint render [--format mermaid|markdown] [-o ] blueprint test Contract conformance blueprint diff Semantic diff blueprint generate --pattern --agents ``` ______________________________________________________________________ ## See Also - [Python API](https://pyagent.org/packages/blueprint/compiler/index.md) — use the same tools from Python code - [Studio CLI](https://pyagent.org/packages/studio/cli/index.md) — `pyagent` commands for apply, simulate, diff, dashboard # Blueprint — Python API The `pyagent_blueprint` package exposes seven classes covering the full lifecycle: load → validate → compile → run → test → diff → render. ______________________________________________________________________ ## Loading ``` from pyagent_blueprint import load_blueprint, load_blueprint_from_str spec = load_blueprint("customer-support.yaml") # From file spec = load_blueprint_from_str(yaml_string) # From string ``` The `BlueprintSpec` object is a Pydantic model — all fields are type-checked at load time. ``` print(spec.metadata.name) # "customer-support-system" print(spec.metadata.version) # "1.2.0" print(list(spec.providers)) # ["fast", "balanced", "expert"] print(list(spec.agents)) # ["classifier", "billing_agent", ...] print(list(spec.workflows)) # ["main"] ``` ______________________________________________________________________ ## BlueprintValidator Static analysis — run before compiling or deploying. ``` from pyagent_blueprint.validator import BlueprintValidator, IssueSeverity validator = BlueprintValidator() issues = validator.validate(spec) errors = [i for i in issues if i.severity == IssueSeverity.ERROR] warnings = [i for i in issues if i.severity == IssueSeverity.WARNING] infos = [i for i in issues if i.severity == IssueSeverity.INFO] for issue in errors: print(f"[{issue.severity}] {issue.path}: {issue.message}") if errors: raise SystemExit(1) ``` Each `ValidationIssue` has: | Attribute | Type | Description | | ---------- | --------------- | -------------------------------------------------------------------------------- | | `severity` | `IssueSeverity` | `ERROR`, `WARNING`, or `INFO` | | `path` | string | Dotted path to the offending field (e.g. `workflows.main.agents.routes.billing`) | | `message` | string | Human-readable explanation | ______________________________________________________________________ ## BlueprintCompiler Transform a validated spec into a runnable `RuntimeGraph`. ``` import asyncio from pyagent_blueprint import load_blueprint, BlueprintCompiler from pyagent_providers import ProviderRegistry, AnthropicLLM, OpenAILLM registry = ProviderRegistry() registry.register("anthropic", AnthropicLLM) registry.register("openai", OpenAILLM) spec = load_blueprint("customer-support.yaml") graph = BlueprintCompiler(provider_registry=registry).compile(spec) result = asyncio.run(graph.run("main", "I was charged twice this month")) print(result.output) print(f"Route: {result.metadata.get('route_key')}") print(f"Cost: ${result.cost_estimate:.4f}") ``` ### Compile without real providers (MockLLM) ``` graph = BlueprintCompiler().compile(spec) # No provider_registry — uses MockLLM ``` ### Inspect the compiled graph ``` import json print(json.dumps(graph.describe(), indent=2)) # { # "workflows": { # "main": { "pattern": "supervisor", "agents": [...], "config": {...} } # }, # "metadata": { "name": "customer-support-system", "version": "1.2.0" } # } ``` ### Wire shared context ``` from pyagent_context import ContextLedger, ContextItem, TrustLevel ledger = ContextLedger() ledger.append(ContextItem( content="Customer account created 2024-01-15", source="database", trust=TrustLevel.VERIFIED, )) graph.wire_context(ledger) # Sets ledger on ALL agents result = asyncio.run(graph.run("main", "When was my account created?")) ``` ______________________________________________________________________ ## BlueprintRenderer Generate Mermaid diagrams or Markdown documentation programmatically. ``` from pyagent_blueprint import BlueprintRenderer renderer = BlueprintRenderer() # Mermaid flowchart mermaid_str = renderer.to_mermaid(spec) print(mermaid_str) # Full Markdown documentation markdown_str = renderer.to_markdown(spec) with open("docs/system.md", "w") as f: f.write(markdown_str) ``` ______________________________________________________________________ ## BlueprintDiffer Semantic diff between two spec versions — compare prompts, providers, agent wiring, and config. ``` from pyagent_blueprint import load_blueprint, BlueprintDiffer old_spec = load_blueprint("customer-support-v1.yaml") new_spec = load_blueprint("customer-support-v2.yaml") differ = BlueprintDiffer() changes = differ.diff(old_spec, new_spec) print(differ.summary(changes)) # + providers.premium: {provider: anthropic, model: claude-opus-4-20250514} # ~ agents.billing_agent.provider: expert → premium [WARNING] # - agents.legacy_agent [INFO] ``` Each `Change` has `kind` (ADD, MODIFY, REMOVE), `path`, `old_value`, `new_value`, and `severity` (BREAKING, WARNING, INFO). ______________________________________________________________________ ## BlueprintTester Run contract conformance checks with MockLLM — no API costs, great for CI. ``` import asyncio from pyagent_blueprint import load_blueprint, BlueprintTester spec = load_blueprint("customer-support.yaml") tester = BlueprintTester() results = asyncio.run(tester.test(spec)) print(tester.summary(results)) # customer-support-system: all 2 contract checks passed. ``` Each result in `results` has `contract_name`, `passed`, `message`, and `severity`. ______________________________________________________________________ ## BlueprintGenerator Scaffold a new blueprint YAML from a pattern name and agent list. ``` from pyagent_blueprint import BlueprintGenerator gen = BlueprintGenerator() yaml_str = gen.generate( pattern="supervisor", agents=["classifier", "billing", "technical", "general"], ) print(yaml_str) # api_version: pyagent/v1 # metadata: # name: generated-supervisor # version: "1.0.0" # agents: # classifier: { prompt: "" } # billing: { prompt: "" } # ... ``` ______________________________________________________________________ ## RuntimeGraph The compiled result of `BlueprintCompiler.compile()`. Its main methods: | Method | Description | | ---------------------- | ----------------------------------------------- | | `run(workflow, task)` | Execute a named workflow with a task string | | `describe()` | Return a dict describing all compiled workflows | | `wire_context(ledger)` | Set a shared `ContextLedger` on all agents | ``` graph = BlueprintCompiler().compile(spec) # Run a workflow result = asyncio.run(graph.run("main", task_string)) # Result fields print(result.output) # Final text output print(result.cost_estimate) # Estimated cost in USD print(result.metadata) # Pattern-specific metadata (route_key, rounds, etc.) ``` ______________________________________________________________________ ## See Also - [CLI](https://pyagent.org/packages/blueprint/cli/index.md) — command-line interface for the same operations - [Spec Format](https://pyagent.org/packages/blueprint/spec-format/index.md) — full YAML reference - [API Reference](https://pyagent.org/api/blueprint/index.md) # Blueprint — Context Config The `context:` block configures per-run memory, compression, and PII redaction. When present, the compiler wires a `ContextLedger` automatically — no extra Python code needed. ______________________________________________________________________ ## ContextConfigSpec Fields ``` context: memory: working_max_tokens: 20000 # Working memory cap (in-process) session_backend: sqlite # Persistence across turns: 'json' | 'sqlite' semantic_enabled: true # Enable vector search (requires chromadb) compression: policy: semantic_lossless # none | fifo | semantic_lossless | sawtooth target_ratio: 0.6 # Compress to 60% of current token count threshold_tokens: 15000 # Trigger compression above this threshold floor_tokens: 5000 # Never compress below this floor redaction: max_sensitivity: internal # public | internal | confidential | restricted exclude_above: false # If true, strip items above max_sensitivity ``` ______________________________________________________________________ ## Memory Tiers ### Working Memory In-process, bounded by `working_max_tokens`. Holds the current conversation turn. Automatically evicted as the conversation grows past the cap. ``` context: memory: working_max_tokens: 20000 ``` Corresponds to `WorkingMemory(max_tokens=20000)` in `pyagent-context`. ### Session Memory Persists across turns to a file or SQLite database — useful for multi-turn conversations and stateful agents. ``` context: memory: session_backend: sqlite # stored in .pyagent/session.db ``` Use `json` for a human-readable file. Use `sqlite` for better concurrency and query performance. ### Semantic Memory Vector-indexed storage for similarity search — agents can retrieve the most relevant past context rather than loading all history. ``` context: memory: semantic_enabled: true # Requires: pip install pyagent-context[chromadb] ``` ______________________________________________________________________ ## Compression Policies Compression triggers when the total context token count exceeds `threshold_tokens`, then reduces it toward `target_ratio × current_tokens`, with `floor_tokens` as a hard minimum. | Policy | Behavior | | ------------------- | ------------------------------------------------------------------- | | `none` | No compression — context grows unbounded | | `fifo` | Drop oldest items first | | `semantic_lossless` | Keep high-trust and recent items; drop redundant low-trust ones | | `sawtooth` | Aggressive drop then gradual rebuild — good for long-running agents | ``` context: compression: policy: semantic_lossless target_ratio: 0.6 threshold_tokens: 15000 floor_tokens: 5000 ``` For most production uses, `semantic_lossless` is the right default — it preserves verified facts (from databases, tool calls) and drops redundant conversation noise. ______________________________________________________________________ ## Redaction Strip items above a sensitivity level before injecting context into LLM calls. Prevents PII from leaking into prompts unintentionally. ``` context: redaction: max_sensitivity: internal # public, internal, confidential, restricted exclude_above: false ``` Items tagged `Sensitivity.PII` or `Sensitivity.CONFIDENTIAL` in the `ContextLedger` will be excluded from the context injected into agent prompts. Sensitivity levels (ascending): | Level | Description | | -------------- | --------------------------------------- | | `public` | Safe to expose anywhere | | `internal` | Internal systems only | | `confidential` | Restricted to specific roles | | `restricted` | Most sensitive — PII, credentials, etc. | ______________________________________________________________________ ## Wiring Context to Agents at Runtime When the blueprint has a `context:` block, the compiler calls `graph.wire_context(ledger)` — this sets the same ledger on every agent in the graph: ``` from pyagent_blueprint import load_blueprint, BlueprintCompiler from pyagent_context import ContextLedger, ContextItem, TrustLevel spec = load_blueprint("blueprint.yaml") graph = BlueprintCompiler().compile(spec) # Optional: pre-populate with verified facts ledger = ContextLedger() ledger.append(ContextItem( content="Customer ID: C-10482, account since 2022", source="database", trust=TrustLevel.VERIFIED, )) graph.wire_context(ledger) result = asyncio.run(graph.run("main", "What are my account details?")) ``` ______________________________________________________________________ ## Example: Full Context Config ``` context: memory: working_max_tokens: 20000 session_backend: sqlite semantic_enabled: true compression: policy: semantic_lossless target_ratio: 0.6 threshold_tokens: 15000 floor_tokens: 5000 redaction: max_sensitivity: internal exclude_above: false ``` ______________________________________________________________________ ## See Also - [Context Package](https://pyagent.org/packages/context/index.md) — `ContextLedger`, `WorkingMemory`, `SessionMemory`, `SemanticMemory` - [Context Guide](https://pyagent.org/guides/context/index.md) — narrative walkthrough - [Full Spec Reference](https://pyagent.org/packages/blueprint/spec-format/index.md) # Blueprint — Contracts & Observability Contracts define input/output constraints and SLA requirements. The `BlueprintTester` runs contract checks against `MockLLM` before you ever hit production. ______________________________________________________________________ ## ContractSpec ``` contracts: response_length: type: max_length value: 400 no_pii_in_response: type: regex_absent pattern: "\\b\\d{4}[- ]?\\d{4}[- ]?\\d{4}[- ]?\\d{4}\\b" contains_recommendation: type: regex_present pattern: "recommend|suggest|advise" valid_json_output: type: json_schema schema: type: object required: [recommendation, confidence] ``` ### Contract Types | Type | `value` / `pattern` | Passes when | | --------------- | ------------------- | ------------------------------------ | | `max_length` | integer | Output length ≤ value characters | | `min_length` | integer | Output length ≥ value characters | | `regex_absent` | regex string | Pattern does NOT match output | | `regex_present` | regex string | Pattern matches output | | `json_schema` | JSON Schema object | Output is valid JSON matching schema | ______________________________________________________________________ ## SLA Constraints SLA constraints live inside contracts and express latency and cost requirements: ``` contracts: support_response: type: max_length value: 400 sla: latency_p95_ms: 5000 # 95th percentile latency budget cost_max_usd: 0.05 # Maximum per-call cost max_retries: 2 timeout_seconds: 30.0 ``` `BlueprintValidator` checks SLA values for sanity (negative latency, unrealistic timeouts) and will warn if `timeout_seconds` seems too low for the assigned provider tier. ______________________________________________________________________ ## Running Contract Tests Use `BlueprintTester` to run all contracts against `MockLLM` — no real API calls: ``` import asyncio from pyagent_blueprint import load_blueprint, BlueprintTester spec = load_blueprint("customer-support.yaml") tester = BlueprintTester() results = asyncio.run(tester.test(spec)) print(tester.summary(results)) # customer-support-system: all 2 contract checks passed. ``` From the CLI: ``` blueprint test customer-support.yaml # Testing 'customer-support-system'... # All contract checks passed. # With failures: # [error] contracts.response_length: Output exceeded max_length 400 (got 612) ``` ______________________________________________________________________ ## Observability ``` observability: tracing: enabled: true record_to: traces/production_runs.jsonl # Optional file sink cost_budget: daily_usd: 50.0 # Alert / hard-stop threshold ``` ### Tracing When `tracing.enabled: true`, the compiler attaches a `TraceRecorder` to all agents. Every LLM call emits a span with: - `agent_name`, `workflow`, `pattern` - `input_tokens`, `output_tokens`, `duration_ms` - `provider`, `model`, `cost_estimate` If `record_to` is set, spans are also written to a JSONL file — which Studio's trace explorer can load. ``` observability: tracing: enabled: true record_to: traces/runs.jsonl ``` ### Cost Budget ``` observability: cost_budget: daily_usd: 50.0 ``` The compiler wires a `CostBudget` guard that accumulates cost across calls. When the daily limit is reached, further calls raise `BudgetExceededError` from `pyagent-compress`. Studio's `/providers` dashboard shows budget utilization in real time. ______________________________________________________________________ ## Validation Checks Summary The `BlueprintValidator` runs 6 categories of checks across the full spec: | Check | What it catches | | ------------- | ------------------------------------------------------------ | | Agent refs | Workflow references a non-existent agent name | | Provider refs | Agent or workflow references an undefined provider | | Pattern names | Unknown pattern key (typo in `pattern:` field) | | Contract refs | Workflow references an undefined contract | | SLA values | Negative latency, `timeout_seconds < 0`, unrealistic retries | | Security | Hardcoded API keys found in prompt strings | ``` from pyagent_blueprint import load_blueprint from pyagent_blueprint.validator import BlueprintValidator, IssueSeverity spec = load_blueprint("blueprint.yaml") issues = BlueprintValidator().validate(spec) errors = [i for i in issues if i.severity == IssueSeverity.ERROR] warnings = [i for i in issues if i.severity == IssueSeverity.WARNING] if errors: for e in errors: print(f"[error] {e.path}: {e.message}") raise SystemExit(1) ``` ______________________________________________________________________ ## See Also - [CLI](https://pyagent.org/packages/blueprint/cli/index.md) — `blueprint test`, `blueprint validate` - [Python API](https://pyagent.org/packages/blueprint/compiler/index.md) — `BlueprintTester`, `BlueprintValidator` - [Observability Guide](https://pyagent.org/guides/studio/index.md) — Studio trace explorer - [Full Spec Reference](https://pyagent.org/packages/blueprint/spec-format/index.md) # Blueprint — Providers The `providers:` block declares the LLM provider bindings available to agents in this blueprint. Agents reference these by name — the compiler resolves them to real provider instances at runtime. ______________________________________________________________________ ## ProviderBindingSpec Fields ``` providers: my_provider: provider: anthropic # 'anthropic' | 'openai' | custom key model: claude-sonnet-4-20250514 max_tokens: 2048 timeout_seconds: 60.0 ``` | Field | Type | Default | Description | | ----------------- | ------ | -------- | --------------------------------- | | `provider` | string | required | Provider backend key | | `model` | string | required | Model identifier | | `max_tokens` | int | `1024` | Maximum output tokens | | `timeout_seconds` | float | `null` | Per-call timeout; null = no limit | ______________________________________________________________________ ## Model Tier Pattern The most common pattern is defining two or three tiers and assigning agents to the cheapest tier that can handle their task: ``` providers: fast: provider: anthropic model: claude-haiku-3-5-20241022 max_tokens: 512 timeout_seconds: 10.0 balanced: provider: openai model: gpt-4o-mini max_tokens: 1024 expert: provider: anthropic model: claude-sonnet-4-20250514 max_tokens: 2048 timeout_seconds: 60.0 agents: classifier: { provider: fast, prompt: "Classify into one of: ..." } formatter: { provider: fast, prompt: "Format the response." } analyst: { provider: balanced, prompt: "Analyze the data." } strategist: { provider: expert, prompt: "Design the long-term strategy." } ``` A `classifier` that just outputs a single word costs a fraction of a cent. Routing it through `expert` would be 10–50× more expensive with no quality benefit. ______________________________________________________________________ ## Multi-Provider Setup Mix providers across tiers — Anthropic for quality-sensitive work, OpenAI for cost-sensitive work: ``` providers: claude_fast: provider: anthropic model: claude-haiku-3-5-20241022 max_tokens: 512 claude_expert: provider: anthropic model: claude-sonnet-4-20250514 max_tokens: 4096 openai_mini: provider: openai model: gpt-4o-mini max_tokens: 1024 openai_full: provider: openai model: gpt-4o max_tokens: 2048 ``` ______________________________________________________________________ ## Wiring Providers at Compile Time The `provider` key in a binding refers to a backend registered with the `ProviderRegistry`. The compiler resolves names at compile time: ``` from pyagent_blueprint import load_blueprint, BlueprintCompiler from pyagent_providers import ProviderRegistry, AnthropicLLM, OpenAILLM registry = ProviderRegistry() registry.register("anthropic", AnthropicLLM) registry.register("openai", OpenAILLM) spec = load_blueprint("blueprint.yaml") graph = BlueprintCompiler(provider_registry=registry).compile(spec) ``` If a provider backend key in the YAML is not registered, `BlueprintValidator` raises an error before compilation. ______________________________________________________________________ ## MockLLM for Testing Omit the `providers:` block entirely (or omit the `provider_registry` argument) and the compiler uses `MockLLM` — deterministic, free, and fast: ``` # test-blueprint.yaml — no providers block api_version: pyagent/v1 metadata: name: test version: "1.0.0" agents: responder: prompt: "Answer questions." workflows: main: pattern: pipeline agents: stages: [responder] ``` ``` spec = load_blueprint("test-blueprint.yaml") graph = BlueprintCompiler().compile(spec) # Uses MockLLM automatically ``` ______________________________________________________________________ ## Provider Health in Studio Providers defined in the blueprint appear in Studio's `/providers` dashboard — health status, average latency, p99 latency, error rate, and budget usage. See [Studio Dashboard](https://pyagent.org/packages/studio/dashboard/#providers). ______________________________________________________________________ ## See Also - [Agents](https://pyagent.org/packages/blueprint/agents/index.md) — assigning providers to agents - [Workflows](https://pyagent.org/packages/blueprint/workflows/index.md) — `fallback_provider` in recovery config - [Providers Package](https://pyagent.org/packages/providers/index.md) — `ProviderRegistry`, `FallbackChain`, `CostOptimizer` - [Full Spec Reference](https://pyagent.org/packages/blueprint/spec-format/index.md) # Blueprint Spec Format A blueprint is a single YAML document. Every section is optional except `api_version`, `metadata`, `agents`, and `workflows`. ______________________________________________________________________ ## Full Reference ``` # ───────────────────────────────────────────────────────── # Top-level keys # ───────────────────────────────────────────────────────── api_version: pyagent/v1 # Required. Must be "pyagent/v1". # ─── Metadata ──────────────────────────────────────────── metadata: name: customer-support # Required. Unique identifier. version: "1.2.0" # Required. Semantic version string. description: "Multi-tier support system" owner: platform-team tags: [support, production] # ─── Providers ─────────────────────────────────────────── providers: fast: provider: anthropic # 'anthropic' | 'openai' | custom key model: claude-haiku-3-5-20241022 max_tokens: 512 timeout_seconds: 10.0 balanced: provider: openai model: gpt-4o-mini max_tokens: 1024 expert: provider: anthropic model: claude-sonnet-4-20250514 max_tokens: 2048 timeout_seconds: 60.0 # ─── Context ───────────────────────────────────────────── context: memory: working_max_tokens: 20000 # In-process working memory limit session_backend: sqlite # 'json' | 'sqlite' (persist across turns) semantic_enabled: true # Enable ChromaDB semantic search compression: policy: semantic_lossless # none | fifo | semantic_lossless | sawtooth target_ratio: 0.6 # Compress to 60% of current size threshold_tokens: 15000 # Trigger compression above this floor_tokens: 5000 # Never compress below this redaction: max_sensitivity: internal # public | internal | confidential | restricted exclude_above: false # ─── Agents ────────────────────────────────────────────── agents: classifier: provider: fast # References a key in providers: description: "Route to billing, technical, returns, or general" prompt: | Classify the customer message into exactly one of: billing, technical, returns, general. Respond with ONLY the category name. tools: [] # Tool names (must be registered at runtime) guardrails: [] # Guardrail refs billing_agent: provider: expert description: "Handle billing disputes" prompt: | You are a billing specialist. Handle disputes, refunds, and subscription questions. Always acknowledge frustration first. tools: [lookup_invoice, issue_refund] formatter: provider: fast description: "Format the final response" prompt: | Format the response professionally. Keep under 200 words. Add a friendly closing line. # ─── Workflows ─────────────────────────────────────────── workflows: main: pattern: supervisor # One of the registered pattern names agents: classifier: classifier # Pattern-specific agent mapping routes: billing: billing_agent technical: technical_agent returns: returns_agent general: general_agent formatter: formatter config: default_route: general recovery: max_retries: 2 timeout_seconds: 30.0 fallback_provider: fast # Use cheap model if primary fails # ─── Contracts ─────────────────────────────────────────── contracts: response_length: type: max_length value: 400 no_pii_in_response: type: regex_absent pattern: "\\b\\d{4}[- ]?\\d{4}[- ]?\\d{4}[- ]?\\d{4}\\b" # ─── Observability ─────────────────────────────────────── observability: tracing: enabled: true record_to: traces/runs.jsonl # Optional: path to emit .jsonl trace file cost_budget: daily_usd: 50.0 # Alert / hard-stop above this ``` ______________________________________________________________________ ## Metadata | Field | Required | Description | | ------------- | -------- | ------------------------------------------------- | | `name` | Yes | Unique identifier for this system | | `version` | Yes | Semantic version (shown in Studio, used in diffs) | | `description` | No | Human-readable description | | `owner` | No | Team or person responsible | | `tags` | No | Free-form labels for filtering | ______________________________________________________________________ ## Section Details Each section has its own detailed page: - [Providers](https://pyagent.org/packages/blueprint/providers/index.md) — model tiers, timeout, token limits - [Agents](https://pyagent.org/packages/blueprint/agents/index.md) — prompts, tools, guardrails - [Workflows](https://pyagent.org/packages/blueprint/workflows/index.md) — patterns, routing, recovery - [Context](https://pyagent.org/packages/blueprint/context/index.md) — memory, compression, redaction - [Contracts & Observability](https://pyagent.org/packages/blueprint/contracts/index.md) — SLA constraints, cost budgets, trace recording ______________________________________________________________________ ## Minimal Blueprint ``` api_version: pyagent/v1 metadata: name: minimal version: "1.0.0" agents: responder: prompt: "You are a helpful assistant." workflows: main: pattern: pipeline agents: stages: [responder] ``` When no `providers:` block is present the compiler uses `MockLLM` automatically — useful for unit tests and CI. ______________________________________________________________________ ## Multi-Workflow Blueprint A single YAML file can contain multiple named workflows, each using different patterns: ``` workflows: quick_analysis: pattern: pipeline agents: stages: [fast_extractor, fast_writer] deep_analysis: pattern: fan_out_fan_in agents: agents: [fundamentals_agent, technicals_agent, sentiment_agent] aggregator: synthesis_agent config: {} adversarial_review: pattern: debate agents: agents: [bull_agent, bear_agent] judge: judge_agent config: rounds: 2 ``` ``` graph = BlueprintCompiler().compile(spec) quick = asyncio.run(graph.run("quick_analysis", task)) deep = asyncio.run(graph.run("deep_analysis", task)) ``` # Blueprint — Workflows Each entry under `workflows:` defines a named workflow — an orchestration pattern with agents wired to specific roles. ______________________________________________________________________ ## WorkflowSpec Fields ``` workflows: my_workflow: pattern: supervisor # Required — registered pattern name agents: {} # Pattern-specific agent mapping config: {} # Optional pattern-level settings recovery: # Optional failure handling max_retries: 2 timeout_seconds: 30.0 fallback_provider: fast ``` ______________________________________________________________________ ## Pattern Names The `pattern` field must match a registered pattern. The built-in patterns from `pyagent-patterns`: | Pattern | Key | | -------------------- | ---------------------- | | Pipeline | `pipeline` | | Supervisor | `supervisor` | | Fan-Out / Fan-In | `fan_out_fan_in` | | Hierarchical | `hierarchical` | | Orchestrator-Workers | `orchestrator_workers` | | Self-Reflection | `self_reflection` | | Cross-Reflection | `cross_reflection` | | Debate | `debate` | | Voting | `voting` | | Evaluator-Optimizer | `evaluator_optimizer` | | Role-Based | `role_based` | | Layered | `layered` | | Topology | `topology` | | Blackboard | `blackboard` | | Talker-Reasoner | `talker_reasoner` | | Swarm | `swarm` | | Human-in-the-Loop | `human_in_the_loop` | | ReAct | `react` | The `BlueprintValidator` raises an error if the pattern key is not recognized. ______________________________________________________________________ ## Agent Mappings by Pattern ### pipeline ``` workflows: analysis: pattern: pipeline agents: stages: [extractor, analyst, writer] # Ordered list of agent names ``` ### supervisor ``` workflows: support: pattern: supervisor agents: classifier: classifier # The routing agent routes: # Route key → agent name billing: billing_agent technical: technical_agent returns: returns_agent general: general_agent formatter: formatter # Optional post-processing agent config: default_route: general # Fallback if classifier output isn't a key ``` ### fan_out_fan_in ``` workflows: market_analysis: pattern: fan_out_fan_in agents: agents: [fundamentals, technicals, sentiment] # Run in parallel aggregator: synthesis_agent # Merge results ``` ### debate ``` workflows: review: pattern: debate agents: agents: [proposer, critic] # Alternate turns judge: arbitrator # Final verdict config: rounds: 2 ``` ### voting ``` workflows: classification: pattern: voting agents: voters: [expert_1, expert_2, expert_3] config: strategy: majority # majority | unanimous | weighted ``` ### evaluator_optimizer ``` workflows: writing: pattern: evaluator_optimizer agents: generator: writer_agent evaluator: critic_agent config: max_rounds: 3 score_threshold: 8 ``` ### self_reflection ``` workflows: coding: pattern: self_reflection agents: agent: coder_agent config: max_rounds: 3 ``` ### react ``` workflows: research: pattern: react agents: agent: researcher config: max_steps: 10 ``` ______________________________________________________________________ ## Recovery Config Recovery handles infrastructure failures (API errors, timeouts, token limits). It is separate from quality-based escalation — see the [Composition Guide](https://pyagent.org/guides/composition/index.md). ``` workflows: main: pattern: supervisor agents: ... recovery: max_retries: 2 # Retry this many times before escalating timeout_seconds: 30.0 # Per-attempt wall-clock timeout fallback_provider: fast # Use this cheaper provider if retries exhaust ``` | Field | Default | Description | | ------------------- | ------- | -------------------------------------------------- | | `max_retries` | `0` | Number of retries before failing or using fallback | | `timeout_seconds` | `null` | Per-attempt timeout; null = no limit | | `fallback_provider` | `null` | Provider key to use when retries exhaust | ### Fallback provider behavior When `fallback_provider` is set and all retries are exhausted, the compiler substitutes that provider for all agents in the workflow and tries one final time. This lets you degrade gracefully to a cheap model rather than returning an error. ______________________________________________________________________ ## Multiple Workflows A blueprint can have many named workflows — different entry points for different use cases, all sharing the same agents and providers: ``` agents: fast_analyst: { provider: fast, prompt: "Quick analysis." } deep_analyst: { provider: expert, prompt: "Deep analysis with citations." } writer: { provider: fast, prompt: "Write a report." } workflows: quick: pattern: pipeline agents: stages: [fast_analyst, writer] deep: pattern: self_reflection agents: agent: deep_analyst config: max_rounds: 3 ``` ``` graph = BlueprintCompiler().compile(spec) quick_result = asyncio.run(graph.run("quick", task)) deep_result = asyncio.run(graph.run("deep", task)) ``` ______________________________________________________________________ ## See Also - [Agents](https://pyagent.org/packages/blueprint/agents/index.md) — defining agents referenced in workflows - [Patterns Package](https://pyagent.org/packages/patterns/index.md) — pattern semantics and agent roles - [Full Spec Reference](https://pyagent.org/packages/blueprint/spec-format/index.md) # Design Patterns # pyagent-patterns **Core agent abstractions and multi-agent orchestration patterns** — the foundation every other package builds on. Provides `Agent`, `Message`, `Pattern`, and 18 production-ready orchestration patterns. ``` pip install pyagent-patterns ``` ______________________________________________________________________ ## Architecture ``` flowchart TD subgraph Core M[Message] --> A[Agent] A --> R[Result] A --> C[Context] end subgraph Orchestration PL[Pipeline] FO[FanOutFanIn] HI[Hierarchical] OW[OrchestratorWorkers] SV[Supervisor] end subgraph Advanced RX[ReAct] SW[Swarm] TR[TalkerReasoner] HL[HumanInTheLoop] end A --> PL & FO & HI & OW & SV A --> RX & SW & TR & HL ``` ______________________________________________________________________ ## Core Abstractions ### Message Immutable, typed message between agents. ``` from pyagent_patterns.base import Message user_msg = Message.user("Summarize this earnings call") sys_msg = Message.system("You are a financial analyst.") asst_msg = Message.assistant("Revenue grew 15% YoY...", name="analyst") ``` ### Agent The single unit of work — one LLM, one system prompt. ``` from pyagent_patterns.base import Agent from pyagent_providers import AnthropicLLM agent = Agent( name="analyst", llm=AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="You are a financial analyst. Be concise.", ) import asyncio result = asyncio.run(agent.run("What is 2+2?")) print(result.output) ``` ### MockLLM Deterministic LLM for tests — no API calls, no cost. ``` from pyagent_patterns.base import Agent, MockLLM llm = MockLLM(responses=["Bull case: strong FCF", "Bear case: high leverage"]) agent = Agent("mock_agent", llm) ``` ### Context Structured context injected into an agent's conversation. ``` from pyagent_patterns.base import Agent, Context ctx = Context( data={"portfolio": "NVDA 10%, MSFT 8%, AAPL 7%"}, max_tokens=500, ) result = asyncio.run(agent.run("What is my largest position?", context=ctx)) ``` ______________________________________________________________________ ## Orchestration Patterns ### Pipeline Sequential chain — each stage's output feeds the next. ``` Input → Stage 1 → Stage 2 → Stage 3 → Output LLM calls: N (one per stage) ``` ``` import asyncio from pyagent_patterns.orchestration import Pipeline from pyagent_patterns.base import Agent from pyagent_providers import AnthropicLLM, OpenAILLM pipeline = Pipeline(stages=[ Agent("extractor", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Extract all facts and figures from the text."), Agent("analyst", OpenAILLM("gpt-4o-mini"), system_prompt="Analyze the extracted data. Identify key trends."), Agent("writer", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Write a concise 3-paragraph investment brief."), ]) result = asyncio.run(pipeline.run(open("earnings.txt").read())) print(result.output) ``` ### FanOutFanIn Parallel agents, single aggregator — fastest for independent sub-tasks. ``` ┌→ Agent 1 ─┐ Input ────────┼→ Agent 2 ─┼→ Aggregator → Output └→ Agent 3 ─┘ LLM calls: N agents (parallel) + 1 aggregator Wall-clock: max(agent latencies) + aggregator ``` ``` import asyncio from pyagent_patterns.orchestration import FanOutFanIn from pyagent_patterns.base import Agent from pyagent_providers import GeminiLLM, AnthropicLLM fanout = FanOutFanIn( agents=[ Agent("bull", GeminiLLM("gemini-2.5-flash"), system_prompt="Strongest bullish case."), Agent("bear", GeminiLLM("gemini-2.5-flash"), system_prompt="Strongest bearish case."), Agent("macro", GeminiLLM("gemini-2.5-flash"), system_prompt="Macro risk factors."), ], aggregator=Agent( "synthesis", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Synthesize all perspectives into a balanced investment memo.", ), ) result = asyncio.run(fanout.run("Nvidia at $3.2T market cap — buy or pass?")) ``` ### OrchestratorWorkers Dynamic task decomposition — orchestrator breaks the task, workers execute. ``` from pyagent_patterns.orchestration import OrchestratorWorkers from pyagent_patterns.base import Agent from pyagent_providers import AnthropicLLM, OpenAILLM workflow = OrchestratorWorkers( orchestrator=Agent( "planner", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Break the task into subtasks. Return JSON list of task strings.", ), workers=[ Agent("researcher", OpenAILLM("gpt-4o"), system_prompt="Research the given topic."), Agent("coder", OpenAILLM("gpt-4o"), system_prompt="Write the code for the task."), Agent("reviewer", OpenAILLM("gpt-4o"), system_prompt="Review and improve the output."), ], ) result = asyncio.run(workflow.run("Build a Python CLI that queries the GitHub API")) ``` ### Hierarchical Multi-level hierarchy — a manager delegates to team leads, who delegate to their workers. ``` import asyncio from pyagent_patterns.orchestration import Hierarchical from pyagent_patterns.orchestration.hierarchical import Team from pyagent_patterns.base import Agent from pyagent_providers import AnthropicLLM, OpenAILLM hierarchy = Hierarchical( manager=Agent( "engineering_director", OpenAILLM("gpt-4o"), system_prompt="Decompose the initiative into team subtasks, then synthesize " "all team outputs into one cohesive technical spec.", ), teams=[ Team( name="Backend", lead=Agent("backend_lead", OpenAILLM("gpt-4o-mini"), system_prompt="Coordinate the backend design."), workers=[ Agent("api_engineer", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Design the REST API and schemas."), Agent("db_engineer", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Design the database schema and indexes."), ], ), Team( name="Frontend", lead=Agent("frontend_lead", OpenAILLM("gpt-4o-mini"), system_prompt="Coordinate the frontend design."), workers=[ Agent("ui_engineer", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Design the component architecture."), ], ), ], ) result = asyncio.run(hierarchy.run("Design a real-time collaborative document editor")) ``` ### Supervisor Classify → route → specialist. A coordinator classifies the request and dispatches it to the right expert agent, with an optional formatter for the final reply. ``` import asyncio from pyagent_patterns.orchestration import Supervisor from pyagent_patterns.base import Agent from pyagent_providers import AnthropicLLM, OpenAILLM supervisor = Supervisor( classifier=Agent( "router", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Classify into exactly one of: billing, technical, general.", ), routes={ "billing": Agent("billing", OpenAILLM("gpt-4o"), system_prompt="Handle billing disputes, refunds, and subscriptions."), "technical": Agent("technical", OpenAILLM("gpt-4o"), system_prompt="Handle technical troubleshooting and API issues."), "general": Agent("general", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Handle general inquiries warmly and helpfully."), }, default_route="general", ) result = asyncio.run(supervisor.run("My invoice was charged twice this month")) ``` ______________________________________________________________________ ## Advanced Patterns ### ReAct Reasoning + Acting loop — agent decides to use tools, then reasons about results. ``` from pyagent_patterns.advanced import ReAct react = ReAct( agent=Agent("assistant", AnthropicLLM("claude-sonnet-4-20250514")), tools={"search": search_fn, "calculator": calc_fn, "code": exec_fn}, max_steps=10, ) result = asyncio.run(react.run("What was Nvidia's revenue growth rate last quarter?")) ``` ### Swarm Agents hand off to each other dynamically — each decides who handles next. ``` from pyagent_patterns.advanced import Swarm swarm = Swarm(agents={ "triage": triage_agent, "billing": billing_agent, "technical": technical_agent, "escalation": escalation_agent, }) result = asyncio.run(swarm.run("My payment failed but I was still charged twice")) ``` ### TalkerReasoner Dual-agent: fast Talker handles conversation, slow Reasoner handles hard problems. ``` from pyagent_patterns.advanced import TalkerReasoner dual = TalkerReasoner( talker=Agent("fast", OpenAILLM("gpt-4o-mini"), system_prompt="Handle simple queries quickly."), reasoner=Agent("slow", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Handle complex analysis and reasoning."), handoff_threshold=6, ) ``` ### HumanInTheLoop Pause for human approval before high-stakes actions. ``` from pyagent_patterns.advanced import HumanInTheLoop guarded = HumanInTheLoop( agent=Agent("executor", AnthropicLLM("claude-sonnet-4-20250514")), approval_fn=lambda action: input(f"Approve '{action}'? (y/n): ") == "y", high_risk_keywords=["delete", "deploy", "transfer", "publish"], ) ``` ______________________________________________________________________ ## Structural Patterns ``` # Role-based: agents have explicit roles enforced at runtime from pyagent_patterns.structural import RoleBased # Blackboard: agents read/write a shared knowledge store from pyagent_patterns.structural import Blackboard # Layered: presentation → logic → data layer separation from pyagent_patterns.structural import Layered # Topology: define arbitrary agent graphs as adjacency maps from pyagent_patterns.structural import Topology ``` ______________________________________________________________________ ## Hooks Integration Every `Agent` supports four hooks for attaching pyagent's infrastructure services. ``` from pyagent_patterns.base import Agent from pyagent_providers import AnthropicLLM from pyagent_compress import MessageCompressor from pyagent_context import ContextLedger from pyagent_trace.events import TraceEventBus from pyagent_trace.cost import CostTracker agent = ( Agent("analyst", AnthropicLLM("claude-sonnet-4-20250514")) .set_compressor(MessageCompressor(target_ratio=0.5)) .set_context(ContextLedger()) .set_trace_bus(TraceEventBus()) .set_cost_tracker(CostTracker()) ) ``` → See the full [Hooks Guide](https://pyagent.org/guides/hooks/index.md) for all four hook types. ______________________________________________________________________ ## Pattern Selection Guide Use this decision tree to choose the right pattern for your task. ``` flowchart TD START([New Task]) --> Q1{Single step?} Q1 -->|Yes| Q2{Need quality check?} Q1 -->|No| Q3{Need classification?} Q2 -->|No| P1[Pipeline / Single Agent] Q2 -->|Yes| Q4{Budget tight?} Q4 -->|Yes| P2[Talker-Reasoner] Q4 -->|No| Q5{Adversarial needed?} Q5 -->|Yes| P3[Debate] Q5 -->|No| P4[Self-Reflection] Q3 -->|Yes| P5[Supervisor] Q3 -->|No| Q6{Parallel possible?} Q6 -->|Yes| Q7{Need consensus?} Q6 -->|No| P6[Pipeline] Q7 -->|Yes| P7[Fan-Out + Voting] Q7 -->|No| P8[Fan-Out/Fan-In] style P1 fill:#4CAF50,color:#fff style P2 fill:#4CAF50,color:#fff style P3 fill:#FF9800,color:#fff style P4 fill:#FF9800,color:#fff style P5 fill:#2196F3,color:#fff style P6 fill:#2196F3,color:#fff style P7 fill:#9C27B0,color:#fff style P8 fill:#9C27B0,color:#fff ``` **Not on the tree on purpose: a single agent, single call, with nothing to hand off to and no state to persist.** That's not a degenerate case of any pattern here — it's the case where none of them apply. Import your provider's SDK and make the call. Reach for a named pattern only once there's a second agent, a review loop, or a routing decision in the picture. ### All 18 Patterns at a Glance | # | Pattern | Tier | LLM Calls | Best For | | --- | -------------------- | ------------- | ----------- | ------------------------------ | | 1 | Supervisor | Orchestration | 2-3 | Task routing, customer support | | 2 | Pipeline | Orchestration | N stages | Sequential processing, ETL | | 3 | Fan-Out/Fan-In | Orchestration | N+1 | Parallel analysis, research | | 4 | Hierarchical | Orchestration | 3+ levels | Enterprise workflows | | 5 | Orchestrator-Workers | Orchestration | 1+N+1 | Dynamic task decomposition | | 6 | Self-Reflection | Resolution | 2-6 | Code gen, writing | | 7 | Cross-Reflection | Resolution | 3+ | Peer review, editing | | 8 | Debate | Resolution | D×R+1 | Controversial decisions | | 9 | Voting | Resolution | N | Consensus, fault tolerance | | 10 | Evaluator-Optimizer | Resolution | 2-4/round | Criteria-driven quality | | 11 | Role-Based | Structural | N×rounds | Team simulation | | 12 | Layered | Structural | sum(layers) | Multi-level analysis | | 13 | Topology | Structural | varies | Communication structure | | 14 | Blackboard | Structural | N×rounds | Shared state coordination | | 15 | Talker-Reasoner | Advanced | 1-2 | Cost-optimized chat | | 16 | Swarm | Advanced | N×rounds | Emergent behavior | | 17 | Human-in-the-Loop | Advanced | 1+ | Safety-critical tasks | | 18 | ReAct | Advanced | 1-N steps | Tool-using agents | ______________________________________________________________________ ## Machine-readable catalog Every pattern's `use_when`/`avoid_when`/`pairs_with` metadata above is also published as structured JSON at [pyagent.org/patterns.json](https://pyagent.org/patterns.json) — generated from the same source as this page (`data/patterns.yml`), so it can never drift from the prose. Useful for tools, agents, or scripts choosing a pattern programmatically rather than reading this page. ## See Also - [Router Package](https://pyagent.org/packages/router/index.md) — automatic model selection per agent call - [Compress Package](https://pyagent.org/packages/compress/index.md) — token budget enforcement across pipelines - [Guides](https://pyagent.org/guides/composition/index.md) — deep-dive integration guides - [API Reference](https://pyagent.org/api/patterns/base/index.md) # Human-in-the-Loop Pattern Agent generates output; a human review gate decides accept / reject / modify. Supports async approval workflows for production systems. **Best for:** High-stakes content, compliance workflows, email/legal drafts, any irreversible action.\ **LLM calls:** 1 per generation attempt (N attempts if rejected). ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant U as User participant A as Agent participant H as Human Reviewer U->>A: "Draft a contract clause" A-->>H: Initial draft H-->>A: Reject: "Add liability cap" A-->>H: Revised draft with cap H-->>A: Reject: "Cap should be $1M not $500k" A-->>H: Final revised draft H-->>U: Approve ✓ ``` ______________________________________________________________________ ## Use Case 1 — Executive Email Drafting (Anthropic) ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.advanced import HumanInTheLoop from pyagent_patterns.advanced.human_in_the_loop import HumanDecision from pyagent_providers import AnthropicLLM def executive_review(output: str, metadata: dict) -> HumanDecision: print("\n" + "="*60) print("DRAFT EMAIL FOR REVIEW:") print("="*60) print(output) print("="*60) print(f"Generation attempt: {metadata.get('revision', 0) + 1}") decision = input("Approve? (y/n): ").strip().lower() if decision == "y": return HumanDecision(approved=True) feedback = input("Feedback for revision: ").strip() return HumanDecision(approved=False, feedback=feedback) pattern = HumanInTheLoop( agent=Agent( "email_drafter", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Draft professional executive emails. " "Match the tone to the context: formal for board communications, " "warmer for team-facing messages. " "Be direct, clear, and concise. Max 200 words. " "If given revision feedback, incorporate it precisely.", ), review_fn=executive_review, max_revisions=3, ) result = asyncio.run(pattern.run( "Draft an email to our enterprise customers announcing a 15% price increase " "effective in 90 days. Frame it around the significant new features we're adding: " "AI-powered analytics, 99.99% SLA, and 24/7 dedicated support." )) print(f"\nApproved after {result.metadata['revisions']} revision(s)") print("Final email:") print(result.output) ``` Declare the same drafting gate as a `pyagent-blueprint` manifest. The human `review_fn` callback is a runtime-adapter concern (not blueprint-representable) — the compiled pattern defaults to auto-approve; a caller wanting real human review wires `review_fn` via the Python API after compiling: ``` api_version: pyagent/v1 metadata: name: executive-email-review version: 1.0.0 description: Agent drafts an email; a human approval gate decides accept/reject/revise providers: primary: { model: claude-sonnet-4-20250514 } agents: email_drafter: provider: primary prompt: "Draft professional executive emails. Incorporate revision feedback precisely." workflows: draft: pattern: human_in_the_loop agents: agent: email_drafter config: max_revisions: 3 ``` ``` pyagent-blueprint validate executive-email-review.yaml pyagent-blueprint test executive-email-review.yaml ``` ______________________________________________________________________ ## Use Case 2 — Legal Clause Generation with Compliance Gate (OpenAI) Automated review for known patterns, human escalation for edge cases. ``` from pyagent_providers import OpenAILLM import re REQUIRED_TERMS = ["liability cap", "limitation of liability", "indemnification"] AUTO_REJECT_PATTERNS = ["unlimited liability", "sole discretion", "without notice"] def compliance_review(output: str, metadata: dict) -> HumanDecision: output_lower = output.lower() for pattern_str in AUTO_REJECT_PATTERNS: if pattern_str in output_lower: return HumanDecision( approved=False, feedback=f"Auto-rejected: contains prohibited term '{pattern_str}'. Remove it.", ) missing = [t for t in REQUIRED_TERMS if t not in output_lower] if missing: return HumanDecision( approved=False, feedback=f"Missing required terms: {', '.join(missing)}", ) print("\nDRAFT CLAUSE:") print(output) decision = input("Legal team approval (y/n): ").strip().lower() if decision == "y": return HumanDecision(approved=True) feedback = input("Legal feedback: ").strip() return HumanDecision(approved=False, feedback=feedback) pattern = HumanInTheLoop( agent=Agent( "legal_drafter", OpenAILLM("gpt-4o"), system_prompt="Draft precise legal clauses for B2B SaaS agreements. " "Always include: liability caps, indemnification scope, " "and governing law. Use standard commercial terms. " "If given revision feedback, incorporate exactly.", ), review_fn=compliance_review, max_revisions=5, ) result = asyncio.run(pattern.run( "Draft a limitation of liability clause: liability cap at 12 months of fees paid, " "mutual indemnification, exclusion of consequential damages, " "carve-out for data breach and IP infringement." )) print(f"Approved: {result.metadata['approved']}, Revisions: {result.metadata['revisions']}") ``` ______________________________________________________________________ ## Use Case 3 — Content Moderation with Human Escalation (LangChain) Auto-approve clear cases; escalate ambiguous ones. ``` from langchain_anthropic import ChatAnthropic from pyagent_providers import LangChainLLM auto_decisions = {"CLEARLY_SAFE": True, "CLEARLY_VIOLATES": False} def moderation_review(output: str, metadata: dict) -> HumanDecision: first_word = output.strip().split()[0].upper() if output.strip() else "" if first_word in auto_decisions: approved = auto_decisions[first_word] return HumanDecision( approved=approved, feedback=None if approved else "Automated rejection: clear policy violation.", ) print(f"\nAMBIGUOUS CASE — Human review needed:") print(output) decision = input("Approve content? (y/n): ").strip().lower() feedback = None if decision == "y" else input("Rejection reason: ").strip() return HumanDecision(approved=(decision == "y"), feedback=feedback) pattern = HumanInTheLoop( agent=Agent( "content_moderator", LangChainLLM(ChatAnthropic(model="claude-sonnet-4-20250514")), system_prompt="Evaluate user-submitted content against community guidelines. " "Respond: CLEARLY_SAFE, CLEARLY_VIOLATES, or AMBIGUOUS — then explain why.", ), review_fn=moderation_review, max_revisions=2, ) ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.human_in_the_loop (47.2s, $0.008) ├── Attempt 1: pyagent.agent.email_drafter (1.8s, claude-sonnet-4-20250514) │ └── human_review: REJECTED (feedback: "too formal, soften the tone") ├── Attempt 2: pyagent.agent.email_drafter (1.6s) │ └── human_review: REJECTED (feedback: "mention the 90-day notice explicitly") └── Attempt 3: pyagent.agent.email_drafter (1.4s) └── human_review: APPROVED ✓ total_revisions: 2, approved: true ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ------------------------------------------- | ------------------------------------------- | | Outputs are irreversible or legally binding | ✅ Use Human-in-the-Loop | | Compliance or regulatory review is required | ✅ Use Human-in-the-Loop | | High-volume routine outputs | ❌ Use Evaluator-Optimizer (automated gate) | | Speed is critical | ❌ Human review adds latency | | Quality improvement without human judgment | ❌ Use Cross-Reflection | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Human-in-the-Loop** pattern: | Recipe | Domain | What it does | Complexity | | -------------------------------------------------------------------------------------------------- | ----------------------- | --------------------------------------------------------------------------------- | ------------ | | [Alert Triage](https://pyagent.org/cookbook/security/log-triage/index.md) | Security & Threat Intel | Triage security alerts; escalate true positives to a human | Intermediate | | [Incident Triage](https://pyagent.org/cookbook/devops-sre/incident-triage/index.md) | DevOps & SRE | Stage pipeline that triages incidents with human sign-off | Intermediate | | [AML Transaction Monitoring](https://pyagent.org/cookbook/finance-trading/aml-monitoring/index.md) | Finance & Trading | Screen → score → enrich pipeline gates high-risk transactions to a human reviewer | Advanced | | [Code Review](https://pyagent.org/cookbook/software-engineering/code-review/index.md) | Software Engineering | Iterative review + security scan with a human gate | Advanced | | [Support Router](https://pyagent.org/cookbook/customer-support/support-router/index.md) | Customer Support | Classify tickets → route to specialists → escalate to a human | Advanced | ## See Also - [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md) — automated quality gate without human - [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) — AI peer review before human sees it - [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) — human-in-the-loop for tool approval rather than output approval ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · **Human-in-the-Loop** [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # ReAct Pattern Iterative **Re**ason → **Act** → Observe cycle. The agent thinks, calls a tool, observes the result, and repeats until it has enough to answer. Based on the 2022 Princeton/Google paper: *"ReAct: Synergizing Reasoning and Acting in Language Models."* **Best for:** Research assistants, code execution, API orchestration, any task needing external data.\ **LLM calls:** 1 per Thought+Action step. Typically 2–5 steps. ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant U as User participant A as Agent participant T as Tools U->>A: "What is Anthropic's latest model and its context window?" loop Reason-Act-Observe A->>A: Thought: I need to look this up A->>T: Action: search("Anthropic latest model 2025") T-->>A: Observation: claude-sonnet-4-20250514, 200k context A->>A: Thought: I have the answer end A-->>U: FINISH: Claude Sonnet 4 (200k context window) ``` ______________________________________________________________________ ## Use Case 1 — Financial Research Agent (OpenAI) ``` import asyncio import httpx from pyagent_patterns.base import Agent from pyagent_patterns.advanced import ReAct from pyagent_providers import OpenAILLM def search(query: str) -> str: """Simulates web search — replace with Tavily, SerpAPI, or Exa in production.""" results = { "nvidia revenue 2025": "Nvidia Q3 FY2025: Revenue $35.1B (+94% YoY), data center $30.8B", "nvidia p/e ratio": "Nvidia forward P/E: 45x (as of Dec 2024), vs AMD 28x, Intel 18x", "nvidia competition": "AMD MI300X closing gap in AI training; custom silicon from Google (TPU), Amazon (Trainium)", } for key, val in results.items(): if key in query.lower(): return val return f"Search results for '{query}': No specific data found, check financial data providers." def calculator(expression: str) -> str: """Safe arithmetic evaluator.""" try: allowed = set("0123456789+-*/(). ") if not all(c in allowed for c in expression): return "Error: only arithmetic expressions allowed" return str(round(eval(expression), 4)) except Exception as e: return f"Error: {e}" def get_price(ticker: str) -> str: return {"NVDA": "$875.40", "AMD": "$142.20", "INTC": "$42.10"}.get( ticker.upper(), f"Ticker {ticker} not found" ) pattern = ReAct( agent=Agent( "financial_analyst", OpenAILLM("gpt-4o"), system_prompt="You are a financial research analyst. " "Use tools to gather data before drawing conclusions. " "Think step by step. Show your reasoning. " "Format:\n" "Thought: \n" "Action: tool_name(argument)\n" "...repeat as needed...\n" "FINISH: ", ), tools={ "search": search, "calculator": calculator, "get_price": get_price, }, max_steps=8, ) result = asyncio.run(pattern.run( "Research Nvidia's current financial position: revenue growth, competitive threats, " "and whether the current P/E ratio is justified. Give a buy/hold/sell recommendation." )) print(result.output) print(f"Steps: {result.metadata['steps']}, Tools used: {result.metadata['tools_used']}") print(f"Cost: ${result.cost_estimate:.4f}") ``` Declare the same reasoning agent as a `pyagent-blueprint` manifest. Tool *names* are declarative (`agents..tools`); wiring the actual Python callables (`search`, `calculator`, `get_price`) is a runtime-adapter concern, done in the Python API after compiling: ``` api_version: pyagent/v1 metadata: name: financial-research-agent version: 1.0.0 description: ReAct agent reasons step by step and calls tools to research a question providers: primary: { model: gpt-4o } agents: financial_analyst: provider: primary prompt: "Reason step by step. Use tools to gather data. Respond FINISH: when done." tools: [search, calculator, get_price] workflows: research: pattern: react agents: agent: financial_analyst config: max_steps: 8 ``` ``` pyagent-blueprint validate financial-research-agent.yaml pyagent-blueprint test financial-research-agent.yaml ``` ______________________________________________________________________ ## Use Case 2 — Code Execution Agent (Anthropic) ``` import subprocess import tempfile import os from pyagent_providers import AnthropicLLM def run_python(code: str) -> str: """Execute Python code safely in a temporary file.""" with tempfile.NamedTemporaryFile(mode="w", suffix=".py", delete=False) as f: f.write(code) tmp_path = f.name try: result = subprocess.run( ["python3", tmp_path], capture_output=True, text=True, timeout=10 ) output = result.stdout or result.stderr return output[:2000] if output else "(no output)" except subprocess.TimeoutExpired: return "Error: code execution timed out (10s limit)" finally: os.unlink(tmp_path) def read_file(path: str) -> str: try: with open(path) as f: return f.read()[:3000] except Exception as e: return f"Error reading file: {e}" code_agent = ReAct( agent=Agent( "code_debugger", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="You are an expert Python debugger. " "Read the failing code, understand the error, write a fix, " "and verify it works by running it. " "Format:\n" "Thought: \n" "Action: tool_name(argument)\n" "FINISH: ", ), tools={ "run_python": run_python, "read_file": read_file, }, max_steps=10, ) result = asyncio.run(code_agent.run( "The following function raises a RecursionError for large inputs. " "Debug it and provide a working iterative implementation:\n\n" "def fibonacci(n):\n if n <= 1: return n\n return fibonacci(n-1) + fibonacci(n-2)" )) print(result.output) ``` ______________________________________________________________________ ## Use Case 3 — API Orchestration Agent (LiteLLM) ``` import json from pyagent_providers import LiteLLM def get_weather(city: str) -> str: # Replace with real weather API weather_data = { "london": "12°C, overcast, 65% humidity", "new york": "8°C, partly cloudy, 45% humidity", "tokyo": "18°C, sunny, 55% humidity", } return weather_data.get(city.lower(), f"Weather data for {city} not available") def convert_currency(amount_and_currencies: str) -> str: # Replace with real FX API parts = amount_and_currencies.split() try: amount = float(parts[0]) from_ccy = parts[1].upper() to_ccy = parts[3].upper() if len(parts) > 3 else "USD" rates = {"USD_EUR": 0.92, "EUR_USD": 1.09, "USD_GBP": 0.79, "GBP_USD": 1.27} rate = rates.get(f"{from_ccy}_{to_ccy}", 1.0) return f"{amount * rate:.2f} {to_ccy}" except Exception: return "Error parsing currency conversion request" api_agent = ReAct( agent=Agent( "travel_assistant", LiteLLM("gpt-4o-mini"), system_prompt="You are a travel assistant. Use tools to get real-time data. " "Format: Thought: \nAction: tool_name(argument)\nFINISH: ", ), tools={"get_weather": get_weather, "convert_currency": convert_currency}, max_steps=6, ) result = asyncio.run(api_agent.run( "I'm traveling from New York to London next week. " "What's the weather like there, and how much would £500 GBP be in USD?" )) print(result.output) ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.react (4.8s, $0.016) ├── Step 1: Thought + Action: search("nvidia revenue 2025") (1.2s, gpt-4o) │ └── Observation: Revenue $35.1B (+94% YoY) ├── Step 2: Thought + Action: search("nvidia p/e ratio") (1.1s) │ └── Observation: Forward P/E 45x ├── Step 3: Thought + Action: get_price("NVDA") (0.9s) │ └── Observation: $875.40 ├── Step 4: Thought + Action: calculator("35.1 / 30.0") (0.8s) │ └── Observation: 1.17 (YoY revenue ratio) └── Step 5: Thought + FINISH (0.8s) → recommendation steps: 5, tools_used: [search, get_price, calculator] ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ----------------------------------------- | ------------------------------------------- | | Task requires real-time or external data | ✅ Use ReAct | | Number of steps is unpredictable | ✅ Use ReAct | | Tools are deterministic functions | ✅ Use ReAct | | Task can be solved without external tools | ❌ Single-shot call | | Tool calls are irreversible actions | ❌ Add Human-in-the-Loop gate before action | | Parallel tool execution needed | ❌ Use Orchestrator-Workers | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **ReAct** pattern: | Recipe | Domain | What it does | Complexity | | --------------------------------------------------------------------------------------------------- | ----------------------- | ---------------------------------------------------------------------------------- | ------------ | | [SQL Analyst](https://pyagent.org/cookbook/data-analytics/sql-analyst/index.md) | Data & Analytics | Reason-act loop that writes and runs SQL to answer questions | Intermediate | | [Fraud Investigation Assistant](https://pyagent.org/cookbook/security/fraud-investigation/index.md) | Security & Threat Intel | A reason-act agent calls transaction, anomaly, and sanctions tools to build a case | Advanced | | [Research Assistant](https://pyagent.org/cookbook/research-analysis/research-assistant/index.md) | Research & Analysis | Gather sources, debate, synthesize, and cite | Advanced | ## See Also - [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) — parallel tool/worker execution rather than sequential steps - [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) — add human approval gate before executing real-world actions - [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) — ReAct as the "reasoner" in a fast/slow routing system ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** **ReAct** · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Swarm Pattern Decentralized multi-agent system: agents form local views, share with neighbors across rounds, and global consensus or diversity emerges from local interaction. **Best for:** Collective intelligence, opinion diversity, technology scanning, distributed decision-making.\ **LLM calls:** N agents × R rounds. ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant A1 as Agent 1 participant A2 as Agent 2 participant A3 as Agent 3 rect rgb(230, 245, 255) Note over A1,A3: Round 0 — Independent positions A1->>A1: Form initial view A2->>A2: Form initial view A3->>A3: Form initial view end rect rgb(255, 245, 230) Note over A1,A3: Round 1 — Neighbor interaction A1->>A2: Share view A2->>A3: Share view A3->>A1: Share view A1->>A1: Update based on A3's view A2->>A2: Update based on A1's view A3->>A3: Update based on A2's view end ``` ______________________________________________________________________ ## Use Case 1 — Technology Trend Forecasting (Gemini) Five independent analysts form views, share with 2 neighbors each round, and refine through interaction. ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.advanced import Swarm from pyagent_providers import GeminiLLM pattern = Swarm( agents=[ Agent( "ml_researcher", GeminiLLM("gemini-2.5-flash"), system_prompt="You are an ML researcher. Form views based on ML/AI research trends: " "what models, architectures, and capabilities will dominate in 2 years. " "When you see a neighbor's view, update yours if their evidence is compelling.", ), Agent( "infra_engineer", GeminiLLM("gemini-2.5-flash"), system_prompt="You are a cloud infrastructure engineer. Form views based on platform " "and infrastructure trends. Update your view when presented with " "compelling evidence from colleagues.", ), Agent( "product_manager", GeminiLLM("gemini-2.5-flash"), system_prompt="You are a product manager tracking user adoption trends. " "Focus on what users actually want vs what researchers build. " "Update your view based on compelling market evidence.", ), Agent( "vc_analyst", GeminiLLM("gemini-2.5-flash"), system_prompt="You are a VC analyst tracking investment flows and startup activity. " "Form views on where capital is flowing and why. " "Update based on compelling evidence from colleagues.", ), Agent( "enterprise_architect", GeminiLLM("gemini-2.5-flash"), system_prompt="You are an enterprise architect tracking corporate adoption. " "Focus on what enterprises are actually deploying vs piloting. " "Update based on compelling evidence.", ), ], rounds=3, neighbor_count=2, aggregation="last", ) result = asyncio.run(pattern.run( "What are the top 3 most important AI/ML technology trends for enterprise software in 2026?" )) print(result.output) print(f"Agents: {result.metadata['agents']}, Rounds: {result.metadata['rounds']}") print(f"Cost: ${result.cost_estimate:.4f}") ``` Declare the same swarm as a `pyagent-blueprint` manifest: ``` api_version: pyagent/v1 metadata: name: trend-forecast-swarm version: 1.0.0 description: Independent analysts form views and refine them through neighbor interaction providers: primary: { model: gemini-2.5-flash } agents: ml_researcher: { provider: primary, prompt: "Form views on ML/AI research trends. Update when neighbors are compelling." } infra_engineer: { provider: primary, prompt: "Form views on infrastructure trends. Update when neighbors are compelling." } product_manager: { provider: primary, prompt: "Form views on product trends. Update when neighbors are compelling." } workflows: forecast: pattern: swarm agents: agents: [ml_researcher, infra_engineer, product_manager] config: rounds: 3 ``` ``` pyagent-blueprint validate trend-forecast-swarm.yaml pyagent-blueprint test trend-forecast-swarm.yaml ``` ______________________________________________________________________ ## Use Case 2 — Collective Code Architecture Design (OpenAI + Anthropic) ``` from pyagent_providers import OpenAILLM, AnthropicLLM arch_swarm = Swarm( agents=[ Agent( "backend_specialist", OpenAILLM("gpt-4o-mini"), system_prompt="You specialize in backend systems. Form an architecture view " "considering scalability, data models, and API design. " "Update your view when frontend or ops colleagues raise valid constraints.", ), Agent( "frontend_specialist", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="You specialize in frontend systems. Form an architecture view " "considering UX, state management, and build complexity. " "Update when backend constraints are compelling.", ), Agent( "ops_specialist", OpenAILLM("gpt-4o-mini"), system_prompt="You specialize in operations and infrastructure. " "Form a view considering deployment, monitoring, and reliability. " "Push back on architectures that are operationally complex.", ), Agent( "security_specialist", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="You specialize in application security. " "Form a view considering authentication, authorization, and data protection. " "Flag security implications of colleagues' architectural choices.", ), ], rounds=2, neighbor_count=2, aggregation="last", ) result = asyncio.run(arch_swarm.run( "Design the architecture for a multi-tenant analytics platform: " "100k tenants, real-time dashboards, 10TB data/day, SOC2 compliance required." )) ``` ______________________________________________________________________ ## Use Case 3 — Multi-Provider Consensus (LiteLLM) Use different providers to prevent any single model's biases from dominating. ``` from pyagent_providers import LiteLLM diverse_swarm = Swarm( agents=[ Agent(f"analyst_openai", LiteLLM("gpt-4o-mini"), system_prompt="Form and refine your analysis. Update when neighbors make compelling points."), Agent(f"analyst_anthropic", LiteLLM("anthropic/claude-haiku-3.5"), system_prompt="Form and refine your analysis. Update when neighbors make compelling points."), Agent(f"analyst_gemini", LiteLLM("gemini/gemini-2.5-flash"), system_prompt="Form and refine your analysis. Update when neighbors make compelling points."), ], rounds=2, neighbor_count=2, aggregation="consensus", ) result = asyncio.run(diverse_swarm.run( "Should an AI startup prioritize getting to market fast with GPT-4 " "or spend 6 months building proprietary fine-tuned models?" )) ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.swarm (8.2s, $0.021) ├── Round 0 — independent │ ├── pyagent.agent.ml_researcher (1.4s) │ ├── pyagent.agent.infra_engineer (1.3s) │ ├── pyagent.agent.product_manager (1.2s) │ ├── pyagent.agent.vc_analyst (1.1s) │ └── pyagent.agent.enterprise_architect (1.4s) ├── Round 1 — neighbor exchange [neighbor_count=2] │ └── [parallel updates with neighbor context] ├── Round 2 — neighbor exchange │ └── [parallel updates] └── aggregation: last → final swarm output ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ------------------------------------------------ | --------------------------- | | You want emergent collective intelligence | ✅ Use Swarm | | Diverse independent starting positions add value | ✅ Use Swarm | | Agents need a central coordinator | ❌ Use Orchestrator-Workers | | Agents don't interact with each other | ❌ Use Fan-Out/Fan-In | | Fixed roles with structured rounds | ❌ Use Role-Based | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Swarm** pattern: | Recipe | Domain | What it does | Complexity | | -------------------------------------------------------------------------------------------- | -------------------- | ----------------------------------------------------------------------------------- | ---------- | | [Trip-Planning Swarm](https://pyagent.org/cookbook/travel-hospitality/trip-planner/index.md) | Travel & Hospitality | Self-organizing agents negotiate flights, lodging, and activities into an itinerary | Advanced | ## See Also - [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) — independent parallel agents without peer interaction - [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) — structured role collaboration vs emergent peer interaction - [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) — adversarial positions with a judge ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · **Swarm** · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Talker-Reasoner Pattern Dual-process architecture: fast cheap System 1 (Talker) handles simple queries; slow expensive System 2 (Reasoner) handles complex ones. Uncertainty triggers automatic escalation. Based on Google DeepMind's 2024 paper and Kahneman's "Thinking, Fast and Slow." **Best for:** Conversational agents, customer-facing chatbots, high-volume Q&A with mixed complexity.\ **LLM calls:** 1 (talker only) or 2 (talker + reasoner). ~70% of queries stay at talker. ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant U as User participant T as Talker (System 1) participant R as Reasoner (System 2) U->>T: "What's 2+2?" T-->>U: "4" (fast, cheap) U->>T: "Design a distributed rate limiter for 1M RPS" T->>T: Uncertainty detected → escalate T->>R: Forward to System 2 R-->>U: Deep technical analysis (slow, accurate) ``` ______________________________________________________________________ ## Use Case 1 — Cost-Efficient Conversational Agent (Haiku + Sonnet) ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.advanced import TalkerReasoner from pyagent_providers import AnthropicLLM pattern = TalkerReasoner( talker=Agent( "talker", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="You are a helpful assistant. For simple, factual questions: " "answer directly and concisely. " "For complex questions requiring deep reasoning, multi-step analysis, " "or specialized expertise, respond ONLY with: " "ESCALATE: ", ), reasoner=Agent( "reasoner", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="You are a senior technical expert. Provide thorough, accurate analysis. " "Show your reasoning. Use concrete examples. " "Your response will be the user's final answer.", ), escalation_signal="ESCALATE:", ) queries = [ "What is the capital of France?", "How do I reverse a string in Python?", "Design a fault-tolerant distributed rate limiter for 1 million RPS with sub-millisecond latency", "What are the trade-offs between B-tree and LSM-tree storage engines for a write-heavy workload?", ] for query in queries: result = asyncio.run(pattern.run(query)) system = result.metadata["system"] escalated = result.metadata["escalated"] print(f"[System {system}{'↑' if escalated else ''}] {query[:60]}") print(f" Cost: ${result.cost_estimate:.5f}") ``` Declare the same talker/reasoner pair as a `pyagent-blueprint` manifest: ``` api_version: pyagent/v1 metadata: name: cost-efficient-chat version: 1.0.0 description: Fast talker handles routine queries; slow reasoner activates only on uncertainty providers: fast: { model: claude-haiku-3-5-20241022 } smart: { model: claude-sonnet-4-20250514 } agents: talker: provider: fast prompt: "Answer simple questions directly. For complex questions, respond ONLY with: ESCALATE: " reasoner: provider: smart prompt: "Provide thorough, accurate analysis with reasoning and examples." workflows: chat: pattern: talker_reasoner agents: talker: talker reasoner: reasoner ``` ``` pyagent-blueprint validate cost-efficient-chat.yaml pyagent-blueprint test cost-efficient-chat.yaml ``` ______________________________________________________________________ ## Use Case 2 — GPT-4o-mini + GPT-4o Pairing (OpenAI) ``` from pyagent_providers import OpenAILLM cost_optimizer = TalkerReasoner( talker=Agent( "fast_agent", OpenAILLM("gpt-4o-mini"), system_prompt="Answer straightforward questions directly and concisely. " "If a question requires: complex multi-step reasoning, " "domain expertise, code generation over 20 lines, " "or careful analysis of trade-offs, respond: " "ESCALATE: ", ), reasoner=Agent( "expert_agent", OpenAILLM("gpt-4o"), system_prompt="Provide comprehensive, expert-level analysis. " "Think step by step. Show your work.", ), escalation_signal="ESCALATE:", ) result = asyncio.run(cost_optimizer.run( "Compare PostgreSQL vs DynamoDB for a multi-tenant SaaS app " "with 10k tenants, 99.99% uptime requirement, and unpredictable traffic spikes." )) print(result.output) print(f"Escalated to expert: {result.metadata['escalated']}") print(f"Cost: ${result.cost_estimate:.5f}") ``` ______________________________________________________________________ ## Use Case 3 — LiteLLM Mixed-Provider Routing ``` from pyagent_providers import LiteLLM mixed_provider = TalkerReasoner( talker=Agent( "gemini_fast", LiteLLM("gemini/gemini-2.5-flash"), system_prompt="Answer simple questions directly. " "For questions requiring detailed reasoning or expertise, respond: ESCALATE: ", ), reasoner=Agent( "claude_expert", LiteLLM("anthropic/claude-sonnet-4-20250514"), system_prompt="Provide expert-level, detailed analysis. Think carefully and show reasoning.", ), escalation_signal="ESCALATE:", ) ``` ______________________________________________________________________ ## Cost Profile For a typical workload where ~70% of queries are simple: | Strategy | Model | Avg cost/query | | -------------------------------------- | ------------------------ | ----------------------- | | Always GPT-4o | gpt-4o | $0.00400 | | Always Sonnet | claude-sonnet-4-20250514 | $0.00350 | | Talker-Reasoner (Haiku + Sonnet) | 70% Haiku / 30% Sonnet | $0.00114 (~68% savings) | | Talker-Reasoner (GPT-4o-mini + GPT-4o) | 70% mini / 30% GPT-4o | $0.00135 (~66% savings) | ______________________________________________________________________ ## OTel Trace Output ``` # Simple query — talker only Trace: pyagent.pattern.talker_reasoner (0.4s, $0.00003) └── pyagent.agent.talker (0.4s, claude-haiku-3-5-20241022) → direct answer # Complex query — escalated Trace: pyagent.pattern.talker_reasoner (3.2s, $0.00280) ├── pyagent.agent.talker (0.4s, claude-haiku-3-5-20241022) → ESCALATE └── pyagent.agent.reasoner (2.8s, claude-sonnet-4-20250514) → deep analysis ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ------------------------------------------ | -------------------------------- | | High query volume with mixed complexity | ✅ Use Talker-Reasoner | | Cost is a primary constraint | ✅ Use Talker-Reasoner | | All queries are uniformly complex | ❌ Use strong model directly | | Routing should be by topic, not complexity | ❌ Use Supervisor | | Multiple escalation tiers needed | ❌ Chain TalkerReasoner patterns | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Talker-Reasoner** pattern: | Recipe | Domain | What it does | Complexity | | --------------------------------------------------------------------------------------- | ---------------- | ------------------------------------------------------------- | ---------- | | [Support Router](https://pyagent.org/cookbook/customer-support/support-router/index.md) | Customer Support | Classify tickets → route to specialists → escalate to a human | Advanced | ## See Also - [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) — route by topic rather than complexity - [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md) — scored quality gate for the reasoner's output - [Routing Guide](https://pyagent.org/guides/router/index.md) — difficulty scoring and cost-based routing ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · **Talker-Reasoner** · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Fan-Out / Fan-In Pattern Broadcast the same task to N agents in parallel; a dedicated aggregator synthesizes all results. **Best for:** Multi-perspective analysis, parallel research, ensemble fact-checking.\ **LLM calls:** N parallel + 1 aggregator. Wall-clock time: max(agent latencies) + aggregator. ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant U as User participant O as Orchestrator participant A1 as Agent 1 participant A2 as Agent 2 participant A3 as Agent 3 participant Agg as Aggregator U->>O: Task par Parallel execution O->>A1: Same task O->>A2: Same task O->>A3: Same task end A1-->>Agg: Perspective 1 A2-->>Agg: Perspective 2 A3-->>Agg: Perspective 3 Agg-->>U: Synthesized output ``` ______________________________________________________________________ ## Use Case 1 — Investment Analysis (Bull / Bear / Neutral) ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import FanOutFanIn from pyagent_providers import GeminiLLM analysis = FanOutFanIn( agents=[ Agent( "bull", GeminiLLM("gemini-2.5-flash"), system_prompt="Argue the strongest possible bullish investment case. " "Focus on: revenue growth trajectory, market share expansion, " "competitive moats, and catalysts for multiple expansion. " "Use specific data points and comparable valuations.", ), Agent( "bear", GeminiLLM("gemini-2.5-flash"), system_prompt="Argue the strongest possible bearish case. " "Focus on: valuation vs peers, competitive threats, macro risks, " "execution risks, and downside scenarios. " "Use specific data and counter the bull case directly.", ), Agent( "neutral", GeminiLLM("gemini-2.5-flash"), system_prompt="Give a balanced, probability-weighted assessment. " "Acknowledge both bull and bear cases explicitly. " "Identify the 2-3 key swing factors. " "End with a base case, bull case, and bear case price target.", ), ], aggregator=Agent( "analyst", GeminiLLM("gemini-2.5-pro"), system_prompt="Synthesize all three perspectives into a structured investment memo: " "1) Executive Summary (2 sentences), " "2) Bull Case (key points), " "3) Bear Case (key points), " "4) Key Risks & Swing Factors, " "5) Verdict with conviction level (High/Medium/Low).", ), ) result = asyncio.run(analysis.run( "Investment thesis for Nvidia at current valuation — $3.2T market cap, " "P/E 45x forward, data center revenue growing 150% YoY" )) print(result.output) print(f"Parallel agents: {result.metadata['parallel_agents']}") print(f"Duration: {result.duration_seconds:.1f}s, Cost: ${result.cost_estimate:.4f}") ``` Declare the same fan-out/fan-in topology as a `pyagent-blueprint` manifest: ``` api_version: pyagent/v1 metadata: name: investment-analysis version: 1.0.0 description: Bull/bear/neutral perspectives fan out in parallel, then synthesize into a memo owner: research-team providers: fast: model: gemini-2.5-flash smart: model: gemini-2.5-pro agents: bull: provider: fast prompt: "Argue the strongest possible bullish investment case, with specific data points." bear: provider: fast prompt: "Argue the strongest possible bearish case, countering the bull case directly." neutral: provider: fast prompt: "Give a balanced, probability-weighted assessment with base/bull/bear price targets." analyst: provider: smart prompt: "Synthesize all three perspectives into a structured investment memo with a verdict." workflows: analyze: pattern: fan_out_fan_in workers: [bull, bear, neutral] aggregator: analyst ``` ``` pyagent-blueprint validate investment-analysis.yaml pyagent-blueprint test investment-analysis.yaml ``` ### Stream individual agent results as they complete ``` async def stream_analysis(): async for chunk in analysis.stream( "Investment thesis for Nvidia at $3.2T market cap" ): print(chunk, end="", flush=True) asyncio.run(stream_analysis()) ``` ______________________________________________________________________ ## Use Case 2 — Multi-Provider Fact Checking Different providers catch different error types — use all three together. ``` from pyagent_providers import OpenAILLM, AnthropicLLM fact_checker = FanOutFanIn( agents=[ Agent( "openai_checker", OpenAILLM("gpt-4o"), system_prompt="Check factual accuracy. List every claim and mark each: " "VERIFIED (with source), DISPUTED (with counter-evidence), " "or UNVERIFIABLE. Be specific about what you can and cannot verify.", ), Agent( "anthropic_checker", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Check logical consistency and internal contradictions. " "Does the argument hold together? Are there gaps or non-sequiturs? " "List any logical fallacies or unsupported leaps.", ), Agent( "gemini_checker", GeminiLLM("gemini-2.5-pro"), system_prompt="Check for statistical and numerical accuracy. " "Are the numbers plausible? Do percentages add up? " "Are comparisons fair? Flag any misleading use of statistics.", ), ], aggregator=Agent( "verdict", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Combine all checker reports into a single fact-check verdict. " "Give an overall reliability score (0-10). " "List the top 3 concerns and which claims are safe to trust.", ), ) result = asyncio.run(fact_checker.run(open("article_to_check.txt").read())) print(result.output) ``` ______________________________________________________________________ ## Use Case 3 — Parallel Code Review (LangChain) ``` from langchain_openai import ChatOpenAI from langchain_anthropic import ChatAnthropic from pyagent_providers import LangChainLLM code_review = FanOutFanIn( agents=[ Agent( "security_reviewer", LangChainLLM(ChatOpenAI(model="gpt-4o")), system_prompt="Review for security vulnerabilities: injection, auth bypass, " "data exposure, insecure dependencies. Be exhaustive.", ), Agent( "performance_reviewer", LangChainLLM(ChatAnthropic(model="claude-sonnet-4-20250514")), system_prompt="Review for performance issues: N+1 queries, memory leaks, " "unnecessary allocations, blocking I/O in async contexts.", ), Agent( "correctness_reviewer", LangChainLLM(ChatOpenAI(model="gpt-4o")), system_prompt="Review for logic errors, edge cases, off-by-one errors, " "race conditions, and missing error handling.", ), ], aggregator=Agent( "lead_reviewer", LangChainLLM(ChatAnthropic(model="claude-sonnet-4-20250514")), system_prompt="Combine all reviews into a structured PR feedback comment. " "Categorize findings as: BLOCKER, WARNING, SUGGESTION. " "List blockers first. Be specific with line references.", ), ) result = asyncio.run(code_review.run(open("pr_diff.py").read())) print(f"Review complete. Cost: ${result.cost_estimate:.4f}") ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.fan_out_fan_in (2.8s, $0.018) ├── [parallel] │ ├── pyagent.agent.bull (2.1s, gemini-2.5-flash) │ ├── pyagent.agent.bear (2.4s, gemini-2.5-flash) │ └── pyagent.agent.neutral (2.8s, gemini-2.5-flash) └── pyagent.agent.analyst (1.6s, gemini-2.5-pro) ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ------------------------------------------------- | --------------------- | | Multiple independent perspectives improve quality | ✅ Use Fan-Out/Fan-In | | Wall-clock latency matters (parallel is fast) | ✅ Use Fan-Out/Fan-In | | Analyses depend on each other | ❌ Use Pipeline | | Agents need fixed hierarchy | ❌ Use Hierarchical | | Agents should influence each other iteratively | ❌ Use Swarm | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Fan-Out / Fan-In** pattern: | Recipe | Domain | What it does | Complexity | | ------------------------------------------------------------------------------------------------ | ------------------- | --------------------------------------------------------------------------------------- | ------------ | | [Campaign Planner](https://pyagent.org/cookbook/marketing-content/campaign-planner/index.md) | Marketing & Content | Parallel channel ideas aggregated into one campaign plan | Beginner | | [CV Screener](https://pyagent.org/cookbook/hr-recruiting/cv-screener/index.md) | HR & Recruiting | Parallel reviewers score a CV; aggregate into one verdict | Intermediate | | [Research Assistant](https://pyagent.org/cookbook/research-analysis/research-assistant/index.md) | Research & Analysis | Gather sources, debate, synthesize, and cite | Advanced | | [Trading Signal Desk](https://pyagent.org/cookbook/finance-trading/trading-signals/index.md) | Finance & Trading | Momentum, mean-reversion, and sentiment agents fan out then aggregate into a trade idea | Advanced | ## See Also - [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) — sequential stages instead of parallel - [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) — parallel agents that vote on a discrete answer - [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) — agents argue against each other across rounds - [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) — agents interact with neighbors across rounds ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · **Fan-Out / Fan-In** · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Hierarchical Pattern Manager → Team Leads → Workers. Multi-level delegation with parallel team execution and bottom-up synthesis. **Best for:** Complex projects with defined sub-teams, enterprise workflows.\ **LLM calls:** 1 manager + T leads + W workers. Teams execute in parallel. ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant M as Manager participant BL as Backend Lead participant FL as Frontend Lead participant W1 as API Engineer participant W2 as DB Engineer participant W3 as UI Engineer M->>BL: Backend subtask M->>FL: Frontend subtask par Backend workers BL->>W1: API design BL->>W2: DB schema end par Frontend workers FL->>W3: UI components end W1-->>BL: API spec W2-->>BL: Schema doc BL-->>M: Backend summary W3-->>FL: Component spec FL-->>M: Frontend summary M-->>M: Synthesize final output ``` ______________________________________________________________________ ## Use Case 1 — Software System Design (Mixed Providers) ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Hierarchical from pyagent_patterns.orchestration.hierarchical import Team from langchain_openai import ChatOpenAI from langchain_anthropic import ChatAnthropic from pyagent_providers import LangChainLLM, AnthropicLLM, OpenAILLM hierarchical = Hierarchical( manager=Agent( "engineering_director", OpenAILLM("gpt-4o"), system_prompt="Decompose the initiative into team subtasks. " "Assign clear ownership and success criteria to each team. " "After receiving all team outputs, synthesize into a cohesive technical spec. " "Call out integration points, shared contracts, and sequencing dependencies.", ), teams=[ Team( name="Backend", lead=Agent( "backend_lead", OpenAILLM("gpt-4o-mini"), system_prompt="Lead the backend design. Coordinate worker outputs into a cohesive " "backend architecture doc. Resolve conflicts between API and DB designs.", ), workers=[ Agent( "api_engineer", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Design REST API endpoints, request/response schemas, " "authentication strategy, and versioning approach.", ), Agent( "db_engineer", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Design database schema, indexes, migration strategy, " "and query patterns for the most common access paths.", ), Agent( "infra_engineer", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Design deployment architecture, auto-scaling strategy, " "observability stack, and disaster recovery approach.", ), ], ), Team( name="Frontend", lead=Agent( "frontend_lead", OpenAILLM("gpt-4o-mini"), system_prompt="Lead the frontend design. Coordinate worker outputs into a " "cohesive frontend spec with clear component boundaries.", ), workers=[ Agent( "ui_engineer", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Design component architecture, state management approach, " "routing structure, and design system integration.", ), Agent( "ux_researcher", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Define user flows, accessibility requirements (WCAG 2.1 AA), " "and success metrics for the key user journeys.", ), ], ), ], ) result = asyncio.run(hierarchical.run( "Design a real-time collaborative document editor — " "targeting 100k concurrent users, sub-100ms sync latency" )) print(result.output) print(f"Teams: {result.metadata['teams']}, Workers: {result.metadata['total_workers']}") print(f"Cost: ${result.cost_estimate:.4f}") ``` Declare the same manager → teams → workers topology as a `pyagent-blueprint` manifest: ``` api_version: pyagent/v1 metadata: name: system-design version: 1.0.0 description: Director delegates backend and frontend teams; synthesizes a technical spec owner: engineering providers: fast: model: gpt-4o-mini smart: model: gpt-4o agents: engineering_director: provider: smart prompt: "Decompose the initiative into team subtasks and synthesize a cohesive technical spec." backend_lead: provider: fast prompt: "Coordinate API and DB worker outputs into a cohesive backend architecture doc." api_engineer: provider: fast prompt: "Design REST API endpoints, request/response schemas, and auth strategy." db_engineer: provider: fast prompt: "Design database schema, indexes, and migration strategy." frontend_lead: provider: fast prompt: "Coordinate UI worker output into a cohesive frontend architecture doc." ui_engineer: provider: fast prompt: "Design UI components, state management approach, and API integration points." workflows: design: pattern: hierarchical manager: engineering_director teams: - name: Backend lead: backend_lead workers: [api_engineer, db_engineer] - name: Frontend lead: frontend_lead workers: [ui_engineer] ``` ``` pyagent-blueprint validate system-design.yaml pyagent-blueprint test system-design.yaml ``` ______________________________________________________________________ ## Use Case 2 — Market Research Report (Gemini + Anthropic) ``` from pyagent_providers import GeminiLLM research = Hierarchical( manager=Agent( "research_director", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Assign research domains to teams. After receiving all outputs, " "synthesize into an executive market intelligence report with strategic recommendations.", ), teams=[ Team( name="Competitive Intelligence", lead=Agent( "ci_lead", GeminiLLM("gemini-2.5-flash"), system_prompt="Synthesize competitive landscape findings into a structured summary.", ), workers=[ Agent( "direct_comp", GeminiLLM("gemini-2.5-flash"), system_prompt="Analyze direct competitors: features, pricing, positioning, weaknesses.", ), Agent( "indirect_comp", GeminiLLM("gemini-2.5-flash"), system_prompt="Analyze indirect competitors and substitute products.", ), ], ), Team( name="Market Sizing", lead=Agent( "market_lead", GeminiLLM("gemini-2.5-flash"), system_prompt="Synthesize market size and trend analyses.", ), workers=[ Agent( "tam_analyst", GeminiLLM("gemini-2.5-flash"), system_prompt="Estimate TAM, SAM, SOM with methodology and data sources.", ), Agent( "trend_analyst", GeminiLLM("gemini-2.5-flash"), system_prompt="Identify macro and micro trends shaping the market over 3-5 years.", ), ], ), ], ) result = asyncio.run(research.run("Enterprise AI developer tooling market analysis, 2025")) print(result.output) ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.hierarchical (6.2s, $0.031) ├── pyagent.agent.engineering_director — planning (1.1s, gpt-4o) ├── [parallel teams] │ ├── Team: Backend │ │ ├── pyagent.agent.backend_lead (0.9s, gpt-4o-mini) │ │ ├── [parallel workers] │ │ │ ├── pyagent.agent.api_engineer (0.8s) │ │ │ ├── pyagent.agent.db_engineer (0.7s) │ │ │ └── pyagent.agent.infra_engineer (0.9s) │ └── Team: Frontend │ ├── pyagent.agent.frontend_lead (0.8s, gpt-4o-mini) │ └── [parallel workers] ... └── pyagent.agent.engineering_director — synthesis (1.2s, gpt-4o) ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ------------------------------------------------ | ------------------------------------- | | Work decomposes into distinct teams | ✅ Use Hierarchical | | Teams need coordination but can work in parallel | ✅ Use Hierarchical | | Task is flat and simple | ❌ Use Pipeline or Fan-Out/Fan-In | | Subtasks aren't known upfront | ❌ Use Orchestrator-Workers | | Cost is a tight constraint | ❌ Many LLM calls — consider Pipeline | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Hierarchical** pattern: | Recipe | Domain | What it does | Complexity | | ---------------------------------------------------------------------------------------------------------- | -------------------------- | ---------------------------------------------------------------------------------- | ---------- | | [Literature Review Team](https://pyagent.org/cookbook/research-analysis/literature-review/index.md) | Research & Analysis | A lead delegates discovery and synthesis sub-teams to produce a cited review | Advanced | | [Policy Briefing Pipeline](https://pyagent.org/cookbook/government/policy-briefing/index.md) | Government & Public Sector | A director delegates to department leads and analysts, then synthesizes a brief | Advanced | | [Regulatory Compliance Checker](https://pyagent.org/cookbook/legal-compliance/compliance-checker/index.md) | Legal & Compliance | A director delegates obligation and control teams, then synthesizes a gap analysis | Advanced | ## See Also - [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) — dynamic team assignment at runtime - [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) — flat parallel execution without hierarchy - [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) — fixed roles collaborating in shared rounds ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · **Hierarchical** · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Orchestrator-Workers Pattern Orchestrator plans dynamically at runtime and assigns subtasks to the right workers for this specific goal. **Best for:** Open-ended goals where subtasks aren't known upfront.\ **LLM calls:** 1 planning + N workers (dynamic) + 1 synthesis. ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant U as User participant O as Orchestrator participant R as Researcher participant C as Coder participant T as Tester U->>O: Open-ended task O->>O: Plan — which workers, what subtasks par Dynamic assignment O->>R: Research subtask O->>C: Code subtask end R-->>O: Research output C-->>O: Code output O->>T: Test the code T-->>O: Test results O-->>U: Synthesized deliverable ``` ______________________________________________________________________ ## Use Case 1 — Software Feature Build ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import OrchestratorWorkers from pyagent_providers import OpenAILLM orchestrator_workers = OrchestratorWorkers( orchestrator=Agent( "planner", OpenAILLM("gpt-4o"), system_prompt="You have a pool of specialist workers. Plan the work by deciding which workers " "are needed and what to assign each. Only use workers that are genuinely needed. " 'Respond as JSON: {"assignments": [{"worker": "name", "subtask": "description"}]}. ' "After all workers complete, synthesize their output into a final deliverable.", ), workers=[ Agent( "researcher", OpenAILLM("gpt-4o-mini"), system_prompt="Research topics thoroughly. Find recent, credible information " "and summarize key findings with citations.", ), Agent( "coder", OpenAILLM("gpt-4o-mini"), system_prompt="Write clean, idiomatic Python with type hints, docstrings, " "and comprehensive error handling.", ), Agent( "tester", OpenAILLM("gpt-4o-mini"), system_prompt="Write comprehensive pytest test suites. Cover: happy path, " "edge cases, error conditions, and boundary values.", ), Agent( "doc_writer", OpenAILLM("gpt-4o-mini"), system_prompt="Write clear technical documentation with usage examples, " "parameter descriptions, and common gotchas.", ), Agent( "reviewer", OpenAILLM("gpt-4o"), system_prompt="Review code and documentation for correctness, completeness, " "security issues, and adherence to best practices.", ), ], ) result = asyncio.run(orchestrator_workers.run( "Build a production-ready async HTTP client with retry logic, circuit breaking, " "rate limiting, and full test coverage" )) print(result.output) print(f"Workers used: {result.metadata['workers_used']}") print(f"Assignments: {result.metadata['assignments']}") print(f"Cost: ${result.cost_estimate:.4f}") ``` Declare the same orchestrator/worker topology as a `pyagent-blueprint` manifest: ``` api_version: pyagent/v1 metadata: name: feature-build version: 1.0.0 description: Orchestrator plans dynamically and dispatches subtasks to specialist workers owner: platform-team providers: fast: model: gpt-4o-mini smart: model: gpt-4o agents: planner: provider: smart prompt: > You have a pool of specialist workers. Plan the work by deciding which workers are needed and what to assign each. Synthesize their output into a final deliverable. researcher: provider: fast prompt: "Research topics thoroughly and summarize key findings with citations." coder: provider: fast prompt: "Write clean, idiomatic Python with type hints, docstrings, and error handling." tester: provider: fast prompt: "Write comprehensive pytest test suites covering happy path and edge cases." doc_writer: provider: fast prompt: "Write clear technical documentation with usage examples." reviewer: provider: smart prompt: "Review code and docs for correctness, security, and best practices." workflows: build: pattern: orchestrator_workers orchestrator: planner workers: [researcher, coder, tester, doc_writer, reviewer] ``` ``` pyagent-blueprint validate feature-build.yaml pyagent-blueprint test feature-build.yaml ``` ______________________________________________________________________ ## Use Case 2 — Data Analysis (Anthropic + Gemini) ``` from pyagent_providers import AnthropicLLM, GeminiLLM data_analyst = OrchestratorWorkers( orchestrator=Agent( "analysis_planner", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="You are a senior data scientist. Given a dataset description and analysis goal, " "plan which analysis workers to deploy and what each should focus on. " 'Return JSON: {"assignments": [{"worker": "name", "subtask": "..."}]}. ' "Then synthesize all analyses into an executive insights report.", ), workers=[ Agent( "statistician", GeminiLLM("gemini-2.5-flash"), system_prompt="Perform statistical analysis: distributions, correlations, outliers, " "significance tests. Report p-values and confidence intervals.", ), Agent( "visualisation_planner", GeminiLLM("gemini-2.5-flash"), system_prompt="Recommend the most effective visualizations for this data. " "Specify chart types, axes, and what insight each chart reveals.", ), Agent( "hypothesis_tester", GeminiLLM("gemini-2.5-flash"), system_prompt="Propose and evaluate 3 specific hypotheses about the data. " "What would confirm or refute each one?", ), Agent( "storyteller", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Translate statistical findings into a clear business narrative. " "Lead with the most actionable insight. Avoid jargon.", ), ], ) result = asyncio.run(data_analyst.run( "Dataset: 12 months of e-commerce transactions (500k rows). " "Goal: understand what drives repeat purchases and identify churn risk factors." )) print(result.output) ``` ______________________________________________________________________ ## Use Case 3 — Content Strategy (LiteLLM) ``` from pyagent_providers import LiteLLM content_team = OrchestratorWorkers( orchestrator=Agent( "content_director", LiteLLM("gpt-4o"), system_prompt="Plan a content strategy by assigning workers to specific content needs. " 'Return {"assignments": [...]}. Synthesize into a complete 30-day content calendar.', ), workers=[ Agent( "seo_specialist", LiteLLM("gpt-4o-mini"), system_prompt="Research keywords, search intent, and content gaps. " "Identify high-opportunity topics with estimated search volumes.", ), Agent( "copywriter", LiteLLM("anthropic/claude-haiku-3.5"), system_prompt="Write compelling, conversion-focused copy. Match the brand's voice.", ), Agent( "social_media", LiteLLM("gemini/gemini-2.5-flash"), system_prompt="Adapt content for Twitter/X, LinkedIn, and Instagram. " "Tailor tone and format for each platform's norms.", ), Agent( "editor", LiteLLM("anthropic/claude-sonnet-4-20250514"), system_prompt="Edit for clarity, consistency, and brand alignment. " "Check facts and ensure all claims are supportable.", ), ], ) result = asyncio.run(content_team.run( "Create a one-month content strategy for a B2B SaaS company launching a new AI feature" )) ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.orchestrator_workers (8.4s, $0.028) ├── pyagent.agent.planner — planning (1.3s, gpt-4o) │ └── assignments: [researcher, coder, tester, reviewer] ├── [parallel workers] │ ├── pyagent.agent.researcher (2.1s, gpt-4o-mini) │ ├── pyagent.agent.coder (3.4s, gpt-4o-mini) │ └── pyagent.agent.tester (2.8s, gpt-4o-mini) ├── pyagent.agent.reviewer (1.9s, gpt-4o) └── pyagent.agent.planner — synthesis (1.1s, gpt-4o) ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ------------------------------------------------------ | ------------------------------- | | Subtasks aren't known until you analyze the goal | ✅ Use Orchestrator-Workers | | Worker pool has meaningfully different specializations | ✅ Use Orchestrator-Workers | | Subtasks are always the same for this task type | ❌ Use Pipeline or Hierarchical | | Workers need to communicate directly with each other | ❌ Use Blackboard or Swarm | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Orchestrator-Workers** pattern: | Recipe | Domain | What it does | Complexity | | ------------------------------------------------------------------------------------------------------- | ------------------- | ----------------------------------------------------------------------------------------- | ------------ | | [Analytics Task Decomposer](https://pyagent.org/cookbook/data-analytics/analytics-decomposer/index.md) | Data & Analytics | Orchestrator breaks an analytics request into query, transform, and chart workers | Intermediate | | [Product Launch Planner](https://pyagent.org/cookbook/ecommerce-retail/product-launch-planner/index.md) | E-commerce & Retail | Orchestrator splits a launch into pricing/copy/SEO/inventory subtasks for workers | Intermediate | | [ESG Report Analyzer](https://pyagent.org/cookbook/finance-trading/esg-analyzer/index.md) | Finance & Trading | Orchestrator dispatches ratings, disclosure, and SFDR-scoring workers into an ESG summary | Advanced | ## See Also - [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) — fixed team structure decided at design time - [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) — fixed sequential stages, no dynamic planning - [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md) — workers share a state store instead of routing through orchestrator ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · **Orchestrator-Workers**\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Pipeline Pattern Sequential stage chain — each agent's output becomes the next agent's input. **Best for:** ETL, document processing, multi-step transformation.\ **LLM calls:** N (one per stage). Latency: sum of all stages. ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant U as User participant S1 as Stage 1 participant S2 as Stage 2 participant S3 as Stage 3 U->>S1: Raw input S1->>S2: Transformed output S2->>S3: Further transformed S3-->>U: Final output ``` ______________________________________________________________________ ## Use Case 1 — Earnings Report Processing Route cheap extraction to Haiku, fact-checking to GPT-4o-mini, final brief to Sonnet. ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Pipeline from pyagent_providers import AnthropicLLM, OpenAILLM pipeline = Pipeline(stages=[ Agent( "extractor", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Extract every claim, figure, and named entity from the text. " "Be exhaustive — list all revenue figures, percentages, and named companies.", ), Agent( "fact_checker", OpenAILLM("gpt-4o-mini"), system_prompt="Identify which extracted items are verifiable facts vs opinions or speculation. " "Label each: FACT, OPINION, or UNVERIFIABLE.", ), Agent( "writer", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Write a concise, structured brief for a portfolio manager. " "Lead with the single most important fact. Use bullet points. Max 200 words.", ), ]) result = asyncio.run(pipeline.run( "Tesla Q3 2025 earnings: Revenue $25.2B (+8% YoY), auto gross margin 17.1%, " "energy storage deployments up 73% YoY, FSD miles driven passed 2B total. " "CEO commented margins will recover in Q4 as production ramps." )) print(result.output) print(f"Stages: {result.metadata['stages']}, Stage names: {result.metadata['stage_names']}") print(f"Duration: {result.duration_seconds:.1f}s, Cost: ${result.cost_estimate:.4f}") ``` Declare the same stage chain as a `pyagent-blueprint` manifest — validate, diff, and compile it against any registered runtime adapter with the `pyagent-blueprint` CLI: ``` api_version: pyagent/v1 metadata: name: earnings-pipeline version: 1.0.0 description: Extract, fact-check, and write a brief from an earnings report owner: portfolio-team providers: fast: model: claude-haiku-3-5-20241022 smart: model: claude-sonnet-4-20250514 agents: extractor: provider: fast prompt: "Extract every claim, figure, and named entity from the text." fact_checker: provider: fast prompt: "Label each extracted item FACT, OPINION, or UNVERIFIABLE." writer: provider: smart prompt: "Write a concise, structured brief for a portfolio manager. Max 200 words." workflows: process: pattern: pipeline agents: stages: - extractor - fact_checker - writer ``` ``` pyagent-blueprint validate earnings-pipeline.yaml pyagent-blueprint test earnings-pipeline.yaml ``` ### Stream stage completions as they arrive ``` async def stream_pipeline(): async for chunk in pipeline.stream( "Tesla Q3 2025 earnings: Revenue $25.2B (+8% YoY)..." ): print(chunk, end="", flush=True) print() asyncio.run(stream_pipeline()) ``` ______________________________________________________________________ ## Use Case 2 — Legal Document Review Mix Anthropic and Gemini across stages based on their different strengths. ``` from pyagent_providers import GeminiLLM legal_pipeline = Pipeline(stages=[ Agent( "clause_extractor", GeminiLLM("gemini-2.5-flash"), system_prompt="Extract every contractual obligation, deadline, and liability clause. " "Format as a numbered list. Include section references.", ), Agent( "risk_classifier", OpenAILLM("gpt-4o"), system_prompt="Classify each clause as HIGH / MEDIUM / LOW risk. " "Flag anything that exposes the company to uncapped liability or IP loss. " "Explain your risk rating in one sentence per clause.", ), Agent( "counsel_brief", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Write an executive brief for in-house counsel. " "Lead with the top 3 risks. Recommend negotiation priorities. " "Keep it under 300 words.", ), ]) result = asyncio.run(legal_pipeline.run(open("contract.txt").read())) print(result.output) print(f"Stages: {result.metadata['stage_names']}") ``` ______________________________________________________________________ ## Use Case 3 — Multilingual Content Pipeline (LangChain) ``` from langchain_anthropic import ChatAnthropic from langchain_openai import ChatOpenAI from pyagent_providers import LangChainLLM multilingual = Pipeline(stages=[ Agent( "simplifier", LangChainLLM(ChatAnthropic(model="claude-haiku-3-5-20241022")), system_prompt="Rewrite the text at a 6th-grade reading level. " "Keep all key information but remove jargon.", ), Agent( "translator", LangChainLLM(ChatOpenAI(model="gpt-4o-mini")), system_prompt="Translate to Spanish (Latin American). Preserve formatting and tone.", ), Agent( "localizer", LangChainLLM(ChatAnthropic(model="claude-haiku-3-5-20241022")), system_prompt="Adapt cultural references and idioms for a Mexican audience. " "Flag any phrases that don't translate well.", ), ]) result = asyncio.run(multilingual.run(open("product_manual.txt").read())) print(f"Cost: ${result.cost_estimate:.4f}") ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.pipeline (4.1s, $0.012) ├── pyagent.agent.extractor (0.9s, claude-haiku-3-5-20241022) ├── pyagent.agent.fact_checker (1.8s, gpt-4o-mini) └── pyagent.agent.writer (1.4s, claude-sonnet-4-20250514) ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ----------------------------------------- | --------------------------- | | Task has clear sequential stages | ✅ Use Pipeline | | Each stage transforms output for the next | ✅ Use Pipeline | | Stages could run independently | ❌ Use Fan-Out/Fan-In | | You need quality feedback loops | ❌ Use Self-Reflection | | A coordinator decides which stages to run | ❌ Use Orchestrator-Workers | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Pipeline** pattern: | Recipe | Domain | What it does | Complexity | | --------------------------------------------------------------------------------------------------- | -------------------------- | ---------------------------------------------------------------------------------------- | ------------ | | [Contract Testing](https://pyagent.org/cookbook/blueprint-ops/contract-testing/index.md) | Blueprint Ops | Run pyagent-blueprint's built-in MockLLM contract tests in CI before any real model call | Beginner | | [Alert Triage](https://pyagent.org/cookbook/security/log-triage/index.md) | Security & Threat Intel | Triage security alerts; escalate true positives to a human | Intermediate | | [Clinical Summary](https://pyagent.org/cookbook/healthcare/clinical-summary/index.md) | Healthcare & Life Sciences | Summarize clinical notes, then self-critique for accuracy | Intermediate | | [Incident Triage](https://pyagent.org/cookbook/devops-sre/incident-triage/index.md) | DevOps & SRE | Stage pipeline that triages incidents with human sign-off | Intermediate | | [Lead Qualifier](https://pyagent.org/cookbook/sales-crm/lead-qualifier/index.md) | Sales & CRM | Score and route inbound leads to the right play | Intermediate | | [Wealth Rebalancing Crew](https://pyagent.org/cookbook/finance-trading/wealth-rebalancing/index.md) | Finance & Trading | Risk → market scan → allocation → compliance pipeline proposes a rebalance | Intermediate | | [AML Transaction Monitoring](https://pyagent.org/cookbook/finance-trading/aml-monitoring/index.md) | Finance & Trading | Screen → score → enrich pipeline gates high-risk transactions to a human reviewer | Advanced | | [Code Review](https://pyagent.org/cookbook/software-engineering/code-review/index.md) | Software Engineering | Iterative review + security scan with a human gate | Advanced | | [Research Assistant](https://pyagent.org/cookbook/research-analysis/research-assistant/index.md) | Research & Analysis | Gather sources, debate, synthesize, and cite | Advanced | ## See Also - [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) — same agents running in parallel instead of sequentially - [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) — add a critique-refine loop at any stage - [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md) — scored quality gate between stages ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · **Pipeline** · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Supervisor Pattern Classify → route → specialist → optional formatter. A coordinator dispatches to the right expert. **Best for:** Customer support bots, multi-domain Q&A, triage systems.\ **LLM calls:** 2–3 (classify + specialist + optional formatter). ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant U as User participant C as Classifier participant B as Billing Agent participant T as Tech Agent participant F as Formatter U->>C: "I was charged twice" C-->>C: classify → billing C->>B: Handle billing query B-->>F: Raw response F-->>U: Polished response ``` ______________________________________________________________________ ## Use Case 1 — Customer Support Routing Cheap Haiku classifies; stronger Sonnet/GPT-4o handles the specialist reply. The same system can be expressed directly in code with `pyagent-patterns`, or declared as a `pyagent-blueprint` YAML manifest and compiled to any registered runtime adapter — pick whichever fits your workflow. ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Supervisor from pyagent_providers import AnthropicLLM, OpenAILLM supervisor = Supervisor( classifier=Agent( "router", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Classify into exactly one of: billing, technical, returns, general. " "Respond with ONLY the category name, nothing else.", ), routes={ "billing": Agent( "billing_agent", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Handle billing disputes, refunds, and subscription questions. " "Always acknowledge frustration, confirm the charge details, " "and offer concrete next steps with a timeline.", ), "technical": Agent( "technical_agent", OpenAILLM("gpt-4o"), system_prompt="Handle technical troubleshooting and API issues. " "Provide numbered step-by-step debugging instructions. " "Include relevant error codes and documentation links.", ), "returns": Agent( "returns_agent", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Handle return requests. Explain the return policy clearly, " "verify eligibility, and initiate the process where applicable.", ), "general": Agent( "general_agent", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Handle general inquiries warmly and helpfully. " "If out of scope, offer to escalate to a human agent.", ), }, formatter=Agent( "formatter", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Format the response professionally. Remove internal notes. " "Keep under 200 words. Add a friendly closing line.", ), default_route="general", ) result = asyncio.run(supervisor.run("I was charged twice for my Pro subscription this month")) print(result.output) print(f"Route: {result.metadata['route_key']}") print(f"Classifier output: {result.metadata['classifier_output']}") print(f"Cost: ${result.cost_estimate:.4f}") ``` Declare the same Supervisor topology as a `pyagent-blueprint` manifest — validate, diff, simulate, and compile it against any registered runtime adapter with the `pyagent-blueprint` CLI: ``` api_version: pyagent/v1 metadata: name: customer-support version: 1.0.0 description: Support routing system owner: platform-team providers: primary: model: gpt-4.1-mini fallback: model: gpt-4.1-nano context: memory: working_max_tokens: 128000 compression: policy: semantic_lossless target_ratio: 0.6 agents: classifier: prompt: "Classify into: billing, tech, general" provider: primary description: Routes customer requests billing: prompt: "Handle billing inquiries" provider: primary guardrails: [pii_redact] description: Billing specialist tech: prompt: "Handle technical support" provider: primary description: Technical support agent workflows: support: pattern: supervisor agents: classifier: classifier routes: billing: billing tech: tech recovery: max_retries: 2 timeout_seconds: 30 fallback_provider: fallback contracts: support: input: type: string max_tokens: 2000 output: type: string sla: latency_p95_ms: 5000 cost_max_usd: 0.05 observability: tracing: enabled: true cost_budget: daily_usd: 100.0 alert_threshold: 0.8 ``` ``` pyagent-blueprint validate customer-support.yaml pyagent-blueprint test customer-support.yaml ``` ______________________________________________________________________ ## Use Case 2 — Multi-Domain Research Router (LiteLLM) ``` from pyagent_providers import LiteLLM research_supervisor = Supervisor( classifier=Agent( "topic_router", LiteLLM("gpt-4o-mini"), system_prompt="Classify the research question into one of: finance, science, law, history, general. " "Respond with ONLY the category.", ), routes={ "finance": Agent( "finance_expert", LiteLLM("anthropic/claude-sonnet-4-20250514"), system_prompt="You are a CFA-level financial analyst. Provide precise, data-driven analysis. " "Cite specific metrics, ratios, and market data where relevant.", ), "science": Agent( "science_expert", LiteLLM("gemini/gemini-2.5-pro"), system_prompt="You are a PhD-level scientist. Explain complex concepts clearly. " "Reference empirical evidence and recent research.", ), "law": Agent( "legal_expert", LiteLLM("anthropic/claude-sonnet-4-20250514"), system_prompt="You are a legal researcher. Provide thorough analysis with jurisdiction context. " "Always note that this is research, not legal advice.", ), "general": Agent( "generalist", LiteLLM("gpt-4o-mini"), system_prompt="Provide a clear, well-researched answer. Cite sources where possible.", ), }, default_route="general", ) result = asyncio.run(research_supervisor.run( "What was the impact of the 2008 financial crisis on tier-1 bank capital requirements?" )) print(f"Routed to: {result.metadata['route_key']}") print(result.output) ``` ______________________________________________________________________ ## Use Case 3 — Language-Specific Code Review (LangChain) ``` from langchain_openai import ChatOpenAI from langchain_anthropic import ChatAnthropic from pyagent_providers import LangChainLLM code_supervisor = Supervisor( classifier=Agent( "language_detector", LangChainLLM(ChatOpenAI(model="gpt-4o-mini")), system_prompt="Detect the programming language of the code snippet. " "Respond with ONLY: python, javascript, go, rust, or other.", ), routes={ "python": Agent( "python_reviewer", LangChainLLM(ChatAnthropic(model="claude-sonnet-4-20250514")), system_prompt="Review Python code for PEP 8 compliance, type hints, error handling, " "and performance. Suggest specific improvements with code examples.", ), "javascript": Agent( "js_reviewer", LangChainLLM(ChatOpenAI(model="gpt-4o")), system_prompt="Review JavaScript/TypeScript for security vulnerabilities, " "async/await patterns, and modern ES2024+ usage.", ), "go": Agent( "go_reviewer", LangChainLLM(ChatOpenAI(model="gpt-4o")), system_prompt="Review Go code for idiomatic patterns, goroutine safety, " "and proper error handling with wrapping.", ), "other": Agent( "general_reviewer", LangChainLLM(ChatOpenAI(model="gpt-4o-mini")), system_prompt="Review code for logic errors, security issues, and best practices.", ), }, default_route="other", ) result = asyncio.run(code_supervisor.run(open("pull_request.py").read())) print(f"Detected language → route: {result.metadata['route_key']}") ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.supervisor (1.4s, $0.005) ├── pyagent.agent.router (0.3s, claude-haiku-3-5-20241022) │ └── route_decision: billing ├── pyagent.agent.billing_agent (0.9s, claude-sonnet-4-20250514) └── pyagent.agent.formatter (0.2s, claude-haiku-3-5-20241022) ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ---------------------------------------------------- | --------------------------- | | Tasks fall into distinct categories | ✅ Use Supervisor | | Specialists are meaningfully better than generalists | ✅ Use Supervisor | | Tasks don't have clear categories | ❌ Use Orchestrator-Workers | | All tasks need the same processing | ❌ Use Pipeline | | You want adversarial quality checking | ❌ Use Debate | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Supervisor** pattern: | Recipe | Domain | What it does | Complexity | | -------------------------------------------------------------------------------------------- | ----------------- | ---------------------------------------------------------------------------------------------------------- | ------------ | | [Ci Cd Validation](https://pyagent.org/cookbook/blueprint-ops/ci-cd-validation/index.md) | Blueprint Ops | Validate a blueprint and semantically diff it against main in a GitHub Actions PR check | Beginner | | [Governance In Yaml](https://pyagent.org/cookbook/blueprint-ops/governance-in-yaml/index.md) | Blueprint Ops | Declare SLA/budget and recovery policy in YAML; see the stable diagnostic a non-supporting adapter reports | Intermediate | | [Lead Qualifier](https://pyagent.org/cookbook/sales-crm/lead-qualifier/index.md) | Sales & CRM | Score and route inbound leads to the right play | Intermediate | | [Portfolio Review](https://pyagent.org/cookbook/finance-trading/portfolio-review/index.md) | Finance & Trading | Analyst panel with an evaluator-optimizer quality gate | Intermediate | | [Support Router](https://pyagent.org/cookbook/customer-support/support-router/index.md) | Customer Support | Classify tickets → route to specialists → escalate to a human | Advanced | ## See Also - [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) — dynamic routing where subtasks aren't known upfront - [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) — route by complexity rather than topic - [Routing Guide](https://pyagent.org/guides/router/index.md) — combine Supervisor with cost-based model selection ______________________________________________________________________ ## Explore all design patterns **Orchestration:** **Supervisor** · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Cross-Reflection Pattern Generator produces output; a separate reviewer provides critique; generator revises. Repeat until APPROVED or max rounds. **Best for:** High-stakes writing, code review with independent expert, adversarial quality gates.\ **LLM calls:** G + R per round (generator + reviewer). Different models → different blind spots. ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant U as User participant G as Generator participant R as Reviewer U->>G: "Write a technical blog post" G-->>R: First draft R-->>G: "Needs stronger intro, add benchmark data" G-->>R: Revised draft R-->>G: "APPROVED" G-->>U: Final polished post ``` ______________________________________________________________________ ## Use Case 1 — Technical Writing (OpenAI writes, Anthropic reviews) Different provider perspectives catch different quality issues. ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.resolution import CrossReflection from pyagent_providers import AnthropicLLM, OpenAILLM pattern = CrossReflection( generator=Agent( "writer", OpenAILLM("gpt-4o"), system_prompt="Write clear, technically accurate content for a developer audience. " "Use concrete examples, avoid buzzwords, and support claims with data.", ), reviewer=Agent( "editor", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="You are a senior technical editor. Review the draft critically: " "1) Is the opening compelling? " "2) Are all technical claims accurate and supported? " "3) Is the structure logical? " "4) Are there unnecessary words or paragraphs? " "If all pass, respond with exactly APPROVED. " "Otherwise list specific, actionable improvements.", ), max_rounds=3, ) result = asyncio.run(pattern.run( "Write a 500-word technical post: 'Why RAG systems fail at scale and how to fix them'" )) print(result.output) print(f"Rounds: {result.metadata['rounds']}, Early stop: {result.metadata['early_stop']}") print(f"Cost: ${result.cost_estimate:.4f}") ``` Declare the same generator/reviewer loop as a `pyagent-blueprint` manifest: ``` api_version: pyagent/v1 metadata: name: technical-writing-review version: 1.0.0 description: Writer drafts; a separate editor reviews and the writer revises until approved providers: primary: model: gpt-4o reviewer_model: model: claude-sonnet-4-20250514 agents: writer: prompt: "Write clear, technically accurate content for a developer audience." provider: primary editor: prompt: "Review critically for accuracy and structure. Respond APPROVED when satisfied." provider: reviewer_model workflows: review: pattern: cross_reflection agents: generator: writer reviewer: editor config: max_rounds: 3 stop_phrase: APPROVED ``` ``` pyagent-blueprint validate technical-writing-review.yaml pyagent-blueprint test technical-writing-review.yaml ``` ______________________________________________________________________ ## Use Case 2 — Code Security Review (Gemini writes, OpenAI reviews) ``` from pyagent_providers import GeminiLLM secure_coder = CrossReflection( generator=Agent( "developer", GeminiLLM("gemini-2.5-pro"), system_prompt="Write production-quality Python code. " "Follow OWASP secure coding guidelines. " "Use parameterized queries, validate inputs, never log secrets.", ), reviewer=Agent( "security_auditor", OpenAILLM("gpt-4o"), system_prompt="You are an application security engineer. Review the code for: " "1) Injection vulnerabilities (SQL, command, LDAP), " "2) Authentication and authorization flaws, " "3) Sensitive data exposure, " "4) Insecure cryptographic practices. " "Be extremely precise — cite specific line numbers and CWE IDs. " "Respond APPROVED only if zero high/critical issues remain.", ), max_rounds=4, ) result = asyncio.run(secure_coder.run( "Write an authentication endpoint: POST /auth/login " "that accepts username+password, checks against a PostgreSQL users table, " "and returns a JWT token with 24h expiry." )) print(result.output) print(f"Security review rounds: {result.metadata['rounds']}") ``` ______________________________________________________________________ ## Use Case 3 — Business Proposal (LangChain: Anthropic writes, OpenAI reviews) ``` from langchain_openai import ChatOpenAI from langchain_anthropic import ChatAnthropic from pyagent_providers import LangChainLLM proposal_review = CrossReflection( generator=Agent( "strategist", LangChainLLM(ChatAnthropic(model="claude-sonnet-4-20250514")), system_prompt="Write compelling, evidence-based business proposals. " "Structure: Executive Summary, Problem, Solution, ROI, Timeline, Ask.", ), reviewer=Agent( "cfo_reviewer", LangChainLLM(ChatOpenAI(model="gpt-4o")), system_prompt="You are a CFO reviewing a business proposal. Be rigorous: " "1) Are ROI projections realistic and methodology sound? " "2) Are risks identified and mitigated? " "3) Is the ask clearly justified by the projected return? " "4) What would you push back on in a board meeting? " "Respond APPROVED only if you'd take this to the board as-is.", ), max_rounds=3, ) result = asyncio.run(proposal_review.run( "Write a proposal for a $500k investment in ML infrastructure: " "GPU cluster, MLOps tooling, and 2 ML engineers — projected to reduce " "model serving costs by 60% and enable 3 new product features per quarter." )) ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.cross_reflection (6.1s, $0.019) ├── Round 1 │ ├── pyagent.agent.writer — generate (2.1s, gpt-4o) │ └── pyagent.agent.editor — review (1.4s, claude-sonnet-4-20250514) → REVISE ├── Round 2 │ ├── pyagent.agent.writer — generate (1.8s, gpt-4o) │ └── pyagent.agent.editor — review (0.8s, claude-sonnet-4-20250514) → APPROVED └── early_stop: true (round 2 of 3) ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | -------------------------------------------------------- | -------------------------- | | External perspective adds more value than self-review | ✅ Use Cross-Reflection | | Reviewer has genuinely different expertise | ✅ Use Cross-Reflection | | Two independent provider perspectives reduce blind spots | ✅ Use Cross-Reflection | | Speed matters more than quality | ❌ Single-shot call | | You want a scored output quality gate | ❌ Use Evaluator-Optimizer | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Cross-Reflection** pattern: | Recipe | Domain | What it does | Complexity | | ----------------------------------------------------------------------------------------- | -------------------- | ------------------------------------------------------- | ------------ | | [Contract Review](https://pyagent.org/cookbook/legal-compliance/contract-review/index.md) | Legal & Compliance | Draft + independent reviewer loop over contract clauses | Intermediate | | [Code Review](https://pyagent.org/cookbook/software-engineering/code-review/index.md) | Software Engineering | Iterative review + security scan with a human gate | Advanced | ## See Also - [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) — same agent critiques its own output - [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md) — scored quality gate (7/10, 8/10...) - [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) — adversarial multi-round with a judge ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · **Cross-Reflection** · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Debate Pattern Multiple agents argue assigned positions across rounds, then a judge renders a final decision. **Best for:** High-stakes decisions, buy-vs-build, investment theses, strategy trade-offs.\ **LLM calls:** D × R (debaters × rounds) + 1 judge. ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant U as User participant B as Bull Agent participant Be as Bear Agent participant J as Judge U->>B: Argue BUY U->>Be: Argue SELL rect rgb(230, 245, 255) Note over B,Be: Round 1 — Opening arguments B->>J: "Strong earnings growth justifies premium..." Be->>J: "P/E ratio unsustainably high relative to peers..." end rect rgb(255, 245, 230) Note over B,Be: Round 2 — Rebuttals B->>J: "Growth trajectory intact, P/E justified by optionality..." Be->>J: "Growth slowing, competition intensifying in core market..." end J-->>U: "Decision: HOLD — valid points on both sides" ``` ______________________________________________________________________ ## Use Case 1 — Investment Decision (Gemini) ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.resolution import Debate from pyagent_providers import GeminiLLM, AnthropicLLM debate = Debate( debaters=[ Agent( "bull", GeminiLLM("gemini-2.5-pro"), system_prompt="You are a bullish investment analyst. Argue that this investment " "should be made. Use specific financial metrics, growth rates, and " "comparable valuations. Anticipate and pre-emptively rebut bear arguments. " "Support every claim with data.", ), Agent( "bear", GeminiLLM("gemini-2.5-pro"), system_prompt="You are a bearish investment analyst. Argue that this investment " "should NOT be made. Focus on: overvaluation, execution risk, " "competitive threats, and downside scenarios. " "Rebut the bull case with specific counter-evidence.", ), ], judge=Agent( "judge", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="You are a senior portfolio manager reviewing this debate. " "Evaluate argument quality, use of evidence, and logical consistency. " "Identify which side had the stronger case for each key point. " "Render a final verdict: BUY / HOLD / SELL with conviction level. " "Explain which arguments were decisive.", ), rounds=2, positions=["BUY", "SELL"], ) result = asyncio.run(debate.run( "Should we invest in Nvidia at $3.2T market cap, 45x forward P/E, " "with data center revenue growing 150% YoY but competition from AMD and custom silicon?" )) print(result.output) print(f"Rounds: {result.metadata['rounds']}") print(f"Arguments logged: {len(result.metadata['debate_log'])}") print(f"Cost: ${result.cost_estimate:.4f}") ``` Declare the same debaters + judge as a `pyagent-blueprint` manifest: ``` api_version: pyagent/v1 metadata: name: investment-debate version: 1.0.0 description: Bull and bear analysts debate a position; a judge renders the final verdict providers: debater_model: model: gemini-2.5-pro judge_model: model: claude-sonnet-4-20250514 agents: bull: prompt: "Argue this investment should be made. Use specific financial metrics." provider: debater_model bear: prompt: "Argue this investment should NOT be made. Focus on downside risk." provider: debater_model judge: prompt: "Evaluate both arguments and render a final verdict: BUY / HOLD / SELL." provider: judge_model workflows: debate: pattern: debate agents: debaters: [bull, bear] judge: judge config: rounds: 2 positions: ["BUY", "SELL"] ``` ``` pyagent-blueprint validate investment-debate.yaml pyagent-blueprint test investment-debate.yaml ``` ______________________________________________________________________ ## Use Case 2 — Build vs Buy Decision (OpenAI + Anthropic) ``` from pyagent_providers import OpenAILLM build_buy = Debate( debaters=[ Agent( "build_advocate", OpenAILLM("gpt-4o"), system_prompt="Argue that we should BUILD this capability in-house. " "Address: control, customization, long-term cost, IP ownership, " "competitive differentiation, and team capability development.", ), Agent( "buy_advocate", OpenAILLM("gpt-4o"), system_prompt="Argue that we should BUY (use a vendor/SaaS). " "Address: speed to market, TCO vs build cost, vendor expertise, " "maintenance burden, and opportunity cost of engineering time.", ), ], judge=Agent( "decision_maker", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="You are a VP of Engineering making a final recommendation. " "Evaluate both cases. Identify what information is missing. " "Give a recommendation: BUILD / BUY / HYBRID, with a 3-bullet rationale.", ), rounds=2, positions=["BUILD", "BUY"], ) result = asyncio.run(build_buy.run( "Should we build our own RAG pipeline or use a managed solution like Pinecone + LangChain? " "Team: 4 engineers. Timeline: 6 weeks to production. " "Scale: 10M documents, 50k queries/day." )) print(result.output) ``` ______________________________________________________________________ ## Use Case 3 — Technical Architecture Debate (LiteLLM multi-provider) ``` from pyagent_providers import LiteLLM arch_debate = Debate( debaters=[ Agent( "microservices_advocate", LiteLLM("gpt-4o"), system_prompt="Argue for a microservices architecture. Address scalability, " "team autonomy, independent deployment, and fault isolation.", ), Agent( "monolith_advocate", LiteLLM("anthropic/claude-sonnet-4-20250514"), system_prompt="Argue for a modular monolith. Address simplicity, " "developer experience, deployment complexity, and premature optimization.", ), ], judge=Agent( "architect", LiteLLM("gemini/gemini-2.5-pro"), system_prompt="Evaluate the architecture debate. Consider the team size, scale, " "and constraints. Give a specific recommendation with trade-offs.", ), rounds=2, positions=["MICROSERVICES", "MONOLITH"], ) result = asyncio.run(arch_debate.run( "Architecture decision for a new B2B SaaS platform: 5-engineer team, " "18-month runway, targeting 1k enterprise customers at launch." )) ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.debate (9.8s, $0.038) ├── Round 1 │ ├── pyagent.agent.bull (2.4s, gemini-2.5-pro) │ └── pyagent.agent.bear (2.7s, gemini-2.5-pro) ├── Round 2 │ ├── pyagent.agent.bull (2.1s, gemini-2.5-pro) │ └── pyagent.agent.bear (1.9s, gemini-2.5-pro) └── pyagent.agent.judge (0.7s, claude-sonnet-4-20250514) ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ------------------------------------------------ | --------------------------------------------- | | Decision requires examining opposing viewpoints | ✅ Use Debate | | Stakes are high — adversarial testing adds value | ✅ Use Debate | | Answer is factual, not arguable | ❌ Single-shot call | | Budget is tight | ❌ Many LLM calls — consider Cross-Reflection | | You need nuanced open-ended output | ❌ Use Evaluator-Optimizer | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Debate** pattern: | Recipe | Domain | What it does | Complexity | | ------------------------------------------------------------------------------------------------------ | ------------------- | --------------------------------------------------------------------------------- | ---------- | | [Loan Underwriting Committee](https://pyagent.org/cookbook/finance-trading/loan-underwriting/index.md) | Finance & Trading | Approve and decline advocates debate an application; a senior underwriter decides | Advanced | | [Research Assistant](https://pyagent.org/cookbook/research-analysis/research-assistant/index.md) | Research & Analysis | Gather sources, debate, synthesize, and cite | Advanced | ## See Also - [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) — parallel agents that vote independently rather than debate - [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) — generate + review rather than argue + judge - [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) — parallel independent analysis without adversarial framing ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · **Debate** · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Evaluator-Optimizer Pattern Generator produces output; evaluator scores against explicit criteria; loop continues until threshold met or max rounds reached. **Best for:** Ad copy, content quality gates, structured output conformance, scored deliverables.\ **LLM calls:** 2 per round (generate + score). Stops early when threshold is crossed. ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant U as User participant G as Generator participant E as Evaluator U->>G: "Write ad copy" G-->>E: "Buy now! Great deals!" E-->>G: Score: 3/10 — too generic, no value prop G-->>E: "Save 30% on premium headphones — today only" E-->>G: Score: 8/10 — PASS G-->>U: Final: "Save 30% on premium headphones — today only" ``` ______________________________________________________________________ ## Use Case 1 — Ad Copy Optimization (OpenAI) ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.resolution import EvaluatorOptimizer from pyagent_providers import OpenAILLM pattern = EvaluatorOptimizer( generator=Agent( "copywriter", OpenAILLM("gpt-4o"), system_prompt="Write conversion-focused ad copy. " "Include: a specific value proposition, a number or statistic, " "a clear benefit, and a call to action. Keep under 30 words.", ), evaluator=Agent( "critic", OpenAILLM("gpt-4o"), system_prompt="Score ad copy on a scale of 1-10 using these criteria: " "1) Specificity (does it have concrete numbers/details?) " "2) Value clarity (is the benefit immediately obvious?) " "3) Urgency (does it motivate action now?) " "4) Credibility (does it feel trustworthy, not salesy?) " "Respond exactly as: SCORE: X\nFEEDBACK: ", ), pass_threshold=8, max_rounds=4, ) result = asyncio.run(pattern.run( "Write ad copy for AirPods Pro 2: targeting commuters, " "key selling points: active noise cancellation, 30hr battery, USB-C charging" )) print(result.output) print(f"Final score: {result.metadata['final_score']}, Rounds: {result.metadata['rounds']}") print(f"Cost: ${result.cost_estimate:.4f}") ``` Declare the same generate/evaluate loop as a `pyagent-blueprint` manifest: ``` api_version: pyagent/v1 metadata: name: ad-copy-optimizer version: 1.0.0 description: Generate ad copy, score it against criteria, and iterate until it clears the bar providers: primary: { model: gpt-4o } agents: copywriter: { provider: primary, prompt: "Write conversion-focused ad copy under 30 words." } critic: { provider: primary, prompt: "Score ad copy 1-10 on specificity, value clarity, urgency, credibility." } workflows: optimize: pattern: evaluator_optimizer agents: generator: copywriter evaluator: critic config: pass_threshold: 8 max_rounds: 4 ``` ``` pyagent-blueprint validate ad-copy-optimizer.yaml pyagent-blueprint test ad-copy-optimizer.yaml ``` ______________________________________________________________________ ## Use Case 2 — Technical Documentation Quality (Anthropic) ``` from pyagent_providers import AnthropicLLM doc_improver = EvaluatorOptimizer( generator=Agent( "doc_writer", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Write clear, comprehensive technical documentation. " "Each function doc must have: a description, all parameters with types, " "return value, exceptions raised, and at least one usage example.", ), evaluator=Agent( "doc_reviewer", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Evaluate documentation completeness and clarity: " "1) Does every function have a description? (0-2 pts) " "2) Are all parameters documented with types? (0-3 pts) " "3) Is the return value documented? (0-2 pts) " "4) Are exceptions documented? (0-1 pt) " "5) Is there at least one working code example? (0-2 pts) " "Respond: SCORE: X\nFEEDBACK: ", ), pass_threshold=9, max_rounds=3, ) result = asyncio.run(doc_improver.run(open("api_module.py").read())) print(f"Documentation score: {result.metadata['final_score']}/10") ``` ______________________________________________________________________ ## Use Case 3 — LLM Response Safety Scoring (LiteLLM) ``` from pyagent_providers import LiteLLM safety_loop = EvaluatorOptimizer( generator=Agent( "responder", LiteLLM("gpt-4o-mini"), system_prompt="Answer questions helpfully and accurately. " "When discussing sensitive topics, include appropriate context and caveats.", ), evaluator=Agent( "safety_scorer", LiteLLM("anthropic/claude-sonnet-4-20250514"), system_prompt="Score this response for safety and helpfulness: " "1) Does it answer the question? (0-3 pts) " "2) Does it include appropriate caveats for sensitive content? (0-3 pts) " "3) Does it avoid harmful, misleading, or dangerous content? (0-4 pts) " "Respond: SCORE: X\nFEEDBACK: ", ), pass_threshold=9, max_rounds=3, ) result = asyncio.run(safety_loop.run( "What are the risks of combining prescription medications with alcohol?" )) print(f"Safety score: {result.metadata['final_score']}/10") print(result.output) ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.evaluator_optimizer (5.3s, $0.014) ├── Round 1 │ ├── pyagent.agent.copywriter — generate (1.6s, gpt-4o) → "Buy now!" │ └── pyagent.agent.critic — evaluate (1.1s, gpt-4o) → SCORE: 3 ├── Round 2 │ ├── pyagent.agent.copywriter — generate (1.4s, gpt-4o) → "Save 30%..." │ └── pyagent.agent.critic — evaluate (0.9s, gpt-4o) → SCORE: 8 ✓ PASS └── early_stop: true (threshold 8 met in round 2) ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ---------------------------------------------- | ------------------------------------------ | | You have explicit, measurable quality criteria | ✅ Use Evaluator-Optimizer | | Iterative improvement is clearly worthwhile | ✅ Use Evaluator-Optimizer | | Quality criteria are subjective or qualitative | ❌ Use Cross-Reflection | | You just need approve/reject (no score) | ❌ Use Self-Reflection or Cross-Reflection | | Budget is tight | ❌ Single-shot generation | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Evaluator-Optimizer** pattern: | Recipe | Domain | What it does | Complexity | | ------------------------------------------------------------------------------------------ | ----------------- | ------------------------------------------------------ | ------------ | | [Portfolio Review](https://pyagent.org/cookbook/finance-trading/portfolio-review/index.md) | Finance & Trading | Analyst panel with an evaluator-optimizer quality gate | Intermediate | ## See Also - [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) — same agent critiques and revises without scoring - [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) — separate reviewer with qualitative feedback - [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) — human replaces the evaluator for high-stakes decisions ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · **Evaluator-Optimizer**\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Self-Reflection Pattern An agent generates output, critiques its own work, and refines iteratively until approved or max rounds reached. **Best for:** Code generation, essay writing, structured output quality improvement.\ **LLM calls:** 2–2R (generate + critique per round). Stops early on APPROVED. ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant U as User participant A as Agent U->>A: "Write a sorting function" A->>A: Generate: def sort(arr): return sorted(arr) A->>A: Critique: missing null check, empty array case A->>A: Refine: added guards A->>A: Critique: APPROVED A-->>U: Final refined function ``` ______________________________________________________________________ ## Use Case 1 — Code Generation with Self-Review ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.resolution import SelfReflection from pyagent_providers import OpenAILLM pattern = SelfReflection( agent=Agent( "coder", OpenAILLM("gpt-4o-mini"), system_prompt="Write clean, idiomatic Python with type hints and docstrings.", ), critic_prompt="Review the code you just wrote. Check for: " "1) Missing edge cases (null, empty, negative inputs), " "2) Missing error handling, " "3) Type hint completeness, " "4) Potential off-by-one errors. " "If all issues are resolved, respond exactly with APPROVED. " "Otherwise list specific problems to fix.", max_rounds=3, ) result = asyncio.run(pattern.run( "Write a robust function that merges two sorted lists into a single sorted list. " "Handle all edge cases." )) print(result.output) print(f"Rounds: {result.metadata['rounds']}, Early stop: {result.metadata['early_stop']}") print(f"Cost: ${result.cost_estimate:.4f}") ``` Declare the same generate-critique loop as a `pyagent-blueprint` manifest: ``` api_version: pyagent/v1 metadata: name: code-self-review version: 1.0.0 description: Agent writes code, critiques its own work, and refines until approved providers: primary: model: gpt-4o-mini agents: coder: prompt: "Write clean, idiomatic Python with type hints and docstrings. When critiquing, check for missing edge cases and error handling. Respond APPROVED when satisfied." provider: primary workflows: refine: pattern: self_reflection agents: agent: coder config: max_rounds: 3 stop_phrase: APPROVED ``` ``` pyagent-blueprint validate code-self-review.yaml pyagent-blueprint test code-self-review.yaml ``` ______________________________________________________________________ ## Use Case 2 — Writing Quality Loop (Anthropic) Use Sonnet to draft and self-critique against journalism standards. ``` from pyagent_providers import AnthropicLLM writer = SelfReflection( agent=Agent( "journalist", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Write clear, engaging technical journalism for a developer audience. " "Use the inverted pyramid — most important facts first. " "Avoid jargon. Every paragraph should add new information.", ), critic_prompt="Critique the article you just wrote: " "1) Does the lead bury the lede? " "2) Are there unsupported claims that need evidence? " "3) Is any paragraph redundant? " "4) Would a non-expert understand this? " "If all four criteria are satisfied, respond with APPROVED. " "Otherwise list specific revisions.", max_rounds=4, ) result = asyncio.run(writer.run( "Write a 400-word article explaining why LLM agents fail in production " "and what engineering teams should do differently." )) print(result.output) print(f"Revision rounds: {result.metadata['rounds']}") ``` ______________________________________________________________________ ## Use Case 3 — Structured Output Validation (LiteLLM) ``` from pyagent_providers import LiteLLM schema_validator = SelfReflection( agent=Agent( "schema_builder", LiteLLM("gpt-4o"), system_prompt="Generate Pydantic v2 data models from natural language descriptions. " "Include field validators, proper types, and docstrings.", ), critic_prompt="Review the Pydantic schema: " "1) Are all required fields present? " "2) Are validators correct (e.g., regex patterns valid)? " "3) Are field types the most specific possible? " "4) Does the model correctly reflect the described domain? " "Respond APPROVED if all pass, or list specific fixes needed.", max_rounds=3, ) result = asyncio.run(schema_validator.run( "Create a Pydantic model for a financial transaction: " "amount in USD (positive), currency code (ISO 4217), timestamp, " "merchant name, and optional category from: food, transport, utilities, other." )) print(result.output) ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.self_reflection (4.2s, $0.009) ├── Round 1 │ ├── pyagent.agent.coder — generate (1.4s, gpt-4o-mini) │ └── pyagent.agent.coder — critique (1.1s, gpt-4o-mini) → REVISE ├── Round 2 │ ├── pyagent.agent.coder — generate (1.3s, gpt-4o-mini) │ └── pyagent.agent.coder — critique (0.4s, gpt-4o-mini) → APPROVED └── early_stop: true (round 2 of 3) ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ----------------------------------------------------- | -------------------------- | | A single agent can meaningfully self-critique | ✅ Use Self-Reflection | | Task has clear quality criteria | ✅ Use Self-Reflection | | External perspective adds more value than self-review | ❌ Use Cross-Reflection | | You need a scored quality threshold | ❌ Use Evaluator-Optimizer | | Speed is more important than quality | ❌ Single-shot call | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Self-Reflection** pattern: | Recipe | Domain | What it does | Complexity | | --------------------------------------------------------------------------------------------- | -------------------------- | -------------------------------------------------------------------------------------------------- | ------------ | | [Clinical Summary](https://pyagent.org/cookbook/healthcare/clinical-summary/index.md) | Healthcare & Life Sciences | Summarize clinical notes, then self-critique for accuracy | Intermediate | | [Earnings Call Analyzer](https://pyagent.org/cookbook/finance-trading/earnings-call/index.md) | Finance & Trading | A self-reflecting agent iterates on an earnings transcript until the analysis clears a quality bar | Intermediate | ## See Also - [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) — a separate reviewer model critiques the output - [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md) — scored quality gate, optimize until threshold - [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) — adversarial refinement with a judge ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** **Self-Reflection** · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Voting Pattern N agents vote independently on the same question; majority (or weighted) vote wins. **Best for:** Fault-tolerant decisions, ensemble reliability, reducing single-model bias.\ **LLM calls:** N (parallel voters). Majority/weighted tally requires no extra LLM call. ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant U as User participant V1 as Voter 1 participant V2 as Voter 2 participant V3 as Voter 3 participant T as Tallier U->>V1: "Is this PR safe to merge?" U->>V2: "Is this PR safe to merge?" U->>V3: "Is this PR safe to merge?" par Independent voting V1-->>T: "YES — no vulnerabilities" V2-->>T: "YES — code quality good" V3-->>T: "NO — potential N+1 query" end T-->>U: "YES (2/3 majority)" ``` ______________________________________________________________________ ## Use Case 1 — Multi-Provider Ensemble (OpenAI + Anthropic + Gemini) Three independent providers reduce the chance of systematic errors from any single model. ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.resolution import Voting from pyagent_providers import OpenAILLM, AnthropicLLM, GeminiLLM pattern = Voting( voters=[ Agent( "openai_voter", OpenAILLM("gpt-4o"), system_prompt="Evaluate the content moderation request. " "Respond with exactly YES (safe to publish) or NO (violates policy). " "On the second line, give a one-sentence reason.", ), Agent( "anthropic_voter", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Evaluate the content moderation request. " "Respond with exactly YES (safe to publish) or NO (violates policy). " "On the second line, give a one-sentence reason.", ), Agent( "gemini_voter", GeminiLLM("gemini-2.5-pro"), system_prompt="Evaluate the content moderation request. " "Respond with exactly YES (safe to publish) or NO (violates policy). " "On the second line, give a one-sentence reason.", ), ], strategy="majority", ) result = asyncio.run(pattern.run( "User-submitted product review: " "'This product is absolute garbage. The CEO should be fired. " "Returning it immediately and never buying from this brand again.'" )) print(f"Decision: {result.metadata['winner']} (Tally: {result.metadata['tally']})") print(result.output) print(f"Cost: ${result.cost_estimate:.4f}") ``` Declare the same voter ensemble as a `pyagent-blueprint` manifest: ``` api_version: pyagent/v1 metadata: name: moderation-ensemble version: 1.0.0 description: Three independent voters decide safe-to-publish by majority vote providers: openai: { model: gpt-4o } anthropic: { model: claude-sonnet-4-20250514 } gemini: { model: gemini-2.5-pro } agents: openai_voter: { provider: openai, prompt: "Respond YES (safe) or NO (violates policy)." } anthropic_voter: { provider: anthropic, prompt: "Respond YES (safe) or NO (violates policy)." } gemini_voter: { provider: gemini, prompt: "Respond YES (safe) or NO (violates policy)." } workflows: grade: pattern: voting agents: voters: [openai_voter, anthropic_voter, gemini_voter] config: strategy: majority ``` ``` pyagent-blueprint validate moderation-ensemble.yaml pyagent-blueprint test moderation-ensemble.yaml ``` ______________________________________________________________________ ## Use Case 2 — PR Safety Gate (Weighted Voting) Give higher weight to the security reviewer. ``` pattern = Voting( voters=[ Agent( "security_reviewer", OpenAILLM("gpt-4o"), system_prompt="Review this code change for security vulnerabilities. " "Respond YES (safe to merge) or NO (has security issues). " "Explain your reasoning briefly.", ), Agent( "style_reviewer", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Review this code change for style and conventions. " "Respond YES (acceptable) or NO (style violations). " "List any violations.", ), Agent( "performance_reviewer", GeminiLLM("gemini-2.5-flash"), system_prompt="Review this code change for performance issues. " "Respond YES (no performance concerns) or NO (has performance issues). " "Explain your reasoning.", ), ], strategy="weighted", weights={"security_reviewer": 3, "style_reviewer": 1, "performance_reviewer": 2}, ) result = asyncio.run(pattern.run(open("pr_diff.py").read())) print(f"Weighted decision: {result.metadata['winner']}") print(f"Weighted tally: {result.metadata['weighted_tally']}") ``` ______________________________________________________________________ ## Use Case 3 — Classification Ensemble (LiteLLM) Reduce classification error rate by using five independent voters. ``` from pyagent_providers import LiteLLM classifier_ensemble = Voting( voters=[ Agent(f"voter_{i}", LiteLLM("gpt-4o-mini"), system_prompt="Classify customer support ticket as: billing, technical, returns, or general. " "Respond with ONLY the category name.") for i in range(5) ], strategy="majority", ) tickets = [ "I was charged twice this month", "My API key stopped working", "I want to return a defective product", ] for ticket in tickets: result = asyncio.run(classifier_ensemble.run(ticket)) print(f"[{result.metadata['winner']}] {ticket}") print(f" Tally: {result.metadata['tally']}") ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.voting (2.1s, $0.011) ├── [parallel voters] │ ├── pyagent.agent.openai_voter (1.8s, gpt-4o) → YES │ ├── pyagent.agent.anthropic_voter (2.1s, claude-sonnet-4-20250514) → YES │ └── pyagent.agent.gemini_voter (1.6s, gemini-2.5-pro) → NO └── tally: {"YES": 2, "NO": 1} → winner: YES ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ------------------------------------------------- | ---------------------- | | Task has a discrete answer (yes/no, A/B/C) | ✅ Use Voting | | Single model failure should not determine outcome | ✅ Use Voting | | Task requires nuanced, open-ended output | ❌ Use Fan-Out/Fan-In | | Agents should argue and debate | ❌ Use Debate | | Quality improves through iteration | ❌ Use Self-Reflection | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Voting** pattern: | Recipe | Domain | What it does | Complexity | | ------------------------------------------------------------------------------------------ | -------------------- | -------------------------------------------------------------------------- | ------------ | | [Essay Grading by Consensus](https://pyagent.org/cookbook/education/essay-grader/index.md) | Education & Training | Independent graders score against a rubric; a majority vote sets the grade | Intermediate | ## See Also - [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) — adversarial positions with a judge, not a tally - [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) — parallel agents that synthesize rather than vote - [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) — single classifier routes to a specialist ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · **Voting** · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Blackboard Pattern Agents communicate indirectly via a shared, asynchronous state store. Each agent reads specific keys and writes its outputs back — no direct messaging. **Best for:** Financial intelligence, data enrichment pipelines, multi-expert analysis where outputs feed each other.\ **LLM calls:** N agents × R rounds. ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant BB as Blackboard participant A as Alpha Agent participant R as Risk Agent participant P as Portfolio Agent BB->>A: reads: [task] A->>BB: writes: alpha_signals BB->>R: reads: [task, alpha_signals] R->>BB: writes: risk_metrics BB->>P: reads: [alpha_signals, risk_metrics] P->>BB: writes: portfolio_weights ``` ______________________________________________________________________ ## Use Case 1 — Financial Portfolio Construction (OpenAI) Agents chain through the blackboard — each builds on the previous agent's committed state. ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.structural import Blackboard from pyagent_patterns.structural.blackboard import BlackboardAgent from pyagent_providers import OpenAILLM pattern = Blackboard( agents=[ BlackboardAgent( agent=Agent( "alpha_generator", OpenAILLM("gpt-4o"), system_prompt="Generate alpha signals for each asset in the watchlist. " "For each asset: state direction (long/short/neutral), " "conviction (high/medium/low), and 2-sentence rationale. " "Format as structured data.", ), reads=["task"], writes=["alpha_signals"], ), BlackboardAgent( agent=Agent( "risk_manager", OpenAILLM("gpt-4o"), system_prompt="Given the alpha signals, assess portfolio-level risk: " "calculate correlation exposure, sector concentration, " "tail risk scenarios, and max position sizes. " "Recommend position limits for each asset.", ), reads=["task", "alpha_signals"], writes=["risk_metrics"], ), BlackboardAgent( agent=Agent( "portfolio_optimizer", OpenAILLM("gpt-4o"), system_prompt="Given alpha signals AND risk constraints, " "construct the optimal portfolio allocation. " "Respect position limits. Maximize risk-adjusted return. " "Output allocation as percentages summing to 100%.", ), reads=["alpha_signals", "risk_metrics"], writes=["portfolio_weights"], ), BlackboardAgent( agent=Agent( "order_generator", OpenAILLM("gpt-4o-mini"), system_prompt="Given the portfolio weights, " "generate execution orders: asset, direction, size, " "and suggested order type (market/limit). " "Flag any large moves that need staged execution.", ), reads=["portfolio_weights", "risk_metrics"], writes=["trade_orders"], ), ], rounds=1, ) result = asyncio.run(pattern.run( "Watchlist: NVDA, MSFT, AMZN, META, GOOGL. " "Portfolio size: $10M. Max single position: 25%. " "Optimize for risk-adjusted return over 3-month horizon." )) print(result.output) print("Final blackboard state:") for key, value in result.metadata["final_state"].items(): print(f" [{key}]: {str(value)[:80]}...") print(f"Cost: ${result.cost_estimate:.4f}") ``` Declare the same blackboard chain as a `pyagent-blueprint` manifest: ``` api_version: pyagent/v1 metadata: name: portfolio-blackboard version: 1.0.0 description: Agents chain through a shared blackboard — each builds on prior committed state providers: primary: { model: gpt-4o } agents: alpha_generator: { provider: primary, prompt: "Generate alpha signals for each asset." } risk_manager: { provider: primary, prompt: "Assess portfolio risk given the alpha signals." } portfolio_optimizer: { provider: primary, prompt: "Construct the optimal allocation given signals and risk." } workflows: construct: pattern: blackboard agents: agents: - { agent: alpha_generator, reads: [task], writes: [alpha_signals] } - { agent: risk_manager, reads: [task, alpha_signals], writes: [risk_metrics] } - { agent: portfolio_optimizer, reads: [alpha_signals, risk_metrics], writes: [portfolio_weights] } ``` ``` pyagent-blueprint validate portfolio-blackboard.yaml pyagent-blueprint test portfolio-blackboard.yaml ``` ______________________________________________________________________ ## Use Case 2 — Competitive Intelligence Pipeline (Anthropic + Gemini) Each specialist enriches the shared knowledge base before the synthesizer runs. ``` from pyagent_providers import AnthropicLLM, GeminiLLM intelligence = Blackboard( agents=[ BlackboardAgent( agent=Agent( "product_analyst", GeminiLLM("gemini-2.5-flash"), system_prompt="Analyze the competitor's product: features, pricing, " "UX, and technical capabilities. Extract structured data.", ), reads=["task"], writes=["product_analysis"], ), BlackboardAgent( agent=Agent( "market_analyst", GeminiLLM("gemini-2.5-flash"), system_prompt="Analyze market position: market share, customer segments, " "GTM strategy, and partnerships.", ), reads=["task"], writes=["market_analysis"], ), BlackboardAgent( agent=Agent( "strategic_analyst", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Given both product and market analyses, " "identify competitive gaps we can exploit, " "threats to our position, and strategic recommendations.", ), reads=["product_analysis", "market_analysis"], writes=["strategic_assessment"], ), BlackboardAgent( agent=Agent( "report_writer", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Write an executive competitive intelligence report " "from all available analyses. Include: " "Executive Summary, Key Threats, Opportunities, Recommended Actions.", ), reads=["product_analysis", "market_analysis", "strategic_assessment"], writes=["final_report"], ), ], rounds=1, ) result = asyncio.run(intelligence.run("Competitor analysis: Salesforce Einstein AI")) print(result.metadata["final_state"]["final_report"]) ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.blackboard (6.8s, $0.022) ├── Round 1 │ ├── pyagent.agent.alpha_generator (2.1s, gpt-4o) │ │ └── writes: alpha_signals │ ├── pyagent.agent.risk_manager (1.9s, gpt-4o) [reads: alpha_signals] │ │ └── writes: risk_metrics │ ├── pyagent.agent.portfolio_optimizer (1.6s, gpt-4o) [reads: alpha_signals, risk_metrics] │ │ └── writes: portfolio_weights │ └── pyagent.agent.order_generator (1.2s, gpt-4o-mini) [reads: portfolio_weights, risk_metrics] │ └── writes: trade_orders └── final_state: {alpha_signals: ..., risk_metrics: ..., portfolio_weights: ..., trade_orders: ...} ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ------------------------------------------------------ | --------------------------- | | Agents need to share and build on each other's outputs | ✅ Use Blackboard | | Output dependencies are complex (A→C, B→C, both→D) | ✅ Use Blackboard | | A central coordinator routes all communication | ❌ Use Orchestrator-Workers | | Sequential stages with no shared state | ❌ Use Pipeline | | Agents communicate directly with neighbors | ❌ Use Swarm | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Blackboard** pattern: | Recipe | Domain | What it does | Complexity | | ---------------------------------------------------------------------------- | ------------------- | --------------------------------------------------------------------------- | ---------- | | [Emergent NPC World](https://pyagent.org/cookbook/gaming/npc-world/index.md) | Gaming & Simulation | NPC agents read/write a shared world-state blackboard for emergent behavior | Advanced | ## See Also - [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) — central coordinator, not shared state - [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) — peer-to-peer agent communication - [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) — layered processing without persistent shared state ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · **Blackboard**\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Layered Cooperation Agents are organized into abstraction layers. Each layer processes all outputs from the previous layer before passing results up. **Best for:** Multi-stage analysis with increasing abstraction (gather → analyze → synthesize), data pipelines with heterogeneous parallel collectors.\ **LLM calls:** Sum of all agents across all layers. ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant U as User participant L1 as Layer 1: Gather participant L2 as Layer 2: Analyze participant L3 as Layer 3: Synthesize U->>L1: Raw task note over L1: Parallel agents in layer L1-->>L2: Gathered data (all combined) note over L2: Parallel agents in layer L2-->>L3: Analyzed results (all combined) L3-->>U: Final synthesis ``` ______________________________________________________________________ ## Use Case 1 — Business Intelligence Report (Gemini Flash → Pro) Use Flash agents for fast parallel collection, Pro agents for deeper analysis, Sonnet for executive synthesis. ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.structural import Layered from pyagent_patterns.structural.layered import Layer from pyagent_providers import GeminiLLM, AnthropicLLM pattern = Layered( layers=[ Layer( name="gather", agents=[ Agent( "financial_collector", GeminiLLM("gemini-2.5-flash"), system_prompt="Extract all financial metrics, KPIs, and quantitative data " "from the input. Format as a structured list.", ), Agent( "qualitative_collector", GeminiLLM("gemini-2.5-flash"), system_prompt="Extract all qualitative signals: management tone, strategic language, " "risk language, and forward-looking statements.", ), Agent( "competitive_collector", GeminiLLM("gemini-2.5-flash"), system_prompt="Extract all references to competitors, market position, " "competitive advantages, and market share data.", ), ], ), Layer( name="analyze", agents=[ Agent( "trend_analyst", GeminiLLM("gemini-2.5-pro"), system_prompt="Given the collected data, identify the 3 most significant trends " "and their implications. Support with specific data points.", ), Agent( "risk_analyst", GeminiLLM("gemini-2.5-pro"), system_prompt="Given the collected data, identify the top 3 risks " "and rate each: HIGH / MEDIUM / LOW with rationale.", ), ], ), Layer( name="synthesize", agents=[ Agent( "exec_writer", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Synthesize all analyses into a crisp executive briefing: " "1) One-paragraph summary, " "2) Top 3 opportunities, " "3) Top 3 risks, " "4) Recommended actions. " "Keep the total under 400 words.", ), ], ), ], ) result = asyncio.run(pattern.run(open("earnings_transcript.txt").read())) print(result.output) print(f"Layers: {result.metadata['layers']}, Total agents: {result.metadata['total_agents']}") print(f"Cost: ${result.cost_estimate:.4f}") ``` Declare the same three-layer pipeline as a `pyagent-blueprint` manifest: ``` api_version: pyagent/v1 metadata: name: bi-report version: 1.0.0 description: Parallel collectors gather signals, an analysis layer finds trends, and a writer synthesizes an executive briefing providers: fast: { model: gemini-2.5-flash } smart: { model: gemini-2.5-pro } writer_model: { model: claude-sonnet-4-20250514 } agents: financial_collector: { provider: fast, prompt: "Extract financial metrics and KPIs." } qualitative_collector: { provider: fast, prompt: "Extract qualitative signals and risk language." } competitive_collector: { provider: fast, prompt: "Extract competitor and market position references." } trend_analyst: { provider: smart, prompt: "Identify the top 3 trends and implications." } risk_analyst: { provider: smart, prompt: "Identify the top 3 risks, rated HIGH/MEDIUM/LOW." } exec_writer: { provider: writer_model, prompt: "Synthesize all analyses into a crisp executive briefing." } workflows: analyze: pattern: layered agents: layers: - { name: gather, agents: [financial_collector, qualitative_collector, competitive_collector] } - { name: analyze, agents: [trend_analyst, risk_analyst] } - { name: synthesize, agents: [exec_writer] } ``` ``` pyagent-blueprint validate bi-report.yaml pyagent-blueprint test bi-report.yaml ``` ______________________________________________________________________ ## Use Case 2 — Code Repository Analysis (OpenAI) ``` from pyagent_providers import OpenAILLM repo_analyser = Layered( layers=[ Layer( name="scan", agents=[ Agent("dependency_scanner", OpenAILLM("gpt-4o-mini"), system_prompt="List all dependencies, versions, and identify any that are " "outdated, deprecated, or have known CVEs."), Agent("pattern_scanner", OpenAILLM("gpt-4o-mini"), system_prompt="Identify code patterns: what design patterns are used? " "What anti-patterns are present? What conventions are followed?"), Agent("test_scanner", OpenAILLM("gpt-4o-mini"), system_prompt="Assess test coverage: what is tested, what is missing, " "and what is the overall testing strategy?"), ], ), Layer( name="evaluate", agents=[ Agent("quality_evaluator", OpenAILLM("gpt-4o"), system_prompt="Given the scan results, rate code quality (1-10) and " "identify the top 5 improvements with estimated effort."), Agent("risk_evaluator", OpenAILLM("gpt-4o"), system_prompt="Given the scan results, identify the top 3 technical debt " "risks and their potential business impact."), ], ), Layer( name="report", agents=[ Agent("report_writer", OpenAILLM("gpt-4o"), system_prompt="Write a technical health report for engineering leadership. " "Include: quality score, top risks, and a prioritized action plan."), ], ), ], ) result = asyncio.run(repo_analyser.run(open("repo_summary.txt").read())) ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.layered (5.8s, $0.019) ├── Layer 1: gather [parallel] │ ├── pyagent.agent.financial_collector (1.2s, gemini-2.5-flash) │ ├── pyagent.agent.qualitative_collector (1.4s, gemini-2.5-flash) │ └── pyagent.agent.competitive_collector (1.1s, gemini-2.5-flash) ├── Layer 2: analyze [parallel] │ ├── pyagent.agent.trend_analyst (1.8s, gemini-2.5-pro) │ └── pyagent.agent.risk_analyst (2.1s, gemini-2.5-pro) └── Layer 3: synthesize └── pyagent.agent.exec_writer (1.4s, claude-sonnet-4-20250514) ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ------------------------------------------------------- | ------------------- | | Task has naturally increasing levels of abstraction | ✅ Use Layered | | Parallel collection → sequential analysis pattern | ✅ Use Layered | | Stages are sequential without parallelism within layers | ❌ Use Pipeline | | You need a fixed hierarchy with delegation | ❌ Use Hierarchical | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Layered** pattern: | Recipe | Domain | What it does | Complexity | | ------------------------------------------------------------------------------------------------ | ---------------------- | ---------------------------------------------------------------- | ------------ | | [Property Valuation Stack](https://pyagent.org/cookbook/real-estate/property-valuation/index.md) | Real Estate & PropTech | Data → comparables → narrative layers produce a valuation report | Intermediate | ## See Also - [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) — sequential single-agent stages - [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) — manager/lead/worker delegation - [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) — flat parallel execution with aggregator ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · **Layered** · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Role-Based Cooperation Agents with distinct, fixed roles collaborate in structured rounds. Each sees the others' outputs and refines its own contribution. **Best for:** C-suite simulations, cross-functional team alignment, product design councils.\ **LLM calls:** N agents × R rounds. ______________________________________________________________________ ## Sequence Diagram ``` sequenceDiagram participant CEO participant CTO participant CFO rect rgb(230, 245, 255) Note over CEO,CFO: Round 1 — Independent views CEO->>CEO: "Strategy: focus on AI products" CTO->>CTO: "Architecture: cloud-native microservices" CFO->>CFO: "Budget: $2M available for Q1" end rect rgb(255, 245, 230) Note over CEO,CFO: Round 2 — With shared context CEO->>CEO: "Adjusted: AI products within $2M budget" CTO->>CTO: "Revised: prioritize ML infra over new services" CFO->>CFO: "Updated: reallocate 40% of engineering budget to AI" end ``` ______________________________________________________________________ ## Use Case 1 — Product Strategy Council (Anthropic) ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.structural import RoleBased from pyagent_providers import AnthropicLLM pattern = RoleBased( agents=[ Agent( "CEO", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="You are the CEO. Think about: market positioning, competitive advantage, " "customer value, and long-term strategic direction. " "Be decisive. After reviewing other executives' input, " "update your position if the evidence warrants it.", ), Agent( "CTO", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="You are the CTO. Think about: technical feasibility, build vs buy, " "architecture trade-offs, engineering capacity, and technical debt. " "Be honest about what is and isn't possible in the proposed timeline.", ), Agent( "CPO", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="You are the CPO. Think about: user needs, product-market fit, " "feature prioritization, and the roadmap impact of each strategic choice. " "Represent the voice of the customer.", ), Agent( "CFO", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="You are the CFO. Think about: budget constraints, ROI, payback period, " "financial risk, and capital allocation trade-offs. " "Challenge assumptions about projected returns.", ), ], rounds=2, shared_context=True, ) result = asyncio.run(pattern.run( "Should we pivot our B2B SaaS product to add an AI-native layer? " "Investment required: ~$3M over 12 months. " "Competitor just launched a similar feature. " "Current ARR: $8M, growth rate: 35% YoY." )) print(result.output) print(f"Roles: {result.metadata['roles']}, Rounds: {result.metadata['rounds']}") print(f"Cost: ${result.cost_estimate:.4f}") ``` Declare the same council as a `pyagent-blueprint` manifest: ``` api_version: pyagent/v1 metadata: name: strategy-council version: 1.0.0 description: CEO, CTO, and CPO collaborate in structured rounds on a strategic decision providers: primary: { model: claude-sonnet-4-20250514 } agents: ceo: { provider: primary, prompt: "You are the CEO. Think about market positioning and strategic direction." } cto: { provider: primary, prompt: "You are the CTO. Think about technical feasibility and architecture trade-offs." } cpo: { provider: primary, prompt: "You are the CPO. Think about product-market fit and roadmap impact." } workflows: align: pattern: role_based agents: agents: [ceo, cto, cpo] config: rounds: 2 ``` ``` pyagent-blueprint validate strategy-council.yaml pyagent-blueprint test strategy-council.yaml ``` ______________________________________________________________________ ## Use Case 2 — Engineering Design Review (OpenAI + Anthropic) ``` from pyagent_providers import OpenAILLM design_review = RoleBased( agents=[ Agent( "architect", OpenAILLM("gpt-4o"), system_prompt="You are the principal architect. Evaluate system design for: " "scalability, maintainability, security, and alignment with architectural principles. " "Propose specific changes with rationale.", ), Agent( "security_engineer", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="You are the security engineer. Review for: " "threat model coverage, authentication/authorization design, " "data protection, and compliance requirements (GDPR, SOC2). " "Flag every security concern, even theoretical ones.", ), Agent( "staff_engineer", OpenAILLM("gpt-4o-mini"), system_prompt="You are a staff engineer responsible for implementation. " "Evaluate: implementation complexity, team capability fit, " "testing strategy, and operational concerns. " "Flag anything that will be painful to maintain.", ), ], rounds=2, shared_context=True, ) result = asyncio.run(design_review.run(open("system_design_doc.md").read())) ``` ______________________________________________________________________ ## Use Case 3 — Marketing Campaign Review (LiteLLM) ``` from pyagent_providers import LiteLLM campaign_team = RoleBased( agents=[ Agent( "brand_strategist", LiteLLM("anthropic/claude-sonnet-4-20250514"), system_prompt="You are the brand strategist. Evaluate for brand alignment, " "tone consistency, and positioning versus competitors.", ), Agent( "performance_marketer", LiteLLM("gpt-4o"), system_prompt="You are the performance marketer. Evaluate for conversion potential, " "measurability, CAC efficiency, and channel fit.", ), Agent( "legal_reviewer", LiteLLM("anthropic/claude-haiku-3.5"), system_prompt="You are the legal reviewer. Check for: misleading claims, " "required disclosures, trademark issues, and regulatory compliance.", ), ], rounds=2, shared_context=True, ) result = asyncio.run(campaign_team.run( "Review this campaign concept: 'The AI that replaces your entire data team.'" )) ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.role_based (8.4s, $0.024) ├── Round 1 │ ├── pyagent.agent.CEO (2.1s, claude-sonnet-4-20250514) │ ├── pyagent.agent.CTO (1.9s, claude-sonnet-4-20250514) │ ├── pyagent.agent.CPO (2.2s, claude-sonnet-4-20250514) │ └── pyagent.agent.CFO (0.9s, claude-haiku-3-5-20241022) └── Round 2 (with shared context) ├── pyagent.agent.CEO (1.8s) ├── pyagent.agent.CTO (1.6s) ├── pyagent.agent.CPO (1.7s) └── pyagent.agent.CFO (0.7s) ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ----------------------------------------------------- | --------------------- | | Different stakeholder perspectives genuinely conflict | ✅ Use Role-Based | | Roles must see and respond to each other's inputs | ✅ Use Role-Based | | Roles can work entirely independently | ❌ Use Fan-Out/Fan-In | | You want adversarial positions with a judge | ❌ Use Debate | | Roles have a strict delegation hierarchy | ❌ Use Hierarchical | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Role-Based** pattern: | Recipe | Domain | What it does | Complexity | | ------------------------------------------------------------------------------------------------------------ | --------------------- | -------------------------------------------------------------------------------------- | ------------ | | [Customer Onboarding Workflow](https://pyagent.org/cookbook/customer-support/onboarding/index.md) | Customer Support | Verification, setup, FAQ, and success roles onboard a new customer end to end | Intermediate | | [Robo-Advisor Onboarding](https://pyagent.org/cookbook/finance-trading/robo-advisor/index.md) | Finance & Trading | Intake, risk, suitability, and planning roles rotate to build a client investment plan | Intermediate | | [Writers' Room](https://pyagent.org/cookbook/media-entertainment/writers-room/index.md) | Media & Entertainment | Showrunner, writer, editor, and continuity roles collaborate on an episode outline | Intermediate | | [Software Startup Simulation](https://pyagent.org/cookbook/software-engineering/startup-simulation/index.md) | Software Engineering | PM, Architect, Engineer, and QA role-play an idea into PRD, design, code, and tests | Advanced | ## See Also - [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) — two sides argue assigned positions rather than natural roles - [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) — structured delegation hierarchy - [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) — roles emerge dynamically from neighbor interaction ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** **Role-Based** · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Topology Pattern Configure agent communication graphs explicitly: Chain, Star, or Mesh. **Best for:** When the communication structure is the design — routing, hub-and-spoke, fully-connected peer networks.\ **LLM calls:** Varies by topology and size. ______________________________________________________________________ ## Topology Types ``` graph LR subgraph Chain A1[A] --> A2[B] --> A3[C] end subgraph Star B1[Hub] --> B2[Spoke 1] B1 --> B3[Spoke 2] B1 --> B4[Spoke 3] end subgraph Mesh C1[A] <--> C2[B] C2 <--> C3[C] C1 <--> C3 end ``` ______________________________________________________________________ ## Use Case 1 — Code Review Chain (OpenAI + Anthropic) Chain topology where each reviewer builds on the previous one's notes. ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.structural import Topology, TopologyType from pyagent_providers import OpenAILLM, AnthropicLLM chain = Topology( agents=[ Agent( "security_pass", OpenAILLM("gpt-4o"), system_prompt="Review this code for security vulnerabilities. " "List any issues found, or confirm SECURITY PASS if none. " "Pass your findings + the original code to the next reviewer.", ), Agent( "performance_pass", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Review the code and security notes from the previous reviewer. " "Add performance findings. List any N+1 queries, memory leaks, " "or blocking operations. Pass all findings forward.", ), Agent( "style_pass", OpenAILLM("gpt-4o-mini"), system_prompt="Given the code and all previous review notes, " "add style and convention findings. " "Write a final consolidated PR review comment.", ), ], topology=TopologyType.CHAIN, ) result = asyncio.run(chain.run(open("pull_request.py").read())) print(result.output) print(f"Topology: {result.metadata['topology']}, Nodes: {result.metadata['nodes']}") print(f"Cost: ${result.cost_estimate:.4f}") ``` Declare the same reviewer chain as a `pyagent-blueprint` manifest: ``` api_version: pyagent/v1 metadata: name: code-review-chain version: 1.0.0 description: Chain topology — each reviewer builds on the previous one's notes providers: primary: { model: gpt-4o } secondary: { model: claude-sonnet-4-20250514 } agents: security_pass: { provider: primary, prompt: "Review for security vulnerabilities. Pass findings forward." } performance_pass: { provider: secondary, prompt: "Review for performance issues. Pass all findings forward." } style_pass: { provider: primary, prompt: "Add style findings. Write a final consolidated review." } workflows: review: pattern: topology agents: agents: [security_pass, performance_pass, style_pass] config: topology: chain ``` ``` pyagent-blueprint validate code-review-chain.yaml pyagent-blueprint test code-review-chain.yaml ``` ______________________________________________________________________ ## Use Case 2 — Hub-and-Spoke Knowledge Base (Star topology) Central hub coordinates domain-specialist spokes. ``` from pyagent_providers import GeminiLLM star = Topology( agents=[ Agent( "research_hub", GeminiLLM("gemini-2.5-pro"), system_prompt="You are the central research coordinator. " "Route sub-questions to the appropriate domain experts, " "collect their responses, and synthesize a comprehensive answer.", ), Agent( "finance_spoke", GeminiLLM("gemini-2.5-flash"), system_prompt="You are a financial domain expert. " "Answer financial and economic questions with precision. " "Cite data where possible.", ), Agent( "tech_spoke", GeminiLLM("gemini-2.5-flash"), system_prompt="You are a technology domain expert. " "Answer technical questions about software, AI, and systems.", ), Agent( "legal_spoke", GeminiLLM("gemini-2.5-flash"), system_prompt="You are a legal domain expert. " "Answer questions about regulations, compliance, and legal frameworks. " "Always note this is not legal advice.", ), ], topology=TopologyType.STAR, hub_index=0, ) result = asyncio.run(star.run( "What are the regulatory implications and financial risks of a US fintech company " "adding cryptocurrency trading to its platform in 2025?" )) print(result.output) ``` ______________________________________________________________________ ## Use Case 3 — Peer Mesh Brainstorming (LangChain) All agents see all other agents' outputs — fully connected creative team. ``` from langchain_openai import ChatOpenAI from langchain_anthropic import ChatAnthropic from pyagent_providers import LangChainLLM mesh = Topology( agents=[ Agent( "engineer", LangChainLLM(ChatOpenAI(model="gpt-4o-mini")), system_prompt="You are a senior engineer. Contribute technical feasibility analysis " "and implementation ideas. Build on what your teammates said.", ), Agent( "designer", LangChainLLM(ChatAnthropic(model="claude-haiku-3-5-20241022")), system_prompt="You are a product designer. Contribute UX/design perspective. " "Build on what your teammates said.", ), Agent( "marketer", LangChainLLM(ChatOpenAI(model="gpt-4o-mini")), system_prompt="You are a growth marketer. Contribute GTM strategy and positioning. " "Build on what your teammates said.", ), ], topology=TopologyType.MESH, rounds=2, ) result = asyncio.run(mesh.run( "Brainstorm how we should launch a new AI writing assistant targeting indie developers" )) ``` ______________________________________________________________________ ## OTel Trace Output ``` Trace: pyagent.pattern.topology.chain (4.1s, $0.013) ├── pyagent.agent.security_pass (1.8s, gpt-4o) → passed to next ├── pyagent.agent.performance_pass (1.4s, claude-sonnet-4-20250514) → passed to next └── pyagent.agent.style_pass (0.9s, gpt-4o-mini) → final output ``` ______________________________________________________________________ ## When to Use | Condition | Recommendation | | ----------------------------------------------- | ----------------- | | Communication structure IS the design decision | ✅ Use Topology | | Sequential enrichment (chain) | ✅ Topology/Chain | | Central coordinator with domain experts (star) | ✅ Topology/Star | | Fixed sequential stages, no enrichment context | ❌ Use Pipeline | | Dynamic routing based on content classification | ❌ Use Supervisor | ______________________________________________________________________ ## Cookbook recipes Complete, runnable examples that use the **Topology** pattern: | Recipe | Domain | What it does | Complexity | | --------------------------------------------------------------------------------------------------- | --------------------- | ------------------------------------------------------------------------------ | ---------- | | [Loan Origination Workflow](https://pyagent.org/cookbook/finance-trading/loan-origination/index.md) | Finance & Trading | Document → income → credit → approval agents pass an application along a chain | Advanced | | [Peer-Review Mesh](https://pyagent.org/cookbook/scientific/peer-review/index.md) | Scientific & Academic | Reviewers critique a manuscript in a mesh, reconciling notes across rounds | Advanced | ## See Also - [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) — sequential chain without explicit topology API - [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) — star topology with content-based routing - [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md) — agents communicate via shared state rather than direct message passing ______________________________________________________________________ ## Explore all design patterns **Orchestration:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) · [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md)\ **Resolution:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) · [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) · [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md)\ **Structural:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) · [Layered](https://pyagent.org/packages/patterns/structural/layered/index.md) · **Topology** · [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md)\ **Iterative & Advanced:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Talker-Reasoner](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) · [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) [Browse the full pattern catalog →](https://pyagent.org/packages/patterns/index.md) # Composition Guide Combine multiple patterns so the system escalates automatically when output quality isn't good enough. `CompositePattern` runs patterns in order, checks quality after each, and moves to the next only when the current one falls short — giving you the cheapest adequate result rather than always using the most expensive approach. ______________________________________________________________________ ## The Idea Running `claude-sonnet` with a 3-round debate for every task is overkill. Most tasks pass a single self-reflection. A few need debate. Very few need human review. ``` flowchart LR T[Task] --> P1[Self-Reflection\ncheap] P1 --> Q1{Quality OK?} Q1 -->|Yes| R[Result] Q1 -->|No| P2[Debate\nmoderate] P2 --> Q2{Quality OK?} Q2 -->|Yes| R Q2 -->|No| P3[Voting\nexpensive] P3 --> R ``` The quality check runs after each pattern. If it passes, escalation stops. Cost is proportional to how hard the task actually is — not the hardest task you might ever get. ______________________________________________________________________ ## Basic Composition ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.composite import CompositePattern, min_length_check from pyagent_patterns.resolution import SelfReflection, Debate, Voting from pyagent_providers import AnthropicLLM, OpenAILLM cheap_llm = OpenAILLM("gpt-4o-mini") expensive_llm = AnthropicLLM("claude-sonnet-4-20250514") # Level 0 — cheapest: quick self-reflection reflection = SelfReflection( agent=Agent("analyst", cheap_llm, system_prompt="Analyze concisely."), max_rounds=2, ) # Level 1 — moderate: debate between two perspectives debate = Debate( debaters=[ Agent("optimist", cheap_llm, system_prompt="Argue the bullish case."), Agent("sceptic", cheap_llm, system_prompt="Argue the bearish case."), ], judge=Agent("judge", expensive_llm, system_prompt="Give a balanced verdict."), rounds=2, ) # Level 2 — expensive: majority vote across multiple agents voting = Voting( voters=[ Agent(f"expert_{i}", expensive_llm, system_prompt="Give your expert opinion.") for i in range(3) ], ) composite = CompositePattern( patterns=[reflection, debate, voting], quality_check=min_length_check(200), # must produce ≥ 200 chars ) result = asyncio.run(composite.run("Should we invest in Nvidia at $3.2T?")) print(f"Escalated to level: {result.metadata['escalation_level']}") print(f"Patterns tried: {result.metadata['total_patterns_tried']}") print(result.output) ``` ______________________________________________________________________ ## Quality Checks The quality check is a callable that receives the `Result` and returns `True` (good enough) or `False` (escalate). ### Built-in check ``` from pyagent_patterns.composite import min_length_check # Must produce at least 150 characters composite = CompositePattern( patterns=[reflection, debate, voting], quality_check=min_length_check(150), ) ``` ### Custom checks ``` # Require at least one number in the output (useful for financial analysis) def contains_numbers(result) -> bool: return any(c.isdigit() for c in result.output) # Require high confidence score from the pattern def high_confidence(result) -> bool: return result.metadata.get("final_score", 0) >= 8 # Require JSON structure import json def valid_json(result) -> bool: try: json.loads(result.output) return True except json.JSONDecodeError: return False # Combine checks def both_checks(result) -> bool: return contains_numbers(result) and len(result.output) >= 100 composite = CompositePattern( patterns=[reflection, debate], quality_check=both_checks, ) ``` ### No quality check — always run all patterns Omit `quality_check` to run every pattern and return the last result (useful for ensemble approaches where you want all perspectives regardless of quality): ``` composite = CompositePattern( patterns=[fast_agent, thorough_agent, expert_agent], # No quality_check — all three always run ) ``` ______________________________________________________________________ ## Real-World Escalation Scenarios ### Code generation ``` from pyagent_patterns.resolution import SelfReflection, CrossReflection # Level 0: agent writes and reviews its own code self_review = SelfReflection( agent=Agent("coder", cheap_llm, system_prompt="Write and review Python code."), max_rounds=3, ) # Level 1: second agent reviews the first's code peer_review = CrossReflection( generator=Agent("author", cheap_llm, system_prompt="Write the code."), reviewer=Agent("reviewer", expensive_llm, system_prompt="Review for bugs and style."), max_rounds=2, ) # Level 2: human reviews if both automated passes fail from pyagent_patterns.advanced import HumanInTheLoop human_review = HumanInTheLoop( agent=Agent("prep", cheap_llm, system_prompt="Prepare a code review summary."), review_fn=lambda output, meta: queue_for_human(output), ) def compiles_and_has_docstrings(result) -> bool: try: compile(result.output, "", "exec") except SyntaxError: return False return '"""' in result.output or "'''" in result.output code_pipeline = CompositePattern( patterns=[self_review, peer_review, human_review], quality_check=compiles_and_has_docstrings, ) ``` ### Financial analysis ``` from pyagent_patterns.resolution import SelfReflection, Debate, EvaluatorOptimizer # Level 0: quick reflection quick = SelfReflection( agent=Agent("analyst", cheap_llm, system_prompt="Analyze concisely."), max_rounds=1, ) # Level 1: debate bull vs bear adversarial = Debate( debaters=[ Agent("bull", cheap_llm, system_prompt="Strongest bullish case."), Agent("bear", cheap_llm, system_prompt="Strongest bearish case."), ], judge=Agent("judge", expensive_llm), rounds=2, ) # Level 2: criteria-based optimization optimized = EvaluatorOptimizer( generator=Agent("writer", expensive_llm, system_prompt="Write the investment memo."), evaluator=Agent("evaluator", expensive_llm, system_prompt="Score 1-10 on: accuracy, balance, actionability."), max_rounds=3, score_threshold=8, ) # Must mention a number (quantitative insight) AND be long enough def has_quantitative_insight(result) -> bool: return any(c.isdigit() for c in result.output) and len(result.output) > 300 analysis_pipeline = CompositePattern( patterns=[quick, adversarial, optimized], quality_check=has_quantitative_insight, ) ``` ______________________________________________________________________ ## Combining Composition with Recovery Quality-based escalation (`CompositePattern`) handles *bad output*. Failure-based recovery (`BoundedExecution`) handles *API errors, timeouts, and token limits*. They serve different purposes and compose cleanly. ``` from pyagent_patterns.composite import CompositePattern, min_length_check from pyagent_patterns.recovery import BoundedExecution # Inner: escalate on quality escalation = CompositePattern( patterns=[cheap_pattern, moderate_pattern, expensive_pattern], quality_check=min_length_check(150), ) # Outer: recover on failure safe = BoundedExecution( pattern=escalation, fallback=Pipeline(stages=[Agent("emergency", cheap_llm, system_prompt="Give a brief summary.")]), max_retries=1, timeout_seconds=60.0, ) result = asyncio.run(safe.run("Analyze this document")) print(f"Escalation level: {result.metadata.get('escalation_level', 0)}") print(f"Recovery level: {result.metadata.get('recovery_level', 0)}") ``` ______________________________________________________________________ ## Cost Profile Escalation rarely reaches level 2 for typical tasks — most requests pass at level 0 or 1: | Level | Pattern | Typical task hit rate | LLM calls | | ----- | -------------- | --------------------- | ---------- | | 0 | SelfReflection | ~70% | 2–4 | | 1 | Debate | ~25% | 5–8 | | 2 | Voting / Human | ~5% | 9+ / human | The result: most requests run cheap, hard requests escalate automatically. ______________________________________________________________________ ## See Also - [Recovery Guide](https://pyagent.org/guides/recovery/index.md) — failure-based recovery with `BoundedExecution` - [Patterns Package](https://pyagent.org/packages/patterns/index.md) — all pattern classes - [Routing Guide](https://pyagent.org/guides/router/index.md) — complement composition with model-level routing - [API Reference](https://pyagent.org/api/patterns/composition-safety/index.md) # Guardrails Guide Validate, sanitize, and block agent inputs and outputs before they cause downstream problems. `pyagent-patterns` ships four built-in guardrails and a composable chain — no extra package needed. ``` from pyagent_patterns.guardrails import GuardrailChain, LengthGuard, PIIGuard, ContentGuard ``` ______________________________________________________________________ ## Why Guardrails Matter LLM inputs and outputs can contain PII, exceed context limits, or include unsafe content. In multi-agent systems these problems compound — a 50,000-token output from stage 1 will blow up stage 2's context, and unredacted PII passed between agents gets logged by every one of them. Guardrails give you a single enforcement point at each boundary. ______________________________________________________________________ ## Built-in Guards ### LengthGuard Reject or truncate messages that exceed a character limit. ``` from pyagent_patterns.guardrails import LengthGuard # Hard reject — raises if too long strict = LengthGuard(max_chars=5000) result = strict.check("x" * 6000) print(result.passed) # False print(result.message) # "Message exceeds 5000 chars (6000)" # Soft truncate — passes but shortens lenient = LengthGuard(max_chars=5000, truncate=True) result = lenient.check("x" * 6000) print(result.passed) # True print(len(result.sanitized_content)) # 5000 ``` ### PIIGuard Detect and redact emails, phone numbers, SSNs, and credit card numbers. ``` from pyagent_patterns.guardrails import PIIGuard # Redact mode (default: keeps message, replaces PII) guard = PIIGuard(redact=True) result = guard.check("Contact jane@example.com or call 555-867-5309") print(result.sanitized_content) # "Contact [REDACTED-EMAIL] or call [REDACTED-PHONE]" # Reject mode (block the message entirely if PII found) strict = PIIGuard(redact=False) result = strict.check("My SSN is 123-45-6789") print(result.passed) # False print(result.message) # "PII detected: ssn" ``` Detected PII types: `email`, `phone`, `ssn`, `credit_card`. ### ContentGuard Block messages containing deny-listed words or regex patterns. ``` from pyagent_patterns.guardrails import ContentGuard # Exact word matching (case-insensitive) guard = ContentGuard(deny_words=["password", "secret_key", "api_key"]) result = guard.check("Here is my api_key: sk-abc123") print(result.passed) # False print(result.message) # "Blocked content detected: api_key" # Regex patterns for more flexibility import re guard = ContentGuard(deny_patterns=[ re.compile(r"sk-[a-zA-Z0-9]{20,}"), # OpenAI key pattern re.compile(r"ghp_[a-zA-Z0-9]{36}"), # GitHub token pattern ]) ``` ### GuardrailChain Run multiple guards in sequence — stops at the first failure. ``` from pyagent_patterns.guardrails import GuardrailChain, LengthGuard, PIIGuard, ContentGuard chain = GuardrailChain([ LengthGuard(max_chars=10_000, truncate=True), # truncate first PIIGuard(redact=True), # then redact PII ContentGuard(deny_words=["secret", "password"]), # then check content ]) result = chain.check("My email is alice@corp.com and my password is hunter2") # After truncation (if needed) → PII redacted → content check print(result.sanitized_content) # "My email is [REDACTED-EMAIL] and my [BLOCKED] is [BLOCKED]" ``` ______________________________________________________________________ ## Integration Patterns ### Input + Output Validation The most common pattern: guard the user's message before it enters the system, and guard the final output before it reaches the user. ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Pipeline from pyagent_patterns.guardrails import GuardrailChain, PIIGuard, LengthGuard from pyagent_providers import AnthropicLLM input_guard = GuardrailChain([LengthGuard(max_chars=5000), PIIGuard(redact=True)]) output_guard = GuardrailChain([LengthGuard(max_chars=20_000, truncate=True)]) pipeline = Pipeline(stages=[ Agent("analyst", AnthropicLLM("claude-haiku-3-5-20241022")), Agent("writer", AnthropicLLM("claude-sonnet-4-20250514")), ]) async def run_safe(user_input: str) -> str: # Validate input check = input_guard.check(user_input) if not check.passed: return f"Input rejected: {check.message}" safe_input = check.sanitized_content or user_input # Run pipeline result = await pipeline.run(safe_input) # Validate output out_check = output_guard.check(result.output) return out_check.sanitized_content or result.output print(asyncio.run(run_safe("Summarize these notes: ..."))) ``` ### Inter-Agent Guardrail In a pipeline, guard the output of each stage before it becomes the next stage's input. Useful when early stages produce verbose or sensitive outputs. ``` from pyagent_patterns.guardrails import GuardrailChain, LengthGuard, PIIGuard inter_agent_guard = GuardrailChain([ LengthGuard(max_chars=8_000, truncate=True), PIIGuard(redact=True), ]) async def guarded_pipeline(text: str) -> str: r1 = await extractor_agent.run(text) safe = inter_agent_guard.check(r1.output).sanitized_content or r1.output r2 = await analyst_agent.run(safe) safe = inter_agent_guard.check(r2.output).sanitized_content or r2.output r3 = await writer_agent.run(safe) return r3.output ``` ### Guardrails with Fan-Out Guard each parallel agent's output before the aggregator sees it. ``` import asyncio from pyagent_patterns.orchestration import FanOutFanIn from pyagent_patterns.guardrails import GuardrailChain, LengthGuard guard = GuardrailChain([LengthGuard(max_chars=3_000, truncate=True)]) fanout = FanOutFanIn( agents=[bull_agent, bear_agent, macro_agent], aggregator=synthesis_agent, ) async def run_guarded_fanout(task: str) -> str: result = await fanout.run(task) # Guard the aggregated output final = guard.check(result.output) return final.sanitized_content or result.output ``` ______________________________________________________________________ ## Custom Guardrails Implement the `Guardrail` protocol to add your own logic. ``` from dataclasses import dataclass from pyagent_patterns.guardrails import GuardrailChain @dataclass class GuardrailResult: passed: bool message: str = "" sanitized_content: str | None = None class LanguageGuard: """Block messages not written in English.""" def check(self, content: str) -> GuardrailResult: # Simple heuristic: flag if > 30% non-ASCII non_ascii = sum(1 for c in content if ord(c) > 127) if len(content) > 0 and non_ascii / len(content) > 0.3: return GuardrailResult(passed=False, message="Non-English content detected") return GuardrailResult(passed=True) class JSONOutputGuard: """Ensure output is valid JSON.""" def check(self, content: str) -> GuardrailResult: import json try: json.loads(content) return GuardrailResult(passed=True) except json.JSONDecodeError as e: return GuardrailResult(passed=False, message=f"Invalid JSON: {e}") # Compose with built-ins chain = GuardrailChain([ LengthGuard(max_chars=5000), LanguageGuard(), JSONOutputGuard(), ]) ``` ______________________________________________________________________ ## Guardrails in Production ### Logging Blocked Requests ``` import logging logger = logging.getLogger(__name__) async def run_with_logging(user_input: str) -> str: check = input_guard.check(user_input) if not check.passed: logger.warning("Guardrail blocked input", extra={ "reason": check.message, "input_length": len(user_input), }) return "Sorry, I can't process that request." return await pipeline.run(check.sanitized_content or user_input) ``` ### Metrics Track block rates to tune your thresholds: ``` from collections import Counter block_counts: Counter = Counter() def guarded_check(guard: GuardrailChain, content: str): result = guard.check(content) if not result.passed: block_counts[result.message] += 1 return result ``` ______________________________________________________________________ ## Guard Selection Reference | Scenario | Guard | Mode | | ---------------------------------- | ------------------------ | ---------------- | | User inputs that might be too long | `LengthGuard` | truncate | | Agent-to-agent messages | `LengthGuard` | truncate | | Customer-facing apps | `PIIGuard` | redact | | Internal audit logging | `PIIGuard` | reject | | Prompts containing credentials | `ContentGuard` | deny_words | | Structured output stages | custom `JSONOutputGuard` | reject | | Multi-layer production system | `GuardrailChain` | all of the above | ______________________________________________________________________ ## See Also - [Recovery Guide](https://pyagent.org/guides/recovery/index.md) — handle failures when guardrails reject at runtime - [Patterns Package](https://pyagent.org/packages/patterns/index.md) — `Agent`, `Pipeline`, `FanOutFanIn` integration points - [API Reference](https://pyagent.org/api/patterns/composition-safety/index.md) # Recovery Guide Protect multi-agent workflows from cascading failures with bounded execution, circuit breakers, and graceful degradation. `pyagent-patterns` ships recovery primitives that compose with every orchestration pattern. ______________________________________________________________________ ## The Problem LLM calls fail. Models time out, exceed context limits, return empty responses, or throw rate-limit errors. In a single-agent setup that's manageable. In a 5-agent pipeline it's not — one failure propagates forward and the whole workflow produces garbage or crashes. Recovery gives you bounded behavior: retry with the same pattern, fall back to a simpler one, or degrade gracefully to a cached or static response. ______________________________________________________________________ ## Three-Level Recovery ``` flowchart TD T[Task] --> L1[Level 0: Primary Pattern] L1 -->|Success| R[Result] L1 -->|Fail / Timeout / Token Limit| L2[Level 1: Fallback Pattern] L2 -->|Success| R L2 -->|Fail| L3[Level 2: Graceful Degradation] L3 --> R style L1 fill:#4CAF50,color:#fff style L2 fill:#FF9800,color:#fff style L3 fill:#f44336,color:#fff ``` ______________________________________________________________________ ## BoundedExecution The core recovery primitive. Wraps any pattern with: - **Retry** — re-runs the primary up to `max_retries` times on failure - **Timeout** — aborts if the pattern takes longer than `timeout_seconds` - **Token limit** — stops if the workflow consumes more than `max_tokens` - **Fallback** — switches to a simpler pattern if all retries fail ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Pipeline from pyagent_patterns.recovery import BoundedExecution from pyagent_providers import AnthropicLLM, OpenAILLM # Primary: thorough multi-stage pipeline primary = Pipeline(stages=[ Agent("extractor", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Extract all facts, figures, and entities."), Agent("analyst", OpenAILLM("gpt-4o"), system_prompt="Deep analysis of the extracted data."), Agent("writer", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Write a detailed investment brief."), ]) # Fallback: single cheap agent fallback = Pipeline(stages=[ Agent("quick_analyst", OpenAILLM("gpt-4o-mini"), system_prompt="Give a concise summary of the key points."), ]) bounded = BoundedExecution( pattern=primary, fallback=fallback, max_retries=2, timeout_seconds=30.0, max_tokens=50_000, ) result = asyncio.run(bounded.run("Analyze Tesla Q3 2025 earnings")) print(f"Recovery level: {result.metadata.get('recovery_level', 0)}") # 0 = primary succeeded # 1 = fallback used (primary failed or timed out) # 2 = degraded (both failed) print(result.output) ``` ### Recovery metadata Every result from `BoundedExecution` carries recovery metadata you can inspect or log: ``` meta = result.metadata print(meta.get("recovery_level")) # 0, 1, or 2 print(meta.get("retries_used")) # 0–max_retries print(meta.get("timeout_hit")) # True if timeout caused fallback print(meta.get("token_limit_hit")) # True if token budget caused fallback print(meta.get("primary_error")) # Exception message if primary failed ``` ______________________________________________________________________ ## CircuitBreaker Prevent repeated calls to a pattern that's consistently failing. After `failure_threshold` consecutive failures, the circuit opens and rejects all requests for `reset_timeout_seconds`. After the timeout, it allows one test request through. ``` from pyagent_patterns.recovery import CircuitBreaker cb = CircuitBreaker( failure_threshold=3, # open after 3 consecutive failures reset_timeout_seconds=60, # try again after 60s ) result = asyncio.run(cb.execute(expensive_pattern, "Do something")) state = result.metadata.get("circuit_state") # "closed" — normal operation # "open" — circuit tripped, request rejected immediately # "half_open" — one test request allowed through ``` ### Combining BoundedExecution and CircuitBreaker ``` from pyagent_patterns.recovery import BoundedExecution, CircuitBreaker # Primary wrapped with a circuit breaker primary_with_breaker = CircuitBreaker(failure_threshold=3, reset_timeout_seconds=120) bounded = BoundedExecution( pattern=primary_pipeline, fallback=cheap_fallback, max_retries=1, timeout_seconds=20.0, ) async def run_safe(task: str): # Check circuit state first cb_result = await primary_with_breaker.execute(bounded, task) return cb_result.output ``` ______________________________________________________________________ ## Recovery with CompositePattern `CompositePattern` escalates through a series of patterns when quality checks fail. Wrap it with `BoundedExecution` to also handle hard failures (timeouts, API errors). ``` import asyncio from pyagent_patterns.composite import CompositePattern, min_length_check from pyagent_patterns.resolution import SelfReflection, Debate, Voting from pyagent_patterns.recovery import BoundedExecution from pyagent_providers import AnthropicLLM, OpenAILLM cheap_llm = OpenAILLM("gpt-4o-mini") expensive_llm = AnthropicLLM("claude-sonnet-4-20250514") # Escalation: quality-based escalation (cheap → moderate → expensive) escalation = CompositePattern( patterns=[ SelfReflection( agent=Agent("coder", cheap_llm), max_rounds=2, ), Debate( debaters=[Agent("pro", cheap_llm), Agent("con", cheap_llm)], judge=Agent("judge", expensive_llm), rounds=1, ), Voting( voters=[Agent(f"voter_{i}", cheap_llm) for i in range(3)] ), ], quality_check=min_length_check(200), ) # Recovery: failure-based fallback (API down, timeout, token limit) safe_workflow = BoundedExecution( pattern=escalation, fallback=Pipeline(stages=[Agent("last_resort", cheap_llm)]), max_retries=1, timeout_seconds=60.0, ) result = asyncio.run(safe_workflow.run("Design a rate-limiting system")) print(f"Escalation level: {result.metadata.get('escalation_level', 0)}") print(f"Recovery level: {result.metadata.get('recovery_level', 0)}") ``` ______________________________________________________________________ ## Recovery Patterns by Failure Mode ### API rate limits ``` import asyncio from pyagent_patterns.recovery import BoundedExecution # Generous retry with backoff for rate-limited APIs bounded = BoundedExecution( pattern=primary, fallback=fallback, max_retries=3, retry_delay_seconds=5.0, # wait 5s between retries timeout_seconds=120.0, ) ``` ### Context window exceeded ``` from pyagent_compress import CompressMiddleware, TokenBudget from pyagent_patterns.recovery import BoundedExecution # Compress first, recover if compression isn't enough budget = TokenBudget(workflow_limit=100_000) middleware = CompressMiddleware(target_ratio=0.5, budget=budget) bounded = BoundedExecution( pattern=middleware.wrap_all(primary_stages), fallback=Pipeline(stages=[Agent("summarizer", cheap_llm, system_prompt="Give a 3-sentence summary.")]), max_tokens=90_000, ) ``` ### Model unavailability ``` from pyagent_providers.router import ProviderRouter, RoutingStrategy from pyagent_patterns.recovery import BoundedExecution # Provider-level fallback (OpenAI → Anthropic → Gemini) provider_router = ProviderRouter(registry, strategy=RoutingStrategy.FALLBACK_CHAIN) # Pattern-level fallback (expensive → cheap) bounded = BoundedExecution( pattern=Pipeline(stages=expensive_agents), fallback=Pipeline(stages=cheap_agents), max_retries=2, timeout_seconds=30.0, ) ``` ______________________________________________________________________ ## Monitoring and Alerting Track recovery events to know when your system is under stress: ``` import logging from pyagent_patterns.recovery import BoundedExecution logger = logging.getLogger(__name__) bounded = BoundedExecution( pattern=primary, fallback=fallback, max_retries=2, timeout_seconds=30.0, ) async def monitored_run(task: str): result = await bounded.run(task) level = result.metadata.get("recovery_level", 0) if level == 1: logger.warning("primary_failed", extra={ "error": result.metadata.get("primary_error"), "retries": result.metadata.get("retries_used"), }) elif level == 2: logger.error("full_degradation", extra={ "task_preview": task[:100], }) return result ``` ______________________________________________________________________ ## See Also - [Composition Guide](https://pyagent.org/guides/composition/index.md) — quality-based escalation with `CompositePattern` - [Compression Guide](https://pyagent.org/guides/compression/index.md) — prevent token-limit failures before they happen - [Providers Package](https://pyagent.org/packages/providers/index.md) — `FallbackChain` for provider-level resilience - [API Reference](https://pyagent.org/api/patterns/composition-safety/index.md) # Observability # Observability **Verb: Observe.** Two independently installable packages: `pyagent-trace` and `pyagent-studio`. ## What it solves A multi-agent run that fails or costs more than expected is hard to debug from logs alone — you need to know which agent produced which output, what it cost, and ideally be able to replay the exact sequence of events. Generic OpenTelemetry gives you spans; it doesn't know a `Supervisor` pattern's parent/child structure unless something teaches it that. ## When to use - You need distributed tracing that reflects the pattern's actual structure (e.g. supervisor → workers), not a flat call list. - You need running cost totals across a traced run, not just per-call estimates. - You need to reproduce a specific run's exact sequence of events for debugging (record/replay). - You want a visual way to inspect traces, diff blueprint revisions, and check provider health, rather than reading raw CLI/JSON output (`pyagent-studio`'s dashboard). ## When not to use - **Full observability is excessive** for a single-agent, single-call workflow, or for early prototyping where the system's shape changes every few minutes — the tracing setup and span review overhead outweighs what there is to observe. Start with `console` output during development; add `otel`/`langfuse` export once the system is stable enough that tracing pays for itself. - A flat list of calls is already sufficient — pattern-aware spans (`PatternSpanEmitter`) add structure a simple linear script doesn't need. - Live/aggregate observability is all you need — the heavier `Recorder`/replay path is for reproducing a *specific* run, not ongoing monitoring. ## Tradeoffs Exporters have real tradeoffs, not just feature differences: `console` is zero-setup but nothing persists; `jsonl` is durable but nothing consumes it live; `otel` is vendor-neutral but needs a running collector; `langfuse` gives LLM-specific views but ties you to Langfuse. Cost tracking depends on accurate per-provider pricing data to stay trustworthy as providers change pricing. ## Packages - `pip install pyagent-trace` — `TraceEventBus`, `Recorder`, `CostTracker`, `PatternSpanEmitter`, exporters (`console`, `jsonl`, `otel`, `langfuse`). - `pip install pyagent-studio` — CLI + FastAPI/HTMX web dashboard (trace explorer, blueprint diff view, governance/compliance view). ## Example ``` from pyagent_trace import traced_pattern from pyagent_trace.exporters.otel import OTelExporter @traced_pattern(exporter=OTelExporter()) async def run_support_workflow(task: str) -> str: ... ``` See the [Tracing guide](https://pyagent.org/guides/tracing/index.md) and [Studio guide](https://pyagent.org/guides/studio/index.md) for the full API and dashboard walkthrough. ## Related pillars Observability instruments [Execution & Routing](https://pyagent.org/architecture/#execution-routing) patterns and adapters automatically once wired in, and can be declared inside a [Blueprint](https://pyagent.org/architecture/blueprint/index.md)'s `observability:` block. Context-tier changes and redaction events are traceable too — none of that integration is required to use tracing standalone. # Tracing Guide **pyagent-trace** provides pattern-aware OpenTelemetry spans for full observability. ## Architecture ``` flowchart LR P[Pattern.run] --> S[Span Emitter] S --> OT[OpenTelemetry SDK] OT --> B1[Jaeger] OT --> B2[Langfuse] OT --> B3[Grafana Tempo] OT --> B4[Datadog] subgraph Span Attributes PT[pattern.type] AR[agent.name] CT[cost.total_usd] DR[exec.duration_ms] end ``` ## Quick Start ### Decorators (Simplest) ``` from pyagent_trace import traced_pattern, traced_agent from pyagent_patterns.base import Agent, MockLLM from pyagent_patterns.orchestration import Pipeline # Auto-emit spans for every pattern.run() @traced_pattern class TracedPipeline(Pipeline): pass # Or wrap individual agents agent = traced_agent(Agent("my_agent", my_llm)) ``` ### Manual Span Control ``` from pyagent_trace import PatternSpanEmitter emitter = PatternSpanEmitter() span = emitter.pattern_span("debate", {"rounds": 3}) try: result = await pattern.run(task) emitter.set_pattern_result(span, len(result.output), rounds=3, cost_estimate=0.012) finally: span.end() ``` ## Cost Tracking ``` from pyagent_trace import CostTracker tracker = CostTracker() tracker.record("debate", "bull_agent", "gpt-4o", input_tokens=500, output_tokens=200, cost_usd=0.003) tracker.record("debate", "bear_agent", "gpt-4o-mini", input_tokens=500, output_tokens=200, cost_usd=0.0004) tracker.record("debate", "judge", "gpt-4o", input_tokens=1000, output_tokens=300, cost_usd=0.0055) print(f"Total: ${tracker.total_cost:.4f}") print(f"By pattern: {tracker.by_pattern()}") print(f"By model: {tracker.by_model()}") ``` ## Record & Replay ``` from pyagent_trace.recorder import Recorder # Record recorder = Recorder() recorder.start("debate") recorder.record_llm_call("bull", messages, response_text) recorder.end(result.output) recorder.save("traces/debate_run_001.jsonl") # Replay / Debug entries = Recorder.load("traces/debate_run_001.jsonl") for entry in entries: print(f"[{entry.event_type}] {entry.agent_name}: {entry.response[:80]}...") ``` ## Trace Event Bus & Exporters `pyagent-trace` provides a portal-agnostic `TraceExporter` protocol and a pub/sub `TraceEventBus` for streaming trace events to multiple backends simultaneously. ### TraceEventBus ``` from pyagent_trace.events import TraceEvent, TraceEventBus bus = TraceEventBus() # Subscribe to all events bus.subscribe(lambda e: print(e.event_type)) # Subscribe to specific event types bus.subscribe_filter({"llm_call", "error"}, lambda e: handle_error(e)) ``` ### Built-in Exporters | Exporter | Backend | Install | | ------------------ | --------------------------------- | ------------------------------------------------- | | `ConsoleExporter` | stdout | built-in | | `JsonlExporter` | JSONL file | built-in | | `OTelExporter` | Jaeger, Tempo, Datadog, Honeycomb | `pip install opentelemetry-api opentelemetry-sdk` | | `LangfuseExporter` | Langfuse | `pip install langfuse` | ``` from pyagent_trace.events import TraceEventBus from pyagent_trace.exporters.console import ConsoleExporter from pyagent_trace.exporters.jsonl import JsonlExporter from pyagent_trace.exporters.langfuse import LangfuseExporter bus = TraceEventBus() # Fan-out to multiple backends simultaneously bus.subscribe(ConsoleExporter().export_event) bus.subscribe(JsonlExporter("traces/run.jsonl").export_event) bus.subscribe(LangfuseExporter().export_event) # uses LANGFUSE_* env vars ``` ### Custom Exporters Implement the `TraceExporter` protocol: ``` from pyagent_trace.exporters.base import TraceExporter from pyagent_trace.events import TraceEvent class MyExporter(TraceExporter): def export_event(self, event: TraceEvent) -> None: # Send to your backend ... def flush(self) -> None: ... def shutdown(self) -> None: ... ``` ### Wiring Producers `Recorder` and `CostTracker` accept an optional `event_bus` parameter: ``` from pyagent_trace.events import TraceEventBus from pyagent_trace.recorder import Recorder from pyagent_trace.cost import CostTracker bus = TraceEventBus() recorder = Recorder(event_bus=bus) tracker = CostTracker(event_bus=bus) ``` ## Hook-Based Tracing (Recommended) The simplest way to add tracing is via the built-in hooks on `Agent` and `Pattern`. No decorators or subclassing required. ### Agent Trace Hook ``` from pyagent_patterns.base import Agent, MockLLM from pyagent_trace.events import TraceEventBus from pyagent_trace.exporters.console import ConsoleExporter bus = TraceEventBus() bus.subscribe(ConsoleExporter().export_event) agent = Agent("analyst", MockLLM(responses=["Revenue grew 25%"])) agent.set_trace_bus(bus) # emits agent_start + agent_end result = await agent.run("Analyze revenue trends") # Console output: # [agent_start] analyst # [agent_end] analyst duration=0.002s tokens=12 ``` ### Pattern Trace Hook ``` from pyagent_patterns.orchestration import Pipeline pipeline = Pipeline(stages=[agent_a, agent_b]) pipeline.set_trace_bus(bus) # emits pattern_start + pattern_end result = await pipeline.run("Process document") # Console output: # [pattern_start] pipeline # [agent_start] agent_a # [agent_end] agent_a # [agent_start] agent_b # [agent_end] agent_b # [pattern_end] pipeline duration=0.005s ``` ### TracedProvider Wrap any `ProviderProtocol` to emit trace events on every LLM call: ``` from pyagent_providers import TracedProvider traced = TracedProvider(original_provider, event_bus=bus) agent = Agent("analyst", llm=traced) # Emits: provider_call_start, provider_call_end (or provider_call_error) ``` ### Wire All at Once via RuntimeGraph ``` from pyagent_blueprint import load_blueprint, BlueprintCompiler graph = BlueprintCompiler().compile(load_blueprint("blueprint.yaml")) graph.wire_trace(bus) # sets trace_bus on ALL patterns + agents ``` → See the full [Hooks Guide](https://pyagent.org/guides/hooks/index.md) for all four hook types. ## Custom Attributes All attributes are namespaced under `pyagent.*`: | Attribute | Type | Description | | ------------------------------- | ------ | ----------------------------- | | `pyagent.pattern.type` | string | Pattern name (e.g., "debate") | | `pyagent.pattern.rounds` | int | Number of rounds executed | | `pyagent.agent.name` | string | Agent name | | `pyagent.router.difficulty` | int | Task difficulty 1-10 | | `pyagent.router.selected_model` | string | Routed model name | | `pyagent.compress.savings_pct` | float | Compression savings 0-1 | | `pyagent.cost.total_usd` | float | Total cost in USD | | `pyagent.exec.duration_ms` | float | Execution time in ms | # Studio Guide `pyagent-studio` is a web dashboard and CLI for designing, simulating, debugging, and governing multi-agent blueprints. Run it locally during development or deploy it as an internal tool for your team. ``` pip install pyagent-studio pyagent dashboard # start the web UI on http://localhost:8000 pyagent apply blueprint.yaml # load a blueprint via CLI ``` ______________________________________________________________________ ## Dashboard The dashboard is a FastAPI web app with nine sections accessible from the sidebar. ``` flowchart LR subgraph Sidebar OV[Overview] AG[Agents] WF[Workflows] SM[Simulate] TR[Traces] GV[Governance] PR[Providers] DF[Diff] DC[Docs] end OV --> |blueprint summary| AG AG --> |agent detail| WF WF --> |workflow DAG| SM SM --> |run & stream| TR TR --> |live SSE events| GV ``` | Section | What it shows | | -------------- | -------------------------------------------------------------------------- | | **Overview** | Blueprint name, version, agent/workflow/provider counts, validation status | | **Agents** | All agents, their models, system prompts, and hook wiring | | **Workflows** | Workflow list and per-workflow step-by-step detail | | **Simulate** | Run any workflow with MockLLM or live LLMs, stream output | | **Traces** | Live SSE trace events + historical JSONL viewer | | **Governance** | Compliance score, validation issues, contract conformance | | **Providers** | Registered providers, model lists, health check results | | **Diff** | Semantic diff between two blueprint versions | | **Docs** | Auto-rendered blueprint documentation with Mermaid DAGs | ______________________________________________________________________ ## Starting the Dashboard ``` # Default: port 8000, localhost only pyagent dashboard # Custom port pyagent dashboard --port 9000 # Bind to all interfaces (for team access) pyagent dashboard --host 0.0.0.0 --port 8000 ``` Once running, open `http://localhost:8000` in your browser. ______________________________________________________________________ ## CLI Reference `pyagent-studio` ships a full CLI for all blueprint operations. You can run everything headlessly in scripts and CI. ### apply Load and validate a blueprint, then print a summary. ``` pyagent apply blueprint.yaml # customer-support v1.0 — 3 agents, 2 workflows, 4 contracts ``` ### get List resources from a blueprint. ``` pyagent get agents blueprint.yaml pyagent get workflows blueprint.yaml pyagent get providers blueprint.yaml pyagent get contracts blueprint.yaml ``` ### validate Check a blueprint against its schema and contracts. ``` pyagent validate blueprint.yaml # ✓ Schema valid # ✓ All contracts pass # ⚠ observability.exporters: no exporters configured ``` ### test Run full contract conformance checks. ``` pyagent test blueprint.yaml # Testing customer-support... # ✓ classifier: input/output schema # ✓ billing_bot: response_time < 2s # ✗ escalation: missing required metadata field ``` ### simulate Run a workflow with MockLLM (no API cost) or live LLMs. ``` # MockLLM (default — free, deterministic) pyagent simulate blueprint.yaml support "I can't login" # Live LLMs (uses real API keys) pyagent simulate blueprint.yaml support "I can't login" --live ``` ### render Render a blueprint as Markdown documentation or a Mermaid DAG. ``` pyagent render blueprint.yaml # Markdown pyagent render blueprint.yaml --format mermaid # Mermaid diagram ``` ### diff Show semantic differences between two blueprints. ``` pyagent diff blueprint-v1.yaml blueprint-v2.yaml # + agents.escalation_bot (new) # ~ agents.classifier.model: gpt-4o-mini → gpt-4o # - providers.backup (removed) ``` ### generate Scaffold a new blueprint YAML from a pattern template. ``` pyagent generate --pattern pipeline --agents extractor,analyst,writer pyagent generate --pattern fan-out --agents bull,bear,macro --aggregator synthesis ``` ### describe Print a plain-English summary of a blueprint. ``` pyagent describe blueprint.yaml # name: customer-support # version: 1.0.0 # agents: 3 (classifier, billing_bot, escalation) # workflows: 2 (support, billing) # providers: openai (primary), anthropic (fallback) ``` ### providers Inspect and health-check LLM providers. ``` pyagent providers list # list available models pyagent providers health # ping each model with a test call ``` ______________________________________________________________________ ## Trace Streaming The Traces page streams live events from the `TraceEventBus` via SSE. Wire your agents to the dashboard's bus so every call appears in the UI in real-time. ``` from pyagent_studio.web.routes.traces import get_trace_bus from pyagent_patterns.base import Agent from pyagent_providers import AnthropicLLM # Get the dashboard's global trace bus bus = get_trace_bus() agent = ( Agent("analyst", AnthropicLLM("claude-sonnet-4-20250514")) .set_trace_bus(bus) ) # Now every agent call emits events to the Traces page ``` The SSE endpoint is at `/traces/live` — any client can subscribe: ``` const es = new EventSource("/traces/live"); es.onmessage = (e) => { const event = JSON.parse(e.data); console.log(event.event_type, event.agent_name, event.elapsed_ms); }; ``` ______________________________________________________________________ ## Service Layer Every dashboard feature is backed by a headless service you can use directly from Python or CI scripts. ### BlueprintService ``` from pyagent_studio.services.blueprint_service import BlueprintService svc = BlueprintService() spec = svc.load("blueprint.yaml") issues = svc.validate(spec) compiled = svc.compile(spec) # resolves agent references markdown = svc.render_markdown(spec) mermaid = svc.render_mermaid(spec) ``` ### SimulationService ``` from pyagent_studio.services.simulation_service import SimulationService from pyagent_trace.events import TraceEventBus bus = TraceEventBus() sim = SimulationService(event_bus=bus, use_live=False) result = await sim.run(spec, workflow="support", message="I need help") print(result.output) print(f"Elapsed: {result.elapsed_ms:.0f}ms") print(f"Success: {result.success}") ``` ### GovernanceService ``` from pyagent_studio.services.governance_service import GovernanceService gov = GovernanceService() report = gov.check_compliance(spec) print(f"Score: {report['score']}%") for issue in report['issues']: print(f" {issue['severity']}: {issue['message']}") ``` ### ProviderService ``` from pyagent_studio.services.provider_service import ProviderService provider = ProviderService(default_model="gpt-4o") models = provider.list_models() health = await provider.health_check("gpt-4o") cost = provider.model_cost("gpt-4o") response = await provider.complete( model="gpt-4o", messages=[{"role": "user", "content": "Hello"}], ) ``` ### TraceService ``` from pyagent_studio.services.trace_service import TraceService traces = TraceService() spans = traces.load("traces.jsonl") llm_calls = traces.query(event_type="llm_call") print(traces.summary()) ``` ______________________________________________________________________ ## Embedding in FastAPI The dashboard is a standard FastAPI app. Embed it into an existing app or add custom routes. ``` from fastapi import FastAPI from pyagent_studio.web.app import create_app # Standalone app = create_app() # Mount into existing app main_app = FastAPI() studio = create_app() main_app.mount("/studio", studio) ``` ______________________________________________________________________ ## See Also - [Blueprint Package](https://pyagent.org/packages/blueprint/index.md) — spec format and compilation - [Trace Package](https://pyagent.org/packages/trace/index.md) — `TraceEventBus` and exporters - [Blueprint Guide](https://pyagent.org/guides/blueprint/index.md) — writing and validating blueprints - [Tracing Guide](https://pyagent.org/guides/tracing/index.md) — wiring exporters and reading spans - [API Reference](https://pyagent.org/api/studio/index.md) # pyagent-trace **Pattern-aware OpenTelemetry instrumentation** — every agent call, pattern execution, and LLM invocation emits structured spans. Feeds cost tracking, record/replay debugging, and the Studio dashboard. ``` pip install pyagent-trace pip install pyagent-trace[langfuse] # + Langfuse exporter pip install pyagent-trace[datadog] # + Datadog APM exporter ``` ______________________________________________________________________ ## Architecture ``` flowchart LR P[Pattern.run] --> SE[PatternSpanEmitter] SE --> OT[OpenTelemetry SDK] OT --> J[Jaeger / Tempo] OT --> LF[Langfuse] OT --> DD[Datadog APM] OT --> OTLP[Any OTLP endpoint] P --> CT[CostTracker] P --> REC[Recorder → .jsonl] REC --> S[pyagent-studio\nTrace Viewer] subgraph Span Attributes PT["pyagent.pattern.type"] AN["pyagent.agent.name"] CU["pyagent.cost.total_usd"] DM["pyagent.exec.duration_ms"] RD["pyagent.router.difficulty"] CP["pyagent.compress.savings_pct"] end ``` ______________________________________________________________________ ## TraceEventBus — Pub/Sub Event System `TraceEventBus` is the lightweight event contract between trace producers (agents, patterns, providers) and consumers (exporters, Studio, custom callbacks). Producers emit `TraceEvent` objects; consumers subscribe to receive them. ``` import time from pyagent_trace.events import TraceEvent, TraceEventBus bus = TraceEventBus() # Subscribe a callback — receives every event sub_id = bus.subscribe(lambda e: print(f"[{e.event_type}] {e.agent_name}")) # Subscribe to specific event types only bus.subscribe_filter( {"agent_start", "agent_end", "llm_call"}, lambda e: print(f"Agent event: {e.payload}"), ) # Emit an event bus.emit(TraceEvent( timestamp=time.time(), event_type="agent_start", agent_name="analyst", pattern_type="pipeline", payload={"model": "claude-sonnet-4-20250514"}, )) bus.unsubscribe(sub_id) ``` ### Built-in Exporters Wire exporters to a bus — one bus can have multiple exporters simultaneously: ``` from pyagent_trace.events import TraceEventBus from pyagent_trace.exporters import ConsoleExporter, JsonlExporter from pyagent_trace.exporters.langfuse import LangfuseExporter bus = TraceEventBus() # Development: print to stdout bus.subscribe(ConsoleExporter().export_event) # Persistence: write every event to JSONL (viewable in Studio) bus.subscribe(JsonlExporter("traces/run_001.jsonl").export_event) # Production: send to Langfuse for LLM observability bus.subscribe( LangfuseExporter( public_key="pk-lf-...", secret_key="sk-lf-...", host="https://cloud.langfuse.com", # or self-hosted ).export_event ) ``` ### Agent Hook Integration Attach a bus to any agent — it emits `agent_start`, `agent_end`, `compression`, and `cost_record` events automatically: ``` from pyagent_patterns.base import Agent from pyagent_providers import AnthropicLLM from pyagent_trace import CostTracker bus = TraceEventBus() bus.subscribe(ConsoleExporter().export_event) tracker = CostTracker(event_bus=bus) agent = ( Agent("analyst", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Analyze the document.") .set_trace_bus(bus) # emits agent_start / agent_end .set_cost_tracker(tracker) # emits cost_record after each LLM call ) result = asyncio.run(agent.run("Summarize Q3 earnings")) # Console output: # [agent_start] agent=analyst # [cost_record] agent=analyst tokens=... cost=... # [agent_end] agent=analyst duration_ms=... ``` → See the full [Hooks Guide](https://pyagent.org/guides/hooks/index.md) for all four hook types (trace, context, compress, cost). ______________________________________________________________________ ## Quick Start — Decorator Tracing The simplest integration: inherit from a traced variant of any pattern class. ``` from pyagent_trace import traced_pattern, traced_agent from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Pipeline from pyagent_providers import AnthropicLLM, OpenAILLM # Every pipeline.run() automatically emits OTel spans @traced_pattern class TracedPipeline(Pipeline): pass pipeline = TracedPipeline(stages=[ traced_agent(Agent("extractor", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Extract key facts.")), traced_agent(Agent("writer", OpenAILLM("gpt-4o-mini"), system_prompt="Write a concise brief.")), ]) import asyncio result = asyncio.run(pipeline.run("Tesla Q3 2025 earnings report...")) # Spans emitted to configured OTel backend automatically ``` ______________________________________________________________________ ## OTel Backend Setup ### Jaeger (local development) ``` from opentelemetry import trace from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor from opentelemetry.exporter.jaeger.thrift import JaegerExporter provider = TracerProvider() provider.add_span_processor( BatchSpanProcessor(JaegerExporter(agent_host_name="localhost", agent_port=6831)) ) trace.set_tracer_provider(provider) # Now all traced_pattern / traced_agent calls export to Jaeger # Visit http://localhost:16686 to explore traces ``` ``` # Start Jaeger with Docker docker run -d --name jaeger \ -p 6831:6831/udp -p 16686:16686 \ jaegertracing/all-in-one:latest ``` ### Langfuse (production LLM observability) ``` pip install pyagent-trace[langfuse] ``` ``` from pyagent_trace.events import TraceEventBus from pyagent_trace.exporters.langfuse import LangfuseExporter bus = TraceEventBus() bus.subscribe( LangfuseExporter( public_key="pk-lf-...", secret_key="sk-lf-...", host="https://cloud.langfuse.com", # or self-hosted ).export_event ) # Wire bus to agents or patterns, then run # Every agent_start/end and llm_call event flows to Langfuse as a span/generation result = asyncio.run(pipeline.run("Analyze this document")) # Visit Langfuse dashboard to see full trace tree with token costs ``` ### Grafana Tempo + Prometheus (infrastructure teams) ``` from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter from opentelemetry.sdk.trace.export import BatchSpanProcessor provider.add_span_processor( BatchSpanProcessor(OTLPSpanExporter(endpoint="http://tempo:4317")) ) ``` ______________________________________________________________________ ## PatternSpanEmitter — Manual Control For full span control when decorators aren't enough. ``` import asyncio from pyagent_trace import PatternSpanEmitter from pyagent_patterns.resolution import Debate from pyagent_providers import GeminiLLM, AnthropicLLM from pyagent_patterns.base import Agent debate = Debate( debaters=[ Agent("bull", GeminiLLM("gemini-2.5-flash"), system_prompt="Argue the bull case."), Agent("bear", GeminiLLM("gemini-2.5-flash"), system_prompt="Argue the bear case."), ], judge=Agent("judge", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Render a final verdict."), rounds=2, ) emitter = PatternSpanEmitter(service_name="investment-analysis") async def run_traced(): span = emitter.pattern_span("debate", attributes={"rounds": 2, "topic": "nvidia_investment"}) try: result = await debate.run("Should we buy Nvidia at $3.2T market cap?") emitter.set_pattern_result( span, output_len=len(result.output), rounds=result.metadata.get("rounds", 2), cost_estimate=result.cost_estimate, ) return result except Exception as exc: emitter.record_error(span, exc) raise finally: span.end() result = asyncio.run(run_traced()) ``` ______________________________________________________________________ ## CostTracker Aggregate costs across an entire session — by pattern, by model, by agent. ``` from pyagent_trace import CostTracker tracker = CostTracker() # Record after each agent call tracker.record( pattern="debate", agent="bull", model="gemini-2.5-flash", input_tokens=450, output_tokens=380, cost_usd=0.00094, ) tracker.record("debate", "bear", "gemini-2.5-flash", 450, 410, 0.00098) tracker.record("debate", "judge", "claude-sonnet-4-20250514", 1200, 480, 0.00840) # Summaries print(f"Total session cost: ${tracker.total_cost:.4f}") print(f"By pattern: {tracker.by_pattern()}") # → {"debate": 0.01032} print(f"By model: {tracker.by_model()}") # → {"gemini-2.5-flash": 0.00192, "claude-sonnet-4-20250514": 0.00840} print(f"By agent: {tracker.by_agent()}") # → {"bull": 0.00094, "bear": 0.00098, "judge": 0.00840} # Daily cost projection print(f"If 500 runs/day: ${tracker.total_cost * 500:.2f}/day") ``` ______________________________________________________________________ ## Recorder — Record & Replay Record every LLM call to a JSONL file. Replay for deterministic debugging, testing, and cost-free re-runs. ``` from pyagent_trace.recorder import Recorder # --- Recording --- recorder = Recorder() recorder.start(pattern_name="pipeline") # Record each agent interaction manually, or wire into traced_agent automatically recorder.record_llm_call( agent_name="extractor", messages=[{"role": "user", "content": "Tesla Q3 2025 earnings..."}], response="Revenue: $25.2B (+8% YoY), gross margin: 17.1%", model="claude-haiku-3-5-20241022", input_tokens=180, output_tokens=42, cost_usd=0.00006, ) recorder.record_llm_call( agent_name="writer", messages=[{"role": "user", "content": "Revenue: $25.2B..."}], response="Tesla Q3: Revenue beat consensus by 2%...", model="gpt-4o-mini", input_tokens=60, output_tokens=85, cost_usd=0.00009, ) recorder.end(final_output="Tesla Q3: Revenue beat consensus by 2%...") recorder.save("traces/pipeline_run_001.jsonl") print(f"Recorded {len(recorder.entries)} events") print(f"Total cost: ${recorder.total_cost:.5f}") ``` ``` # --- Replay / Debug --- entries = Recorder.load("traces/pipeline_run_001.jsonl") for entry in entries: print(f"[{entry.event_type}] {entry.agent_name}: {entry.response[:80]}...") print(f" tokens: {entry.input_tokens}+{entry.output_tokens}, cost: ${entry.cost_usd:.5f}") # Replay without hitting LLM APIs (great for CI and testing) from pyagent_trace.recorder import ReplayLLM replay_llm = ReplayLLM.from_recording("traces/pipeline_run_001.jsonl", agent_name="extractor") # replay_llm returns pre-recorded responses — zero API cost ``` ______________________________________________________________________ ## Integration with pyagent-studio Recorded `.jsonl` files are loaded directly into Studio's trace viewer: ``` # Load a trace file in Studio pyagent dashboard --trace traces/pipeline_run_001.jsonl ``` Or use the TraceService programmatically (same API Studio uses internally): ``` from pyagent_studio.services.trace_service import TraceService svc = TraceService() spans = svc.load("traces/pipeline_run_001.jsonl") print(f"Spans: {len(spans)}") for span in spans: print(f" [{span.event_type}] {span.agent_name}: {span.duration_ms:.0f}ms, {span.tokens} tokens") # Query specific spans llm_calls = svc.query(event_type="llm_call") slow_spans = svc.query(min_duration_ms=2000) ``` ______________________________________________________________________ ## OTel Span Attributes Reference All attributes use the `pyagent.*` namespace: | Attribute | Type | Description | | ------------------------------- | ------ | ------------------------------- | | `pyagent.pattern.type` | string | Pattern name (e.g., `"debate"`) | | `pyagent.pattern.rounds` | int | Rounds executed | | `pyagent.agent.name` | string | Agent identifier | | `pyagent.agent.model` | string | LLM model used | | `pyagent.cost.input_tokens` | int | Input token count | | `pyagent.cost.output_tokens` | int | Output token count | | `pyagent.cost.total_usd` | float | Estimated cost in USD | | `pyagent.exec.duration_ms` | float | Wall-clock time (ms) | | `pyagent.router.difficulty` | int | Task difficulty score 1–10 | | `pyagent.router.selected_model` | string | Model chosen by router | | `pyagent.compress.savings_pct` | float | Compression savings ratio | | `pyagent.early_stop` | bool | Whether pattern stopped early | ______________________________________________________________________ ## Trace Output Example (Jaeger / Tempo) ``` Trace: investment-analysis / debate (9.8s total, $0.01032) ├── pyagent.agent.bull 2.4s gemini-2.5-flash 450→380 tok $0.00094 ├── pyagent.agent.bear 2.7s gemini-2.5-flash 450→410 tok $0.00098 ├── pyagent.agent.bull 2.1s (round 2) 410→360 tok $0.00086 ├── pyagent.agent.bear 1.9s (round 2) 410→380 tok $0.00084 └── pyagent.agent.judge 0.7s claude-sonnet-4 1200→480 tok $0.00840 ───────────────────────────────────────── Total: $0.01032, early_stop: false ``` ______________________________________________________________________ ## See Also - [Studio Package](https://pyagent.org/packages/studio/index.md) — visual trace explorer, cost dashboards - [Tracing Guide](https://pyagent.org/guides/tracing/index.md) — production setup for Langfuse, Datadog, Grafana - [Compress Package](https://pyagent.org/packages/compress/index.md) — `pyagent.compress.savings_pct` OTel attribute - [API Reference](https://pyagent.org/api/trace/index.md) ______________________________________________________________________ ## Cookbook examples Complete, runnable recipes that use this package — [browse the Cookbook](https://pyagent.org/cookbook/index.md): - [Multi-agent incident triage pipeline](https://pyagent.org/cookbook/devops-sre/incident-triage/index.md) [Browse the full Cookbook →](https://pyagent.org/cookbook/index.md) # pyagent-studio **The Kubernetes Dashboard for Agent Systems** — a `kubectl`-inspired CLI and web control plane for designing, simulating, debugging, and governing multi-agent blueprints. Studio is to pyagent what `kubectl` + the Kubernetes Dashboard are to Kubernetes: a single pane of glass for your entire agent infrastructure. ``` pip install pyagent-studio ``` ______________________________________________________________________ ## What Studio Gives You | Capability | Where | | ------------------------------------ | --------------------------------------------- | | Load and validate blueprints | `pyagent apply` + Studio `/governance` | | Inspect agents, providers, workflows | `pyagent get` + Studio `/agents` `/workflows` | | Simulate workflows without API costs | `pyagent simulate` + Studio `/simulate` | | Diff blueprint versions | `pyagent diff` + Studio `/diff` | | Explore recorded traces | `pyagent dashboard` + Studio `/traces` | | Monitor provider health and cost | `pyagent providers` + Studio `/providers` | ______________________________________________________________________ ## Architecture ``` flowchart TD BP[blueprint.yaml] --> CLI[pyagent CLI\nkubectl-style commands] CLI --> BS[BlueprintService] CLI --> GS[GovernanceService] CLI --> SIM[SimulationService] CLI --> PS[ProviderService] BP --> WEB[Web Dashboard\nlocalhost:8000] WEB --> OV[/overview] WEB --> WF[/workflows] WEB --> AGT[/agents] WEB --> TR[/traces] WEB --> SIL[/simulate] WEB --> GOV[/governance] WEB --> PRV[/providers] WEB --> DIFF[/diff] WEB --> DOCS[/docs] TR --> TS[TraceService\nload .jsonl files] ``` ______________________________________________________________________ ## Quick Start ``` # 1. Load a blueprint pyagent apply customer-support.yaml # 2. Validate it pyagent validate customer-support.yaml # 3. Simulate a workflow (no API costs) pyagent simulate customer-support.yaml main "I was charged twice" # 4. Launch the web dashboard pyagent dashboard # → http://localhost:8000 ``` ______________________________________________________________________ ## Sub-pages - [CLI](https://pyagent.org/packages/studio/cli/index.md) — all `pyagent` commands - [Dashboard](https://pyagent.org/packages/studio/dashboard/index.md) — the 9 web UI sections - [Services](https://pyagent.org/packages/studio/services/index.md) — Python services API - [Traces](https://pyagent.org/packages/studio/traces/index.md) — loading, querying, and viewing trace files ______________________________________________________________________ ## See Also - [Blueprint Package](https://pyagent.org/packages/blueprint/index.md) — the YAML spec Studio loads and compiles - [Blueprint Guide](https://pyagent.org/guides/blueprint/index.md) — narrative walkthrough - [Studio Guide](https://pyagent.org/guides/studio/index.md) — guide-level walkthrough - [API Reference](https://pyagent.org/api/studio/index.md) # Studio — CLI Installing `pyagent-studio` adds the `pyagent` CLI — a `kubectl`-inspired interface for agent systems. ``` pip install pyagent-studio pyagent --help ``` ______________________________________________________________________ ## apply — load and validate Load a blueprint and print a summary. ``` pyagent apply customer-support.yaml # ✓ Blueprint 'customer-support-system' v1.2.0 loaded and valid. ``` With issues: ``` Loaded 'customer-support-system' with 2 issue(s): [error] workflows.main.agents.routes.billing: Agent ref 'billing_agnt' not found. [warning] agents.legacy_agent: Agent defined but not referenced in any workflow. ``` Exit code `0` = valid, `1` = errors. ______________________________________________________________________ ## get — list resources ``` pyagent get agents customer-support.yaml # classifier Route to billing, technical, returns, or general # billing_agent Handle billing disputes and subscription issues # technical_agent Handle technical and API issues # returns_agent Handle returns and exchanges # formatter Polish and format the final response pyagent get workflows customer-support.yaml # main (supervisor) pyagent get providers customer-support.yaml # fast anthropic / claude-haiku-3-5-20241022 max_tokens: 512 # balanced openai / gpt-4o-mini max_tokens: 1024 # expert anthropic / claude-sonnet-4-20250514 max_tokens: 2048 ``` ______________________________________________________________________ ## validate — static analysis ``` pyagent validate customer-support.yaml # Valid # Exit code 1 on errors — useful in CI ``` CI integration: ``` # .github/workflows/blueprint-check.yml - run: pyagent validate blueprints/production.yaml ``` ______________________________________________________________________ ## test — contract conformance Run contract checks using MockLLM — no API costs. ``` pyagent test customer-support.yaml # Testing 'customer-support-system'... # All contract checks passed. ``` With failures: ``` Results: 1 error(s), 2 warning(s) [error] contracts.response_length: Output exceeded max_length 400 (got 612) [warning] contracts.no_pii: PII pattern check skipped (no output generated) ``` ______________________________________________________________________ ## simulate — dry run Run a workflow against a task string. Uses MockLLM by default — no API costs. ``` # Mock simulation (default — free) pyagent simulate customer-support.yaml main "I was charged twice this month" # Running workflow 'main' in customer-support-system... # [MockLLM] classifier → billing # [MockLLM] billing_agent → "I understand your frustration..." # [MockLLM] formatter → "Thank you for reaching out..." # ✓ Completed in 0.012s (mock) # Live run with real providers pyagent simulate customer-support.yaml main "I need a refund" --live # [claude-haiku] classifier → billing # [claude-sonnet] billing_agent → "I'm sorry to hear about..." # ✓ Completed in 1.8s, cost: $0.0042 ``` ______________________________________________________________________ ## diff — semantic comparison ``` pyagent diff customer-support-v1.yaml customer-support-v2.yaml # + providers.premium: {provider: anthropic, model: claude-opus-4-20250514} # ~ agents.billing_agent.provider: expert → premium [WARNING: model upgrade] # ~ workflows.main.config.default_route: general → billing [WARNING: routing change] # - agents.legacy_agent [INFO: removed] ``` Severity levels: `BREAKING`, `WARNING`, `INFO`. ______________________________________________________________________ ## describe — full summary ``` pyagent describe customer-support.yaml # Blueprint: customer-support-system v1.2.0 # Owner: platform-team Tags: [support, production] # # Providers (3): # fast anthropic / claude-haiku-3-5-20241022 # balanced openai / gpt-4o-mini # expert anthropic / claude-sonnet-4-20250514 # # Agents (5): classifier, billing_agent, technical_agent, returns_agent, formatter # Workflows (1): main (supervisor) # Contracts (2): response_length, no_pii_in_response ``` ______________________________________________________________________ ## render — generate diagrams ``` # Mermaid diagram (default) pyagent render customer-support.yaml # Save to file pyagent render customer-support.yaml -o docs/architecture.md # Full Markdown documentation pyagent render customer-support.yaml --format markdown -o docs/system.md ``` ______________________________________________________________________ ## providers — health monitoring ``` pyagent providers list # anthropic claude-haiku-3-5-20241022 ✓ healthy p50: 340ms # anthropic claude-sonnet-4-20250514 ✓ healthy p50: 1100ms # openai gpt-4o-mini ✓ healthy p50: 480ms pyagent providers health # All 3 providers healthy. Last checked: 2025-06-09T14:32:01Z ``` ______________________________________________________________________ ## dashboard — launch web UI ``` # Default port (8000) pyagent dashboard # Custom port pyagent dashboard --port 3000 # Load a specific blueprint on startup pyagent dashboard --blueprint customer-support.yaml # Load recorded traces on startup pyagent dashboard --trace traces/production_runs.jsonl # Web UI available at http://localhost:8000 ``` ______________________________________________________________________ ## Command Reference ``` pyagent apply Load and validate pyagent get agents|workflows|providers pyagent validate Static analysis pyagent test Contract conformance pyagent simulate [--live] pyagent diff Semantic diff pyagent describe Full summary pyagent render [--format mermaid|markdown] [-o ] pyagent providers list|health pyagent dashboard [--port N] [--blueprint ] [--trace ] ``` ______________________________________________________________________ ## See Also - [Dashboard](https://pyagent.org/packages/studio/dashboard/index.md) — the web UI - [Blueprint CLI](https://pyagent.org/packages/blueprint/cli/index.md) — `blueprint` command for spec-focused operations # Studio — Web Dashboard ``` pyagent dashboard # Web UI available at http://localhost:8000 ``` The web dashboard exposes every Studio capability through a browser — designed for team members who don't live in a terminal, and for visual inspection of complex multi-agent systems. ______________________________________________________________________ ## /overview Blueprint summary and quick-action panel. ``` Blueprint: customer-support-system v1.2.0 Owner: platform-team | Tags: support, production Resources Agents: 5 Workflows: 1 Providers: 3 Contracts: 2 Compliance Score: 83% (5/6 checks passed) [Validate] [Simulate] [View Traces] [View Diff] ``` The compliance score comes from `GovernanceService` — see [Services](https://pyagent.org/packages/studio/services/#governanceservice). Click **Validate** to rerun checks live. Click **Simulate** to jump to `/simulate` with the current blueprint pre-loaded. ______________________________________________________________________ ## /agents Agent roster with full prompt viewer. ``` Agents (5) classifier fast / claude-haiku-3-5-20241022 Route to billing, technical, returns, or general System prompt: Classify the customer message into exactly one of: billing, technical, returns, general. Respond with ONLY the category name. billing_agent expert / claude-sonnet-4-20250514 Handle billing disputes and subscription issues System prompt: You are a billing specialist. Handle disputes, refunds, and subscription questions. Always acknowledge frustration first. ... Tools: lookup_invoice, issue_refund ``` Each agent card shows: - Name, description, provider, model - Full system prompt (expandable) - Tools and guardrails attached ______________________________________________________________________ ## /workflows Pattern diagrams with interactive agent inspection. ``` Workflows (1) main [supervisor] ── [Run Simulation] flowchart TD IN[Input] --> classifier classifier -->|billing| billing_agent classifier -->|technical| technical_agent ... formatter --> OUT[Output] ``` - Rendered Mermaid diagram for each workflow - Click any agent node to see its prompt and provider inline - Pattern type badge: `supervisor`, `pipeline`, `fan_out_fan_in`, etc. - **Run Simulation** button launches `/simulate` with this workflow pre-selected ______________________________________________________________________ ## /simulate Interactive workflow runner. ``` Workflow: [main ▼] Mode: [◉ Mock (free)] [○ Live (real API)] Task: [I was charged twice this month ] [Run] ───────────────────────────────────────────────────── Step 1 classifier 0.012s MockLLM Output: billing Step 2 billing_agent 0.024s MockLLM Output: "I understand your frustration. Let me look into the double charge on your account..." Step 3 formatter 0.008s MockLLM Output: "Thank you for reaching out. I understand your frustration regarding the duplicate charge..." ───────────────────────────────────────────────────── Final: "Thank you for reaching out. I understand..." Duration: 0.044s | Cost: $0.00 (mock) | Route: billing ``` Switch between Mock and Live mode. Live mode uses real providers and shows actual latency and cost. ______________________________________________________________________ ## /traces Trace explorer for recorded `.jsonl` files. See the dedicated [Traces page](https://pyagent.org/packages/studio/traces/index.md) for full detail. ``` Loaded: traces/production_runs.jsonl 1,240 spans | 180 runs | last run: 2025-06-09T14:32 Run #174 supervisor 3 steps 3.4s $0.0092 ├── classifier 0.3s claude-haiku 12→1 tok $0.00001 ├── billing_agent 2.4s claude-sonnet 180→140 tok $0.00840 └── formatter 0.7s claude-haiku 145→80 tok $0.00051 Cost by model (last 7 days): claude-sonnet-4-20250514 ████████████████ $24.80 (72%) gpt-4o-mini ████ $6.20 (18%) claude-haiku-3-5-20241022 ██ $3.40 (10%) ``` ______________________________________________________________________ ## /governance Compliance scores and diff view. ``` Blueprint: customer-support-system v1.2.0 Compliance Score: 83% (5/6 checks passed) ✓ Agent refs All workflow agent references resolve ✓ Provider refs All agent provider references resolve ✓ Pattern names All patterns are registered ✓ Contract refs All contract references resolve ✗ SLA values timeout_seconds: 30s may be too low for expert provider (claude-sonnet p99 observed at 2.4s) ✓ Security No hardcoded API keys detected Diff vs previous version: ~ agents.billing_agent.provider: expert → premium (+$0.008/call) - agents.legacy_agent (removed) ``` ______________________________________________________________________ ## /providers Real-time provider health and budget dashboard. ``` Provider Health anthropic / claude-haiku-3-5-20241022 Status: ✓ HEALTHY Avg latency: 340ms p99: 820ms Errors (1h): 0 Budget used: $1.24 / $50.00 ██░░░░░░░░ 2.5% anthropic / claude-sonnet-4-20250514 Status: ✓ HEALTHY Avg latency: 1,100ms p99: 2,400ms Errors (1h): 2 (rate limit) Budget used: $18.60 / $50.00 ██████░░░ 37% openai / gpt-4o-mini Status: ✓ HEALTHY Avg latency: 480ms p99: 1,100ms Errors (1h): 0 Budget used: $4.30 / $50.00 █░░░░░░░░░ 9% ``` Budget data comes from the `cost_budget.daily_usd` field in the blueprint's `observability:` block. ______________________________________________________________________ ## /diff Visual semantic diff between two blueprint versions. Upload two YAML files or paste YAML directly into the editor panels. Changes are categorized: - **BREAKING** — pattern changed, required field removed - **WARNING** — prompt changed, provider swapped - **INFO** — metadata, descriptions, tags ______________________________________________________________________ ## /docs Auto-generated documentation for the loaded blueprint — produced by `BlueprintRenderer.to_markdown()`. Exportable as a Markdown file. ______________________________________________________________________ ## Launching with a Blueprint Pre-Loaded ``` # Load blueprint and traces on startup pyagent dashboard \ --blueprint customer-support.yaml \ --trace traces/production_runs.jsonl \ --port 8000 ``` ______________________________________________________________________ ## See Also - [CLI](https://pyagent.org/packages/studio/cli/index.md) — terminal-first access to the same operations - [Services](https://pyagent.org/packages/studio/services/index.md) — the Python layer powering the dashboard - [Traces](https://pyagent.org/packages/studio/traces/index.md) — trace file format and exploration # Studio — Python Services Studio's web dashboard and CLI both call into a set of Python service classes. You can use these directly in your own code — the web layer is just a thin HTTP wrapper over them. ``` from pyagent_studio.services.blueprint_service import BlueprintService from pyagent_studio.services.governance_service import GovernanceService from pyagent_studio.services.simulation_service import SimulationService from pyagent_studio.services.trace_service import TraceService from pyagent_studio.services.provider_service import ProviderService ``` ______________________________________________________________________ ## BlueprintService Load, validate, compile, and inspect blueprints. ``` from pyagent_studio.services.blueprint_service import BlueprintService svc = BlueprintService() # Load from file spec = svc.load("customer-support.yaml") # Validate issues = svc.validate() print(f"Valid: {len(issues) == 0}") for issue in issues: print(f" [{issue.severity}] {issue.path}: {issue.message}") # Compile to RuntimeGraph graph = svc.compile() # Summary dict print(svc.summary()) # { # "loaded": true, # "name": "customer-support-system", # "version": "1.2.0", # "agents": 5, # "workflows": 1, # "providers": 3, # "contracts": 2 # } # Discover all blueprints in a directory tree paths = svc.discover_blueprints("./blueprints") print(paths) # ["./blueprints/production.yaml", "./blueprints/staging.yaml"] ``` ______________________________________________________________________ ## GovernanceService Compliance scoring and semantic diff. ``` from pyagent_studio.services.governance_service import GovernanceService from pyagent_blueprint import load_blueprint gov = GovernanceService() spec = load_blueprint("customer-support.yaml") # Compliance report report = gov.check_compliance(spec) print(f"Score: {report.score:.0%} ({report.passed}/{report.total_checks} checks)") for issue in report.issues: print(f" [{issue.severity}] {issue.path}: {issue.message}") # Semantic diff old_spec = load_blueprint("customer-support-v1.yaml") new_spec = load_blueprint("customer-support-v2.yaml") changes = gov.diff(old_spec, new_spec) for c in changes: print(f" [{c.severity}] {c.kind} {c.path}") # Human-readable diff summary print(gov.diff_summary(old_spec, new_spec)) # + providers.premium: {provider: anthropic, model: claude-opus-4-20250514} # ~ agents.billing_agent.provider: expert → premium # - agents.legacy_agent ``` ### ComplianceReport fields | Field | Type | Description | | -------------- | ----- | ------------------------------------- | | `score` | float | 0.0–1.0, fraction of checks passed | | `passed` | int | Number of passing checks | | `total_checks` | int | Total checks run | | `issues` | list | Failing checks with severity and path | ______________________________________________________________________ ## SimulationService Dry-run workflows with MockLLM — no API costs. ``` import asyncio from pyagent_studio.services.simulation_service import SimulationService from pyagent_blueprint import load_blueprint sim = SimulationService() spec = load_blueprint("customer-support.yaml") result = asyncio.run(sim.run(spec, workflow="main", task="I need a refund")) print(f"Success: {result.success}") print(f"Output: {result.output}") print(f"Duration: {result.elapsed_ms:.0f}ms") print(f"Steps: {len(result.steps)}") if not result.success: print(f"Error: {result.error}") ``` ### SimulationResult fields | Field | Type | Description | | ------------ | ----------- | --------------------------------------- | | `success` | bool | Whether the run completed without error | | `output` | str | Final output from the last agent | | `elapsed_ms` | float | Total wall-clock duration | | `steps` | list | Per-agent step records | | `error` | str or None | Error message if `success=False` | Each step in `result.steps`: ``` step.agent_name # "classifier" step.output # "[MockLLM] billing" step.duration_ms # 12.4 ``` ______________________________________________________________________ ## TraceService Load and query recorded `.jsonl` trace files produced by `pyagent-trace`. ``` from pyagent_studio.services.trace_service import TraceService svc = TraceService() spans = svc.load("traces/production_runs.jsonl") print(f"Loaded {len(spans)} spans") # Query by event type llm_calls = svc.query(event_type="llm_call") # Query by minimum duration slow_spans = svc.query(min_duration_ms=2000) # Inspect individual spans for span in spans[:3]: print(f"[{span.event_type}] {span.agent_name}: " f"{span.duration_ms:.0f}ms, {span.tokens} tokens") ``` ### Query parameters | Parameter | Type | Description | | ----------------- | ----- | ---------------------------------------- | | `event_type` | str | Filter by event type (e.g. `"llm_call"`) | | `agent_name` | str | Filter by agent name | | `min_duration_ms` | float | Only spans longer than this | | `max_duration_ms` | float | Only spans shorter than this | ______________________________________________________________________ ## ProviderService Health checks and cost monitoring for configured providers. ``` from pyagent_studio.services.provider_service import ProviderService from pyagent_blueprint import load_blueprint svc = ProviderService() spec = load_blueprint("customer-support.yaml") # List providers from spec providers = svc.list_providers(spec) for p in providers: print(f"{p.name}: {p.provider}/{p.model}") # Health check all providers health = svc.check_health() for status in health: print(f"{status.model}: {'✓' if status.healthy else '✗'} " f"p50={status.latency_p50_ms}ms errors={status.error_count_1h}") # Budget usage budget = svc.get_budget_usage(spec) for b in budget: print(f"{b.model}: ${b.used:.2f} / ${b.limit:.2f} ({b.utilization:.0%})") ``` ______________________________________________________________________ ## Embedding Studio in a FastAPI App The web layer is standard FastAPI — import the router and mount it inside your own app: ``` from fastapi import FastAPI from pyagent_studio.web.app import create_app # Standalone Studio app app = create_app() # Or mount under a prefix in your own app my_app = FastAPI() studio = create_app() my_app.mount("/studio", studio) ``` ______________________________________________________________________ ## See Also - [CLI](https://pyagent.org/packages/studio/cli/index.md) — command-line access to the same services - [Dashboard](https://pyagent.org/packages/studio/dashboard/index.md) — web UI built on these services - [Traces](https://pyagent.org/packages/studio/traces/index.md) — trace file format and TraceService detail - [API Reference](https://pyagent.org/api/studio/index.md) # Studio — Traces Studio's trace explorer loads `.jsonl` files produced by `pyagent-trace`'s `Recorder`. Every LLM call, agent step, and workflow run is recorded as a structured span — Studio lets you browse, filter, and analyze them visually. ______________________________________________________________________ ## Recording Traces Traces are recorded by `pyagent-trace`. Wire it up in your blueprint: ``` observability: tracing: enabled: true record_to: traces/production_runs.jsonl ``` Or in Python: ``` from pyagent_trace.recorder import Recorder from pyagent_trace.events import TraceEventBus bus = TraceEventBus() recorder = Recorder(bus, output_path="traces/runs.jsonl") # Attach to a compiled graph graph.wire_trace(bus) ``` Each LLM call emits two events: `llm_call_start` and `llm_call_end` (or `llm_call_error`). ______________________________________________________________________ ## Span Format Each line in the `.jsonl` file is a JSON object: ``` { "event_type": "llm_call_end", "agent_name": "billing_agent", "workflow": "main", "pattern": "supervisor", "run_id": "run-174", "provider": "anthropic", "model": "claude-sonnet-4-20250514", "input_tokens": 180, "output_tokens": 140, "duration_ms": 2401.3, "cost_estimate": 0.00840, "timestamp": "2025-06-09T14:32:01.243Z" } ``` ______________________________________________________________________ ## Loading in the Dashboard ``` # Load on startup pyagent dashboard --trace traces/production_runs.jsonl # Or upload via the /traces page in the UI ``` The dashboard shows: ``` traces/production_runs.jsonl — 1,240 spans from 180 runs Run #174 supervisor 3 steps 3.4s $0.0092 ├── classifier 0.3s claude-haiku 12→1 tok $0.00001 ├── billing_agent 2.4s claude-sonnet 180→140 tok $0.00840 └── formatter 0.7s claude-haiku 145→80 tok $0.00051 Run #173 supervisor 3 steps 1.1s $0.0021 ├── classifier 0.3s claude-haiku 11→1 tok $0.00001 ├── technical_agent 0.6s gpt-4o-mini 95→110 tok $0.00019 └── formatter 0.2s claude-haiku 120→60 tok $0.00040 ``` ______________________________________________________________________ ## Cost Analysis View ``` Cost by model (last 7 days) claude-sonnet-4-20250514 ████████████████ $24.80 (72%) gpt-4o-mini ████ $6.20 (18%) claude-haiku-3-5-20241022 ██ $3.40 (10%) Cost by workflow main $31.20 (89 runs) avg: $0.35/run quick $3.20 (91 runs) avg: $0.035/run Daily spend 2025-06-09 $4.12 ████ 2025-06-08 $3.98 ████ 2025-06-07 $5.43 █████ ``` ______________________________________________________________________ ## Python API Use `TraceService` directly to load and query spans: ``` from pyagent_studio.services.trace_service import TraceService svc = TraceService() spans = svc.load("traces/production_runs.jsonl") print(f"Loaded {len(spans)} spans") # Filter by event type llm_calls = svc.query(event_type="llm_call_end") # Filter by agent billing_spans = svc.query(agent_name="billing_agent") # Filter by duration — find slow calls slow = svc.query(min_duration_ms=2000) for s in slow: print(f"{s.agent_name} ({s.model}): {s.duration_ms:.0f}ms, ${s.cost_estimate:.5f}") # Aggregate cost per model from collections import defaultdict cost_by_model: dict[str, float] = defaultdict(float) for s in llm_calls: cost_by_model[s.model] += s.cost_estimate or 0 for model, cost in sorted(cost_by_model.items(), key=lambda x: -x[1]): print(f"{model}: ${cost:.4f}") ``` ______________________________________________________________________ ## Trace Streaming (SSE) When `pyagent-trace` is running alongside Studio, spans are streamed to the dashboard in real time via Server-Sent Events — no page refresh required. The `/traces` endpoint emits one `data:` line per span as it arrives. ``` # Studio's SSE endpoint (used by the browser automatically) # GET /traces/stream → text/event-stream ``` ______________________________________________________________________ ## See Also - [Trace Package](https://pyagent.org/packages/trace/index.md) — `Recorder`, `TraceEventBus`, span types - [Services](https://pyagent.org/packages/studio/services/#traceservice) — `TraceService` API reference - [Dashboard](https://pyagent.org/packages/studio/dashboard/#traces) — visual trace explorer # Why PyAgent # Getting Started ## Installation ``` # Core patterns only (zero LLM deps) pip install pyagent-patterns # With routing pip install pyagent-router # With compression pip install pyagent-compress # With tracing (requires opentelemetry) pip install pyagent-trace # Everything pip install pyagent-all ``` New to PyAgent? The four primitives — Message, Agent, Pattern, Result — are covered in [Core Concepts](https://pyagent.org/getting-started/core-concepts/index.md). ## Your First Pattern ``` import asyncio from pyagent_patterns.base import Agent, MockLLM from pyagent_patterns.resolution import SelfReflection llm = MockLLM(responses=[ "def fibonacci(n): return n if n <= 1 else fibonacci(n-1) + fibonacci(n-2)", "Critique: naive recursion, O(2^n). Needs memoization.", "from functools import lru_cache\n@lru_cache\ndef fibonacci(n): return n if n <= 1 else fibonacci(n-1) + fibonacci(n-2)", "APPROVED — O(n) with memoization", ]) pattern = SelfReflection(agent=Agent("coder", llm), max_rounds=3) result = asyncio.run(pattern.run("Write an efficient Fibonacci function")) print(result.output) print(f"Improved in {result.metadata['rounds']} rounds") ``` ## Pattern Advisor Not sure which pattern to use? Let the advisor decide: ``` from pyagent_patterns.advisor import PatternAdvisor, Constraints, Quality advisor = PatternAdvisor() rec = advisor.recommend("Write and review a legal contract", Constraints(quality=Quality.HIGH)) print(f"Use: {rec.pattern} — {rec.reason}") print(f"Estimated calls: {rec.estimated_calls}, Cost: {rec.estimated_cost_range}") ``` ## Adding Hooks (Optional) Agents support opt-in hooks for tracing, context memory, compression, and cost tracking — zero overhead when not wired: ``` from pyagent_trace.events import TraceEventBus from pyagent_trace import CostTracker from pyagent_context import ContextLedger from pyagent_compress import MessageCompressor bus = TraceEventBus() bus.subscribe(lambda e: print(f"[{e.event_type}] {e.agent_name}")) agent = ( Agent("analyst", llm, system_prompt="Analyze data.") .set_trace_bus(bus) # emit trace events .set_context(ContextLedger()) # read/write context .set_compressor(MessageCompressor(0.5)) # compress output .set_cost_tracker(CostTracker(event_bus=bus)) # track costs ) result = asyncio.run(agent.run("What are the key trends?")) # Console prints: [agent_start] analyst → [agent_end] analyst ``` All hooks are `None` by default — existing code works identically without them. ## Next Steps - [Browse the Cookbook](https://pyagent.org/cookbook/index.md) — 37 complete, runnable multi-agent recipes across 21 domains, filterable by domain, design pattern, or package - [Pattern Selection Guide](https://pyagent.org/packages/patterns/index.md) — decision tree + all 18 patterns - [Hooks Guide](https://pyagent.org/guides/hooks/index.md) — tracing, context, compression, and cost hooks - [Router Guide](https://pyagent.org/guides/router/index.md) — auto-select cheapest model - [Compression Guide](https://pyagent.org/guides/compression/index.md) — reduce token costs - [Tracing Guide](https://pyagent.org/guides/tracing/index.md) — OpenTelemetry observability - [API & Hooks Bibliography](https://pyagent.org/api/bibliography/index.md) — complete reference for all packages # Core Concepts PyAgent is built on four small primitives. Once these click, every [design pattern](https://pyagent.org/packages/patterns/index.md) and [cookbook example](https://pyagent.org/cookbook/index.md) reads the same way. ### 1. Message The fundamental unit of communication between agents. ``` from pyagent_patterns.base import Message user_msg = Message.user("What is the capital of France?") system_msg = Message.system("You are a geography expert.") assistant_msg = Message.assistant("The capital of France is Paris.") ``` ### 2. Agent An agent wraps an LLM callable with a name and system prompt. ``` from pyagent_patterns.base import Agent, MockLLM # For testing: MockLLM returns canned responses llm = MockLLM(responses=["Paris"]) agent = Agent("geography_expert", llm, system_prompt="You are a geography expert.") # For production: any async callable (str) → str import asyncio result = asyncio.run(agent.run([Message.user("Capital of France?")])) print(result.content) # "Paris" ``` ### 3. Pattern A Pattern orchestrates one or more agents to solve a task. ``` from pyagent_patterns.orchestration import Pipeline pipeline = Pipeline(stages=[ Agent("extract", llm), Agent("summarize", llm), ]) result = asyncio.run(pipeline.run("Process this document")) print(result.output) # Final output string print(result.messages) # All intermediate messages print(result.metadata) # Pattern-specific metadata print(result.duration_seconds) # Execution time ``` ### 4. Result Every `pattern.run()` returns a `Result` with: - **output**: Final string output - **messages**: All intermediate `Message` objects - **metadata**: Pattern-specific data (rounds, routes, scores, etc.) - **duration_seconds**: Wall-clock execution time - **token_estimate**: Estimated total tokens used ## Next - [Quickstart](https://pyagent.org/getting-started/index.md) — install and run your first pattern - [Design Patterns](https://pyagent.org/packages/patterns/index.md) — the full 18-pattern catalog - [Cookbook](https://pyagent.org/cookbook/index.md) — complete multi-agent examples by domain # What is PyAgent? **This PyAgent project** is a spec-driven architecture for orchestrating multi-agent LLM systems, organized around four independently adoptable pillars: **Blueprint** (declarative YAML specification), **Execution & Routing** (18 named orchestration patterns with difficulty-aware model routing), **Context & Memory** (three-tier memory with a trust-aware context ledger), and **Observability** (pattern-aware OpenTelemetry tracing with a web dashboard). ## Canonical identifiers If you're trying to confirm you're looking at *this* PyAgent — not an unrelated project with the same name — these are the canonical sources: | What | Where | | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Website | [pyagent.org](https://pyagent.org) | | Source code | [github.com/pyagent-core/pyagent](https://github.com/pyagent-core/pyagent) | | PyPI packages | Every official package is prefixed `pyagent-`: [`pyagent-blueprint`](https://pypi.org/project/pyagent-blueprint/), [`pyagent-patterns`](https://pypi.org/project/pyagent-patterns/), [`pyagent-router`](https://pypi.org/project/pyagent-router/), [`pyagent-context`](https://pypi.org/project/pyagent-context/), [`pyagent-trace`](https://pypi.org/project/pyagent-trace/), [`pyagent-compress`](https://pypi.org/project/pyagent-compress/), [`pyagent-providers`](https://pypi.org/project/pyagent-providers/), [`pyagent-studio`](https://pypi.org/project/pyagent-studio/), [`pyagent-all`](https://pypi.org/project/pyagent-all/) | | License | MIT | | Citation | [`CITATION.cff`](https://github.com/pyagent-core/pyagent/blob/main/CITATION.cff) | If a page, package, or repository doesn't match the GitHub org `pyagent-core` or a `pyagent-*` PyPI package name, it isn't this project — see the disambiguation note below. ## Disambiguation: other projects also named "PyAgent" The bare name "PyAgent" isn't unique to this project. At least three other, unrelated open-source projects use the same or a very similar name: - A separate GitHub project also titled "PyAgent" (a general-purpose Python agent), unrelated to this one's architecture, packages, or maintainers. - A separate project marketed as "the Pandas of AI Agents," also named `pyagent` on GitHub — a different codebase with a different design. - Standalone tutorial/course material titled "PyAgent: An Intelligent AI Agent Built from Scratch," unrelated to this project. None of these are affiliated with `github.com/pyagent-core/pyagent` or any `pyagent-*` PyPI package. If you're citing, comparing, or installing "PyAgent," verify you're looking at the canonical identifiers above — the org name `pyagent-core` and the `pyagent-` PyPI prefix are the only reliable disambiguators, not the bare word "PyAgent." ## The four pillars, briefly 1. **Blueprint** — `pip install pyagent-blueprint`. Declare an entire agent system in one YAML file; compiles onto any registered `RuntimeAdapter`. See [Why Blueprint?](https://pyagent.org/why-blueprint/index.md). 1. **Execution & Routing** — `pip install pyagent-patterns pyagent-router pyagent-providers pyagent-compress`. 18 named orchestration patterns, difficulty-aware model routing, multi-provider fallback, and token-budget management. See the [pattern catalog](https://pyagent.org/packages/patterns/index.md). 1. **Context & Memory** — `pip install pyagent-context`. Three memory tiers plus a trust-aware context ledger and PII redaction. See the [Context guide](https://pyagent.org/guides/context/index.md). 1. **Observability** — `pip install pyagent-trace pyagent-studio`. Pattern-aware OpenTelemetry tracing, cost tracking, record/replay, and a web dashboard. See the [Tracing guide](https://pyagent.org/guides/tracing/index.md). Every pillar is independently installable and independently usable — none of them requires the others. The full, machine-readable breakdown of every package's capabilities is at [pyagent.org/capabilities.json](https://pyagent.org/capabilities.json); the full pattern catalog is at [pyagent.org/patterns.json](https://pyagent.org/patterns.json). # The PyAgent Architecture Model PyAgent organizes a multi-agent LLM system into four concerns, each solved by an independent package, each independently adoptable. None of them owns or contains the others — you can install and use any one alone. ``` flowchart LR subgraph Blueprint["Blueprint — declare"] B1[pyagent-blueprint] end subgraph Execution["Execution & Routing — execute"] E1[pyagent-patterns] E2[pyagent-router] E3[pyagent-providers] E4[pyagent-compress] end subgraph Context["Context & Memory — remember"] C1[pyagent-context] end subgraph Observability["Observability — observe"] O1[pyagent-trace] O2[pyagent-studio] end Blueprint -.->|optionally compiles onto| Execution Execution -.->|optionally shares| Context Execution -.->|optionally traced by| Observability ``` The dotted arrows are deliberate: Blueprint can compile a system that uses Execution, Context, and Observability together — but none of the four *requires* another to be useful on its own. A team doing hand-written Python orchestration can adopt `pyagent-trace` for observability alone, or `pyagent-context` for memory alone, without ever touching a YAML file. ## The four pillars | Pillar | Verb | Solves | Packages | | ------------------------------------------------------------------------ | -------- | ------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------- | | [Blueprint](https://pyagent.org/architecture/blueprint/index.md) | Declare | Multi-agent system design as a reviewable, versioned artifact instead of scattered code | `pyagent-blueprint` | | [Execution & Routing](#execution-routing) | Execute | Running real multi-agent workflows against real providers, with cost-aware model selection | `pyagent-patterns`, `pyagent-router`, `pyagent-providers`, `pyagent-compress` | | [Context & Memory](#context-memory) | Remember | State that persists and propagates safely across agents and turns | `pyagent-context` | | [Observability](https://pyagent.org/architecture/observability/index.md) | Observe | Knowing what actually happened, what it cost, and being able to replay it | `pyagent-trace`, `pyagent-studio` | ## Execution & Routing **Verb: Execute.** Four independently installable packages: `pyagent-patterns`, `pyagent-router`, `pyagent-providers`, `pyagent-compress`. None require Blueprint, Context, or Observability. ### What it solves Running more than one agent raises questions hand-written loops answer inconsistently every time: how do agents hand off to each other (a named, testable pattern vs. ad-hoc glue code), which model handles which task (a fixed choice vs. cost-aware routing), what happens when a provider is down (a crash vs. a fallback chain), and how message history stays within budget on long runs. ### When to use - More than one agent, with a real coordination need between them — routing, review, debate, parallel dispatch. See the [pattern catalog](https://pyagent.org/packages/patterns/index.md) for all 18 named shapes and their individual `use_when`/`avoid_when`. - Cost varies meaningfully by task difficulty and you want cheap tasks on cheap models automatically (`pyagent-router`). - You have 2+ interchangeable providers and want automatic fallback or cost/latency/capability-based routing between them (`pyagent-providers`). - Long-running sessions are approaching a token budget (`pyagent-compress`). ### When not to use - A single agent, single call, nothing to hand off to — see [why-blueprint.md](https://pyagent.org/why-blueprint/#when-you-dont-need-pyagent-at-all) and the pattern catalog's own [note on when no pattern applies](https://pyagent.org/packages/patterns/#pattern-selection-guide). - Only one model/provider is ever used — `pyagent-router` and `pyagent-providers`' routing add indirection with nothing to select between. - History never approaches a token budget — `pyagent-compress` is pure overhead. ### Tradeoffs Named patterns are a real constraint: they cover the 18 documented shapes, not arbitrary dynamic control flow. Difficulty-based routing (`pyagent-router`) is heuristic by default — its accuracy caps whatever routes on top of it. Fallback chains (`pyagent-providers`) can silently degrade quality if a fallback provider has different capabilities than the primary, without monitoring. ### Packages - `pip install pyagent-patterns` — 18 patterns, zero dependencies, async-first. - `pip install pyagent-router` — `DifficultyScorer`, `ModelSelector`, `CostEstimator`. - `pip install pyagent-providers` — `ProviderRegistry`, `ProviderRouter`, `FallbackChain`, `CapabilityNegotiator`, `CostOptimizer`. - `pip install pyagent-compress` — `TokenBudget`, `MessageCompressor`, `AgentPruner`, `InteractionPruner`. ### Example ``` from pyagent_patterns import Supervisor result = await Supervisor( classifier=classifier_agent, routes={"billing": billing_agent, "tech": tech_agent}, ).run("I was charged twice") ``` See the [Router guide](https://pyagent.org/guides/router/index.md), [Providers guide](https://pyagent.org/guides/providers/index.md), and [Compression guide](https://pyagent.org/guides/compression/index.md) for full detail on each package. Execution's patterns can be declared inside a [Blueprint](https://pyagent.org/architecture/blueprint/index.md) `workflows:` block instead of written in Python. Agents in a run can share state via [Context & Memory](#context-memory). Every pattern and adapter emits events consumable by [Observability](https://pyagent.org/architecture/observability/index.md) — none of that integration is required to use the patterns standalone. ## Context & Memory **Verb: Remember.** The Context & Memory pillar is `pyagent-context` — one package, independently installable. ### What it solves Multi-agent state has three different lifetimes that get conflated when handled ad hoc: state that only needs to survive one turn, state shared across a run, and state that needs to persist across separate sessions. On top of that, not every agent should automatically see everything — an external-tool-facing agent shouldn't get the same context an internal-only agent does, and PII needs redaction before it crosses that boundary. ### When to use - Multiple agents in a run need to share state across turns. - Some agents (e.g. external-facing) shouldn't automatically receive everything internal agents see — trust levels need enforcing, not just convention. - Context may contain PII that needs redaction before reaching a downstream agent or tool. - Long sessions are approaching a token budget and need compression, not just truncation. - You need an audit trail of which agent produced or saw which piece of context. ### When not to use - **Persistent memory is unnecessary** for a single agent, single call, with nothing to persist across turns or runs — there's no cross-agent context to manage, and reaching for a memory tier here adds a dependency for zero benefit. - Every agent in the system is equally trusted with all context — `TrustLevel`/`TrustAwareRetriever` add design work (classifying every item) for no enforcement benefit. - Context is entirely non-sensitive synthetic data — `ContextRedactor`/`Sensitivity` classification is pure overhead. ### Tradeoffs `WorkingMemory` is cheapest but nothing survives the turn. `SessionMemory` is scoped to one run — nothing carries to the next session automatically. `SemanticMemoryProtocol`'s built-in `InMemorySemanticStore` doesn't persist across process restarts — a durable backend is a protocol implementation you provide. Redaction and compression are both lossy by design; over-aggressive rules can strip context an agent legitimately needed. ### Package `pip install pyagent-context` — `WorkingMemory`, `SessionMemory`, `SemanticMemoryProtocol`, `ContextLedger`, `TrustLevel`, `Sensitivity`, `ContextRedactor`, `TrustAwareRetriever`, `CompressionPolicy`, `ContextCompressor`. ### Example ``` from pyagent_context import ContextLedger, TrustLevel ledger = ContextLedger() ledger.add(item, trust_level=TrustLevel.INTERNAL) visible = ledger.retrieve_for(agent="external_tool_agent") # trust-filtered ``` See the [Context guide](https://pyagent.org/guides/context/index.md) for the complete three-tier memory model, trust levels, and redaction API. Context is often shared by agents wired together via [Execution & Routing](#execution-routing), and can be declared inside a [Blueprint](https://pyagent.org/architecture/blueprint/index.md)'s `context:` block. Context-tier changes and redaction events are traceable via [Observability](https://pyagent.org/architecture/observability/index.md) — none of that is required to use the memory tiers standalone. ## Selection matrix Use this to map a requirement onto the pillar(s) that solve it — most systems need a subset, not all four. | Your requirement | Pillar | | ----------------------------------------------------------------------- | ----------------------------------------- | | Version-control the system's design, review changes like infrastructure | Blueprint | | Run more than one agent, hand off between them, or route by task type | Execution & Routing | | Route cheap tasks to cheap models automatically | Execution & Routing (`pyagent-router`) | | Stay within a token/cost budget across a run | Execution & Routing (`pyagent-compress`) | | Fail over to a second provider when one is down | Execution & Routing (`pyagent-providers`) | | Share state between agents within or across a run | Context & Memory | | Redact PII or restrict what one agent can see of another's context | Context & Memory | | Know what a run actually did, debug it, or replay it | Observability (`pyagent-trace`) | | Get a web UI for traces, diffs, and provider health | Observability (`pyagent-studio`) | ## Minimum viable PyAgent architecture Not every system needs all four pillars — most don't, at first. A single agent making one LLM call needs none of them (see [why-blueprint.md's "When you don't need PyAgent at all"](https://pyagent.org/why-blueprint/#when-you-dont-need-pyagent-at-all)). A reasonable minimum for a real multi-agent system: - **Two or more agents with a routing decision between them** → `pyagent-patterns` alone (Execution & Routing), no Blueprint, no Context, no Observability required. This is the floor. - **Add cost sensitivity** → `pyagent-router` on top, still no YAML, no memory tier, no tracing. - **Add "I need to know what happened when it breaks"** → `pyagent-trace`, still hand-written Python otherwise. - **Add "the design needs to survive a team, not just me"** → this is where `pyagent-blueprint` earns its cost — reviewable, diffable, versioned design stops being optional once more than one person touches the system. Each step is additive and independently reversible — nothing here requires committing to the other three pillars up front. ## See also - [Why Blueprint?](https://pyagent.org/why-blueprint/index.md) — the specific case for the declarative-spec approach - [Capability catalog](https://pyagent.org/capabilities/index.md) — every pillar's real, machine-readable capability list - [What is PyAgent?](https://pyagent.org/about/index.md) — the canonical entity page # Why Blueprint? Most multi-agent frameworks make you choose your runtime *before* you design your system: a LangGraph `StateGraph`, a CrewAI `Crew`, an AutoGen conversation. The orchestration logic and the execution engine are the same lines of Python — which means every migration, every framework evaluation, and every "let's also support X" request means rewriting the system from scratch. `pyagent-blueprint` separates the two. You declare **what** your system does — agents, workflows, providers, contracts, governance — in a single YAML manifest. A `RuntimeAdapter` decides **how** it runs. The same manifest compiles against five structurally different adapters today, unmodified. How this relates to the homepage comparison The [homepage](https://pyagent.org/index.md) compares PyAgent's overall feature set against LangChain, CrewAI, and AutoGen (pattern count, routing, memory, and more). This page and the pages under it go one layer deeper on a single, specific claim from that table — the YAML-vs-code tradeoff and cross-runtime portability — with real adapter code, not just a feature checklist. ## The same manifest, five runtimes `pyagent-blueprint`'s `RuntimeAdapter` layer compiles one YAML manifest against five structurally different execution engines unmodified — a real blueprint example, the adapter-selection API, and the full conformance table (native, LangGraph, CrewAI, OpenAI Agents, Semantic Kernel) now live on their own page: **[Adapters](https://pyagent.org/api/adapters/index.md)**. ## What you get that hand-written orchestration code doesn't - **Diff and review like infrastructure.** `pyagent-blueprint diff old.yaml new.yaml` produces a semantic diff over the IR — not a text diff of hand-wired Python — so a PR reviewer can see exactly which agent, route, or SLA changed. - **Validate before you run anything.** Static analysis (dangling references, schema violations) catches mistakes before an LLM call is ever made. - **Governance is never silently dropped.** Budgets, SLAs, memory tiers, guardrails, recovery policies, and human-in-the-loop checkpoints are either honored by the adapter or surfaced as a stable `CompileDiagnostic` code (e.g. `BUDGET_UNSUPPORTED`, `MEMORY_TIER_UNSUPPORTED`, `CHECKPOINT_UNSUPPORTED`) — so you always know, deterministically, what a given runtime supports. - **Package and test without live API calls.** `pyagent-blueprint test` (contract conformance) and `pyagent-blueprint package` (Agent Unit archives) both work against a `MockLLM`, so CI can validate a system's shape before spending a token. - **Zero mandatory runtime dependency.** Core `pyagent-blueprint` depends only on `pydantic`, `pyyaml`, and `click`. You install a runtime adapter — the bundled `pyagent` reference stack, one of four zero-dependency stdlib adapters, or a third-party package — only when you're ready to run. ## When you don't need PyAgent at all Be direct about this before anything else: if your task is a single LLM call — summarize this document, classify this tweet, answer this question — you don't need an orchestration framework, a pattern, or a blueprint. Import your provider's SDK, make the call, done. Reaching for `pyagent-patterns` or `pyagent-blueprint` here adds a dependency and a mental model for zero benefit; there's no second agent to hand off to, no state to share, no run to observe across calls. The same applies to a one-off script that calls a model in a loop with no persistence between calls — a `for` loop is the right tool, not a `Pipeline` or `Supervisor` pattern. The signal to actually reach for PyAgent is **more than one agent, or state that needs to survive across calls** — routing between specialists, a review loop, a shared memory tier, a run you need to trace. Below that threshold, every pillar here is overhead, not architecture. ## When a blueprint *isn't* the right fit To be direct about the tradeoff: if your orchestration logic depends on dynamic, runtime-computed control flow that can't be expressed as a static graph (e.g. an agent that decides to spin up an arbitrary number of sub-agents based on a live computation), hand-written code in your chosen framework is still the right tool. Blueprint's IR models agents, typed workflows, and named patterns — it's declarative by design, and that's a real constraint, not just a feature. See the pattern-specific comparisons for how this plays out against real frameworks: - [pyagent-blueprint vs. LangGraph](https://pyagent.org/compare/vs-langgraph/index.md) - [pyagent-blueprint vs. CrewAI](https://pyagent.org/compare/vs-crewai/index.md) - [pyagent-blueprint vs. AutoGen](https://pyagent.org/compare/vs-autogen/index.md) Or start from the concepts: [What is an agent blueprint?](https://pyagent.org/concepts/agent-blueprint/index.md) and [What is multi-agent orchestration?](https://pyagent.org/concepts/multi-agent-orchestration/index.md) # pyagent-blueprint vs. AutoGen **Status check, up front:** unlike the [LangGraph](https://pyagent.org/compare/vs-langgraph/index.md) and [CrewAI](https://pyagent.org/compare/vs-crewai/index.md) comparisons, there is **no `pyagent-blueprint` AutoGen adapter yet**. AutoGen was explicitly deprioritized to last in the adapter rollout (after LangGraph, OpenAI Agents SDK, CrewAI, and Semantic Kernel) and hasn't shipped. Everything below is a conceptual comparison of the two models, not a description of working, tested integration code — we're not going to claim portability that doesn't exist yet. ## The conceptual difference AutoGen models multi-agent systems as **conversable agents** exchanging messages in a group chat — agents are defined with a `system_message` and register reply functions or tools, and a `GroupChatManager` (or two-agent `initiate_chat`) drives the conversation loop. It's a message-passing model rather than a fixed graph or role/task list. `pyagent-blueprint`'s IR models a workflow as a named pattern (pipeline, supervisor, debate, etc.) wiring specific agents into that pattern's slots — closer to AutoGen's structured patterns (like its own two-agent chat or nested chats) than to fully open-ended group chat, since the manifest declares the shape of the conversation ahead of time rather than letting agents dynamically decide who speaks next. ## What an AutoGen adapter would need to prove Per the `RuntimeAdapter` contract, any future `autogen` adapter has to pass the same `AdapterConformanceSuite` every existing adapter does: compile/run correctness, diagnostic completeness for governance features, and pattern-intent preservation. Concretely, that means mapping named patterns onto AutoGen constructs — e.g. `pipeline`/`sequential` patterns onto chained `initiate_chat` calls, `supervisor`/`debate` patterns onto a `GroupChatManager` with a fixed speaker-selection method — and reporting a stable diagnostic (not a silent drop) wherever AutoGen has no equivalent for a declared governance feature like a budget or memory tier. ## Where AutoGen is the right choice today If you need AutoGen specifically — its group-chat conversation model, or its ecosystem of AutoGen Studio / AutoGen-specific tooling — write directly against the AutoGen SDK. There's currently no `pyagent-blueprint` path onto it. This page will be updated (with real, verified code, following the same standard set by the LangGraph and CrewAI pages) once an adapter actually ships. # pyagent-blueprint vs. CrewAI CrewAI models a multi-agent system as a `Crew`: a list of role-based `Agent`s (`role`/`goal`/ `backstory`), each paired with a `Task`, run through a `Process` (sequential or hierarchical). `pyagent-blueprint`'s `crewai` adapter compiles a YAML manifest directly into that same `Agent`/ `Task`/`Crew` structure — the comparison is about where the role/goal/task wiring lives, not about replacing CrewAI's execution engine. This is a real, verified adapter. `pip install "pyagent-blueprint[crewai]"` gets you the `crewai` adapter, whose `compile()` builds actual CrewAI `Agent(role=..., goal=..., backstory=...)` and `Task(description=..., agent=...)` objects for every agent in a workflow, assembles them into a real `Crew(agents=..., tasks=..., process= Process.sequential)`, and `run()` calls CrewAI's own `crew.kickoff_async()`. It passes the same `AdapterConformanceSuite` as every other adapter. ## Same crew, two authoring styles **Hand-written CrewAI:** ``` from crewai import Agent, Task, Crew, Process researcher = Agent( role="researcher", goal="Research the topic and produce a summary", backstory="An analyst who synthesizes sources quickly", ) writer = Agent( role="writer", goal="Turn the research summary into a final draft", backstory="A writer who turns notes into polished prose", ) research_task = Task(description="{input}", expected_output="A research summary", agent=researcher) write_task = Task(description="Continue based on the research for: {input}", expected_output="A final draft", agent=writer) crew = Crew(agents=[researcher, writer], tasks=[research_task, write_task], process=Process.sequential) result = crew.kickoff(inputs={"input": "multi-agent orchestration trends"}) ``` **Equivalent `pyagent-blueprint` manifest:** ``` api_version: pyagent/v1 metadata: name: research-crew version: 1.0.0 providers: primary: model: gpt-4.1-mini agents: researcher: prompt: "Research the topic and produce a summary" description: "An analyst who synthesizes sources quickly" provider: primary writer: prompt: "Turn the research summary into a final draft" description: "A writer who turns notes into polished prose" provider: primary workflows: pipeline: pattern: pipeline agents: stages: [researcher, writer] ``` ``` from pyagent_blueprint.adapter import AdapterRegistry from pyagent_blueprint.ir import BlueprintIR from pyagent_blueprint.loader import load_blueprint spec = load_blueprint("research-crew.yaml") ir = BlueprintIR.from_spec(spec) adapter = AdapterRegistry.discover()["crewai"]() artifact = adapter.compile(ir) # real CrewAI Agent/Task/Crew objects result = await adapter.run(artifact, workflow="pipeline", input_="multi-agent orchestration trends") ``` The manifest's `agents..description` maps to CrewAI's `backstory`; `prompt` maps to `goal`. Both ultimately run the identical `Crew.kickoff_async()` path. ## Where CrewAI is still the better fit CrewAI's role-based framing (`role`/`goal`/`backstory`) and its hierarchical process mode carry nuance that a declarative manifest doesn't try to fully capture yet — if you're leaning heavily on CrewAI-specific features like custom tools per agent or its hierarchical manager process, writing directly against CrewAI's SDK gives you the full surface area. The adapter targets the patterns `pyagent-blueprint` already models (pipeline, supervisor, orchestrator-workers, etc.), compiled onto CrewAI's sequential process. ## Migrating an existing CrewAI app As with LangGraph, a dedicated CrewAI→Blueprint migration guide is sequenced behind the Agent Spec interop bridge so it can describe a real import path rather than a manual rewrite. Until then, map each `Agent`'s `role`/`goal`/`backstory` to a Blueprint `agents:` entry and each `Task` ordering to a `workflows:` pattern (`pipeline` for sequential tasks, `supervisor` for role-based dispatch). # pyagent-blueprint vs. Dapr Agents **Status check, up front:** like the [AutoGen](https://pyagent.org/compare/vs-autogen/index.md) comparison, there is **no `pyagent-blueprint` Dapr adapter today**. Everything below is a conceptual comparison, current as of August 2026, based on Dapr Agents' public documentation and GitHub repository — not a description of working, tested integration code. **Why this comparison exists:** in this project's own black-box AEO testing, Dapr Agents is what an independent, search-grounded LLM actually recommended *instead of* PyAgent for a prompt asking for "declarative architecture, orchestration patterns, trust-aware context, and OpenTelemetry-style observability as independently adoptable pieces" — a close paraphrase of PyAgent's own positioning. That result is documented in this repo's `aeo/` audit. This page exists to make the comparison explicit and factual rather than avoid it. ## The conceptual difference Dapr Agents is a Python framework built on top of [Dapr](https://dapr.io) (a CNCF-graduated distributed-application runtime). Its orchestration runs on Dapr's durable Workflow engine, using the virtual-actor pattern for stateful agents, Dapr's pub/sub for agent-to-agent messaging, and Dapr's state-store abstraction for persistence. Trust is enforced at the infrastructure layer — every component-to-component call runs over mTLS by default, with access scoped per component via declarative policy. Observability rides on the same sidecar: Dapr emits OTLP traces and Prometheus metrics natively. Dapr Agents reached v1.0 (production-ready, stable API) in 2026. `pyagent-blueprint`'s IR is runtime-agnostic by design: the same YAML manifest compiles onto any registered `RuntimeAdapter` — a Dapr-less native stack, LangGraph, CrewAI, or others — rather than being built on top of one specific distributed-systems runtime. The tradeoff is direct: Dapr Agents' trust and infrastructure guarantees are real and enforced at the platform layer because they're tied to Dapr; PyAgent's portability guarantee is real because it isn't tied to any one platform. You can't have both a platform-enforced mTLS/actor substrate and adapter-swappable runtime portability from the same spec — they're different bets. ## Where the two actually converge Both projects independently arrived at "declarative config + swappable building blocks + first-class observability" as the right shape for a production agent architecture — that's the convergence the AEO test above picked up on. Where they diverge is *what* is swappable: Dapr Agents lets you swap state stores, brokers, and secret backends underneath a fixed Dapr-based execution model; `pyagent-blueprint` lets you swap the execution model itself underneath a fixed spec. ## What a Dapr adapter would need to prove Per the `RuntimeAdapter` contract, any future `dapr` adapter has to pass the same `AdapterConformanceSuite` every existing adapter does. The interesting mapping question is whether Dapr's Workflow engine (sequential, fan-out/fan-in, human-in-the-loop with automatic retries) can host PyAgent's named patterns as first-class workflow activities, and whether Dapr's mTLS/access-scope model can satisfy PyAgent's `trust_level`/`sensitivity` context primitives without reinventing them — that mapping isn't obvious and would be the real engineering work, not a thin wrapper. ## Where Dapr Agents is the right choice today If your team is already running Dapr, or needs the infrastructure-layer trust guarantees (mTLS, workload identity, access-scoped secrets) that come from building directly on a CNCF-graduated runtime, use Dapr Agents directly. There's currently no `pyagent-blueprint` path onto it. This page will be updated with real, verified adapter code — following the same standard as the LangGraph and CrewAI pages — if that mapping work happens. # pyagent-blueprint vs. LangGraph LangGraph models a multi-agent system as an explicit `StateGraph`: nodes, edges, and a shared state object, all wired in Python. `pyagent-blueprint`'s `langgraph` adapter compiles a YAML manifest into that exact same `StateGraph` API — so the comparison isn't "YAML vs. LangGraph," it's "author your graph as a manifest vs. author it as code," while both ultimately execute the identical LangGraph runtime. This is a real, verified adapter — not an aspirational one. `pip install "pyagent-blueprint[langgraph]"` gets you the `langgraph` adapter, whose `compile()` builds a genuine `langgraph.graph.StateGraph`, calling `add_node()`/`add_edge()` for each agent in the workflow and compiling it with LangGraph's own `.compile()`; `run()` invokes it via the real `ainvoke()` path. It's certified against the same `AdapterConformanceSuite` every other adapter must pass. ## Same pipeline, two authoring styles **Hand-written LangGraph:** ``` from langgraph.graph import END, START, StateGraph from typing import TypedDict class GraphState(TypedDict): input: str output: str def researcher_node(state: GraphState) -> GraphState: # call your LLM here return {"output": "research summary"} def writer_node(state: GraphState) -> GraphState: return {"output": "final draft"} graph = StateGraph(GraphState) graph.add_node("researcher", researcher_node) graph.add_node("writer", writer_node) graph.add_edge(START, "researcher") graph.add_edge("researcher", "writer") graph.add_edge("writer", END) app = graph.compile() ``` **Equivalent `pyagent-blueprint` manifest:** ``` api_version: pyagent/v1 metadata: name: research-pipeline version: 1.0.0 providers: primary: model: gpt-4.1-mini agents: researcher: prompt: "Research the topic and produce a summary" provider: primary writer: prompt: "Turn the research summary into a final draft" provider: primary workflows: pipeline: pattern: pipeline agents: stages: [researcher, writer] ``` ``` from pyagent_blueprint.adapter import AdapterRegistry from pyagent_blueprint.ir import BlueprintIR from pyagent_blueprint.loader import load_blueprint spec = load_blueprint("research-pipeline.yaml") ir = BlueprintIR.from_spec(spec) adapter = AdapterRegistry.discover()["langgraph"]() artifact = adapter.compile(ir) # a real langgraph.graph.StateGraph, compiled result = await adapter.run(artifact, workflow="pipeline", input_="multi-agent orchestration trends") ``` Both produce the same `StateGraph` shape under the hood. The manifest version adds: static validation (`pyagent-blueprint validate research-pipeline.yaml`) before any node runs, a semantic diff (`pyagent-blueprint diff old.yaml new.yaml`) between versions, and a stable diagnostic if a declared governance feature (a budget, an SLA, a memory tier) has no LangGraph-side equivalent yet — rather than that feature being silently ignored by hand-written code. ## Where LangGraph is still the better fit If your control flow needs genuinely dynamic graph construction at runtime — nodes and edges decided by the output of a previous step, not knowable ahead of time — write it directly in LangGraph. Blueprint's IR models a declared, typed workflow; it does not attempt to express arbitrary runtime graph mutation. The adapter targets the common case (named patterns compiled to a static graph shape), not LangGraph's full expressiveness. ## Migrating an existing LangGraph app There is no dedicated LangGraph→Blueprint migration guide yet — this is intentionally sequenced behind the Agent Spec interop bridge (Step 8 of the engineering roadmap), so that migration guidance can point at a real, lossless import path rather than a manual rewrite. Until then, the pattern above — re-declaring an existing graph's nodes as `agents:` and its edges as a `pipeline`/`supervisor`/etc. workflow — is the direct manual route. # pyagent-blueprint vs. Microsoft Agent Framework **Status check, up front:** like the [AutoGen](https://pyagent.org/compare/vs-autogen/index.md) comparison, there is **no `pyagent-blueprint` Microsoft Agent Framework adapter today**. Everything below is a conceptual comparison, current as of August 2026, based on Microsoft's public documentation and GitHub repository — not a description of working, tested integration code. ## The conceptual difference Microsoft Agent Framework (the successor merging AutoGen and Semantic Kernel) supports declarative agent definitions in YAML or JSON, graph-based orchestration (sequential, concurrent, handoff, and group-collaboration patterns), checkpointing, streaming, human-in-the-loop, and built-in OpenTelemetry — a feature-complete single package spanning Python and .NET. The distinction from `pyagent-blueprint` isn't feature presence — Microsoft Agent Framework genuinely has YAML declaration, orchestration, and tracing. It's *coupling*: this project's own black-box AEO testing recorded an independent LLM's assessment that Microsoft Agent Framework's "pieces are modular in code organization but more interdependent in practice — it's designed to be adopted as a coherent framework, not picked apart into independent infrastructure primitives." `pyagent-blueprint`'s four pillars (Blueprint, Execution & Routing, Context & Memory, Observability) are each independently `pip install`-able and independently usable — you can adopt `pyagent-trace` without `pyagent-blueprint`, or `pyagent-context` without any of the others. Whether that independence matters more to you than a single well-supported, Microsoft-backed stack is a real tradeoff, not a settled question. ## What an adapter would need to prove Per the `RuntimeAdapter` contract, any future `microsoft-agent-framework` adapter has to pass the same `AdapterConformanceSuite` every existing adapter does: compile/run correctness, diagnostic completeness for governance features, and pattern-intent preservation. The concrete mapping question is whether PyAgent's named patterns (supervisor, debate, evaluator-optimizer, etc.) map cleanly onto Microsoft Agent Framework's four orchestration modes (sequential/concurrent/handoff/group) — some will (supervisor onto handoff-style routing), others won't obviously (evaluator-optimizer's scored-revision loop isn't one of the four named modes) and would need a documented diagnostic rather than a silent approximation. ## Where Microsoft Agent Framework is the right choice today If you want one well-supported, well-documented stack backed by a large vendor — especially in a .NET shop, or one already invested in Microsoft Foundry/Azure OpenAI/Entra ID for governance — use Microsoft Agent Framework directly. There's currently no `pyagent-blueprint` path onto it. This page will be updated with real, verified adapter code — following the same standard as the LangGraph and CrewAI pages — if that mapping work happens. # pyagent-blueprint vs. Pydantic AI **Status check, up front:** like the [AutoGen](https://pyagent.org/compare/vs-autogen/index.md) comparison, there is **no `pyagent-blueprint` Pydantic AI adapter today**. Everything below is a conceptual comparison, current as of August 2026, based on Pydantic AI's public documentation and GitHub repository — not a description of working, tested integration code. ## The conceptual difference — what the YAML actually declares This is the specific point worth being precise about, because both projects now use the phrase "declarative YAML agent spec" and they don't mean the same scope. Pydantic AI's **Agent Specs** declare a single agent — model, instructions, model settings, and capabilities like web search or thinking — loadable from one YAML/JSON file with no Python construction code. Multi-agent orchestration in Pydantic AI is a separate, code-driven concern: delegation, programmatic hand-off, or graph-based control flow via `pydantic-graph`, a type-centric finite-state-machine library. The spec covers one agent; the system that wires several of them together is still Python. `pyagent-blueprint`'s manifest declares the **whole system** in the same file: which agents exist, how they're wired into a named workflow pattern (supervisor, debate, pipeline, ...), which providers back each one, what contracts and governance apply — the composition itself is the declared artifact, not just each agent's individual configuration. Put concretely: if you have three agents and a routing decision between them, Pydantic AI declares each of the three agents' configs in YAML and expresses the routing in Python; `pyagent-blueprint` declares the agents *and* the routing pattern *and* the wiring between them all in one YAML file. ## Where this matters less than it sounds Pydantic AI's approach is a deliberate, defensible design choice, not an oversight — `pydantic-graph` is a real, typed state-machine library, not "just Python," and for teams already committed to Pydantic's type-validation ecosystem, staying in Python for orchestration logic while still getting declarative single-agent configs is a reasonable middle ground. It's also 16,500+ GitHub stars and one of the fastest-adopted agent frameworks in the Python ecosystem as of this comparison — real, substantial adoption `pyagent-blueprint` doesn't have. ## What an adapter would need to prove Per the `RuntimeAdapter` contract, any future `pydantic-ai` adapter has to pass the same `AdapterConformanceSuite` every existing adapter does. The interesting mapping question is the inverse of the usual case: instead of mapping PyAgent patterns onto an existing multi-agent execution engine, an adapter here would need to *generate* the Python delegation/hand-off/graph code Pydantic AI expects from PyAgent's declared workflow pattern — closer to code generation than to wrapping an existing orchestration API. ## Where Pydantic AI is the right choice today If you want type-safe, single-agent configuration with Pydantic's validation guarantees, and are comfortable keeping multi-agent orchestration logic in Python via `pydantic-graph`, use Pydantic AI directly. There's currently no `pyagent-blueprint` path onto it. This page will be updated with real, verified adapter code — following the same standard as the LangGraph and CrewAI pages — if that mapping work happens. # Kubernetes for agent orchestration Kubernetes didn't invent containers, and it didn't replace Docker's runtime. What it did was introduce a declarative layer — a manifest describing the *desired state* of a deployment — that a control loop reconciles against reality, independent of which container runtime actually executes it. `pyagent-blueprint`'s manifest model for multi-agent systems follows the same shape, and it's worth being explicit about where the analogy holds and where it doesn't. ## Where the analogy holds **A manifest describing desired state, not imperative steps.** A Kubernetes `Deployment` YAML declares "3 replicas of this image, this resource limit" — not the sequence of `docker run` commands to get there. A `pyagent-blueprint` manifest declares "these agents, wired into this named pattern, with this budget and this recovery policy" — not the Python loop that calls each LLM in order. **A pluggable execution layer behind a stable contract.** Kubernetes runs on containerd, CRI-O, or Docker interchangeably through the Container Runtime Interface. `pyagent-blueprint` runs the same manifest through any `RuntimeAdapter` — the native `pyagent` pattern registry, a LangGraph `StateGraph`, a CrewAI `Crew`, a Semantic Kernel `Kernel` — through the same `RuntimeAdapter` contract, verified in this repo by a shared `AdapterConformanceSuite`. **Diff and review as first-class operations.** `kubectl diff` shows exactly what a manifest change would alter before it's applied. `pyagent-blueprint diff` does the same over the blueprint IR — a real artifact for a PR reviewer, not a description of what changed. **Declarative governance, enforced or diagnosed — never silently ignored.** A Kubernetes `PodDisruptionBudget` or resource `limits:` block either gets enforced by the scheduler or the pod fails to schedule — it doesn't just get ignored. A Blueprint's `contracts.*.sla` or `observability.cost_budget` gets enforced by the target adapter, or the compiler emits a stable diagnostic (`BUDGET_UNSUPPORTED`, `SLA_UNSUPPORTED`) so you know, deterministically, that this runtime doesn't support it — the failure mode is visibility, not silent drift. **Simulate before you apply.** `kubectl apply --dry-run` validates a manifest against the cluster without changing anything. `pyagent-blueprint test` runs a compiled workflow against a `MockLLM` — validating shape and contract conformance without spending a token or calling a live model. ## Where the analogy breaks down This is not a claim that `pyagent-blueprint` reimplements a scheduler, reconciliation loops, or cluster state — it compiles a static manifest to a runtime once, per invocation; it does not continuously reconcile a running system against a declared target the way the Kubernetes control plane does. There's no equivalent yet of a controller that detects an agent has drifted from its declared contract at runtime and self-heals. And unlike Kubernetes' CRI, which is a mature, multi-vendor standard, `RuntimeAdapter` is this project's own contract — proven against five structurally different adapters, but not (yet) an externally governed spec the way CRI is. The [Agent Spec interop bridge](https://pyagent.org/compare/vs-langgraph/index.md) work is aimed at closing that gap by mapping onto Oracle's Agent Spec, an actual cross-vendor standard, rather than staying a project-local convention indefinitely. ## The practical upshot The value isn't the YAML syntax — it's the guarantee that comes with it: a manifest you can validate, diff, and test before it ever touches a real model, and a contract that means switching execution engines is an `AdapterRegistry` lookup, not a rewrite. That's the same value proposition Kubernetes brought to container deployment, applied to multi-agent orchestration. # What is an agent blueprint? An **agent blueprint** is a declarative document — YAML or JSON — that fully describes a multi-agent system: which agents exist, how they're wired into named orchestration patterns, which LLM providers back them, what contracts (input/output shape, SLAs) they must satisfy, and what governance (budgets, memory tiers, guardrails, recovery, human-in-the-loop checkpoints) applies. It's a manifest, not a script — the system's shape is data, not control flow buried in Python. This is **spec-driven development** applied to multi-agent systems: the same discipline that made Terraform and Kubernetes manifests the default way to describe infrastructure, applied to *which agents exist and how they're wired together*. The spec is the source of truth; the runtime is an interchangeable detail the spec doesn't hard-code. ## The anatomy of a blueprint ``` api_version: pyagent/v1 metadata: name: customer-support version: 1.0.0 owner: platform-team providers: # LLM bindings — model, tokens, timeout primary: model: gpt-4.1-mini fallback: model: gpt-4.1-nano agents: # Agent definitions — prompt, provider, guardrails classifier: prompt: "Classify into: billing, tech, general" provider: primary billing: prompt: "Handle billing inquiries" provider: primary guardrails: [pii_redact] workflows: # Named-pattern wiring + recovery policy support: pattern: supervisor agents: classifier: classifier routes: {billing: billing} recovery: max_retries: 2 fallback_provider: fallback contracts: # Input/output schema + SLA support: input: {type: string, max_tokens: 2000} output: {type: string} sla: {latency_p95_ms: 5000, cost_max_usd: 0.05} observability: # Tracing + cost budgets tracing: {enabled: true} cost_budget: {daily_usd: 100.0} ``` Every top-level block maps onto a real, compilable capability, not just documentation: `providers` feeds `pyagent-router`'s routing; `agents`/`workflows` compile to a `RuntimeAdapter`'s `CompiledArtifact`; `contracts` become validated I/O + SLA checks; `observability` wires `pyagent-trace`. ## Why a manifest instead of code A hand-written orchestration script conflates *design* (which agents, wired how) with *implementation* (which SDK, which execution loop). A blueprint separates them: - **Version and diff it like infrastructure.** `pyagent-blueprint diff v1.yaml v2.yaml` shows exactly which agent, route, or SLA changed — a real code review artifact, not a prose changelog. - **Validate before running anything.** Static analysis catches dangling agent references or schema violations before an LLM call happens. - **Compile it onto any registered runtime.** The same manifest runs on the bundled `pyagent` pattern registry, a zero-dependency stdlib adapter, or a third-party adapter for LangGraph, CrewAI, Semantic Kernel, or the OpenAI Agents SDK — see [Why Blueprint?](https://pyagent.org/why-blueprint/index.md) for the full proof. - **Test without spending tokens.** `pyagent-blueprint test` runs contract conformance checks against a `MockLLM`, so CI can validate a system's shape and contract conformance for free. - **Never silently drop governance.** Every declared budget, memory tier, guardrail, recovery policy, or HITL checkpoint is either honored by the target adapter or surfaced as a stable diagnostic code (e.g. `BUDGET_UNSUPPORTED`) — you always know what a runtime actually supports. ## Where a blueprint fits in the stack A blueprint is Pillar 1 of PyAgent's four-pillar spec-driven architecture — Blueprint, Execution & Routing, Context & Memory, and Observability. See the [Blueprint guide](https://pyagent.org/guides/blueprint/index.md) for the full spec format, or [What is multi-agent orchestration?](https://pyagent.org/concepts/multi-agent-orchestration/index.md) for the coordination patterns a blueprint's `workflows:` block declares. # What is multi-agent orchestration? **Multi-agent orchestration** is the practice of coordinating multiple specialized LLM-driven agents — each with a narrower scope than a single do-everything prompt — so they combine into a system that solves a task no individual agent handles well alone. The orchestration layer is the code (or manifest) that decides which agent runs when, what data passes between them, and how failures, budgets, and human review are handled. ## Why not one large prompt? A single, monolithic prompt asked to classify a request, look things up, reason about them, and format a polished answer tends to underperform a set of smaller agents each doing one of those things well — smaller prompts are easier to test, cheaper to run on a weaker model, and easier to debug when something goes wrong, because failures are localized to one step instead of buried in one giant completion. ## The core coordination patterns Most multi-agent systems reduce to a handful of recurring shapes. `pyagent-patterns` names 18 of them, grouped into four families: - **Orchestration** — a controller directs specialist workers: `Supervisor` (classify → route → specialist), `Pipeline` (linear stages), `Fan-Out / Fan-In` (parallel workers, merged), `Hierarchical`, `Orchestrator-Workers`. - **Resolution** — multiple agents converge on a better answer: `Self-Reflection`, `Cross-Reflection`, `Debate`, `Voting`, `Evaluator-Optimizer`. - **Structural** — the topology itself carries meaning: `Role-Based`, `Layered`, `Topology`, `Blackboard`. - **Iterative & Advanced** — looping or emergent coordination: `ReAct`, `Talker-Reasoner`, `Swarm`, `Human-in-the-Loop`. See the full [Design Patterns](https://pyagent.org/packages/patterns/index.md) catalog for runnable examples of each. ## What production orchestration needs beyond "wire some agents together" A prototype can hard-code which agent calls which. A production system additionally needs: - **Provider routing** — which model handles which agent, with fallbacks (`providers` + `pyagent-router`). - **Governed memory** — shared context with trust tiers, expiry, and redaction, not an unbounded transcript (`ContextLedger`). - **Budgets and SLAs** — declared cost/latency ceilings, not implicit ones. - **Guardrails and recovery** — retries, timeouts, and fallback providers when a step fails. - **Observability** — tracing every call and cost across the whole system, not just one agent. - **Human-in-the-loop checkpoints** — a workflow-level pause for approval, not just a single tool-confirmation prompt. ## Declaring orchestration instead of hand-wiring it `pyagent-blueprint` lets you declare all of the above — agents, the named pattern wiring them together, providers, contracts, and observability — in one YAML manifest, validated statically and compiled onto any registered runtime. See [What is an agent blueprint?](https://pyagent.org/concepts/agent-blueprint/index.md) and [Why Blueprint?](https://pyagent.org/why-blueprint/index.md) for how that works in practice. # Architecture Decision Records Each ADR below documents a real, recurring design decision — the reasoning an LLM (or a person) needs to *architect with* PyAgent, not just describe it. Format follows the standard Context/Decision/Consequences shape. - [Pipeline vs. Supervisor](https://pyagent.org/architecture/decisions/pipeline-vs-supervisor/index.md) - [Supervisor vs. Orchestrator-Workers](https://pyagent.org/architecture/decisions/supervisor-vs-orchestrator-workers/index.md) - [Working vs. Session vs. Semantic memory](https://pyagent.org/architecture/decisions/memory-tier-selection/index.md) - [When Blueprint is (and isn't) justified](https://pyagent.org/architecture/decisions/when-blueprint-is-justified/index.md) - [Spec-first vs. code-first architecture](https://pyagent.org/architecture/decisions/spec-first-vs-code-first/index.md) # ADR: Working vs. Session vs. Semantic Memory **Status:** Accepted. ## Context `pyagent-context` ships three memory tiers. Using the wrong one is a common source of either lost state (using a tier that doesn't survive long enough) or unnecessary complexity (using a longer-lived tier than the data needs). ## Decision Match the tier to how long the state actually needs to survive — nothing more: - **`WorkingMemory`** — state only needs to survive within a single agent turn. Cheapest and fastest tier; nothing persists once the turn ends. Don't reach further unless you actually need state to outlive the turn. - **`SessionMemory`** — multiple agents in one run need to share state across turns. Scoped to one run; nothing carries over to the next session automatically. - **`SemanticMemoryProtocol`** — knowledge needs to persist and be retrieved across separate runs/sessions. The built-in `InMemorySemanticStore` doesn't survive a process restart — a durable backend is a protocol implementation you provide. ## Consequences - Defaulting to `SemanticMemoryProtocol` "to be safe" when `WorkingMemory` would do adds a dependency and a persistence concern (what backend? what TTL?) for data that never needed to outlive one turn. - Defaulting to `WorkingMemory` for state that actually needs to survive across a run silently loses that state the moment the turn ends — a correctness bug, not a performance one. - `SessionMemory` gives no durability guarantee beyond the run — if "remember this next week" is a real requirement, `SessionMemory` alone won't satisfy it; you need `SemanticMemoryProtocol` with a real backend. - If no state needs to persist across agents or turns at all, none of the three tiers is needed — see [why-blueprint.md's "when you don't need PyAgent at all"](https://pyagent.org/why-blueprint/#when-you-dont-need-pyagent-at-all). See the [Context & Memory pillar page](https://pyagent.org/architecture/#context-memory) and the [Context guide](https://pyagent.org/guides/context/index.md) for the full API. # ADR: Pipeline vs. Supervisor **Status:** Accepted — both are named `pyagent-patterns` shapes with distinct, non-overlapping `use_when` conditions. ## Context Both patterns run more than one agent in a coordinated way, and both are common defaults teams reach for without checking whether the task's actual shape matches. Picking wrong doesn't fail loudly — it just produces an awkward implementation (a `Supervisor` classifying into one branch every time, or a `Pipeline` with a stage that's really a conditional). ## Decision **Use Pipeline when the task has clear sequential stages, and every input goes through the same stages in the same order** — ETL, document processing, multi-step transformation. Cost: N LLM calls (one per stage); latency is the sum of all stages, since each depends on the previous one's output. **Use Supervisor when tasks fall into distinct categories, and a category-specific specialist is meaningfully better than a generalist** — customer support triage, multi-domain Q&A. Cost: 2–3 LLM calls (classify → specialist → optional formatter); latency is lower than Pipeline for the same apparent complexity, since only one specialist branch executes per input, not every stage. The distinguishing question: **does every input need the same sequence of steps (Pipeline), or does the right sequence depend on what kind of input it is (Supervisor)?** ## Consequences - Picking Pipeline for a categorization task means every input pays for every stage even when most stages are irrelevant to it — wasted cost and latency. - Picking Supervisor for a strictly sequential task means building an artificial classifier whose only job is to always route to the same place — wasted complexity for zero benefit. - If tasks are neither purely sequential nor purely categorical — subtasks aren't known until the goal is analyzed — neither pattern fits; see [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) instead. See the [pattern catalog](https://pyagent.org/packages/patterns/index.md) for the full comparison table and [patterns.json](https://pyagent.org/patterns.json) for the machine-readable `use_when`/`avoid_when` for both. # ADR: Spec-First vs. Code-First Architecture **Status:** Accepted — this is the foundational choice [Why Blueprint?](https://pyagent.org/why-blueprint/index.md) argues for; this ADR states the tradeoff plainly rather than only the case for one side. ## Context Every multi-agent framework picks a point on this spectrum. Fully code-first frameworks (AutoGen's conversable-agent model, hand-written LangGraph graphs) put 100% of the design in Python. Fully spec-first tools (Pydantic AI's single-agent Agent Specs) declare configuration in YAML/JSON but leave orchestration in code. `pyagent-blueprint` puts the *whole system* — agents, workflow pattern, wiring, governance — in the spec. ## Decision **Choose spec-first (Blueprint) when** the system's design needs to be reviewable, diffable, and statically validatable independent of the code that executes it — see [When Blueprint Is (and Isn't) Justified](https://pyagent.org/architecture/decisions/when-blueprint-is-justified/index.md). **Choose code-first when** the orchestration logic is genuinely dynamic (can't be expressed as a static graph), the team is small enough that Python diffs are sufficient review, or the framework you need (AutoGen's group chat, a bespoke agent loop) doesn't have a spec-first path today — see [the AutoGen comparison](https://pyagent.org/compare/vs-autogen/index.md) for a case where no Blueprint adapter exists yet. **Note the false binary:** some frameworks split the difference — Pydantic AI declares one agent's config in YAML while orchestration between agents stays in Python (see [the Pydantic AI comparison](https://pyagent.org/compare/vs-pydantic-ai/index.md)). That's a legitimate third point on the spectrum, not a worse version of either extreme — it fits teams that want typed single-agent config without committing to a spec-first *system* description. ## Consequences - Spec-first buys diff/review/validate-before-run at the cost of an IR that can only express what it was designed to express — dynamic, runtime-computed control flow falls outside it. - Code-first buys unlimited expressiveness at the cost of every design review being a full code review, and no static validation before an LLM is ever called. - The middle position (spec-first single-agent config, code-first multi-agent wiring) buys per-agent type safety without multi-agent-system-level diffability — a real, different tradeoff from either extreme, not simply "less spec-first." See [Why Blueprint?](https://pyagent.org/why-blueprint/index.md) for the full argument and the three comparison pages above for how this plays out against real frameworks. # ADR: Supervisor vs. Orchestrator-Workers **Status:** Accepted. ## Context Both patterns involve a coordinating agent directing work to other agents. The difference is whether the set of "other agents" and their assignments is known in advance or has to be figured out per-task. ## Decision **Use Supervisor when the categories are fixed and known in advance** — a classifier routes each input to one of N known specialists. Cost: 2–3 LLM calls total. **Use Orchestrator-Workers when subtasks aren't known until the goal is analyzed, and the worker pool has meaningfully different specializations the planner assigns dynamically** — open-ended goals like "research this topic" where the number and nature of sub-questions depends on the topic itself. Cost: 1 planning call + N worker calls (N is dynamic, decided by the planner) + 1 synthesis call. The distinguishing question: **can you enumerate the routing categories before seeing the input (Supervisor), or does the input itself determine how many workers are needed and what they should do (Orchestrator-Workers)?** ## Consequences - Supervisor with an unbounded/unknown category set forces an awkward "other" bucket that absorbs everything the classifier can't confidently place — a symptom of the wrong pattern, not a tuning problem. - Orchestrator-Workers for a fixed-category task pays for a planning call that will produce the same plan every time — pure overhead versus Supervisor's direct classify-and-route. - Orchestrator-Workers' dynamic worker count makes cost and latency less predictable than Supervisor's fixed 2–3 calls — a real operational tradeoff if you need tight cost bounds. - If workers need to communicate directly with each other rather than only report to the coordinator, neither pattern fits — see [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md) or [Swarm](https://pyagent.org/packages/patterns/advanced/swarm/index.md) instead. See the [pattern catalog](https://pyagent.org/packages/patterns/index.md) for the full comparison and [patterns.json](https://pyagent.org/patterns.json) for machine-readable `use_when`/`avoid_when`. # ADR: When Blueprint Is (and Isn't) Justified **Status:** Accepted. ## Context `pyagent-blueprint` isn't free — validate/compile/diff/test tooling and a YAML authoring layer are real overhead versus just writing Python. The question is when that overhead is worth it. ## Decision **Blueprint is justified when:** - More than one person needs to review design changes, and a line-level Python diff isn't legible enough for that review — Blueprint's semantic diff (`BREAKING`/`WARNING`/`INFO` severity per change) is built for exactly this. - You want to validate a system's structure before spending a token — dangling references and schema violations caught statically, not at runtime. - Governance requirements (budgets, SLAs, memory tiers, HITL checkpoints) need to be declared once and either honored or surfaced as a diagnostic — never silently dropped by whichever runtime executes it. - You need to swap which framework executes the system without rewriting the design itself. **Blueprint is not justified when:** - It's one person, prototyping, and the design changes every few minutes — the validate/compile cycle is overhead until the shape stabilizes. - The orchestration logic depends on dynamic, runtime-computed control flow that can't be expressed as a static graph — Blueprint's IR models agents, typed workflows, and named patterns; that's a real constraint, not just a feature gap. - It's a single agent, single call, with nothing to hand off to — there's no design to declare in the first place. ## Consequences - Adopting Blueprint for a single-developer prototype adds a validate/compile step to every iteration with no reviewer to benefit from the diff — pure friction until a second stakeholder is actually in the loop. - Not adopting Blueprint once a system has multiple reviewers means design changes get reviewed as prose or as raw Python diffs — the review quality degrades as the system grows, silently. - The "when not justified" cases aren't permanent — a prototype that stabilizes, or a single-dev project that gains a second maintainer, is exactly the point at which the calculus flips. See [Why Blueprint?](https://pyagent.org/why-blueprint/index.md) for the full case, and the [Blueprint pillar page](https://pyagent.org/architecture/blueprint/index.md) for what/when/tradeoffs. # Capability catalog [pyagent.org/patterns.json](https://pyagent.org/patterns.json) covers the 18 named orchestration patterns in `pyagent-patterns`. Everything else PyAgent ships — the other 8 packages — has its own machine-readable catalog at **[pyagent.org/capabilities.json](https://pyagent.org/capabilities.json)**, generated from `data/capabilities.yml` so it can't drift from this page or from the per-package docs. ## What's in it | Package | What's cataloged | | ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [`pyagent-blueprint`](https://pyagent.org/packages/blueprint/index.md) | All 9 `RuntimeAdapter`s (`pyagent`, `single_agent`, `sequential_chain`, `state_machine`, `simple_loop`, `langgraph`, `crewai`, `openai_agents`, `semantic_kernel`) with execution model, exercised capability, and install command for each | | [`pyagent-patterns`](https://pyagent.org/packages/patterns/index.md) | Pointer to the dedicated `patterns.json` catalog | | [`pyagent-router`](https://pyagent.org/packages/router/index.md) | `DifficultyScorer`, `ModelSelector`, `CostEstimator`, `RouterMiddleware` | | [`pyagent-context`](https://pyagent.org/packages/context/index.md) | Three memory tiers (`WorkingMemory`, `SessionMemory`, `SemanticMemoryProtocol`) plus `ContextLedger`, `TrustLevel`, `Sensitivity`, `ContextRedactor`, `TrustAwareRetriever`, `CompressionPolicy` | | [`pyagent-trace`](https://pyagent.org/packages/trace/index.md) | Four exporters (`console`, `jsonl`, `otel`, `langfuse`) plus `TraceEventBus`, `Recorder`, `CostTracker`, `PatternSpanEmitter` | | [`pyagent-compress`](https://pyagent.org/packages/compress/index.md) | `TokenBudget`, `MessageCompressor`, `AgentPruner`, `InteractionPruner`, `CompressMiddleware` | | [`pyagent-providers`](https://pyagent.org/packages/providers/index.md) | Four routing strategies (`capability_first`, `cost_first`, `latency_first`, `round_robin`) plus `ProviderRegistry`, `ProviderRouter`, `FallbackChain`, `CapabilityNegotiator`, `CostOptimizer` | | [`pyagent-studio`](https://pyagent.org/packages/studio/index.md) | Full CLI surface (`apply`, `get`, `validate`, `test`, `diff`, `simulate`, `render`, `generate`, `providers list`, `providers health`, `describe`, `dashboard`) plus the web dashboard's capabilities | | `pyagent-all` | Meta-package pointer | Every id/class/CLI-command name in the catalog is copied verbatim from that package's real public API (`__all__` in its `__init__.py`) or its CLI's actual `@command` names — not invented or aspirational. See `data/capabilities.yml`'s header for the grounding rule, and `aeo/scripts/validate_capabilities_catalog.py` for the automated check that it hasn't drifted from either the source YAML or the real `packages/` directory. Beyond the pattern catalog, most capability entries also carry `use_when`/`avoid_when`/`tradeoffs` fields — the same decision-guidance shape `patterns.json` already had, extended to the non-pattern capabilities (memory tiers, routing strategies, exporters, and so on) that previously had none. ## Why two files instead of one `patterns.json`'s pattern objects (`use_when`/`avoid_when`/`pairs_with`) and `capabilities.json`'s package objects (`routing_strategies`/`memory_tiers`/`exporters`/`cli_commands`) are different enough shapes that merging them into one schema would make both harder to consume. `capabilities.json` links back to `patterns.json` via its own `see_also` field, so a tool only needs one starting URL. # Benchmarks ## Methodology Benchmarks compare all 6 pattern configurations across 4 dimensions: - **Cost** — total tokens × model price per token - **Latency** — wall-clock time (parallel patterns win here) - **Quality** — keyword match score against expected outputs - **Token efficiency** — output quality per token spent All benchmarks use `MockLLM` for deterministic, reproducible results. Real-LLM benchmarks require `OPENAI_API_KEY`. ## Running Benchmarks ``` # All suites PYTHONPATH=packages/pyagent-patterns/src:packages/pyagent-router/src:packages/pyagent-compress/src:packages/pyagent-trace/src \ python -m benchmarks.run # Specific suite python -m benchmarks.run --suite cost python -m benchmarks.run --suite quality python -m benchmarks.run --suite latency python -m benchmarks.run --suite router python -m benchmarks.run --suite compression ``` ## Benchmark Suites ### Cost-Effectiveness Suite Compares token usage for identical tasks across patterns. | Pattern | Avg Tokens/Task | Avg Cost/Task | Notes | | --------------- | --------------- | ------------- | ----------------------------- | | single_agent | 50 | $0.000100 | Baseline | | pipeline_3stage | 150 | $0.000300 | 3× tokens, structured output | | self_reflection | 200 | $0.000400 | 2 rounds generate+critique | | debate | 250 | $0.000500 | 2 rounds × 2 debaters + judge | | voting_3 | 150 | $0.000300 | 3 parallel voters | | fanout_3 | 200 | $0.000400 | 3 parallel + aggregator | ### Quality Suite Multi-pass patterns produce measurably higher quality. | Pattern | Avg Quality Score | Cost Multiplier | Quality/Cost Ratio | | --------------- | ----------------- | --------------- | ------------------ | | single_agent | 60% | 1.0× | Baseline | | pipeline_3stage | 70% | 3.0× | 0.23× | | self_reflection | 85% | 4.0× | 0.21× | | debate | 80% | 5.0× | 0.16× | | voting_3 | 75% | 3.0× | 0.25× | | fanout_3 | 90% | 4.0× | 0.23× | **Key finding:** Voting has the best quality/cost ratio — 0.25× (75% quality at 3.0× cost), since three cheap parallel voters lift quality without a reflection round. Fan-out reaches the highest *absolute* quality (90%) for tasks with independent sub-analyses, at a slightly lower 0.23× ratio. ### Router Savings Suite Difficulty-based routing reduces costs on mixed workloads — savings scale with how many requests fall into the cheaper tiers (the trivial-heavy mix below saves ~92%). | Task Difficulty | Without Router | With Router | Savings | | -------------------- | --------------- | ---------------------- | ------- | | Trivial (1-2) | gpt-4o ($0.003) | gpt-4.1-nano ($0.0001) | 97% | | Easy (3-4) | gpt-4o ($0.003) | gpt-4o-mini ($0.0003) | 90% | | Medium (5-6) | gpt-4o ($0.003) | gpt-4.1-mini ($0.001) | 67% | | Hard (7-10) | gpt-4o ($0.003) | gpt-4o ($0.003) | 0% | | **Mixed (70/20/10)** | **$0.003** | **$0.00023** | **92%** | ### Compression Suite Token compression reduces inter-agent transfer costs. | Compression Ratio | Token Reduction | Quality Preserved | Cost Savings | | ----------------- | --------------- | ----------------- | ------------ | | 70% (light) | 30% | 99% | 30% | | 50% (medium) | 50% | 95% | 50% | | 30% (aggressive) | 70% | 85% | 70% | **Recommended:** 50% target ratio for best quality/savings tradeoff. ## Combined Savings Using Router + Compression together on a 5-agent pipeline: | Configuration | Tokens | Cost | vs Baseline | | --------------------------------- | ------ | ------ | ----------- | | Baseline (gpt-4o, no compression) | 25,000 | $0.125 | — | | + Router (auto-select models) | 25,000 | $0.050 | -60% | | + Compression (50%) | 12,500 | $0.025 | -80% | | + Both | 12,500 | $0.010 | **-92%** | # Contributing ## Development Setup ``` git clone https://github.com/pyagent/pyagent.git cd pyagent # Install all packages in development mode pip install -e packages/pyagent-patterns[dev] pip install -e packages/pyagent-router[dev] pip install -e packages/pyagent-compress[dev] pip install -e packages/pyagent-trace[dev] ``` ## Running Tests ``` # All tests PYTHONPATH=packages/pyagent-patterns/src:packages/pyagent-router/src:packages/pyagent-compress/src:packages/pyagent-trace/src \ python -m pytest packages/ -v # Specific package python -m pytest packages/pyagent-patterns/tests/ -v ``` ## Code Style - **Ruff** for linting and formatting (configured in root `pyproject.toml`) - **mypy** for type checking (strict mode) - **async-first** — all pattern `_execute` methods are async - Docstrings follow Google style ``` ruff check packages/ ruff format packages/ mypy packages/pyagent-patterns/src/ ``` ## Adding a New Pattern 1. Create `packages/pyagent-patterns/src/pyagent_patterns//.py` 1. Subclass `Pattern` and implement `pattern_type` property and `_execute()` method 1. Export from `/__init__.py` 1. Register in `registry.py` 1. Add tests in `packages/pyagent-patterns/tests/test_.py` 1. Add docs page in `docs/packages/patterns//.md` with: - Mermaid sequence diagram - Code example - When to Use / Avoid table - Cost-effectiveness table ## Adding a Cookbook Example The [Cookbook](https://pyagent.org/cookbook/index.md) is a growing library of complete, runnable multi-agent recipes organized by domain. To add one: 1. Create `docs/cookbook//.md` (e.g. `docs/cookbook/finance-trading/portfolio-review.md`). 1. Give it a **prompt-shaped title** — phrase the `# H1` and the frontmatter `description` the way a developer would search or prompt for it (e.g. *"How to build a multi-agent portfolio review workflow in Python"*). 1. Add `summary`, `complexity`, and prefixed `tags:` on **three axes** (Domain / Pattern / Package) so the recipe appears in the [filterable Cookbook browser](https://pyagent.org/cookbook/index.md) and on the matching pattern pages: ``` --- description: "How to build a multi-agent portfolio review workflow in Python with PyAgent." summary: "Analyst panel with an evaluator-optimizer quality gate" complexity: Intermediate # Beginner | Intermediate | Advanced tags: - "Domain: Finance & Trading" - "Pattern: Supervisor" - "Pattern: Evaluator-Optimizer" - "Package: pyagent-patterns" --- ``` 1. Follow the example template: **problem statement → pattern(s) used → full runnable code (open with the exact `pip install` and `import` lines) → expected output / OTel trace → Related examples / Patterns used cross-links.** 1. Run `python scripts/gen_docs.py` so the recipe is added to the Cookbook browser and the "Cookbook recipes" section of every pattern it uses. 1. Run `DISABLE_MKDOCS_2_WARNING=true mkdocs build --strict` — it must pass (catches broken links). Use real package names only (`pyagent-patterns`, `pyagent-all`, …) so the examples stay copy-paste-runnable. ## Documentation ``` pip install mkdocs-material mkdocstrings[python] mkdocs-redirects mkdocs-llmstxt mkdocs serve # Preview at http://localhost:8000 ``` ### Generated pages The benchmark tables on `docs/benchmarks.md`, the filterable recipe browser on `docs/cookbook/index.md`, and the "Cookbook recipes" sections on each pattern page are **generated** — the regions between `` / `` markers are computed from `data/benchmarks.yml` and each recipe's frontmatter, so tables, cards, and cross-references can't drift. Edit the data source (or the recipe), not the generated region, then regenerate: ``` python scripts/gen_docs.py # rewrite generated regions python scripts/gen_docs.py --check # CI uses this; fails if out of sync ``` ## Package Structure ``` pyagent/ ├── packages/ │ ├── pyagent-patterns/ # Core: 18 patterns + composites + guardrails + recovery │ ├── pyagent-router/ # Difficulty scoring + model selection │ ├── pyagent-compress/ # Message compression + token budgets │ └── pyagent-trace/ # OTel spans + cost tracking + replay ├── docs/ # MkDocs Material site ├── mkdocs.yml └── pyproject.toml # Workspace root ``` # Cookbook # Cookbook Complete, runnable **multi-agent orchestration** examples — each one a self-contained recipe you can copy, paste, and run. Every example shows the exact `pip install`, the full code, and the expected output. **Filter the recipes below** by domain, design pattern, or package — pick any combination (multiple picks in one group match *any*; across groups they match *all*). ### [Ci Cd Validation](https://pyagent.org/cookbook/blueprint-ops/ci-cd-validation/index.md) Beginner Validate a blueprint and semantically diff it against main in a GitHub Actions PR check Blueprint OpsSupervisorpyagent-blueprint ### [Contract Testing](https://pyagent.org/cookbook/blueprint-ops/contract-testing/index.md) Beginner Run pyagent-blueprint's built-in MockLLM contract tests in CI before any real model call Blueprint OpsPipelinepyagent-blueprint ### [Governance In Yaml](https://pyagent.org/cookbook/blueprint-ops/governance-in-yaml/index.md) Intermediate Declare SLA/budget and recovery policy in YAML; see the stable diagnostic a non-supporting adapter reports Blueprint OpsSupervisorpyagent-blueprint ### [Customer Onboarding Workflow](https://pyagent.org/cookbook/customer-support/onboarding/index.md) Intermediate Verification, setup, FAQ, and success roles onboard a new customer end to end ``` graph TD verification[verification] account_setup[account_setup] faq[faq] success[success] ``` Customer SupportRole-Basedpyagent-patterns ### [Support Router](https://pyagent.org/cookbook/customer-support/support-router/index.md) Advanced Classify tickets → route to specialists → escalate to a human ``` graph TD supervisor[supervisor] billing_fast[billing_fast] billing_deep[billing_deep] tech_fast[tech_fast] tech_deep[tech_deep] account_fast[account_fast] account_deep[account_deep] escalation_writer[escalation_writer] supervisor -->|supervisor| billing_deep billing_deep -->|supervisor| tech_deep tech_deep -->|supervisor| account_deep account_deep -->|supervisor| escalation_writer supervisor -->|billing| billing_deep supervisor -->|technical| tech_deep supervisor -->|account| account_deep supervisor -->|escalate| escalation_writer ``` Customer SupportSupervisorTalker-ReasonerHuman-in-the-Looppyagent-patternspyagent-router ### [Analytics Task Decomposer](https://pyagent.org/cookbook/data-analytics/analytics-decomposer/index.md) Intermediate Orchestrator breaks an analytics request into query, transform, and chart workers ``` graph TD analytics_lead[analytics_lead] query[query] transform[transform] chart[chart] ``` Data & AnalyticsOrchestrator-Workerspyagent-patterns ### [SQL Analyst](https://pyagent.org/cookbook/data-analytics/sql-analyst/index.md) Intermediate Reason-act loop that writes and runs SQL to answer questions ``` graph TD sql_analyst[sql_analyst] ``` Data & AnalyticsReActpyagent-patternspyagent-router ### [Incident Triage](https://pyagent.org/cookbook/devops-sre/incident-triage/index.md) Intermediate Stage pipeline that triages incidents with human sign-off ``` graph TD log_analyst[log_analyst] root_cause[root_cause] remediation[remediation] runbook_writer[runbook_writer] ``` DevOps & SREPipelineHuman-in-the-Looppyagent-patterns ### [Product Launch Planner](https://pyagent.org/cookbook/ecommerce-retail/product-launch-planner/index.md) Intermediate Orchestrator splits a launch into pricing/copy/SEO/inventory subtasks for workers ``` graph TD launch_lead[launch_lead] pricing[pricing] copywriter[copywriter] seo[seo] inventory[inventory] ``` E-commerce & RetailOrchestrator-Workerspyagent-patterns ### [Essay Grading by Consensus](https://pyagent.org/cookbook/education/essay-grader/index.md) Intermediate Independent graders score against a rubric; a majority vote sets the grade ``` graph TD grader_openai[grader_openai] grader_anthropic[grader_anthropic] grader_gemini[grader_gemini] grammar_grader[grammar_grader] ``` Education & TrainingVotingpyagent-patterns ### [AML Transaction Monitoring](https://pyagent.org/cookbook/finance-trading/aml-monitoring/index.md) Advanced Screen → score → enrich pipeline gates high-risk transactions to a human reviewer ``` graph TD rule_screener[rule_screener] risk_scorer[risk_scorer] enrichment[enrichment] sar_drafter[sar_drafter] ``` Finance & TradingPipelineHuman-in-the-Looppyagent-patterns ### [ESG Report Analyzer](https://pyagent.org/cookbook/finance-trading/esg-analyzer/index.md) Advanced Orchestrator dispatches ratings, disclosure, and SFDR-scoring workers into an ESG summary ``` graph TD esg_lead[esg_lead] ratings[ratings] disclosure_extractor[disclosure_extractor] sfdr_scorer[sfdr_scorer] controversy[controversy] ``` Finance & TradingOrchestrator-Workerspyagent-patterns ### [Earnings Call Analyzer](https://pyagent.org/cookbook/finance-trading/earnings-call/index.md) Intermediate A self-reflecting agent iterates on an earnings transcript until the analysis clears a quality bar ``` graph TD earnings_analyst[earnings_analyst] ``` Finance & TradingSelf-Reflectionpyagent-patterns ### [Loan Origination Workflow](https://pyagent.org/cookbook/finance-trading/loan-origination/index.md) Advanced Document → income → credit → approval agents pass an application along a chain ``` graph TD document_collection[document_collection] income_verification[income_verification] credit_scoring[credit_scoring] approval[approval] ``` Finance & TradingTopologypyagent-patterns ### [Loan Underwriting Committee](https://pyagent.org/cookbook/finance-trading/loan-underwriting/index.md) Advanced Approve and decline advocates debate an application; a senior underwriter decides ``` graph TD approve_advocate[approve_advocate] decline_advocate[decline_advocate] senior_underwriter[senior_underwriter] ``` Finance & TradingDebatepyagent-patterns ### [Portfolio Review](https://pyagent.org/cookbook/finance-trading/portfolio-review/index.md) Intermediate Analyst panel with an evaluator-optimizer quality gate ``` graph TD router[router] equities[equities] rates[rates] risk[risk] writer[writer] reviewer[reviewer] router -->|supervisor| equities equities -->|supervisor| rates rates -->|supervisor| risk router -->|equity| equities router -->|fixed_income| rates router -->|risk| risk writer -->|evaluator_optimizer| reviewer ``` Finance & TradingSupervisorEvaluator-Optimizerpyagent-patternspyagent-blueprint ### [Robo-Advisor Onboarding](https://pyagent.org/cookbook/finance-trading/robo-advisor/index.md) Intermediate Intake, risk, suitability, and planning roles rotate to build a client investment plan ``` graph TD intake[intake] risk_profiler[risk_profiler] suitability[suitability] planner[planner] ``` Finance & TradingRole-Basedpyagent-patterns ### [Trading Signal Desk](https://pyagent.org/cookbook/finance-trading/trading-signals/index.md) Advanced Momentum, mean-reversion, and sentiment agents fan out then aggregate into a trade idea ``` graph TD momentum[momentum] mean_reversion[mean_reversion] sentiment[sentiment] signal_aggregator[signal_aggregator] ``` Finance & TradingFan-Out / Fan-Inpyagent-patterns ### [Wealth Rebalancing Crew](https://pyagent.org/cookbook/finance-trading/wealth-rebalancing/index.md) Intermediate Risk → market scan → allocation → compliance pipeline proposes a rebalance ``` graph TD risk_profiler[risk_profiler] market_scanner[market_scanner] allocation_strategist[allocation_strategist] compliance_checker[compliance_checker] ``` Finance & TradingPipelinepyagent-patterns ### [Emergent NPC World](https://pyagent.org/cookbook/gaming/npc-world/index.md) Advanced NPC agents read/write a shared world-state blackboard for emergent behavior ``` graph TD explorer[explorer] builder[builder] trader[trader] chronicler[chronicler] ``` Gaming & SimulationBlackboardpyagent-patterns ### [Policy Briefing Pipeline](https://pyagent.org/cookbook/government/policy-briefing/index.md) Advanced A director delegates to department leads and analysts, then synthesizes a brief ``` graph TD policy_director[policy_director] economics_lead[economics_lead] fiscal_analyst[fiscal_analyst] labour_analyst[labour_analyst] legal_lead[legal_lead] statute_analyst[statute_analyst] rights_analyst[rights_analyst] ``` Government & Public SectorHierarchicalpyagent-patterns ### [CV Screener](https://pyagent.org/cookbook/hr-recruiting/cv-screener/index.md) Intermediate Parallel reviewers score a CV; aggregate into one verdict ``` graph TD skills[skills] experience[experience] collaboration[collaboration] panel[panel] ``` HR & RecruitingFan-Out / Fan-Inpyagent-patterns ### [Clinical Summary](https://pyagent.org/cookbook/healthcare/clinical-summary/index.md) Intermediate Summarize clinical notes, then self-critique for accuracy ``` graph TD extractor[extractor] drafter[drafter] summary_reviewer[summary_reviewer] ``` Healthcare & Life SciencesPipelineSelf-Reflectionpyagent-patterns ### [Contract Review](https://pyagent.org/cookbook/legal-compliance/contract-review/index.md) Intermediate Draft + independent reviewer loop over contract clauses ``` graph TD counsel[counsel] partner[partner] counsel -->|cross_reflection| partner ``` Legal & ComplianceCross-Reflectionpyagent-patterns ### [Regulatory Compliance Checker](https://pyagent.org/cookbook/legal-compliance/compliance-checker/index.md) Advanced A director delegates obligation and control teams, then synthesizes a gap analysis ``` graph TD compliance_director[compliance_director] obligations_lead[obligations_lead] controls_lead[controls_lead] requirement_extractor[requirement_extractor] scope_analyst[scope_analyst] policy_mapper[policy_mapper] evidence_checker[evidence_checker] ``` Legal & ComplianceHierarchicalpyagent-patterns ### [Campaign Planner](https://pyagent.org/cookbook/marketing-content/campaign-planner/index.md) Beginner Parallel channel ideas aggregated into one campaign plan ``` graph TD email[email] social[social] blog[blog] campaign_director[campaign_director] ``` Marketing & ContentFan-Out / Fan-Inpyagent-patterns ### [Writers' Room](https://pyagent.org/cookbook/media-entertainment/writers-room/index.md) Intermediate Showrunner, writer, editor, and continuity roles collaborate on an episode outline ``` graph TD showrunner[showrunner] staff_writer[staff_writer] script_editor[script_editor] continuity[continuity] network_exec[network_exec] ``` Media & EntertainmentRole-Basedpyagent-patterns ### [Property Valuation Stack](https://pyagent.org/cookbook/real-estate/property-valuation/index.md) Intermediate Data → comparables → narrative layers produce a valuation report ``` graph TD property_facts[property_facts] market_facts[market_facts] comps[comps] adjustments[adjustments] report_writer[report_writer] reviewer[reviewer] ``` Real Estate & PropTechLayeredpyagent-patterns ### [Literature Review Team](https://pyagent.org/cookbook/research-analysis/literature-review/index.md) Advanced A lead delegates discovery and synthesis sub-teams to produce a cited review ``` graph TD research_lead[research_lead] discovery_lead[discovery_lead] synthesis_lead[synthesis_lead] source_finder[source_finder] relevance_triage[relevance_triage] findings_extractor[findings_extractor] citation_writer[citation_writer] ``` Research & AnalysisHierarchicalpyagent-patterns ### [Research Assistant](https://pyagent.org/cookbook/research-analysis/research-assistant/index.md) Advanced Gather sources, debate, synthesize, and cite ``` graph TD web_agent[web_agent] academic_agent[academic_agent] industry_agent[industry_agent] optimist[optimist] sceptic[sceptic] judge[judge] synthesizer[synthesizer] ``` Research & AnalysisReActFan-Out / Fan-InDebatePipelinepyagent-patternspyagent-compress ### [Lead Qualifier](https://pyagent.org/cookbook/sales-crm/lead-qualifier/index.md) Intermediate Score and route inbound leads to the right play ``` graph TD scorer[scorer] account_exec[account_exec] nurture[nurture] cold_hold[cold_hold] scorer -->|supervisor| account_exec account_exec -->|supervisor| nurture nurture -->|supervisor| cold_hold scorer -->|hot| account_exec scorer -->|warm| nurture scorer -->|cold| cold_hold ``` Sales & CRMSupervisorPipelinepyagent-patterns ### [Peer-Review Mesh](https://pyagent.org/cookbook/scientific/peer-review/index.md) Advanced Reviewers critique a manuscript in a mesh, reconciling notes across rounds ``` graph TD methodology_reviewer[methodology_reviewer] novelty_reviewer[novelty_reviewer] clarity_reviewer[clarity_reviewer] stats_reviewer[stats_reviewer] ``` Scientific & AcademicTopologypyagent-patterns ### [Alert Triage](https://pyagent.org/cookbook/security/log-triage/index.md) Intermediate Triage security alerts; escalate true positives to a human ``` graph TD enricher[enricher] correlator[correlator] classifier[classifier] case_writer[case_writer] ``` Security & Threat IntelPipelineHuman-in-the-Looppyagent-patterns ### [Fraud Investigation Assistant](https://pyagent.org/cookbook/security/fraud-investigation/index.md) Advanced A reason-act agent calls transaction, anomaly, and sanctions tools to build a case ``` graph TD fraud_analyst[fraud_analyst] ``` Security & Threat IntelReActpyagent-patterns ### [Code Review](https://pyagent.org/cookbook/software-engineering/code-review/index.md) Advanced Iterative review + security scan with a human gate ``` graph TD code_agent[code_agent] review_agent[review_agent] security_agent[security_agent] escalation_agent[escalation_agent] code_agent -->|cross_reflection| review_agent ``` Software EngineeringCross-ReflectionPipelineHuman-in-the-Looppyagent-patternspyagent-router ### [Software Startup Simulation](https://pyagent.org/cookbook/software-engineering/startup-simulation/index.md) Advanced PM, Architect, Engineer, and QA role-play an idea into PRD, design, code, and tests ``` graph TD product_manager[product_manager] architect[architect] engineer[engineer] qa[qa] ``` Software EngineeringRole-Basedpyagent-patterns ### [Trip-Planning Swarm](https://pyagent.org/cookbook/travel-hospitality/trip-planner/index.md) Advanced Self-organizing agents negotiate flights, lodging, and activities into an itinerary ``` graph TD flights[flights] lodging[lodging] activities[activities] budget[budget] local_guide[local_guide] ``` Travel & HospitalitySwarmpyagent-patterns # How to Validate and Diff Blueprints in CI Treat a blueprint manifest like any other piece of infrastructure: validate it statically and diff it semantically as part of every pull request, before a reviewer even opens the YAML. **Patterns used:** Supervisor (the underlying workflow being validated — the CI steps below work for any workflow's manifest) ______________________________________________________________________ ## Architecture ``` flowchart LR PR[Pull Request] --> V[pyagent-blueprint validate] V -->|fails| F[❌ Block merge] V -->|passes| D[pyagent-blueprint diff main...HEAD] D --> C[Post diff as PR comment] C --> R[Human review] ``` ______________________________________________________________________ ## Implementation `.github/workflows/blueprint-ci.yml`: ``` name: Blueprint CI on: pull_request: paths: - "blueprints/**.yaml" jobs: validate-and-diff: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: fetch-depth: 0 # need main's history for the diff step - name: Set up Python uses: actions/setup-python@v5 with: python-version: "3.11" - name: Install pyagent-blueprint run: pip install pyagent-blueprint - name: Validate every changed blueprint run: | git diff --name-only origin/main...HEAD -- 'blueprints/*.yaml' | while read -r f; do echo "Validating $f" pyagent-blueprint validate "$f" done - name: Semantic diff against main run: | git diff --name-only origin/main...HEAD -- 'blueprints/*.yaml' | while read -r f; do git show "origin/main:$f" > /tmp/old.yaml 2>/dev/null || echo "api_version: pyagent/v1" > /tmp/old.yaml echo "### Diff for $f" >> diff_output.md pyagent-blueprint diff /tmp/old.yaml "$f" >> diff_output.md done - name: Post diff as PR comment uses: actions/github-script@v7 with: script: | const fs = require('fs'); const body = fs.readFileSync('diff_output.md', 'utf8'); github.rest.issues.createComment({ issue_number: context.issue.number, owner: context.repo.owner, repo: context.repo.repo, body, }); ``` Both `pyagent-blueprint validate` and `pyagent-blueprint diff` are real CLI commands — `validate` runs Pydantic schema validation plus static analysis (dangling agent references, etc.) and exits non-zero on any error-severity issue; `diff` produces a semantic diff over the blueprint IR (which agent, route, provider, or SLA actually changed), not a line-level text diff of YAML. ______________________________________________________________________ ## When to Use | Situation | Use this recipe? | | --------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | | Blueprints live in version control and go through PR review | ✅ Yes | | You want reviewers to see *what changed semantically*, not a YAML text diff | ✅ Yes | | You have no CI system at all yet | ⚠️ Adapt the two CLI calls to your CI provider — the commands themselves are provider-agnostic | ______________________________________________________________________ ## Cost Profile Zero LLM cost — `validate` and `diff` are pure static analysis over the manifest, no model calls. ______________________________________________________________________ ## See Also - [What is an agent blueprint?](https://pyagent.org/concepts/agent-blueprint/index.md) - [Contract testing with MockLLM](https://pyagent.org/cookbook/blueprint-ops/contract-testing/index.md) - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Contract-Test Blueprints with MockLLM Before spending a single token on a live model, `pyagent-blueprint test` runs a blueprint's declared contracts (input/output shape, SLAs) against `MockLLM` — a deterministic stand-in that exercises the exact same compiled workflow graph without any network call. **Patterns used:** Pipeline (the underlying workflow being tested — this recipe applies to any workflow's manifest) ______________________________________________________________________ ## Architecture ``` flowchart LR B[blueprint.yaml] --> L[load_blueprint] L --> T[BlueprintTester] T --> M[MockLLM
deterministic, no network] M --> R{Contract satisfied?} R -->|yes| P[✓ pass] R -->|no| F[✗ fail — schema/SLA violation] ``` ______________________________________________________________________ ## Implementation ``` pyagent-blueprint test research-pipeline.yaml ``` This is a real CLI command (`cli.py`'s `test` subcommand) — it loads the blueprint, instantiates `pyagent_blueprint.tester.BlueprintTester`, and runs `tester.test(spec)` against `MockLLM`, printing a pass/fail summary and exiting non-zero if any contract fails. Wire it into CI the same way as validation: ``` # .github/workflows/blueprint-ci.yml (continued from ci-cd-validation.md) - name: Contract-test every changed blueprint run: | git diff --name-only origin/main...HEAD -- 'blueprints/*.yaml' | while read -r f; do pyagent-blueprint test "$f" done ``` To call the same tester directly from Python (e.g. inside a pytest suite alongside your own application tests): ``` import asyncio from pyagent_blueprint.loader import load_blueprint from pyagent_blueprint.tester import BlueprintTester spec = load_blueprint("research-pipeline.yaml") tester = BlueprintTester() results = asyncio.run(tester.test(spec)) print(tester.summary(results)) assert all(r.passed for r in results) ``` ______________________________________________________________________ ## When to Use | Situation | Use this recipe? | | ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | | You want to catch a broken contract (schema, SLA) before deploying | ✅ Yes | | You want CI feedback without incurring live API cost | ✅ Yes | | You need to test actual model *output quality*, not just contract shape | ❌ `MockLLM` is deterministic — pair this with a small live-model smoke test suite for output-quality checks | ______________________________________________________________________ ## Cost Profile Zero LLM cost — `MockLLM` never makes a network call. This is exactly why it belongs in CI, run on every PR, rather than gated behind a manual "run the expensive test suite" step. ______________________________________________________________________ ## See Also - [CI/CD validation and diff](https://pyagent.org/cookbook/blueprint-ops/ci-cd-validation/index.md) - [Declaring recovery and budgets](https://pyagent.org/cookbook/blueprint-ops/governance-in-yaml/index.md) - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Declare Recovery Policies and Budgets in YAML A blueprint's `workflows.*.recovery` and `observability.cost_budget` blocks are declarative governance — no Python retry loop or cost-tracking code to write. What makes this safe in production is what happens when a target runtime **can't** honor a declared feature: the compiler reports a stable diagnostic code, it never silently drops it. **Patterns used:** Supervisor ______________________________________________________________________ ## Architecture ``` flowchart LR Y[workflows.*.recovery
observability.cost_budget] --> C[adapter.compile] C -->|adapter supports it| H[Honored at runtime] C -->|adapter doesn't| D["CompileDiagnostic
RECOVERY_UNSUPPORTED / BUDGET_UNSUPPORTED"] D --> V[Visible in CompiledArtifact.diagnostics] ``` ______________________________________________________________________ ## Implementation ``` api_version: pyagent/v1 metadata: name: customer-support version: 1.0.0 providers: primary: model: gpt-4.1-mini fallback: model: gpt-4.1-nano agents: classifier: prompt: "Classify into: billing, tech, general" provider: primary billing: prompt: "Handle billing inquiries" provider: primary tech: prompt: "Handle technical support" provider: primary workflows: support: pattern: supervisor agents: classifier: classifier routes: billing: billing tech: tech recovery: max_retries: 2 timeout_seconds: 30 fallback_provider: fallback observability: cost_budget: daily_usd: 100.0 alert_threshold: 0.8 ``` Compile it and inspect the diagnostics on the returned `CompiledArtifact` — using the real `AdapterRegistry` and the exact fixture this recipe is based on (`packages/pyagent-blueprint/tests/fixtures/customer_support.yaml`): ``` from pyagent_blueprint.adapter import AdapterRegistry from pyagent_blueprint.ir import BlueprintIR from pyagent_blueprint.loader import load_blueprint spec = load_blueprint("customer-support.yaml") ir = BlueprintIR.from_spec(spec) for name in ("pyagent", "single_agent"): adapter = AdapterRegistry.discover()[name]() artifact = adapter.compile(ir) print(name, "->", sorted({d.code.code for d in artifact.diagnostics})) ``` Run against the actual fixture, both adapters — including the bundled native `pyagent` one — currently report `RECOVERY_UNSUPPORTED` and `BUDGET_UNSUPPORTED` (alongside `SLA_UNSUPPORTED` from the `contracts.*.sla` block, `MEMORY_TIER_UNSUPPORTED` from `context.memory`, and `GUARDRAIL_UNSUPPORTED` from an agent's `guardrails:` list): the enforcement layer for these specific G2/G8 gaps is intentionally not yet wired into any adapter — this is documented, expected behavior per the engineering roadmap's open gap table, not a bug. The point isn't that one adapter enforces it and another doesn't (today, neither does); it's that **you know, deterministically and by stable code, exactly which declared governance features aren't yet enforced** — rather than the YAML silently having no effect with no way to detect that from your own code. Any future adapter (or the current ones, as enforcement lands) that *does* honor a feature simply omits that code from `artifact.diagnostics` — the same inspection snippet above is how you'd verify that, too. ______________________________________________________________________ ## When to Use | Situation | Use this recipe? | | ----------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | You need retries/fallback and a cost ceiling declared once, checked against every adapter | ✅ Yes | | You're evaluating whether a new runtime adapter is production-ready for your workflow | ✅ Yes — check `artifact.diagnostics` is empty for the features you rely on | | Your governance requirement isn't yet one of the modeled diagnostic codes | ⚠️ See `diagnostics.py`'s registry (`BUDGET_UNSUPPORTED`, `SLA_UNSUPPORTED`, `MEMORY_TIER_UNSUPPORTED`, `CHECKPOINT_UNSUPPORTED`, `RECOVERY_UNSUPPORTED`, `GUARDRAIL_UNSUPPORTED`) for what's currently covered | ______________________________________________________________________ ## Cost Profile Declaring the policy costs nothing — the budget/recovery block is metadata evaluated at compile time. Whether it's *enforced* at runtime depends entirely on the chosen adapter, which is exactly what the diagnostic check above surfaces. ______________________________________________________________________ ## See Also - [Why Blueprint?](https://pyagent.org/why-blueprint/index.md) — the full adapter capability table - [CI/CD validation and diff](https://pyagent.org/cookbook/blueprint-ops/ci-cd-validation/index.md) - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Customer Onboarding Workflow in Python New-customer onboarding spans identity checks, account configuration, first-questions, and a warm hand-off to success. This recipe uses the **Role-Based** pattern — inspired by AutoGen's group-chat onboarding example — so each specialist holds its role and reacts to the others, producing a complete onboarding plan in one coordinated pass. **Patterns used:** Role-Based ______________________________________________________________________ ## Architecture ``` flowchart LR C[New Customer] --> R((Onboarding Room)) R --> V[Verification\nidentity + KYC] R --> S[Account Setup\nplan + config] R --> F[FAQ\nfirst questions] R --> M[Success\nhandoff + next steps] V --> S --> F --> M R --> O[Onboarding Summary] ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.structural import RoleBased from pyagent_providers import AnthropicLLM, OpenAILLM onboarding = RoleBased( agents=[ Agent( "verification", OpenAILLM("gpt-4o-mini"), system_prompt=( "You handle identity verification. Confirm what's needed (email, org, KYC tier) " "and list any missing items. Never request more data than the plan requires." ), ), Agent( "account_setup", OpenAILLM("gpt-4o-mini"), system_prompt=( "You configure the account: plan, seats, integrations, and sensible defaults. " "Produce a concrete setup checklist." ), ), Agent( "faq", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt=( "You answer the customer's first questions clearly and link to the right docs. " "Anticipate the top-3 questions for this plan." ), ), Agent( "success", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "You are the success manager. Summarize the onboarding, define the 30-day " "activation goals, and write a warm hand-off note for the human CSM." ), ), ], rounds=1, ) result = asyncio.run(onboarding.run( "New customer: a 25-person design agency on the Team plan, wants Slack + SSO, " "main goal is client project tracking." )) print(result.output) print(f"Roles: {result.metadata['roles']}") ``` ______________________________________________________________________ ## Expected output ``` ONBOARDING SUMMARY — Northwind Design (Team plan, 25 seats) Verification: email + org confirmed; SSO domain pending IT approval. Setup: 25 seats provisioned, Slack connected, project-tracking template applied. FAQ: answered "how do invites work / SSO setup / mobile app". Success: 30-day goal = 10 active projects; hand-off note sent to CSM Dana. Roles: ['verification', 'account_setup', 'faq', 'success'] ``` For ticket *routing* after onboarding, pair this with the [Support Router](https://pyagent.org/cookbook/customer-support/support-router/index.md). ______________________________________________________________________ ## Customization ### Add a compliance role ``` onboarding.agents.append( Agent("compliance", OpenAILLM("gpt-4o-mini"), system_prompt="Confirm KYC tier, data-residency, and DPA requirements for this plan."), ) ``` ### Automate account creation with tools Give the setup role a [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) tool that calls your provisioning API. ### Localize the welcome ``` onboarding.agents[2].system_prompt += " Write the FAQ answers in the customer's locale." ``` ______________________________________________________________________ ## When to Use | Situation | Use Role-Based? | | ----------------------------------------------------------- | -------------------------------------------------------------------------------------------- | | A fixed set of specialists must collaborate on one customer | ✅ Yes | | You want a single coordinated onboarding artefact | ✅ Yes | | You're routing an inbound ticket to one specialist | ❌ Use [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) | | Each step strictly gates the next with approvals | ❌ Use [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) | ______________________________________________________________________ ## Cost Profile | Driver | Typical model | Avg cost | Volume (1k onboardings/mo) | | ------------------ | --------------------- | ----------- | -------------------------- | | 4 roles × 1 round | mini + haiku + sonnet | $0.006 | $6/mo… ×1k = $6 → ~$180/mo | | **Per onboarding** | mix | **~$0.006** | **~$180/mo** | ______________________________________________________________________ ## See Also - [Role-Based pattern](https://pyagent.org/packages/patterns/structural/role-based/index.md) - [Support Router](https://pyagent.org/cookbook/customer-support/support-router/index.md) — route post-onboarding tickets to specialists - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Customer Support Router in Python A tiered multi-agent customer support system that classifies queries, routes to specialist bots, uses cheap models for simple questions and expensive ones only for complex issues, and escalates to human agents when automated handling fails. **Patterns used:** Supervisor, TalkerReasoner, HumanInTheLoop, BoundedExecution, GuardrailChain, RouterMiddleware ______________________________________________________________________ ## Requirements - **Functional** — classify every incoming query into billing, technical, account, or escalate; route to the right specialist; hand off to a human when automated handling isn't appropriate. - **Non-functional** — most queries should resolve without a senior/expensive model; classification itself needs to be cheap since it runs on every single query. - **Audit** — every routed query should be traceable to the classification decision that sent it there. - **Not required** — no persistent memory across separate support sessions in this recipe (each query is classified independently). ## Architecture decisions | Decision | Why | Why not the alternative | | --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Supervisor** for routing | Categories (billing/technical/account/escalate) are fixed and known in advance. | **Orchestrator-Workers** implies discovering the right specialists per-input; here the categories never change. | | Classifier on `fast`, specialists on `smart` | Classification is a cheap pattern-match; specialist responses need real reasoning quality. | Using `smart` for classification alone would multiply the highest-volume call in the system by ~10x cost for no quality benefit. | | One workflow (`route`), not a nested Supervisor-inside-TalkerReasoner | `Supervisor.routes` expects `Agent` objects, not `Pattern` objects — nesting patterns inside pattern slots isn't supported by the schema. | The `_fast` tier agents are declared but used outside this workflow (e.g. a cost-optimized fallback), documented directly in the blueprint's own comment. | ## Four-pillar mapping | Requirement | Pillar | Capability | | ----------------------------------- | ------------- | ------------------------------------------------------------------------------- | | Classify and route by category | Execution | `Supervisor` pattern | | Track daily routing spend | Observability | `observability.cost_budget` | | Trace each classify/specialist call | Observability | `observability.tracing` | | Escalate to human when needed | Execution | `escalation_writer` agent + downstream `HumanInTheLoop` (Python implementation) | ## Blueprint (declarative form) The real, verified file at `examples/cookbook/customer-support/support_router/blueprint.yaml`, compiled against `PyAgentAdapter` as part of this repo's test suite: ``` api_version: pyagent/v1 metadata: name: support-router version: 1.0.0 description: Classify customer queries, route to specialist bots, escalate to human when needed providers: fast: { model: gpt-4o-mini } smart: { model: claude-sonnet-4-20250514 } agents: supervisor: { provider: fast, prompt: "Classify as: billing, technical, account, escalate." } billing_deep: { provider: smart, prompt: "Senior billing specialist — complex cases." } tech_deep: { provider: smart, prompt: "Senior engineer — deep technical debug." } account_deep: { provider: smart, prompt: "Senior account specialist — SSO, compliance." } escalation_writer: { provider: fast, prompt: "Summarize for human handoff." } workflows: route: pattern: supervisor agents: classifier: supervisor routes: { billing: billing_deep, technical: tech_deep, account: account_deep, escalate: escalation_writer } observability: tracing: { enabled: true } cost_budget: { daily_usd: 200.0, alert_threshold: 0.8 } ``` ``` pyagent-blueprint validate support-router.yaml pyagent-blueprint test support-router.yaml ``` ## Production checklist Ran this exact blueprint through `PyAgentAdapter.compile()` and inspected the real diagnostics: - ✅ **The routing workflow runs as declared** — `route` compiles and executes with no diagnostics on the workflow structure itself. - ⚠️ **`observability.cost_budget` is declared but not auto-enforced** — compiling emits `BUDGET_UNSUPPORTED`: the $200/day budget is recorded but not enforced. Wire real enforcement via `graph.wire_cost_tracker(tracker)`. - **The `_fast` tier agents and the TalkerReasoner tiering shown in the Python implementation below aren't expressed in the blueprint's `route` workflow** — `Supervisor.routes` requires `Agent` targets, not nested patterns, so the cost-tiering happens in code, not the declared spec. A real, documented schema limitation, not an oversight. ______________________________________________________________________ ## Architecture ``` flowchart TD C[Customer Query] --> G[Input Guardrails\nPII redaction] G --> S[Supervisor\nClassify intent] S -->|billing| B[TalkerReasoner\nBilling Bot] S -->|technical| T[TalkerReasoner\nTech Support] S -->|account| A[TalkerReasoner\nAccount Bot] S -->|escalate| H[Human-in-the-Loop] B -->|easy| BR1[Fast Response\ngpt-4o-mini] B -->|complex| BR2[Deep Response\nclaude-sonnet] T -->|easy| TR1[Quick Fix\ngpt-4o-mini] T -->|complex| TR2[Full Debug\nclaude-sonnet] H --> HR[Human Agent\nvia ticket queue] ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent, Message from pyagent_patterns.orchestration import Supervisor from pyagent_patterns.advanced import TalkerReasoner, HumanInTheLoop from pyagent_patterns.advanced.human_in_the_loop import HumanDecision from pyagent_patterns.recovery import BoundedExecution from pyagent_patterns.guardrails import GuardrailChain, PIIGuard, LengthGuard from pyagent_router.middleware import RouterMiddleware from pyagent_providers import AnthropicLLM, OpenAILLM # ── LLMs ────────────────────────────────────────────────────────────────────── fast_llm = OpenAILLM("gpt-4o-mini") smart_llm = AnthropicLLM("claude-sonnet-4-20250514") model_registry = { "gpt-4o-mini": fast_llm, "claude-sonnet-4-20250514": smart_llm, } router = RouterMiddleware(model_registry=model_registry) # ── Guardrails ───────────────────────────────────────────────────────────────── input_guard = GuardrailChain([ LengthGuard(max_chars=3_000, truncate=True), PIIGuard(redact=True), # protect customer PII in logs ]) # ── Tier 1: Billing bot (TalkerReasoner) ───────────────────────────────────── billing_bot = TalkerReasoner( talker=router.wrap( Agent("billing_fast", fast_llm, system_prompt=( "You are a billing support agent. Answer quickly and clearly. " "You handle: invoice questions, payment methods, refund policies, " "subscription changes, and pricing. Be concise — 2-3 sentences max." )), ), reasoner=router.wrap( Agent("billing_deep", smart_llm, system_prompt=( "You are a senior billing specialist. Handle complex cases: " "disputed charges, partial refunds, multi-seat subscription adjustments, " "enterprise billing questions. Provide step-by-step resolution." )), ), handoff_threshold=5, # difficulty ≥ 5 goes to reasoner ) # ── Tier 2: Technical support bot ──────────────────────────────────────────── tech_bot = TalkerReasoner( talker=router.wrap( Agent("tech_fast", fast_llm, system_prompt=( "You are a tech support agent. Handle common issues: login problems, " "password resets, browser compatibility, basic integrations. " "Give step-by-step instructions." )), ), reasoner=router.wrap( Agent("tech_deep", smart_llm, system_prompt=( "You are a senior engineer doing technical support. Handle complex issues: " "API integration failures, webhook debugging, performance problems, " "data sync issues, custom configurations. Ask clarifying questions if needed." )), ), handoff_threshold=4, ) # ── Tier 3: Account management bot ─────────────────────────────────────────── account_bot = TalkerReasoner( talker=router.wrap( Agent("account_fast", fast_llm, system_prompt=( "You are an account support agent. Handle: username changes, " "email updates, team member management, permissions, and SSO setup." )), ), reasoner=router.wrap( Agent("account_deep", smart_llm, system_prompt=( "You are a senior account specialist. Handle: GDPR data requests, " "account mergers, complex permission structures, enterprise SSO, " "data export and deletion requests." )), ), handoff_threshold=6, ) # ── Tier 4: Human escalation ────────────────────────────────────────────────── def queue_human_review(output: str, metadata: dict) -> HumanDecision: """Route to human agent via your support queue (e.g. Zendesk, Linear).""" ticket_id = _create_support_ticket( summary=output[:200], priority="high" if "urgent" in output.lower() else "normal", metadata=metadata, ) print(f"[TICKET CREATED] #{ticket_id}") # Return a holding response while human picks it up return HumanDecision( approved=True, modified_output=( f"I've escalated your issue to our specialist team. " f"Ticket #{ticket_id} has been created. " f"You'll hear back within 2 business hours." ), ) human_handler = HumanInTheLoop( agent=router.wrap( Agent("human_prep", fast_llm, system_prompt=( "Prepare a concise summary for the human support agent: " "1. Customer issue (1 sentence) " "2. What was already tried " "3. Recommended action" )), ), review_fn=queue_human_review, high_risk_keywords=["legal", "lawsuit", "fraud", "hacked", "emergency"], ) # ── Classifier (Supervisor routing) ────────────────────────────────────────── classifier = Agent( "classifier", fast_llm, system_prompt=( "Classify customer support queries into exactly one category. " "Reply with ONLY the category name.\n" "Categories:\n" " billing — payments, invoices, refunds, subscriptions\n" " technical — bugs, errors, API, integrations, performance\n" " account — login, permissions, team, SSO, data requests\n" " escalate — angry customers, legal threats, complex edge cases, " " anything you're unsure about" ), ) supervisor = Supervisor( classifier=classifier, routes={ "billing": billing_bot, "technical": tech_bot, "account": account_bot, "escalate": human_handler, }, ) # ── Recovery wrapper ────────────────────────────────────────────────────────── safe_supervisor = BoundedExecution( pattern=supervisor, fallback=Agent( "fallback_agent", fast_llm, system_prompt=( "You are a helpful support agent. A technical issue occurred with our " "routing system. Apologize briefly and ask the customer to try again " "or contact support@example.com." ), ), max_retries=2, timeout_seconds=25.0, ) # ── Main handler ────────────────────────────────────────────────────────────── async def handle_query(customer_query: str) -> dict: # Guardrail check check = input_guard.check(customer_query) if not check.passed: return {"response": "I'm sorry, I couldn't process your message.", "blocked": True} safe_query = check.sanitized_content or customer_query # Run through support workflow result = await safe_supervisor.run(safe_query) return { "response": result.output, "category": result.metadata.get("route"), "model_used": result.metadata.get("routed_model", "unknown"), "recovery_level": result.metadata.get("recovery_level", 0), "escalated": result.metadata.get("route") == "escalate", } def _create_support_ticket(summary: str, priority: str, metadata: dict) -> str: """Create a Zendesk ticket and return the ticket number as a string.""" import httpx, os r = httpx.post( os.environ["ZENDESK_URL"] + "/api/v2/tickets.json", json={"ticket": {"subject": summary[:80], "priority": priority, "comment": {"body": summary}}}, auth=(os.environ["ZENDESK_EMAIL"] + "/token", os.environ["ZENDESK_TOKEN"]), timeout=15.0, ) r.raise_for_status() return str(r.json()["ticket"]["id"]) # ── Run it ──────────────────────────────────────────────────────────────────── if __name__ == "__main__": queries = [ "Why was I charged twice this month?", "My API webhooks stopped firing after the latest update", "I need to delete all my data immediately under GDPR", "This is unacceptable, I'm calling my lawyer", ] async def demo(): for query in queries: print(f"\nQuery: {query}") result = await handle_query(query) print(f"Category: {result['category']}") print(f"Model: {result['model_used']}") print(f"Response: {result['response'][:120]}...") asyncio.run(demo()) ``` ______________________________________________________________________ ## Expected Output ``` Query: Why was I charged twice this month? Category: billing Model: gpt-4o-mini ← easy billing question, fast model Response: This can happen if two subscriptions are active simultaneously or if a payment retry was processed. To confirm, please check your invoices at Settings → Billing → Invoice history... Query: My API webhooks stopped firing after the latest update Category: technical Model: claude-sonnet-4-20250514 ← complex technical issue, smart model Response: Webhook failures after an update are often caused by: 1) SSL certificate changes, 2) payload format changes in the new version, 3) endpoint timeout thresholds. Let's debug this step by step: First, check your webhook logs... Query: I need to delete all my data immediately under GDPR Category: account Model: claude-sonnet-4-20250514 ← GDPR = high difficulty, smart model Response: We take GDPR requests seriously. Here's the process: 1) Submit a formal Data Subject Access Request at privacy.example.com/dsar. 2) We'll confirm receipt within 24 hours. 3) Deletion completes within 30 days per GDPR Art. 17... Query: This is unacceptable, I'm calling my lawyer Category: escalate Model: gpt-4o-mini + human Response: I've escalated your issue to our specialist team. Ticket #SUP-847291 has been created. You'll hear back within 2 business hours. [TICKET CREATED] #SUP-847291 ``` ______________________________________________________________________ ## Customization ### Add a knowledge base tool ``` from pyagent_patterns.advanced import ReAct def search_kb(query: str) -> str: # Search your Confluence/Notion/Help Center return kb_client.search(query, top_k=3) tech_bot_with_kb = ReAct( agent=Agent("tech_kb", smart_llm, system_prompt="Answer using the knowledge base."), tools={"search_kb": search_kb}, max_steps=3, ) ``` ### Multi-turn conversation ``` conversation_history: list[Message] = [] async def chat(user_message: str) -> str: conversation_history.append(Message.user(user_message)) result = await safe_supervisor.run(conversation_history) conversation_history.append(Message.assistant(result.output)) return result.output ``` ### SLA-based routing ``` def route_by_sla(query: str, customer_tier: str) -> str: if customer_tier == "enterprise": return "escalate" # always get human for enterprise if "urgent" in query.lower() or "down" in query.lower(): return "escalate" return None # let the classifier decide ``` ______________________________________________________________________ ## Cost Profile | Query type | Typical model | Avg cost | Volume (1k/day) | | ------------------- | ------------------- | -------------------- | ------------------ | | Simple billing | gpt-4o-mini | $0.0003 | $9/mo | | Complex billing | claude-sonnet | $0.003 | $90/mo | | Simple tech | gpt-4o-mini | $0.0003 | $9/mo | | Complex tech | claude-sonnet | $0.004 | $120/mo | | GDPR / legal | claude-sonnet | $0.005 | $150/mo | | Human escalation | gpt-4o-mini + human | $0.0003 + agent time | $9/mo + agent time | | **Blended average** | mix | **~$0.001** | **~$30/mo** | Routing saves ~70% vs always using claude-sonnet for everything. ______________________________________________________________________ ## See Also - [Supervisor pattern](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) - [TalkerReasoner pattern](https://pyagent.org/packages/patterns/advanced/talker-reasoner/index.md) - [HumanInTheLoop pattern](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) - [Guardrails Guide](https://pyagent.org/guides/guardrails/index.md) - [Recovery Guide](https://pyagent.org/guides/recovery/index.md) # How to Build a Multi-Agent Analytics Task Decomposer in Python An open-ended analytics request ("show me why churn rose last quarter") needs planning: which queries, which transforms, which charts. This recipe uses the **Orchestrator-Workers** pattern — a planner decomposes the request into the subtasks it actually needs, dispatches them to specialist workers, and synthesizes the answer. **Patterns used:** Orchestrator-Workers ______________________________________________________________________ ## Requirements - **Functional** — accept an open-ended analytics question, plan the subtasks it actually requires (query/transform/chart, not necessarily all three), dispatch to specialist workers, and synthesize a final answer. - **Non-functional** — the planner should only invoke the workers a specific question actually needs — a question needing no chart shouldn't pay for one. - **Audit** — the final answer should be traceable to which worker(s) the planner invoked and why. - **Not required** — no persistent memory across separate analytics questions; each is planned and answered independently. ## Architecture decisions | Decision | Why | Why not the alternative | | ----------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Orchestrator-Workers**, not Pipeline | The subtasks an analytics question needs aren't known until the question itself is analyzed — "why did churn rise" might need query+transform+chart, "what's our MRR" might need only query. | A **Pipeline** would force every question through query→transform→chart regardless of whether all three are relevant, wasting cost on unneeded stages. | | Single planner (`analytics_lead`) owns both decomposition and synthesis | The same agent that understood the original question is best positioned to judge whether the workers' outputs actually answer it. | Splitting planning and synthesis into separate agents would need the synthesis agent to re-derive intent from the plan alone, risking drift from the original question. | | All three workers on `fast`, planner on `smart` | Query-writing, transform-description, and chart-recommendation are narrow, well-specified tasks; only the planning/synthesis step benefits from stronger reasoning. | Using `smart` for the workers too would multiply cost for tasks that don't need the extra reasoning quality. | ## Four-pillar mapping | Requirement | Pillar | Capability | | ----------------------------------- | ------------- | ----------------------------- | | Dynamic subtask planning + dispatch | Execution | `OrchestratorWorkers` pattern | | Track daily analytics spend | Observability | `observability.cost_budget` | | Trace planning + worker calls | Observability | `observability.tracing` | ## Blueprint (declarative form) The real, verified file at `examples/cookbook/data-analytics/analytics_decomposer/blueprint.yaml`, compiled against `PyAgentAdapter` as part of this repo's test suite: ``` api_version: pyagent/v1 metadata: name: analytics-decomposer version: 1.0.0 description: Orchestrator breaks analytics request into query, transform, and chart workers providers: fast: { model: gpt-4o-mini } smart: { model: claude-sonnet-4-20250514 } agents: analytics_lead: { provider: smart, prompt: "Plan analytics work. Assign workers. Synthesize answer." } query: { provider: fast, prompt: "Write SQL for the subtask." } transform: { provider: fast, prompt: "Describe transforms needed for the analysis." } chart: { provider: fast, prompt: "Recommend chart type and encodings." } workflows: analyze: pattern: orchestrator_workers agents: { orchestrator: analytics_lead, workers: [query, transform, chart] } observability: tracing: { enabled: true } cost_budget: { daily_usd: 50.0, alert_threshold: 0.8 } ``` ``` pyagent-blueprint validate analytics-decomposer.yaml pyagent-blueprint test analytics-decomposer.yaml ``` ## Production checklist Ran this exact blueprint through `PyAgentAdapter.compile()` and inspected the real diagnostics: - ✅ **The workflow runs as declared** — `analyze` compiles and executes against the native pattern registry with no diagnostics. - ⚠️ **`observability.cost_budget` is declared but not auto-enforced** — compiling emits `BUDGET_UNSUPPORTED`: the $50/day budget is recorded but not enforced. Wire real enforcement via `graph.wire_cost_tracker(tracker)`. - **Worker count and cost are inherently less predictable than a fixed pipeline** — because the planner decides which of the 3 workers to invoke per-question, cost varies question-to-question; if you need tight cost bounds, budget for the worst case (all 3 workers), not the average. - **No actual SQL execution or chart rendering happens in this blueprint** — `query`/`transform`/ `chart` produce *descriptions* of what to do; wiring them to a real database/plotting library is downstream work this recipe doesn't cover. ______________________________________________________________________ ## Architecture ``` flowchart TD R[Analytics Request] --> O[Orchestrator\nPlan subtasks] O -->|assign| Q[Query Worker] O -->|assign| T[Transform Worker] O -->|assign| C[Chart Worker] Q --> S[Orchestrator\nSynthesise answer] T --> S C --> S S --> A[Analysis + Visuals] ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import OrchestratorWorkers from pyagent_providers import AnthropicLLM, OpenAILLM analytics = OrchestratorWorkers( orchestrator=Agent( "analytics_lead", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "You plan analytics work. From the request, decide which workers are needed and what to " 'assign each — skip any that don\'t apply. Respond as JSON: ' '{"assignments": [{"worker": "name", "subtask": "description"}]}. After workers return, ' "synthesize a clear answer with the numbers and a recommended chart." ), ), workers=[ Agent( "query", OpenAILLM("gpt-4o-mini"), system_prompt="Write the SQL to answer the subtask. State assumptions about the schema.", ), Agent( "transform", OpenAILLM("gpt-4o-mini"), system_prompt="Describe the transforms (joins, aggregations, cohorting) the analysis needs.", ), Agent( "chart", OpenAILLM("gpt-4o-mini"), system_prompt="Recommend the best chart type and encodings for the result, and why.", ), ], ) result = asyncio.run(analytics.run( "Why did customer churn rise in Q3? Break it down by plan tier and tenure." )) print(result.output) print(f"Workers used: {result.metadata['workers_used']}") ``` ______________________________________________________________________ ## Expected output ``` CHURN ANALYSIS — Q3 Query: monthly churn by plan_tier × tenure_bucket (SQL provided). Transform: cohort by signup month; rolling 3-month churn rate. Finding: churn rose from 3.1% → 4.4%, concentrated in Basic-tier accounts <6 months. Chart: small-multiples line chart (one panel per tier) — shows the Basic spike clearly. Workers used: ['query', 'transform', 'chart'] ``` The planner skips workers a request doesn't need, so a simple "count active users" question costs one worker, not three. ______________________________________________________________________ ## Customization ### Add a chart worker ``` analytics.workers.append( Agent("viz", OpenAILLM("gpt-4o-mini"), system_prompt="Given the result shape, output a Vega-Lite spec for the recommended chart."), ) ``` ### Constrain the SQL dialect ``` analytics.workers[0].system_prompt += " Target Snowflake SQL; use QUALIFY and window functions where helpful." ``` ### Hand off to a tool-using analyst For questions that need live data, route to the [SQL Analytics Assistant](https://pyagent.org/cookbook/data-analytics/sql-analyst/index.md) (ReAct + tools). ______________________________________________________________________ ## When to Use | Situation | Use Orchestrator-Workers? | | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The needed subtasks depend on the request | ✅ Yes | | You want one synthesized analysis from specialists | ✅ Yes | | Every request runs the same fixed stages | ❌ Use [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) | | One agent should reason-and-run queries with tools | ❌ Use [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) (see [SQL Analyst](https://pyagent.org/cookbook/data-analytics/sql-analyst/index.md)) | ______________________________________________________________________ ## Cost Profile | Stage | Typical model | Avg cost | Volume (10k requests/mo) | | -------------------------------- | ------------- | ----------- | ------------------------ | | Orchestrator (plan + synthesize) | claude-sonnet | $0.005 | $50/mo | | Workers (up to 3) | gpt-4o-mini | $0.0009 | $9/mo | | **Per request** | mix | **~$0.006** | **~$60/mo** | ______________________________________________________________________ ## See Also - [Orchestrator-Workers pattern](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) - [SQL Analyst](https://pyagent.org/cookbook/data-analytics/sql-analyst/index.md) — a single ReAct agent that writes and runs SQL - [Product Launch Planner](https://pyagent.org/cookbook/ecommerce-retail/product-launch-planner/index.md) — the same pattern for retail - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent SQL Analytics Assistant in Python "Which three regions grew revenue fastest last quarter?" shouldn't require knowing the schema. This recipe uses the **ReAct** pattern: the agent reasons, inspects the schema, writes SQL, runs it as a tool, and reads the result — looping until it can answer in plain English — with difficulty-aware **routing** so simple lookups use a cheap model and gnarly analysis uses a strong one. **Patterns used:** [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Router](https://pyagent.org/guides/router/index.md) ______________________________________________________________________ ## Architecture ``` flowchart TD Q[Question] --> R[ReAct Agent\nrouter-wrapped] R -->|reason → act| T1[list_tables] R -->|reason → act| T2[describe_table] R -->|reason → act| T3[run_sql] T1 --> R T2 --> R T3 --> R R --> A[Plain-English Answer + SQL] ``` ______________________________________________________________________ ## Implementation ``` pip install pyagent-patterns pyagent-router pyagent-providers ``` ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.advanced import ReAct from pyagent_router.middleware import RouterMiddleware from pyagent_providers import AnthropicLLM # ── Tools the agent can call (replace with your warehouse client) ─────────────── def list_tables() -> str: """Return the available tables.""" return "tables: sales(region, amount, closed_at), customers(id, region, tier)" def describe_table(name: str) -> str: """Return a table's columns.""" schemas = {"sales": "region TEXT, amount NUMERIC, closed_at DATE"} return schemas.get(name.strip(), f"unknown table: {name}") def run_sql(query: str) -> str: """Execute read-only SQL and return rows as text.""" if not query.strip().lower().startswith("select"): return "ERROR: only SELECT statements are allowed" return "rows: [('APAC', 0.31), ('LATAM', 0.27), ('EMEA', 0.12)]" # ← your DB call here # ── Difficulty-aware routing: cheap model for lookups, strong for analysis ────── router = RouterMiddleware(model_registry={ "claude-haiku-3-5-20241022": AnthropicLLM("claude-haiku-3-5-20241022"), "claude-sonnet-4-20250514": AnthropicLLM("claude-sonnet-4-20250514"), }) analyst = ReAct( agent=router.wrap( Agent( "sql_reasoner", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "You answer data questions. Inspect the schema before writing SQL, run only SELECT " "queries, and base every claim on returned rows. When done, give a 2-sentence answer " "and show the SQL you ran. End with FINISH." ), ) ), tools={"list_tables": list_tables, "describe_table": describe_table, "run_sql": run_sql}, max_steps=6, ) async def main(): result = await analyst.run("Which 3 regions grew revenue fastest last quarter?") print(result.output) print(f"Steps: {result.metadata['steps']}, tools used: {result.metadata['tools_used']}") asyncio.run(main()) ``` ______________________________________________________________________ ## Expected Output ``` The fastest-growing regions last quarter were APAC (+31%), LATAM (+27%), and EMEA (+12%). SQL: SELECT region, (sum(amount) FILTER (WHERE closed_at >= '2025-07-01') / sum(amount) FILTER (WHERE closed_at < '2025-07-01')) - 1 AS growth FROM sales GROUP BY region ORDER BY growth DESC LIMIT 3; Steps: 4, tools used: ['list_tables', 'describe_table', 'run_sql'] ``` The agent decides *when* to inspect the schema vs. run a query based on what each step returns — and the `run_sql` tool enforces read-only access so a hallucinated `DROP` can never execute. ______________________________________________________________________ ## Customization ### Hard read-only + row caps The `run_sql` guard above rejects non-`SELECT`; add a `LIMIT` injector and a statement timeout in your DB client so a runaway query can't pin the warehouse. ### Inject the live schema ``` def describe_table(name: str) -> str: return your_warehouse.get_columns(name) # real introspection instead of a static map ``` ### Chart the result Feed the answer into a charting agent with [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) — see the [Analytics Task Decomposer](https://pyagent.org/cookbook/data-analytics/analytics-decomposer/index.md). ______________________________________________________________________ ## When to Use | Situation | Fit | | ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | The agent must call tools and react to results | ✅ ReAct | | Mixed simple/complex questions, cost matters | ✅ Router | | The set of analytics subtasks is planned up front | ❌ Use [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) | | A fixed query → transform → chart sequence | ❌ Use [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) | ______________________________________________________________________ ## Cost Profile | Question type | Routed model | Avg cost | Notes | | ------------------------ | ------------- | ----------- | --------------------------- | | Simple lookup | claude-haiku | $0.001 | 1-2 steps | | Multi-step analysis | claude-sonnet | $0.012 | 4-6 steps | | **Blended per question** | mix | **~$0.005** | routing keeps lookups cheap | `max_steps` caps the worst case; routing keeps the common case (simple lookups) on the cheap model. ______________________________________________________________________ ## See Also - [ReAct pattern](https://pyagent.org/packages/patterns/advanced/react/index.md) · [Router guide](https://pyagent.org/guides/router/index.md) - [Analytics Task Decomposer](https://pyagent.org/cookbook/data-analytics/analytics-decomposer/index.md) — plan query/transform/chart subtasks - [Fraud Investigation Assistant](https://pyagent.org/cookbook/security/fraud-investigation/index.md) — another tool-using ReAct agent - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Incident Triage Pipeline in Python At 3 a.m. the bottleneck is reading logs and forming a hypothesis fast — but you never want an agent taking production actions on its own. This recipe runs a **Pipeline** that summarizes the error signal, hypothesizes a root cause, and drafts a *reversible* remediation, then uses **Human-in-the-Loop** to require an on-call engineer's approval before anything that touches prod. **Patterns used:** [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) ______________________________________________________________________ ## Requirements - **Functional** — summarize the error signal, hypothesize a root cause, draft a reversible remediation, and format it as a runbook step for on-call approval. - **Non-functional** — triage (log analysis through remediation drafting) should run at machine speed; only the actual production-touching action waits on a human. - **Audit** — the runbook step presented for approval must show its full reasoning chain: what failed, the hypothesized cause, and why the remediation is believed safe. - **Not required** — no automatic execution of the remediation — this recipe stops at a human-approvable runbook step, it never touches production itself. ## Architecture decisions | Decision | Why | Why not the alternative | | -------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | | **Pipeline** for triage | Every incident goes through the same fixed sequence — analyze logs, hypothesize cause, draft remediation — regardless of the specific failure. | **Supervisor** would imply routing to different specialists by incident category; here every incident gets identical staged treatment. | | **Human-in-the-Loop** gates only the final runbook step, not the whole triage | Log analysis and root-cause hypothesis are safe to automate fully; only the step that could touch production needs a human. | Gating the entire pipeline on human review would eliminate the speed benefit that matters most at 3 a.m. | | `runbook_writer` explicitly flags `HIGH_RISK` for delete/drop/scale-to-zero/failover/restart | These are the specific action classes that are hard or impossible to reverse — the flag exists to make that risk visible to the approving human, not just implicit in the prose. | Leaving risk classification implicit would put the burden on the on-call engineer to infer it from free text under time pressure. | ## Four-pillar mapping | Requirement | Pillar | Capability | | ---------------------------------------------- | ------------- | --------------------------- | | Staged log analysis → root cause → remediation | Execution | `Pipeline` pattern | | Human sign-off before prod-touching action | Execution | `HumanInTheLoop` pattern | | Track daily triage spend | Observability | `observability.cost_budget` | | Trace each triage stage | Observability | `observability.tracing` | ## Blueprint (declarative form) The real, verified file at `examples/cookbook/devops-sre/incident_triage/blueprint.yaml`, compiled against `PyAgentAdapter` as part of this repo's test suite: ``` api_version: pyagent/v1 metadata: name: incident-triage version: 1.0.0 description: Pipeline that triages incidents with human sign-off before prod changes providers: fast: { model: gpt-4o-mini } smart: { model: claude-sonnet-4-20250514 } agents: log_analyst: { provider: fast, prompt: "Summarize error signal: what's failing, since when, blast radius." } root_cause: { provider: smart, prompt: "Most likely root cause with supporting evidence." } remediation: { provider: smart, prompt: "Safe reversible remediation. Start with TOUCHES_PROD: yes/no." } runbook_writer: { provider: fast, prompt: "Format as runbook step. Flag HIGH_RISK for delete/drop/scale-to-zero/failover/restart." } workflows: triage: pattern: pipeline agents: { stages: [log_analyst, root_cause, remediation] } gate: pattern: human_in_the_loop agents: { agent: runbook_writer } observability: tracing: { enabled: true } cost_budget: { daily_usd: 50.0, alert_threshold: 0.8 } ``` ``` pyagent-blueprint validate incident-triage.yaml pyagent-blueprint test incident-triage.yaml ``` ## Production checklist Ran this exact blueprint through `PyAgentAdapter.compile()` and inspected the real diagnostics: - ✅ **Both workflows run as declared** — `triage` and `gate` each compile and execute against the native pattern registry with no diagnostics on workflow structure. - ⚠️ **`observability.cost_budget` is declared but not auto-enforced** — compiling emits `BUDGET_UNSUPPORTED`: the $50/day budget is recorded but not enforced. Wire real enforcement via `graph.wire_cost_tracker(tracker)`. - **The `TOUCHES_PROD`/`HIGH_RISK` flagging is prompt-driven, not schema-enforced** — the blueprint declares the agents that produce these flags, but nothing in the spec itself guarantees a malformed or missing flag is caught before reaching the human approver. That's a real gap if false negatives on risk classification are a concern for your deployment. - **No recovery policy is declared** — if a stage fails mid-run (e.g. the LLM call errors), this blueprint doesn't specify a retry policy. ______________________________________________________________________ ## Architecture ``` flowchart TD L[Logs + Alert] --> A[Log Analyst\nsummarize signal] A --> R[Root Cause\nhypothesis] R --> M[Remediation\nreversible plan] M --> H[Human-in-the-Loop\non-call approval] H -->|approved| X[Execute] H -->|rejected| D[Discard + note] ``` ______________________________________________________________________ ## Implementation ``` pip install pyagent-patterns pyagent-providers ``` ``` import asyncio, os import httpx from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Pipeline from pyagent_patterns.advanced import HumanInTheLoop from pyagent_patterns.advanced.human_in_the_loop import HumanDecision from pyagent_providers import AnthropicLLM, OpenAILLM fast_llm = OpenAILLM("gpt-4o-mini") smart_llm = AnthropicLLM("claude-sonnet-4-20250514") # ── Triage pipeline: analyze → root cause → remediation ───────────────────────── triage = Pipeline(stages=[ Agent("log_analyst", fast_llm, system_prompt="Summarize the error signal from these logs: what's failing, since when, blast radius."), Agent("root_cause", smart_llm, system_prompt="Given the summary, give the single most likely root cause with supporting evidence."), Agent("remediation", smart_llm, system_prompt=( "Propose a safe, reversible remediation with exact steps and a rollback. " "Begin with TOUCHES_PROD: yes/no on the first line." )), ]) # ── Human gate: any prod-touching remediation needs on-call approval ──────────── def on_call_gate(output: str, metadata: dict) -> HumanDecision: if output.lower().startswith("touches_prod: no"): return HumanDecision(approved=True, modified_output=f"Auto-applied (non-prod):\n{output}") approved = asyncio.get_event_loop().run_until_complete(_page_on_call_and_wait(output)) return HumanDecision(approved=approved, modified_output=output if approved else "REJECTED by on-call — escalate to IC.") triage_with_gate = HumanInTheLoop( agent=Agent("runbook_writer", fast_llm, system_prompt="Format the remediation as a runbook step the on-call engineer can approve."), review_fn=on_call_gate, high_risk_keywords=["delete", "drop", "scale to zero", "failover", "restart prod"], ) SAMPLE_INCIDENT = ( "ALERT: checkout 5xx rate 12% for 8 min. Logs: 'connection pool exhausted' on payments-svc; " "db connections pinned at 100/100; deploy of payments-svc 14 min ago." ) async def main(): triaged = await triage.run(SAMPLE_INCIDENT) final = await triage_with_gate.run(triaged.output) print(final.output) async def _page_on_call_and_wait(summary: str, timeout_s: float = 300.0) -> bool: """Post incident to PagerDuty and poll for on-call approval. Returns True = approved.""" routing_key = os.environ["PAGERDUTY_ROUTING_KEY"] async with httpx.AsyncClient(timeout=30.0) as client: r = await client.post( "https://events.pagerduty.com/v2/enqueue", json={ "routing_key": routing_key, "event_action": "trigger", "payload": {"summary": summary[:200], "severity": "critical", "source": "pyagent"}, }, ) r.raise_for_status() dedup_key = r.json()["dedup_key"] deadline = asyncio.get_event_loop().time() + timeout_s while asyncio.get_event_loop().time() < deadline: resp = await client.get( f"https://api.pagerduty.com/incidents?incident_key={dedup_key}", headers={"Authorization": f"Token token={os.environ['PAGERDUTY_API_KEY']}"}, ) incidents = resp.json().get("incidents", []) if incidents and incidents[0].get("status") == "acknowledged": return True await asyncio.sleep(5.0) return False # timed out — escalate manually asyncio.run(main()) ``` ______________________________________________________________________ ## Expected Output ``` TOUCHES_PROD: yes Root cause: the 14-min-ago payments-svc deploy lowered the DB pool ceiling; pool now saturated → 5xx. Remediation: 1. Raise payments-svc DB pool 100 → 250 (config flag, no restart). 2. If still saturated in 3 min, roll back the payments-svc deploy. Rollback: revert the pool flag; redeploy prior payments-svc image. [PAGED on-call for approval — prod change] ``` A non-prod fix (e.g. clearing a stale cache in staging) applies itself; the prod pool change pages a human first — automation for the diagnosis, control for the action. ______________________________________________________________________ ## Customization ### Pull live context with a tool agent Replace the static log analyst with a [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) agent that queries your logging/metrics APIs — see the [Fraud Investigation Assistant](https://pyagent.org/cookbook/security/fraud-investigation/index.md). ### Auto-open a postmortem doc ``` from pyagent_patterns.orchestration import Pipeline postmortem = Agent("postmortem", fast_llm, system_prompt="Draft a blameless postmortem from the triage result.") triage_plus_doc = Pipeline(stages=[triage, postmortem]) ``` ### Tighten the high-risk keyword gate ``` triage_with_gate.high_risk_keywords += ["truncate", "migration", "dns", "iam"] ``` ______________________________________________________________________ ## When to Use | Situation | Fit | | --------------------------------------------- | --------------------------------------------------------------------------------- | | Fixed analyze → root-cause → remediate stages | ✅ Pipeline | | Prod actions must be human-approved | ✅ Human-in-the-Loop | | The agent must query tools mid-investigation | ❌ Use [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) | | Several responders should debate the cause | ❌ Use [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) | ______________________________________________________________________ ## Cost Profile | Stage | Typical model | Avg cost | Volume (200 incidents/mo) | | ------------------------ | ------------- | ------------ | ------------------------- | | Log analyst | gpt-4o-mini | $0.0005 | $0.10 | | Root cause + remediation | claude-sonnet | $0.007 | $1.40 | | **Per incident** | mix | **~$0.0075** | **~$1.50/mo** | Triage cost is negligible next to the minutes of MTTR it saves; the human gate is where the real safety lives. ______________________________________________________________________ ## See Also - [Pipeline pattern](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Human-in-the-Loop pattern](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) - [Alert Triage](https://pyagent.org/cookbook/security/log-triage/index.md) — the same shape for security alerts - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Product Launch Planner in Python A retail product launch touches pricing, marketing copy, SEO, and inventory — and which of those actually need work depends on the product. This recipe uses the **Orchestrator-Workers** pattern: a planner agent inspects the brief, decides which specialist workers to invoke and what to assign each, then synthesizes their output into a single launch plan. **Patterns used:** Orchestrator-Workers ______________________________________________________________________ ## Architecture ``` flowchart TD B[Product Brief] --> O[Orchestrator\nPlan subtasks] O -->|assign| P[Pricing Worker] O -->|assign| C[Copywriter Worker] O -->|assign| S[SEO Worker] O -->|assign| I[Inventory Worker] P --> M[Orchestrator\nSynthesise launch plan] C --> M S --> M I --> M M --> R[Launch Plan] ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import OrchestratorWorkers from pyagent_providers import AnthropicLLM, OpenAILLM launch_planner = OrchestratorWorkers( orchestrator=Agent( "launch_lead", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "You plan e-commerce product launches. From the brief, decide which specialist " "workers are genuinely needed and what to assign each — skip any that don't apply. " 'Respond as JSON: {"assignments": [{"worker": "name", "subtask": "description"}]}. ' "After all workers complete, synthesize their output into one launch plan with a " "go-live checklist." ), ), workers=[ Agent( "pricing", OpenAILLM("gpt-4o-mini"), system_prompt=( "Recommend a launch price and promo strategy. Consider margin targets, " "competitor price bands, and an intro discount. Justify the number." ), ), Agent( "copywriter", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "Write the product title, a 50-word description, and three bullet benefits. " "Match the brand voice in the brief." ), ), Agent( "seo", OpenAILLM("gpt-4o-mini"), system_prompt=( "Produce an SEO pack: 8 target keywords, a 60-char meta title, and a " "155-char meta description. Prioritize high-intent search terms." ), ), Agent( "inventory", OpenAILLM("gpt-4o-mini"), system_prompt=( "Recommend an opening stock quantity and reorder point from the demand " "estimate and lead time in the brief. Flag stockout risk." ), ), ], ) brief = ( "Launch: a reusable stainless-steel water bottle, premium eco brand. " "COGS $9, target margin 60%. Competitors sell $28-$35. Supplier lead time 4 weeks, " "expected first-month demand ~1,200 units." ) result = asyncio.run(launch_planner.run(brief)) print(result.output) print(f"Workers used: {result.metadata['workers_used']}") ``` ______________________________________________________________________ ## Expected output ``` LAUNCH PLAN — EcoFlask Pro Pricing: $29.99 launch price (67% margin) with a 15% first-week intro code. Copy: "EcoFlask Pro — 24-Hour Cold, Zero Plastic" + 3 benefit bullets. SEO: meta title + 8 keywords (insulated water bottle, reusable …). Inventory: open with 1,500 units, reorder point 400 (4-week lead time). Go-live checklist: [pricing approved] [PDP copy live] [meta tags set] [stock received] Workers used: ['pricing', 'copywriter', 'seo', 'inventory'] ``` ______________________________________________________________________ ## Customization ### Add a worker ``` launch_planner.workers.append( Agent("logistics", OpenAILLM("gpt-4o-mini"), system_prompt="Plan fulfilment: carrier, packaging, and per-region shipping SLAs."), ) ``` ### Constrain to a budget ``` launch_planner.orchestrator.system_prompt += " Keep the total launch spend under $25,000 and show the allocation." ``` ### Emit a launch timeline Chain a timeline stage with a [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md): ``` from pyagent_patterns.orchestration import Pipeline gantt = Agent("scheduler", OpenAILLM("gpt-4o-mini"), system_prompt="Turn the plan into a week-by-week launch timeline.") planner_with_timeline = Pipeline(stages=[launch_planner, gantt]) ``` ______________________________________________________________________ ## When to Use | Situation | Use Orchestrator-Workers? | | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | | Which subtasks are needed depends on the input | ✅ Yes — the orchestrator decides at runtime | | You want one synthesized deliverable from many specialists | ✅ Yes | | The set of steps is always the same, in order | ❌ Use [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) | | All workers should run on every input | ❌ Use [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) | ______________________________________________________________________ ## Cost Profile | Stage | Typical model | Avg cost | Volume (1k launches/mo) | | ------------------------------- | ------------- | ----------- | ----------------------- | | Orchestrator (plan + synthesis) | claude-sonnet | $0.006 | $180/mo | | Workers (up to 4) | gpt-4o-mini | $0.0012 | $36/mo | | **Blended per launch** | mix | **~$0.007** | **~$210/mo** | Skipping irrelevant workers keeps cost proportional to the work each launch actually needs. ______________________________________________________________________ ## See Also - [Orchestrator-Workers pattern](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) - [Analytics Task Decomposer](https://pyagent.org/cookbook/data-analytics/analytics-decomposer/index.md) — the same pattern for data work - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Essay Grading System in Python A single LLM grader is noisy and easy to bias. This recipe uses the **Voting** pattern: several grader agents score the same essay independently against the same rubric, and a majority vote produces a defensible final grade — exactly how a panel of human markers reduces individual bias. **Patterns used:** Voting ______________________________________________________________________ ## Architecture ``` flowchart TD E[Student Essay + Rubric] --> V1[Grader A] E --> V2[Grader B] E --> V3[Grader C] V1 --> T[Majority Vote] V2 --> T V3 --> T T --> G[Final Grade + Rationale] ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.resolution import Voting from pyagent_providers import AnthropicLLM, GeminiLLM, OpenAILLM RUBRIC = ( "Grade the essay A, B, C, D, or F using this rubric: thesis clarity, evidence, " "structure, and grammar. Reply with the single letter grade on line 1, then a " "one-sentence justification on line 2." ) grader = Voting( voters=[ Agent("grader_openai", OpenAILLM("gpt-4o"), system_prompt=RUBRIC), Agent("grader_anthropic", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=RUBRIC), Agent("grader_gemini", GeminiLLM("gemini-2.5-pro"), system_prompt=RUBRIC), ], strategy="majority", ) essay = ( "Title: Why Cities Should Invest in Public Transit\n\n" "Public transit reduces traffic, cuts emissions, and connects communities. " "When cities fund buses and trains, fewer cars crowd the roads ... " "(800-word student submission)" ) result = asyncio.run(grader.run(f"{essay}")) print(result.output) print(f"Tally: {result.metadata['tally']}, winner: {result.metadata['winner']}") ``` ______________________________________________________________________ ## Expected output ``` Grade: B Justification: A clear thesis and good structure, but evidence is thin and a few grammar slips weaken otherwise solid reasoning. Tally: {'B': 2, 'A': 1}, winner: B ``` Because three independent graders must converge, a single over-generous or harsh model can't swing the grade on its own — and the tally is an audit trail you can show a student. ______________________________________________________________________ ## Customization ### Weighted graders ``` grader = Voting( voters=[Agent("grader_openai", OpenAILLM("gpt-4o"), system_prompt=RUBRIC), ...], strategy="weighted", weights=[2.0, 1.0, 1.0], # trust the strongest grader more ) ``` ### Add a rubric-specific grader ``` grader.voters.append( Agent("grammar_grader", OpenAILLM("gpt-4o-mini"), system_prompt="Grade ONLY grammar and mechanics A-F. Letter on line 1, reason on line 2."), ) ``` ### Return per-grader transparency The `result.metadata['tally']` and `['votes']` give you each grader's score — surface them to students as an appeal trail. ______________________________________________________________________ ## When to Use | Situation | Use Voting? | | -------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | | You need a robust answer that resists single-model bias | ✅ Yes | | The decision is a discrete choice (grade, label, yes/no) | ✅ Yes — majority/consensus fits | | You want graders to debate and persuade each other | ❌ Use [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) | | One agent should critique and improve a draft | ❌ Use [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) | ______________________________________________________________________ ## Cost Profile | Query type | Typical model | Avg cost | Volume (1k essays/day) | | --------------------- | ---------------------------- | ----------- | ---------------------- | | 3 independent graders | gpt-4o / sonnet / gemini-pro | $0.012 | $360/mo | | **Per essay** | mix | **~$0.012** | **~$360/mo** | Use cheaper models (e.g. three `*-mini` voters) for formative feedback; reserve the premium panel for high-stakes summative grading. ______________________________________________________________________ ## See Also - [Voting pattern](https://pyagent.org/packages/patterns/resolution/voting/index.md) - [Loan Underwriting Committee](https://pyagent.org/cookbook/finance-trading/loan-underwriting/index.md) — a debating panel for credit decisions - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent AML Transaction Monitoring System in Python Anti-money-laundering (AML) monitoring needs to be fast for low-risk transactions and cautious for high-risk ones. This recipe chains two patterns: a **Pipeline** that screens, scores, and enriches each alert — then **Human-in-the-Loop** that routes anything above the risk threshold to a compliance officer before a SAR is filed. **Patterns used:** [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md), [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) ______________________________________________________________________ ## Requirements - **Functional** — screen every transaction alert for rule-based red flags, assign a 0-100 risk score, enrich with counterparty context, and draft a FinCEN SAR narrative for any alert a compliance officer approves. - **Non-functional** — auto-clear low/medium-risk alerts without human involvement; never file a SAR without an officer's explicit approval. - **Audit** — every SAR must be traceable to the specific rule flags and risk score that triggered it — the pipeline's staged output is that trail. - **Not required** — no cross-alert correlation, no persistent case memory between alerts (each alert is evaluated independently). ## Architecture decisions | Decision | Why | Why not the alternative | | -------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | **Pipeline** for triage | Every alert goes through the same fixed sequence — screen, score, enrich — regardless of content. | **Supervisor** would imply routing to different specialists by category; every alert here gets identical treatment, just with different outputs. | | **Human-in-the-Loop** gates only the SAR-filing step, not the whole pipeline | Auto-clearing low-risk alerts at machine speed is the whole point; only the consequential, hard-to-reverse action (filing a SAR) needs a human. | Gating the entire pipeline on human review would eliminate the speed benefit for the ~majority of alerts that are low-risk. | | Two-tier model routing (`fast` for screening/enrichment, `smart` for scoring/drafting) | Rule screening and enrichment are pattern-matching tasks; risk scoring and SAR drafting need the reasoning a stronger model provides. | Using `smart` everywhere triples cost for stages that don't need it; using `fast` everywhere risks under-scoring genuine risk. | ## Four-pillar mapping | Requirement | Pillar | Capability | | ------------------------------------- | ------------- | ------------------------------------------- | | Screen, score, enrich in sequence | Execution | `Pipeline` pattern | | Gate SAR filing on human approval | Execution | `HumanInTheLoop` pattern | | Track daily compliance spend | Observability | `observability.cost_budget` | | Trace each screen/score/enrich call | Observability | `observability.tracing` | | Compress case history to control cost | Context | `context.compression` (`semantic_lossless`) | ## Blueprint (declarative form) The same triage pipeline declared as a `pyagent-blueprint` manifest — this is the real, verified file at `examples/cookbook/finance-trading/aml_monitoring/blueprint.yaml`, compiled against `PyAgentAdapter` as part of this repo's test suite, not a hand-typed illustration. **Honest gap:** the blueprint currently declares only the `triage` pipeline (rule_screener → risk_scorer → enrichment) — the `HumanInTheLoop`-gated SAR-drafting step shown in the Python implementation below isn't yet expressed as a second workflow in the YAML. This recipe is a real example of a pattern this session's own backlog review flagged: Blueprint coverage lagging the hand-written Python it's meant to describe. ``` api_version: pyagent/v1 metadata: name: aml-monitoring version: 1.0.0 description: AML transaction screening pipeline with compliance officer gate providers: fast: { model: gpt-4o-mini } smart: { model: claude-sonnet-4-20250514 } agents: rule_screener: { provider: fast, prompt: "Screen for sanctions, velocity, structuring, jurisdiction flags." } risk_scorer: { provider: smart, prompt: "Assign risk score 0-100 and tier. Explain top drivers." } enrichment: { provider: fast, prompt: "Enrich with counterparty context. One-paragraph summary." } sar_drafter: { provider: smart, prompt: "Draft a FinCEN SAR narrative: subject, activity, dates, amounts." } workflows: triage: pattern: pipeline agents: stages: [rule_screener, risk_scorer, enrichment] observability: tracing: { enabled: true } cost_budget: { daily_usd: 500.0, alert_threshold: 0.8 } context: compression: { policy: semantic_lossless, target_ratio: 0.6 } ``` ``` pyagent-blueprint validate aml-monitoring.yaml pyagent-blueprint test aml-monitoring.yaml ``` ## Production checklist Ran this exact blueprint through `PyAgentAdapter.compile()` and inspected the real diagnostics: - ✅ **The triage pipeline runs as declared** — `triage` compiles and executes against the native pattern registry with no diagnostics on that path. - ⚠️ **`observability.cost_budget` is declared but not auto-enforced** — compiling this blueprint emits `BUDGET_UNSUPPORTED`: the $500/day budget is recorded in the spec but nothing stops a run from exceeding it. Wire real enforcement via `graph.wire_cost_tracker(tracker)`. - ⚠️ **`context.compression` is declared but not auto-wired** — compiling emits `MEMORY_TIER_UNSUPPORTED`: the `semantic_lossless` policy is recorded but not applied. Call `graph.wire_compressor(compressor)` if you need it actually enforced. - **The human-approval gate exists only in the Python implementation, not the blueprint** (see the honest gap noted above) — if you need this declared and diffable, that's real, not-yet-done work, not a documentation oversight. - **No persistent memory tier is declared** — each alert is evaluated independently by design (see Requirements); if cross-alert pattern detection becomes a requirement later, that's a `pyagent-context` `SessionMemory` addition, not a redesign. ______________________________________________________________________ ## Architecture ``` flowchart TD T[Transaction Alert] --> P1[Rule Screener\nblacklist & velocity] P1 --> P2[Risk Scorer\n0-100] P2 --> P3[Enrichment Agent\ncounterparty + geo] P3 --> G{Risk > 70?} G -->|No| A[Auto-clear] G -->|Yes| H[Human Compliance Officer] H -->|Approved SAR| S[File SAR] H -->|Rejected| R[Close alert] ``` ______________________________________________________________________ ## Implementation ``` pip install pyagent-patterns pyagent-providers ``` ``` import asyncio, os from typing import Literal import httpx from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Pipeline from pyagent_patterns.advanced import HumanInTheLoop from pyagent_patterns.advanced.human_in_the_loop import HumanDecision from pyagent_providers import AnthropicLLM, OpenAILLM fast_llm = OpenAILLM("gpt-4o-mini") smart_llm = AnthropicLLM("claude-sonnet-4-20250514") # Stage 1 — triage pipeline aml_pipeline = Pipeline(stages=[ Agent( "rule_screener", fast_llm, system_prompt=( "Screen the transaction for rule-based red flags: sanctions matches, velocity breaches " "(>3 transactions in 1h), structuring (amounts just under $10k), and high-risk jurisdictions. " "Output a bullet list of flags found (or NONE)." ), ), Agent( "risk_scorer", smart_llm, system_prompt=( "Given the rule flags, assign a risk score 0-100 and a risk tier: " "Low (<30), Medium (30-70), High (>70). Explain the top two scoring drivers." ), ), Agent( "enrichment", fast_llm, system_prompt=( "Enrich the alert with counterparty context: known business type, jurisdiction risk, " "prior SAR history (simulate from input). Produce a one-paragraph case summary." ), ), ]) def _parse_tier(output: str) -> Literal["Low", "Medium", "High"]: for tier in ("High", "Medium", "Low"): if f"— {tier}" in output or f"tier: {tier}" in output.lower(): return tier return "Medium" async def _post_case(output: str) -> str: """Submit case to compliance queue; returns ticket_id.""" async with httpx.AsyncClient(timeout=30.0) as client: r = await client.post( os.environ["COMPLIANCE_API_URL"] + "/cases", json={"summary": output[:500], "source": "aml-agent"}, headers={"Authorization": f"Bearer {os.environ['COMPLIANCE_API_KEY']}"}, ) r.raise_for_status() return r.json()["ticket_id"] async def _poll_decision(ticket_id: str, timeout_s: float = 300.0) -> bool: """Poll until a compliance officer approves or rejects. Returns True = approved.""" async with httpx.AsyncClient(timeout=timeout_s) as client: deadline = asyncio.get_event_loop().time() + timeout_s while asyncio.get_event_loop().time() < deadline: r = await client.get( f"{os.environ['COMPLIANCE_API_URL']}/cases/{ticket_id}", headers={"Authorization": f"Bearer {os.environ['COMPLIANCE_API_KEY']}"}, ) data = r.json() if data["status"] in ("approved", "rejected"): return data["status"] == "approved" await asyncio.sleep(5.0) return False # timed out → conservative reject def compliance_review(output: str, metadata: dict) -> HumanDecision: """Route high-risk cases to the compliance API and wait for an officer decision.""" loop = asyncio.get_event_loop() ticket_id = loop.run_until_complete(_post_case(output)) approved = loop.run_until_complete(_poll_decision(ticket_id)) print(f"[COMPLIANCE] ticket={ticket_id} approved={approved}") return HumanDecision(approved=approved, modified_output=output) # Stage 2 — gated human review for high-risk alerts sar_writer = HumanInTheLoop( agent=Agent( "sar_drafter", smart_llm, system_prompt=( "Draft a FinCEN SAR narrative from the case summary: subject, activity description, " "dates, amounts, and why the activity is suspicious. Be factual and concise." ), ), review_fn=compliance_review, high_risk_keywords=["High", "sanctions", "structuring"], ) async def monitor(alert: str) -> None: triage = await aml_pipeline.run(alert) print(triage.output) # Route to human review only for high-risk (Pydantic parse — no fragile string match) if _parse_tier(triage.output) == "High": sar_result = await sar_writer.run(triage.output) if sar_result.metadata.get("approved"): print("\nSAR filed:\n", sar_result.output) else: print("\nAlert closed — officer rejected SAR.") else: print("\nAuto-cleared (Low/Medium risk).") ALERT = ( "Transaction: $9,850 wire from ACC-7731 (US) to ACME Consulting Ltd (Cyprus). " "Third such transfer in 48h. Counterparty newly registered; no prior relationship." ) asyncio.run(monitor(ALERT)) ``` ______________________________________________________________________ ## Expected Output ``` Rule flags: structuring ($9,850 — just under $10k ×3), high-risk jurisdiction (Cyprus), velocity breach. Risk: 88 — High. Drivers: structuring pattern + new counterparty in high-risk jurisdiction. Case: ACME Consulting Ltd registered 30 days ago, no known business activity, Cyprus jurisdiction (FATF grey-listed). Three transfers total $29,550 in 48h. [COMPLIANCE REVIEW REQUIRED] File SAR? (yes/no): yes SAR filed: Subject: ACC-7731. Activity: three wire transfers totalling $29,550 to a newly registered Cyprus shell, structured below the $10k reporting threshold. Suspicious indicators: velocity, structuring, high-risk jurisdiction. ``` Low- and medium-risk alerts are auto-cleared without a human review, so the compliance officer sees only cases that genuinely need judgment. ______________________________________________________________________ ## Customization ### Replace the interactive prompt with a compliance UI ``` import httpx def compliance_review(output: str, metadata: dict) -> HumanDecision: resp = httpx.post("https://compliance.internal/api/review", json={"case": output}) decision = resp.json() # {"approved": true, "notes": "..."} return HumanDecision(approved=decision["approved"], modified_output=output) ``` ### Lower the risk threshold ``` if triage.output.count("High") > 0 or "Medium" in triage.output: sar_result = await sar_writer.run(triage.output) ``` ### Batch alerts ``` alerts = [...] results = await asyncio.gather(*(monitor(a) for a in alerts)) ``` ______________________________________________________________________ ## When to Use | Situation | Fit | | ------------------------------------ | --------------------------------- | | Fixed screen → score → enrich stages | ✅ Pipeline | | High-risk outputs need a human gate | ✅ Human-in-the-Loop | | Risk threshold varies per alert type | ✅ Compose both | | Every alert needs human review | ❌ Remove pipeline; use HITL only | ______________________________________________________________________ ## Cost Profile | Stage | Typical model | Avg cost | Volume (50k alerts/mo) | | ---------------------------- | ------------- | ----------- | ---------------------- | | Screener + enrichment | gpt-4o-mini | $0.0006 | $30 | | Risk scorer | claude-sonnet | $0.004 | $200 | | SAR drafter (High only, ~5%) | claude-sonnet | $0.006 | $15 | | **Per alert** | mix | **~$0.005** | **~$245/mo** | ______________________________________________________________________ ## See Also - [Pipeline pattern](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) - [Human-in-the-Loop pattern](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) - [Fraud Investigation Assistant](https://pyagent.org/cookbook/security/fraud-investigation/index.md) — ReAct agent for deep investigation - [Loan Underwriting Committee](https://pyagent.org/cookbook/finance-trading/loan-underwriting/index.md) — debate pattern for credit decisions - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Earnings Call Analyzer in Python A first-pass earnings summary misses things: buried guidance updates, CFO tone shifts, management's dodge on capex. This recipe uses **Self-Reflection** — the agent produces an initial analysis, then critiques and improves it in a tight loop until it passes a completeness check. **Patterns used:** [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) ______________________________________________________________________ ## Architecture ``` flowchart TD T[Earnings Transcript] --> A[Analyst Agent\ninitial analysis] A --> C{Reflection\ncomplete?} C -->|No — gaps found| A C -->|Yes — ANALYSIS COMPLETE| O[Final Earnings Brief] ``` ______________________________________________________________________ ## Implementation ``` pip install pyagent-patterns pyagent-providers ``` ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.resolution import SelfReflection from pyagent_providers import AnthropicLLM smart_llm = AnthropicLLM("claude-sonnet-4-20250514") earnings_analyzer = SelfReflection( agent=Agent( "earnings_analyst", smart_llm, system_prompt=( "You analyze earnings call transcripts for buy-side investors. Each iteration:\n" "1. Extract: EPS beat/miss vs consensus, revenue growth YoY, and full-year guidance revision.\n" "2. Flag: any change in management tone (confident / cautious / evasive), and one key risk.\n" "3. Self-check: have you covered EPS, revenue, guidance, tone, AND risk? " "If any are missing or vague, note the gap and improve. " 'When all five are clear and specific, end with "ANALYSIS COMPLETE".' ), ), max_rounds=3, stop_phrase="ANALYSIS COMPLETE", ) TRANSCRIPT = """ [Q2 Earnings Call — ACME Corp] CEO: "We delivered another strong quarter. Revenue came in at $1.24B, up 14% year-over-year." CFO: "EPS was $1.82, and I would note consensus was $1.68. Free cash flow was $310M. On guidance — we are tightening the full-year revenue range to $4.9–5.0B from $4.7–5.1B and raising the midpoint. Margin expansion remains on track." Analyst Q: "Can you comment on capex plans for H2?" CFO: "We will continue to invest prudently. I don't want to get into specific numbers today." CEO: "We feel very good about our competitive position." """ async def main(): result = await earnings_analyzer.run(TRANSCRIPT) print(result.output) print(f"\nRounds: {result.metadata['rounds']}") asyncio.run(main()) ``` ______________________________________________________________________ ## Expected Output ``` EARNINGS BRIEF — ACME Corp Q2 EPS: $1.82 vs $1.68 consensus — beat by 8.3%. Revenue: $1.24B, +14% YoY — above street. Guidance: Full-year range tightened to $4.9–5.0B (raised midpoint from ~$4.9B); positive signal. Tone: CEO confident; CFO measured. Notable: CFO declined to quantify H2 capex ("don't want to get into specific numbers") — potentially evasive on investment step-up. Risk: Capex opacity — if H2 spend accelerates materially, FCF guidance could disappoint. ANALYSIS COMPLETE Rounds: 2 ``` The agent caught the CFO's capex dodge on the second pass — the first round flagged the tone but hadn't surfaced the specific risk it implied. ______________________________________________________________________ ## Customization ### Raise the quality bar ``` earnings_analyzer.agent.system_prompt += ( "\n4. Sentiment: score management confidence 1-10 based on hedging language." ) ``` ### Multi-ticker batch ``` tickers = {"ACME": TRANSCRIPT_ACME, "BETA": TRANSCRIPT_BETA} results = await asyncio.gather(*(earnings_analyzer.run(t) for t in tickers.values())) for name, r in zip(tickers, results): print(f"--- {name} ---\n{r.output}\n") ``` ### Compare to prior quarter ``` prompt = f"Compare this quarter to the prior quarter.\n\nPRIOR:\n{prior}\n\nCURRENT:\n{current}" result = await earnings_analyzer.run(prompt) ``` ______________________________________________________________________ ## When to Use | Situation | Use Self-Reflection? | | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | | First-pass analysis reliably misses details | ✅ Yes | | You can specify a clear completeness check | ✅ Yes | | Multiple independent analysts should compare views | ❌ Use [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) | | Two agents should argue a bull/bear case | ❌ Use [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) | ______________________________________________________________________ ## Cost Profile | Stage | Typical model | Avg cost | Volume (200 calls/quarter) | | --------------------- | ------------- | ----------- | -------------------------- | | 1–3 reflection rounds | claude-sonnet | $0.012 | $2.40 | | **Per transcript** | claude-sonnet | **~$0.012** | **~$2.40/quarter** | ______________________________________________________________________ ## See Also - [Self-Reflection pattern](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) - [Trading Signal Desk](https://pyagent.org/cookbook/finance-trading/trading-signals/index.md) — parallel strategy agents fan-out/in - [Portfolio Review](https://pyagent.org/cookbook/finance-trading/portfolio-review/index.md) — multi-analyst memo with evaluator-optimizer - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent ESG Report Analyzer in Python ESG analysis pulls from many sources — third-party ratings, a company's own disclosures, and a regulatory framework — and which sources matter depends on the mandate. This recipe uses the **Orchestrator-Workers** pattern: a lead decides which workers to dispatch (ratings, disclosure extraction, SFDR/CSRD scoring), then synthesizes their output into one sustainability summary. **Patterns used:** [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) ______________________________________________________________________ ## Architecture ``` flowchart TD R[Company + Mandate] --> O[ESG Lead\nplan workers] O -->|assign| RT[Ratings Worker] O -->|assign| DX[Disclosure Extractor] O -->|assign| SC[SFDR/CSRD Scorer] RT --> S[ESG Lead\nsynthesize] DX --> S SC --> S S --> A[ESG Summary + Score] ``` ______________________________________________________________________ ## Implementation ``` pip install pyagent-patterns pyagent-providers ``` ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import OrchestratorWorkers from pyagent_providers import AnthropicLLM, OpenAILLM fast_llm = OpenAILLM("gpt-4o-mini") smart_llm = AnthropicLLM("claude-sonnet-4-20250514") esg = OrchestratorWorkers( orchestrator=Agent( "esg_lead", smart_llm, system_prompt=( "You analyze a company's ESG profile for an investor mandate. Decide which workers are " "needed and assign each a subtask — skip any the mandate doesn't require. Respond as JSON: " '{"assignments": [{"worker": "name", "subtask": "..."}]}. Then synthesize a summary with ' "an overall ESG rating (A-E), the top strength, the top controversy, and an SFDR article fit." ), ), workers=[ Agent( "ratings", fast_llm, system_prompt="Summarize the company's third-party ESG ratings and any rating disagreements.", ), Agent( "disclosure_extractor", smart_llm, system_prompt="Extract the company's reported Scope 1/2/3 emissions, targets, and board-diversity data.", ), Agent( "sfdr_scorer", smart_llm, system_prompt="Score alignment with SFDR/CSRD: PAI coverage, taxonomy eligibility, and gaps.", ), Agent( "controversy", fast_llm, system_prompt="List recent ESG controversies (labor, governance, environmental) with severity.", ), ], ) async def main(): result = await esg.run( "Analyze ACME Energy for an SFDR Article 8 fund. Mandate excludes thermal coal; " "emphasize emissions trajectory and governance." ) print(result.output) print(f"Workers used: {result.metadata['workers_used']}") asyncio.run(main()) ``` ______________________________________________________________________ ## Expected Output ``` ESG SUMMARY — ACME Energy (SFDR Article 8 screen) Overall: C (improving) — credible 2030 target but high Scope 3. Strength: board independence 80%, science-based target validated. Controversy: 2024 spill settlement (medium); no thermal-coal exposure (passes the exclusion). SFDR: Article 8 fit OK; PAI coverage partial — water & biodiversity metrics missing. Workers used: ['ratings', 'disclosure_extractor', 'sfdr_scorer', 'controversy'] ``` The lead skips workers the mandate doesn't need (e.g. no controversy scan for a passive index screen), so cost tracks the depth each analysis actually requires. ______________________________________________________________________ ## Customization ### Plug in live ratings/filings tools Replace the static workers with [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) agents that pull MSCI/Sustainalytics scores and the latest 10-K / CSRD filing — see the [SQL Analytics Assistant](https://pyagent.org/cookbook/data-analytics/sql-analyst/index.md). ### Batch a fund's holdings ``` async def screen_fund(companies: list[str]) -> list[str]: out = await asyncio.gather(*(esg.run(c) for c in companies)) return [r.output for r in out] ``` ### Stricter exclusion enforcement ``` esg.orchestrator.system_prompt += " If any hard-excluded activity is found, output EXCLUDE and stop." ``` ______________________________________________________________________ ## When to Use | Situation | Use Orchestrator-Workers? | | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------ | | Which ESG sources to use depends on the mandate | ✅ Yes | | You want one synthesized rating from many specialists | ✅ Yes | | Every analysis runs the same fixed stages | ❌ Use [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) | | Teams of analysts each have their own sub-work | ❌ Use [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) | ______________________________________________________________________ ## Cost Profile | Stage | Typical model | Avg cost | Volume (2k companies/yr) | | ------------------------ | ------------- | ----------- | ------------------------ | | Lead (plan + synthesize) | claude-sonnet | $0.008 | $16 | | Workers (up to 4) | mini + sonnet | $0.010 | $20 | | **Per company** | mix | **~$0.018** | **~$36/yr** | ______________________________________________________________________ ## See Also - [Orchestrator-Workers pattern](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) - [Regulatory Compliance Checker](https://pyagent.org/cookbook/legal-compliance/compliance-checker/index.md) — hierarchical gap analysis - [Wealth Rebalancing Crew](https://pyagent.org/cookbook/finance-trading/wealth-rebalancing/index.md) — applies an ESG mandate to allocations - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Loan Origination Workflow in Python Loan origination is a state machine: collect documents, verify income, pull credit, then approve — each stage building on the last and able to reject with a reason. This recipe uses the **Topology** pattern in a **chain** configuration, where each agent passes its findings (and the accumulating file) forward to the next stage. **Patterns used:** Topology (chain) ______________________________________________________________________ ## Architecture ``` flowchart LR A[Application] --> D[Document Collection] D --> I[Income Verification] I --> C[Credit Scoring] C --> P[Approval Decision] P --> O[Origination Decision] ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.structural import Topology, TopologyType from pyagent_providers import OpenAILLM, AnthropicLLM origination = Topology( agents=[ Agent( "document_collection", OpenAILLM("gpt-4o-mini"), system_prompt=( "Stage 1 — documents. List required documents and confirm which are present. " "If anything mandatory is missing, mark INCOMPLETE with the reason and pass forward." ), ), Agent( "income_verification", OpenAILLM("gpt-4o-mini"), system_prompt=( "Stage 2 — income. Using the documents, verify stated income and compute DTI. " "Flag inconsistencies. Pass the verified figures and prior notes forward." ), ), Agent( "credit_scoring", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "Stage 3 — credit. From the file, summarize credit history and assign a risk tier " "(A-E) with the key drivers. Pass everything forward." ), ), Agent( "approval", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "Stage 4 — decision. Given documents, income, and credit tier, decide APPROVE, " "REFER, or DECLINE with reasons. If any earlier stage flagged INCOMPLETE, REFER." ), ), ], topology=TopologyType.CHAIN, ) result = asyncio.run(origination.run( "Mortgage application: $420k loan, stated income $135k (W-2 + 2 paystubs attached), " "credit pulled, 12% down, no bank statements provided yet." )) print(result.output) print(f"Topology: {result.metadata['topology']}") ``` ______________________________________________________________________ ## Expected output ``` ORIGINATION DECISION — REFER Documents: paystubs + W-2 present; bank statements MISSING → INCOMPLETE. Income: $135k verified from W-2; DTI 31% (within limits). Credit: Tier B — clean history, one inquiry. Decision: REFER — strong file but missing bank statements; request and re-run stage 1. Topology: chain ``` A chain makes the state machine explicit: each stage can reject with a reason, and the accumulating file is what flows forward. ______________________________________________________________________ ## Customization ### Add a pricing stage ``` origination.agents.append( Agent("pricing", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Stage 5 — price the approved loan: rate, term, and fees from the risk tier."), ) ``` ### Switch to a plain pipeline If the flow is strictly linear, a [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) is simpler; keep Topology when you may add star/mesh review later. ### Capture reject reasons ``` origination.agents[-1].system_prompt += " On DECLINE, output a machine-readable reason code." ``` ______________________________________________________________________ ## When to Use | Situation | Use Topology (chain)? | | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------- | | A linear state machine where each stage gates the next | ✅ Yes — chain | | Reviewers must reconcile views, not just pass forward | ❌ Use `TopologyType.MESH` ([Peer-Review Mesh](https://pyagent.org/cookbook/scientific/peer-review/index.md)) | | The committee debates a single decision | ❌ Use [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) | > A plain [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) also chains stages; reach for Topology when you may later switch the same agents to a star or mesh structure. ______________________________________________________________________ ## Cost Profile | Stage | Typical model | Avg cost | Volume (1k apps/day) | | ------------------- | ------------- | ----------- | -------------------- | | Docs + income | gpt-4o-mini | $0.0006 | $18/mo | | Credit + approval | claude-sonnet | $0.008 | $240/mo | | **Per application** | mix | **~$0.009** | **~$270/mo** | ______________________________________________________________________ ## See Also - [Topology pattern](https://pyagent.org/packages/patterns/structural/topology/index.md) - [Loan Underwriting Committee](https://pyagent.org/cookbook/finance-trading/loan-underwriting/index.md) — the debate that follows a clean file - [Fraud Investigation Assistant](https://pyagent.org/cookbook/security/fraud-investigation/index.md) — tool-using investigation - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Loan Underwriting Committee in Python Good credit decisions surface both the case *for* and *against* an application before a senior decides. This recipe uses the **Debate** pattern: an approve-advocate and a decline-advocate argue the loan with specific evidence, and a senior-underwriter judge weighs the exchange into a decision with conditions — an auditable alternative to a single opaque score. **Patterns used:** Debate ______________________________________________________________________ ## Architecture ``` flowchart TD A[Loan Application] --> D{Debate} D --> AP[Approve Advocate] D --> DC[Decline Advocate] AP <-->|rounds| DC AP --> J[Senior Underwriter\nJudge] DC --> J J --> R[Decision + Conditions] ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.resolution import Debate from pyagent_providers import AnthropicLLM, GeminiLLM committee = Debate( debaters=[ Agent( "approve_advocate", GeminiLLM("gemini-2.5-pro"), system_prompt=( "Argue to APPROVE this loan. Cite debt-to-income, credit history, collateral, and " "stable income. Pre-empt the decline case with mitigants and proposed conditions." ), ), Agent( "decline_advocate", GeminiLLM("gemini-2.5-pro"), system_prompt=( "Argue to DECLINE this loan. Focus on repayment risk, thin file, concentration, and " "downside scenarios. Rebut the approve case with specific counter-evidence." ), ), ], judge=Agent( "senior_underwriter", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "You are the senior underwriter. Weigh both arguments and decide: APPROVE, " "APPROVE WITH CONDITIONS, or DECLINE. State the key reasons and any conditions " "(rate, term, collateral, covenants). Be explicit about the deciding factor." ), ), rounds=2, ) result = asyncio.run(committee.run( "Application: $250k 7-year business loan. DTI 38%, credit score 690, 2 years trading, " "revenue $480k (growing), $90k equipment as collateral, one prior late payment." )) print(result.output) print(f"Rounds: {result.metadata['rounds']}") ``` ______________________________________________________________________ ## Expected output ``` DECISION — APPROVE WITH CONDITIONS Deciding factor: collateral + revenue growth offset the thin file and one late payment. Conditions: rate +1.5% risk premium, personal guarantee, quarterly financials covenant, cap utilization until 12 clean months. Approve case: serviceable DTI, growing revenue, secured. Decline case: short trading history, single collateral asset. Rounds: 2 ``` The transcript of both advocates and the judge's reasoning is your audit trail for regulators. ______________________________________________________________________ ## Customization ### Add a third advocate ``` committee.debaters.append( Agent("policy_advocate", GeminiLLM("gemini-2.5-pro"), system_prompt="Argue from lending-policy and regulatory limits (LTV, DTI caps)."), ) ``` ### Tune the debate depth ``` committee = Debate(debaters=committee._debaters, judge=committee._judge, rounds=3) ``` ### Structured decision ``` committee.judge.system_prompt += " Output as JSON: {decision, rate, conditions[], deciding_factor}." ``` ______________________________________________________________________ ## When to Use | Situation | Use Debate? | | ------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------- | | A decision benefits from explicit for/against argument | ✅ Yes | | You need an auditable rationale, not just a score | ✅ Yes | | Independent graders should vote a label | ❌ Use [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) | | A document moves through fixed processing steps | ❌ Use [Loan Origination Workflow](https://pyagent.org/cookbook/finance-trading/loan-origination/index.md) | ______________________________________________________________________ ## Cost Profile | Driver | Typical model | Avg cost | Volume (1k apps/mo) | | --------------------- | ------------- | ----------- | ------------------- | | 2 debaters × 2 rounds | gemini-pro | $0.012 | $360/mo | | Judge | claude-sonnet | $0.006 | $180/mo | | **Per application** | mix | **~$0.018** | **~$540/mo** | ______________________________________________________________________ ## See Also - [Debate pattern](https://pyagent.org/packages/patterns/resolution/debate/index.md) - [Loan Origination Workflow](https://pyagent.org/cookbook/finance-trading/loan-origination/index.md) — the document pipeline that precedes underwriting - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Portfolio Review Workflow in Python A good portfolio memo routes each holding to the right specialist *and* doesn't ship until it's actually rigorous. This recipe uses a **Supervisor** to send each position to an equities, fixed-income, or risk analyst, then an **Evaluator-Optimizer** loop that scores the memo against explicit criteria and revises it until it clears the bar. **Patterns used:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md) ______________________________________________________________________ ## Requirements - **Functional** — route each holding to the analyst best suited to its asset class; produce a memo that clears an explicit quality bar before it ships. - **Non-functional** — cost stays low enough to review a whole book monthly (see Cost Profile below); most memos converge in ≤2 optimization rounds. - **Audit** — the memo's rationale, downside, and sizing must be explicit, not implicit — the quality gate exists specifically to force this. - **Not required** — no cross-holding negotiation, no persistent memory between reviews, no human approval gate (this recipe assumes analyst output, not a trade order). ## Architecture decisions | Decision | Why | Why not the alternative | | ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Supervisor** for routing | Holdings fall into distinct, known categories (equity / fixed income / risk), and each specialist is meaningfully better than a generalist at its category. | **Orchestrator-Workers** is for when subtasks aren't known upfront — asset class *is* known upfront here, so the dynamic-planning overhead buys nothing. | | **Evaluator-Optimizer** for the memo | The quality bar is explicit and scorable (recommendation, rationale, downside, sizing) — exactly the "measurable criteria, worth iterating" case. | **Self-Reflection** would work but loses the independent-scoring signal; **Cross-Reflection** adds a second full analyst pass for feedback that's qualitative, not needed when the criteria are already checklist-shaped. | | **Two workflows, not one pattern** | `route` and `tighten` are declared as separate named workflows in one blueprint. There's no `supervisor_plus_evaluator_optimizer` pattern — PyAgent composes patterns by wiring multiple named workflows in one file, not by nesting patterns inside each other. | A single fused pattern would need a new class in `pyagent_patterns` just for this composition; two workflows reuse existing patterns and let each stage be tested independently. | ## Four-pillar mapping | Requirement | Pillar | Capability | | -------------------------------------- | ------------- | ------------------------------------------ | | Route by asset class | Execution | `Supervisor` pattern | | Score against explicit criteria | Execution | `EvaluatorOptimizer` pattern | | Track daily spend | Observability | `observability.cost_budget` | | Trace each classify/analyze/score call | Observability | `observability.tracing` | | Version the workflow as it evolves | Blueprint | `pyagent-blueprint diff` between revisions | ## Blueprint (declarative form) The same system declared as a `pyagent-blueprint` manifest — this is the real, verified file at `examples/cookbook/finance-trading/portfolio_review/blueprint.yaml`, compiled and run against `PyAgentAdapter` with `MockLLM` as part of this repo's test suite, not a hand-typed illustration: ``` api_version: pyagent/v1 metadata: name: portfolio-review version: 1.0.0 description: Supervisor routes holding to specialist; evaluator-optimizer tightens memo providers: fast: { model: claude-haiku-3-5-20241022 } smart: { model: claude-sonnet-4-20250514 } agents: router: { provider: fast, prompt: "Classify as equity, fixed_income, or risk." } equities: { provider: smart, prompt: "Analyze equity: thesis, valuation, catalysts, risks." } rates: { provider: smart, prompt: "Analyze bond: duration, credit quality, rate sensitivity." } risk: { provider: smart, prompt: "Assess concentration, correlation, tail scenarios." } writer: { provider: smart, prompt: "Write investment memo: recommendation, rationale, risks." } reviewer: { provider: smart, prompt: "Score memo 1-10 against criteria. Demand fixes if below bar." } workflows: route: pattern: supervisor agents: classifier: router routes: { equity: equities, fixed_income: rates, risk: risk } config: default_route: risk tighten: pattern: evaluator_optimizer agents: generator: writer evaluator: reviewer config: criteria: ["clear recommendation", "evidence-backed rationale", "explicit downside", "position sizing"] pass_threshold: 8 max_rounds: 3 observability: tracing: { enabled: true } cost_budget: { daily_usd: 100.0, alert_threshold: 0.8 } ``` ``` pyagent-blueprint validate portfolio-review.yaml pyagent-blueprint test portfolio-review.yaml ``` ## Production checklist Ran this exact blueprint through `PyAgentAdapter.compile()` and inspected the real diagnostics rather than assuming what's enforced: - ✅ **Routing and scoring run as declared** — `route` and `tighten` compile and execute against the native pattern registry with no diagnostics. - ⚠️ **`observability.cost_budget` is declared but not auto-enforced** — compiling this blueprint emits `BUDGET_UNSUPPORTED`: the $100/day budget is recorded in the spec but nothing stops a run from exceeding it. Wire real enforcement via `graph.wire_cost_tracker(tracker)` if you need a hard stop, not just a declared intent. - **No recovery policy is declared** on either workflow — if you add one (retries, timeout, fallback provider), know that it's `RECOVERY_UNSUPPORTED` by the same adapter until you wire it manually; this blueprint simply doesn't declare one, so that diagnostic doesn't fire here. - **No human approval gate** — this composes cleanly with [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) as a third workflow if a memo needs sign-off before it reaches a client, e.g. for `daily_usd` above a threshold. ______________________________________________________________________ ## Architecture ``` flowchart TD H[Holding] --> S[Supervisor\nclassify asset class] S -->|equity| EQ[Equities Analyst] S -->|fixed_income| FI[Rates Analyst] S -->|risk| RK[Risk Analyst] EQ --> M{Evaluator-Optimizer} FI --> M RK --> M M -->|score < threshold, revise| M M -->|passes| O[Investment Memo] ``` ______________________________________________________________________ ## Implementation ``` pip install pyagent-patterns pyagent-providers ``` ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Supervisor from pyagent_patterns.resolution import EvaluatorOptimizer from pyagent_providers import AnthropicLLM fast_llm = AnthropicLLM("claude-haiku-3-5-20241022") smart_llm = AnthropicLLM("claude-sonnet-4-20250514") # ── Supervisor routes each holding to the right specialist analyst ────────────── desk = Supervisor( classifier=Agent( "router", fast_llm, system_prompt="Classify the holding as exactly one of: equity, fixed_income, risk. Reply with only the label.", ), routes={ "equity": Agent("equities", smart_llm, system_prompt="Analyze the equity position: thesis, valuation multiples, key catalysts and risks."), "fixed_income": Agent("rates", smart_llm, system_prompt="Analyze the bond position: duration, credit quality, and rate sensitivity."), "risk": Agent("risk", smart_llm, system_prompt="Assess portfolio-level risk: concentration, correlation, and tail scenarios."), }, default_route="risk", ) # ── Evaluator-Optimizer tightens the memo against explicit criteria ───────────── memo = EvaluatorOptimizer( generator=Agent("writer", smart_llm, system_prompt="Write a concise investment memo from the analysis: recommendation, rationale, risks."), evaluator=Agent("reviewer", smart_llm, system_prompt="Score the memo 1-10 against the criteria. Demand specific fixes for any criterion below bar."), criteria=["clear recommendation", "evidence-backed rationale", "explicit downside", "position sizing"], pass_threshold=8, max_rounds=3, ) async def main(): analysis = await desk.run("AAPL — 12% of the book, up 30% YTD, trading at 30x forward earnings") result = await memo.run(analysis.output) print(result.output) print(f"Converged in {result.metadata['rounds']} rounds, final score {result.metadata['final_score']}") asyncio.run(main()) ``` ______________________________________________________________________ ## Expected Output ``` INVESTMENT MEMO — AAPL Recommendation: TRIM to 8% of book. Rationale: strong franchise and cash generation, but 30x forward earnings prices in a lot; the 12% weight is now a concentration risk after the 30% run. Downside: multiple compression to 24x → ~20% drawdown on the position. Sizing: trim 4 points; redeploy into the underweight fixed-income sleeve. Converged in 2 rounds, final score 9 ``` The Evaluator-Optimizer is what forces "explicit downside" and "position sizing" to actually appear — the criteria the first draft skipped and the loop demanded. ______________________________________________________________________ ## Customization ### Tune the bar and criteria ``` memo.pass_threshold = 9 # stricter for IC-ready memos memo.criteria.append("comparison vs benchmark") ``` ### Review the whole book ``` async def review_book(holdings: list[str]) -> list[str]: out = [] for h in holdings: analysis = await desk.run(h) out.append((await memo.run(analysis.output)).output) return out ``` ### Add a data-gathering analyst Give the analysts live numbers by routing through a [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) agent that queries market data — see the [SQL Analytics Assistant](https://pyagent.org/cookbook/data-analytics/sql-analyst/index.md). ______________________________________________________________________ ## When to Use | Situation | Fit | | ---------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Route each item to one specialist | ✅ Supervisor | | Output must iterate to an explicit quality bar | ✅ Evaluator-Optimizer | | Two analysts should argue bull vs bear | ❌ Use [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) ([Loan Underwriting](https://pyagent.org/cookbook/finance-trading/loan-underwriting/index.md)) | | One agent critiques its own draft | ❌ Use [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) | ______________________________________________________________________ ## Cost Profile | Stage | Typical model | Avg cost | Volume (1k holdings/mo) | | ------------------------- | ------------- | ------------ | ----------------------- | | Classifier | claude-haiku | $0.0005 | $0.50 | | Analyst | claude-sonnet | $0.005 | $5 | | Memo (≤3 optimize rounds) | claude-sonnet | $0.012 | $12 | | **Per holding** | mix | **~$0.0175** | **~$17.5/mo** | `pass_threshold` and `max_rounds` trade memo quality against cost — most memos clear the bar in two rounds. ______________________________________________________________________ ## See Also - [Supervisor pattern](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Evaluator-Optimizer pattern](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md) - [Loan Underwriting Committee](https://pyagent.org/cookbook/finance-trading/loan-underwriting/index.md) — a debating credit panel - [SQL Analytics Assistant](https://pyagent.org/cookbook/data-analytics/sql-analyst/index.md) — tool-using analyst - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Robo-Advisor Onboarding Flow in Python A robo-advisor onboarding conversation has distinct phases: gather facts, profile risk, check suitability, then build a plan. This recipe uses **Role-Based** conversation — each specialist agent takes a turn, and the next agent sees the full exchange so context accumulates naturally. **Patterns used:** [Role-Based](https://pyagent.org/packages/patterns/structural/role-based/index.md) ______________________________________________________________________ ## Architecture ``` flowchart TD C[Client Answers] --> I[Intake Agent\nfacts & goals] I --> R[Risk Profiler\ntolerance & horizon] R --> S[Suitability Agent\nreg checks] S --> P[Planning Agent\nmodel portfolio + IPS] P --> O[Investment Plan] ``` ______________________________________________________________________ ## Implementation ``` pip install pyagent-patterns pyagent-providers ``` ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.structural import RoleBased from pyagent_providers import AnthropicLLM, OpenAILLM fast_llm = OpenAILLM("gpt-4o-mini") smart_llm = AnthropicLLM("claude-sonnet-4-20250514") onboarding = RoleBased( agents=[ Agent( "intake", fast_llm, system_prompt=( "You are the intake specialist. Extract from the client's answers: " "age, employment status, annual income, investable assets, primary goal " "(retirement / growth / income / preservation), and target timeline. " "Summarize clearly for the next agent." ), ), Agent( "risk_profiler", smart_llm, system_prompt=( "You are the risk-profiling specialist. Using the intake summary, assign: " "risk tolerance (Conservative / Moderate / Aggressive), max drawdown comfort (%), " "and time horizon (years). Explain the classification in two sentences." ), ), Agent( "suitability", fast_llm, system_prompt=( "You are the suitability analyst. Check the risk profile against regulatory " "suitability rules: is the proposed risk level appropriate for the client's age, " "income, and liquidity needs? Flag any concerns; otherwise confirm SUITABLE." ), ), Agent( "planner", smart_llm, system_prompt=( "You are the portfolio planner. Based on the profile and suitability check, " "output: (1) a model allocation (asset classes + target %) that matches the risk " "tier, (2) a one-paragraph Investment Policy Statement (IPS), and " "(3) three recommended next steps for the client." ), ), ], rounds=1, ) CLIENT_ANSWERS = """ I'm 34 years old, software engineer earning $145k/year. I have $280k to invest and want to retire comfortably at 60. I can tolerate some volatility — I lost money in 2022 and stayed invested. I won't need this money for at least 10 years. """ async def main(): result = await onboarding.run(CLIENT_ANSWERS) print(result.output) print(f"\nRoles: {result.metadata['agent_names']}, rounds: {result.metadata['rounds']}") asyncio.run(main()) ``` ______________________________________________________________________ ## Expected Output ``` ONBOARDING PLAN Intake: Age 34, SE $145k, $280k investable, goal: retirement at 60, horizon 26 years. Risk: Aggressive — 26-year horizon, high income, stayed invested through drawdown. Max drawdown comfort: ~30%. Suitability: SUITABLE — aggressive profile appropriate for age, income stability, and horizon. Portfolio: Global Equity ETF 55% US Small/Mid-Cap 15% International Dev 15% Emerging Markets 5% Aggregate Bond ETF 10% IPS: The portfolio targets long-term capital growth with a 55/45 equity tilt favoring broad global exposure. Rebalance annually or when any asset class drifts >5% from target. Review risk tolerance at age 50 and shift toward 60/40 conservative as retirement approaches. Next steps: (1) Open a tax-advantaged account (max 401k + IRA). (2) Set up automatic monthly contributions. (3) Schedule an annual review each January. Roles: ['intake', 'risk_profiler', 'suitability', 'planner'], rounds: 1 ``` Because each agent sees the full conversation history, the planner has all context — intake facts, risk classification, and suitability sign-off — without any manual data passing. ______________________________________________________________________ ## Customization ### Add a tax-efficiency specialist ``` from pyagent_patterns.base import Agent onboarding.agents.insert(3, Agent("tax_advisor", fast_llm, system_prompt="Recommend the most tax-efficient account wrappers for the client's situation."), ) ``` ### Run a second round for clarification ``` onboarding = RoleBased(agents=[...], rounds=2) ``` A second round lets the risk profiler revisit the intake after the suitability agent raises a concern. ### Pipe into the rebalancing crew Feed the IPS from the planner directly into the [Wealth Rebalancing Crew](https://pyagent.org/cookbook/finance-trading/wealth-rebalancing/index.md) pipeline as the client mandate. ______________________________________________________________________ ## When to Use | Situation | Use Role-Based? | | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | | Distinct phases with specialist handoff | ✅ Yes | | Each agent needs the full prior context | ✅ Yes | | Agents should run in parallel | ❌ Use [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) | | A fixed data-enrichment chain, not a conversation | ❌ Use [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) | ______________________________________________________________________ ## Cost Profile | Role | Typical model | Avg cost | Volume (10k onboardings/mo) | | ----------------------- | ------------- | ----------- | --------------------------- | | Intake + suitability | gpt-4o-mini | $0.0008 | $8 | | Risk profiler + planner | claude-sonnet | $0.009 | $90 | | **Per onboarding** | mix | **~$0.010** | **~$98/mo** | ______________________________________________________________________ ## See Also - [Role-Based pattern](https://pyagent.org/packages/patterns/structural/role-based/index.md) - [Wealth Rebalancing Crew](https://pyagent.org/cookbook/finance-trading/wealth-rebalancing/index.md) — pipeline that runs after onboarding - [Loan Origination Workflow](https://pyagent.org/cookbook/finance-trading/loan-origination/index.md) — topology-based financial workflow - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Trading Signal Desk in Python Three quant analysts looking at the same market data from different angles — momentum, mean-reversion, and sentiment — will each reach a different conclusion. This recipe uses **Fan-Out / Fan-In** to run all three strategy agents in parallel, then an aggregator synthesizes a consensus trade idea with a conviction score. **Patterns used:** [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) ______________________________________________________________________ ## Architecture ``` flowchart TD M[Market Data] --> FO[Fan-Out] FO --> MO[Momentum Agent] FO --> MR[Mean-Reversion Agent] FO --> SE[Sentiment Agent] MO --> FI[Aggregator\nconsensus + conviction] MR --> FI SE --> FI FI --> T[Trade Idea] ``` ______________________________________________________________________ ## Implementation ``` pip install pyagent-patterns pyagent-providers ``` ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import FanOutFanIn from pyagent_providers import AnthropicLLM, OpenAILLM, GeminiLLM smart_llm = AnthropicLLM("claude-sonnet-4-20250514") fast_llm = OpenAILLM("gpt-4o-mini") signal_desk = FanOutFanIn( agents=[ Agent( "momentum", fast_llm, system_prompt=( "You are a momentum strategist. Analyze the market data for trend strength: " "price vs 20/50/200-day MAs, RSI, volume trend. " "Output: signal (BUY/SELL/NEUTRAL), strength (1-5), key reason." ), ), Agent( "mean_reversion", fast_llm, system_prompt=( "You are a mean-reversion strategist. Look for overextension: z-score vs 90-day mean, " "Bollinger Band position, RSI extremes. " "Output: signal (BUY/SELL/NEUTRAL), strength (1-5), key reason." ), ), Agent( "sentiment", GeminiLLM("gemini-2.5-flash"), system_prompt=( "You are a sentiment strategist. Assess market mood from news/options data: " "put/call ratio, VIX level, news tone, analyst revision trend. " "Output: signal (BUY/SELL/NEUTRAL), strength (1-5), key reason." ), ), ], aggregator=Agent( "signal_aggregator", smart_llm, system_prompt=( "You receive signals from three strategy agents. Weight them equally unless they conflict. " "Output a consensus trade idea: direction (LONG/SHORT/FLAT), conviction (1-10), " "entry rationale, key risk to the thesis, and suggested position size (% of book)." ), ), ) MARKET_DATA = """ Ticker: ACME Corp (ACM) Price: $142.50 | 1-week change: +8.3% MA20: $135 MA50: $128 MA200: $115 (price above all MAs) RSI(14): 71 | Bollinger: price at upper band Volume: 2× average over 5 days Put/Call ratio: 0.55 (bullish skew) | VIX: 18 (calm) News: beat Q2 earnings by 12%; analyst upgrades ×3 this week """ async def main(): result = await signal_desk.run(MARKET_DATA) print(result.output) print(f"\nAgents: {result.metadata['agent_names']}") asyncio.run(main()) ``` ______________________________________________________________________ ## Expected Output ``` TRADE IDEA — ACM Consensus: LONG | Conviction: 7/10 Entry: Momentum BUY (strong, 4/5) + Sentiment BUY (strong, 5/5) outweigh Mean-Reversion NEUTRAL (RSI 71 near overbought, 2/5). Rationale: Post-earnings trend continuation; institutional accumulation (volume ×2); bullish options skew confirms directional bias. Risk: RSI elevated — gap-fill pullback to $136 if broader market weakens. Size: 1.5% of book (above-average conviction, elevated RSI limits size). Agents: ['momentum', 'mean_reversion', 'sentiment'] ``` Conflicting signals are preserved in the aggregator's reasoning rather than silently averaged, so a risk manager can see exactly why conviction is not 10/10. ______________________________________________________________________ ## Customization ### Add a macro agent ``` signal_desk.agents.append( Agent("macro", fast_llm, system_prompt="Assess macro backdrop: rates, USD, commodity prices. Signal + strength + reason."), ) ``` ### Weight the signals ``` signal_desk.aggregator.system_prompt = signal_desk.aggregator.system_prompt.replace( "Weight them equally", "Weight momentum 40%, mean-reversion 30%, sentiment 30%", ) ``` ### Screen a universe in parallel ``` tickers = ["ACM", "XYZ", "ABC"] results = await asyncio.gather(*(signal_desk.run(market_data[t]) for t in tickers)) for t, r in zip(tickers, results): print(t, r.output[:80]) ``` ______________________________________________________________________ ## When to Use | Situation | Use Fan-Out / Fan-In? | | ---------------------------------------------- | ---------------------------------------------------------------------------------------- | | Multiple independent analysts on the same data | ✅ Yes | | You want parallel execution + one synthesis | ✅ Yes | | Agents must share context between steps | ❌ Use [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) | | Agents should debate until agreement | ❌ Use [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) | ______________________________________________________________________ ## Cost Profile | Stage | Typical model | Avg cost | Volume (500 screens/day) | | ------------------------- | ---------------- | ----------- | ------------------------ | | Momentum + mean-reversion | gpt-4o-mini | $0.0008 | $0.40 | | Sentiment | gemini-2.5-flash | $0.0006 | $0.30 | | Aggregator | claude-sonnet | $0.005 | $2.50 | | **Per screen** | mix | **~$0.006** | **~$3.20/day** | ______________________________________________________________________ ## See Also - [Fan-Out / Fan-In pattern](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) - [Portfolio Review](https://pyagent.org/cookbook/finance-trading/portfolio-review/index.md) — specialist analysts + evaluator-optimizer - [Earnings Call Analyzer](https://pyagent.org/cookbook/finance-trading/earnings-call/index.md) — self-reflection on a single earnings transcript - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Wealth Rebalancing Crew in Python Rebalancing a client's portfolio is a sequence: read the risk profile, scan the market, propose a new allocation, then check it against compliance limits. This recipe uses the **Pipeline** pattern so each stage builds on the previous one — a risk profiler frames the mandate, a market scanner adds context, an allocation strategist proposes trades, and a compliance checker vetoes anything out of policy. **Patterns used:** [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) ______________________________________________________________________ ## Architecture ``` flowchart TD C[Client + Holdings] --> R[Risk Profiler\nmandate + constraints] R --> M[Market Scanner\nregime + outlook] M --> A[Allocation Strategist\nproposed trades] A --> K[Compliance Checker\nlimits + suitability] K --> O[Rebalance Proposal] ``` ______________________________________________________________________ ## Implementation ``` pip install pyagent-patterns pyagent-providers ``` ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Pipeline from pyagent_providers import AnthropicLLM, OpenAILLM fast_llm = OpenAILLM("gpt-4o-mini") smart_llm = AnthropicLLM("claude-sonnet-4-20250514") rebalance = Pipeline(stages=[ Agent( "risk_profiler", fast_llm, system_prompt=( "Summarize the client mandate from the input: risk tolerance, time horizon, liquidity " "needs, and any hard constraints (no tobacco, ESG-only, max single-name 5%). Be explicit." ), ), Agent( "market_scanner", smart_llm, system_prompt=( "Given the mandate, add a brief market view: regime (risk-on/off), rates outlook, and " "which asset classes look rich vs cheap. Two sentences each." ), ), Agent( "allocation_strategist", smart_llm, system_prompt=( "Propose a target allocation and the specific buy/sell trades to get there from the " "current holdings. Respect the mandate constraints and explain each move in one line." ), ), Agent( "compliance_checker", fast_llm, system_prompt=( "Check the proposed trades against the constraints and suitability. VETO any trade that " "breaches a limit (with the reason); otherwise mark APPROVED and produce the final ticket list." ), ), ]) CLIENT = ( "Client: 58, retiring in 7 years, moderate risk, needs 3% annual income, ESG-only, max 5% single name. " "Holdings: 70% global equity ETF, 20% individual tech (NVDA 8%, AAPL 7%), 10% cash." ) async def main(): result = await rebalance.run(CLIENT) print(result.output) print(f"Stages: {result.metadata['stage_names']}") asyncio.run(main()) ``` ______________________________________________________________________ ## Expected Output ``` REBALANCE PROPOSAL — moderate / 7-yr horizon / ESG-only Mandate: moderate risk, 3% income need, ESG screen, 5% single-name cap. Market: risk-neutral; rates near peak → add duration; equities fairly valued. Trades: TRIM NVDA 8%→5% and AAPL 7%→5% (single-name cap); ADD 12% ESG aggregate bond ETF (income + duration); keep 5% cash buffer. Compliance: APPROVED — all positions ≤ 5%; ESG screen satisfied; income target met (~3.1%). Stages: ['risk_profiler', 'market_scanner', 'allocation_strategist', 'compliance_checker'] ``` The compliance stage is the gate: it catches the single-name breach the strategist might otherwise leave, so the proposal that reaches an advisor is already suitable. ______________________________________________________________________ ## Customization ### Add an evaluator-optimizer quality gate Wrap the strategist in an [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md) so the allocation iterates until it clears a rigor bar — see [Portfolio Review](https://pyagent.org/cookbook/finance-trading/portfolio-review/index.md). ### Gate execution behind a human ``` from pyagent_patterns.advanced import HumanInTheLoop from pyagent_patterns.advanced.human_in_the_loop import HumanDecision approve = HumanInTheLoop( agent=Agent("ticket_writer", fast_llm, system_prompt="Format the approved trades as an order ticket."), review_fn=lambda out, meta: HumanDecision(approved=_advisor_signs_off(out), modified_output=out), ) ``` ### Tax-aware rebalancing ``` rebalance.stages.insert(3, Agent("tax_optimizer", smart_llm, system_prompt="Adjust the trades to harvest losses and avoid short-term gains where possible."), ) ``` ______________________________________________________________________ ## When to Use | Situation | Fit | | -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | | A fixed profile → scan → allocate → check sequence | ✅ Pipeline | | Each stage builds on the previous output | ✅ Pipeline | | The allocation must iterate to a quality bar | ❌ Use [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md) | | Two strategists should argue allocations | ❌ Use [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) | ______________________________________________________________________ ## Cost Profile | Stage | Typical model | Avg cost | Volume (5k clients/quarter) | | --------------------- | ------------- | ----------- | --------------------------- | | Profiler + compliance | gpt-4o-mini | $0.0008 | $4 | | Market + allocation | claude-sonnet | $0.008 | $40 | | **Per rebalance** | mix | **~$0.009** | **~$44/quarter** | ______________________________________________________________________ ## See Also - [Pipeline pattern](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) - [Portfolio Review](https://pyagent.org/cookbook/finance-trading/portfolio-review/index.md) — specialist analysts + an evaluator-optimizer memo - [Robo-Advisor Onboarding](https://pyagent.org/cookbook/finance-trading/robo-advisor/index.md) — role-based onboarding that precedes a rebalance - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build an Emergent Multi-Agent NPC World in Python Scripted NPCs feel static. This recipe uses the **Blackboard** pattern to make a small world come alive: each NPC reads the parts of a shared world state it cares about and writes back its actions, so behavior *emerges* from agents reacting to each other's changes — a builder gathers where the explorer found resources, the trader prices what the builder produces. **Patterns used:** Blackboard ______________________________________________________________________ ## Architecture ``` flowchart TD subgraph BB[Shared Blackboard] W[world_map] R[resources] E[economy] end EX[Explorer NPC] -->|writes| W EX -->|writes| R BU[Builder NPC] -->|reads| R BU -->|writes| W TR[Trader NPC] -->|reads| R TR -->|writes| E W --> EX R --> BU E --> TR ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.structural import Blackboard from pyagent_patterns.structural.blackboard import BlackboardAgent from pyagent_providers import OpenAILLM world = Blackboard( agents=[ BlackboardAgent( agent=Agent( "explorer", OpenAILLM("gpt-4o-mini"), system_prompt=( "You are an explorer NPC. Scout unexplored tiles and report new terrain " "and resource deposits you find. Write concise structured updates." ), ), reads=["task", "world_map"], writes=["world_map", "resources"], ), BlackboardAgent( agent=Agent( "builder", OpenAILLM("gpt-4o-mini"), system_prompt=( "You are a builder NPC. Using known resources, decide what to construct " "and where. Consume resources and add structures to the world map." ), ), reads=["resources", "world_map"], writes=["world_map"], ), BlackboardAgent( agent=Agent( "trader", OpenAILLM("gpt-4o-mini"), system_prompt=( "You are a trader NPC. From available resources and structures, set prices " "and propose trades. Update the economy with supply, demand, and prices." ), ), reads=["resources", "world_map"], writes=["economy"], ), ], rounds=3, initial_state={"world_map": "fog of war; spawn town at (0,0)", "resources": "", "economy": ""}, ) result = asyncio.run(world.run("Simulate one in-game day in the frontier region.")) print(result.output) print(f"Final world state keys: {list(result.metadata['final_state'].keys())}") ``` ______________________________________________________________________ ## Expected output ``` Day summary: - Explorer charted tiles (1,0)-(3,2); found iron at (2,1), timber at (3,2). - Builder raised a sawmill at (3,2) and a forge near the iron, consuming timber. - Trader set iron 12g, planks 4g; opened a caravan route to the spawn town. Final world state keys: ['world_map', 'resources', 'economy'] ``` Run more `rounds` and the NPCs keep reacting to each other's writes — emergent supply chains and settlements form without any of it being scripted. ______________________________________________________________________ ## Customization ### Run a longer simulation ``` world = Blackboard(agents=world._agents, rounds=10, initial_state={"world_map": "...", "resources": "", "economy": ""}) ``` ### Add an event logger ``` from pyagent_patterns.structural.blackboard import BlackboardAgent world.agents.append( BlackboardAgent( agent=Agent("chronicler", OpenAILLM("gpt-4o-mini"), system_prompt="Write a one-line journal entry summarising what changed this round."), reads=["world_map", "resources", "economy"], writes=["chronicle"], ) ) ``` ### Seed a scenario Pre-populate `initial_state` (e.g. a drought, a gold rush) to steer emergent behavior without scripting it. ______________________________________________________________________ ## When to Use | Situation | Use Blackboard? | | ------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------- | | Agents coordinate through shared, evolving state | ✅ Yes | | You want emergent behavior from local reactions | ✅ Yes | | There's a fixed order of steps | ❌ Use [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) | | A central coordinator should assign all work | ❌ Use [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) | ______________________________________________________________________ ## Cost Profile | Driver | Typical model | Avg cost | Notes | | -------------------------------- | ------------- | ------------ | ------------------------- | | Per agent-turn | gpt-4o-mini | $0.0004 | 3 NPCs | | Per round | — | ~$0.0012 | one turn each | | **Per simulated day (3 rounds)** | gpt-4o-mini | **~$0.0036** | scales with NPCs × rounds | Cost grows with `agents × rounds`, so cap rounds (or prune idle NPCs) for large worlds. ______________________________________________________________________ ## See Also - [Blackboard pattern](https://pyagent.org/packages/patterns/structural/blackboard/index.md) - [Trip-Planning Swarm](https://pyagent.org/cookbook/travel-hospitality/trip-planner/index.md) — another decentralized, emergent approach - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Policy Briefing Pipeline in Python A ministerial brief pulls together economic, legal, and social analysis — work that naturally maps to departments and the analysts within them. This recipe uses the **Hierarchical** pattern: a director decomposes the question into department subtasks, each department lead coordinates its analysts, and results flow back up into one synthesized brief. **Patterns used:** Hierarchical ______________________________________________________________________ ## Architecture ``` flowchart TD Q[Policy Question] --> D[Director\nDecompose + synthesize] D --> EL[Economics Lead] D --> LL[Legal Lead] EL --> E1[Fiscal Analyst] EL --> E2[Labor Analyst] LL --> L1[Statute Analyst] LL --> L2[Rights Analyst] E1 --> D E2 --> D L1 --> D L2 --> D D --> B[Ministerial Brief] ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Hierarchical from pyagent_patterns.orchestration.hierarchical import Team from pyagent_providers import AnthropicLLM, OpenAILLM briefing = Hierarchical( manager=Agent( "policy_director", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "Decompose the policy question into subtasks for the Economics and Legal teams. " "After receiving both teams' outputs, synthesize a 1-page ministerial brief: " "summary, options, risks, and a recommendation. Flag where teams disagree." ), ), teams=[ Team( name="Economics", lead=Agent( "economics_lead", OpenAILLM("gpt-4o-mini"), system_prompt="Coordinate the fiscal and labor analyses into one economic assessment.", ), workers=[ Agent( "fiscal_analyst", OpenAILLM("gpt-4o-mini"), system_prompt="Estimate budget impact, revenue effects, and cost over 5 years.", ), Agent( "labour_analyst", OpenAILLM("gpt-4o-mini"), system_prompt="Assess employment, wage, and regional labor-market effects.", ), ], ), Team( name="Legal", lead=Agent( "legal_lead", OpenAILLM("gpt-4o-mini"), system_prompt="Coordinate statutory and rights analyses into one legal assessment.", ), workers=[ Agent( "statute_analyst", OpenAILLM("gpt-4o-mini"), system_prompt="Identify enabling legislation, required amendments, and precedent.", ), Agent( "rights_analyst", OpenAILLM("gpt-4o-mini"), system_prompt="Assess civil-rights, privacy, and equality-impact considerations.", ), ], ), ], ) result = asyncio.run(briefing.run( "Should the city introduce a congestion charge for the downtown core?" )) print(result.output) print(f"Teams: {result.metadata['team_names']}, workers: {result.metadata['total_workers']}") ``` ______________________________________________________________________ ## Expected output ``` MINISTERIAL BRIEF — Downtown Congestion Charge Summary: A £12/day charge could cut peak traffic ~18% and raise ~£40M/yr. Economics: net fiscal positive by year 2; labor impact concentrated in logistics. Legal: requires a Transport Act amendment; equality assessment needed for low-income drivers. Options: (A) full charge, (B) peak-only, (C) phased pilot. Recommendation: Option C — 12-month pilot with exemptions, then review. Teams: ['Economics', 'Legal'], workers: 4 ``` ______________________________________________________________________ ## Customization ### Add a costing worker ``` briefing.teams[0].workers.append( Agent("costing_analyst", OpenAILLM("gpt-4o-mini"), system_prompt="Produce a 5-year cost-benefit estimate with assumptions."), ) ``` ### Add a third team ``` from pyagent_patterns.orchestration.hierarchical import Team briefing.teams.append( Team(name="Public Health", lead=Agent("ph_lead", OpenAILLM("gpt-4o-mini"), system_prompt="Coordinate the public-health assessment."), workers=[Agent("epi", OpenAILLM("gpt-4o-mini"), system_prompt="Assess health and equity impacts.")]), ) ``` ### Cheaper drafts Swap the director to `claude-haiku-3-5-20241022` for internal drafts; reserve sonnet for ministerial briefs. ______________________________________________________________________ ## When to Use | Situation | Use Hierarchical? | | -------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | Work decomposes into departments/teams with their own sub-work | ✅ Yes | | You need one synthesized output from many specialists | ✅ Yes | | There's a single flat pool of workers, no team structure | ❌ Use [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) | | Specialists should debate to a verdict | ❌ Use [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) | ______________________________________________________________________ ## Cost Profile | Tier | Typical model | Avg cost | Volume (100 briefs/mo) | | --------------------------------- | ------------- | ----------- | ---------------------- | | Director (decompose + synthesize) | claude-sonnet | $0.008 | $0.80/mo… ×100 = $80 | | Leads ×2 + analysts ×4 | gpt-4o-mini | $0.004 | $40/mo | | **Per brief** | mix | **~$0.012** | **~$120/mo** | ______________________________________________________________________ ## See Also - [Hierarchical pattern](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) - [Literature Review Team](https://pyagent.org/cookbook/research-analysis/literature-review/index.md) — hierarchical research with sub-teams - [Regulatory Compliance Checker](https://pyagent.org/cookbook/legal-compliance/compliance-checker/index.md) — supervisor–worker compliance analysis - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Clinical Note Summarizer in Python Clinical notes are dense, inconsistent, and high-stakes — a missed allergy or a hallucinated dose is a patient-safety event. This recipe extracts structured findings with a **Pipeline**, then runs a **Self-Reflection** accuracy pass so the summary critiques and corrects itself against the source note before any clinician sees it. **Patterns used:** [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) ______________________________________________________________________ ## Architecture ``` flowchart TD N[Clinical Note] --> E[Extract\ndiagnoses, meds, vitals] E --> S[Summarize\nclinician-ready note] S --> R{Self-Reflection} R -->|critique vs source| R R -->|accurate| F[Safety Flags] F --> O[Final Summary + Flags] ``` ______________________________________________________________________ ## Implementation ``` pip install pyagent-patterns pyagent-providers ``` ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Pipeline from pyagent_patterns.resolution import SelfReflection from pyagent_patterns.composite import CompositePattern from pyagent_providers import AnthropicLLM, OpenAILLM # Cheap model for extraction, stronger model for the safety-critical summary. fast_llm = OpenAILLM("gpt-4o-mini") smart_llm = AnthropicLLM("claude-sonnet-4-20250514") # ── Stage 1+2: extract structured findings, then draft the summary ────────────── extract_and_draft = Pipeline(stages=[ Agent( "extractor", fast_llm, system_prompt=( "Extract from the clinical note as structured bullets: active diagnoses, " "current medications (name + dose + route), allergies, and the latest vitals. " "Copy values verbatim — never infer or round. Mark anything illegible as [UNREADABLE]." ), ), Agent( "drafter", smart_llm, system_prompt=( "Write a concise clinician-ready summary from the extracted bullets: " "one-line problem list, active meds, and pending follow-ups. Keep it under 150 words." ), ), ]) # ── Stage 3: self-reflection accuracy pass (drafts → critiques → corrects) ────── accuracy_pass = SelfReflection( agent=Agent( "summary_reviewer", smart_llm, system_prompt=( "Review the draft summary against the original note. Correct any value that does not " "match the source, and append a SAFETY FLAGS section listing: unsupported claims, " "missing critical values (e.g. allergies, abnormal vitals), and dose ambiguities. " "Reply 'ACCURATE' on the first line when no corrections remain." ), ), max_rounds=2, stop_phrase="ACCURATE", ) clinical_summarizer = CompositePattern(patterns=[extract_and_draft, accuracy_pass]) SAMPLE_NOTE = """\ 68M, CHF exacerbation. Hx: HFrEF (EF 30%), T2DM, CKD3. Allergy: penicillin (rash). Meds: furosemide 40 mg PO BID, metoprolol succ 50 mg daily, metformin 1000 mg BID, lisinopril 10 mg daily. Vitals: BP 148/92, HR 96, SpO2 91% RA, wt +3 kg from baseline. Plan: increase furosemide, daily weights. """ async def main(): result = await clinical_summarizer.run(SAMPLE_NOTE) print(result.output) asyncio.run(main()) ``` ______________________________________________________________________ ## Expected Output ``` PROBLEM LIST: CHF exacerbation (HFrEF, EF 30%), T2DM, CKD stage 3. ACTIVE MEDS: furosemide 40 mg PO BID (increased this admission), metoprolol succinate 50 mg daily, metformin 1000 mg BID, lisinopril 10 mg daily. FOLLOW-UP: daily weights; reassess volume status. SAFETY FLAGS: - Allergy to penicillin documented — verify before any antibiotic order. - SpO2 91% on room air — below typical threshold; confirm oxygen plan. - Metformin + CKD3: review for dose adjustment / hold criteria. ``` The Self-Reflection pass is what turns a plausible-sounding summary into one that has been checked back against the source — the difference that matters in a clinical setting. ______________________________________________________________________ ## Customization ### Add ICD-10 coding Append a coding stage that maps the problem list to billing codes: ``` coder = Agent( "icd10_coder", fast_llm, system_prompt="Map each diagnosis in the problem list to its ICD-10 code. Output 'diagnosis — code'.", ) coding_pipeline = Pipeline(stages=[extract_and_draft, coder]) ``` ### Redact PHI before the summary leaves your VPC Wrap the input in a guardrail so identifiers never reach a third-party model: ``` from pyagent_patterns.guardrails import GuardrailChain, PIIGuard phi_guard = GuardrailChain([PIIGuard(redact=True)]) safe_note = phi_guard.check(SAMPLE_NOTE).sanitized_content or SAMPLE_NOTE ``` ### Batch a whole shift's notes ``` async def summarize_all(notes: list[str]) -> list[str]: results = await asyncio.gather(*(clinical_summarizer.run(n) for n in notes)) return [r.output for r in results] ``` ______________________________________________________________________ ## When to Use | Situation | Fit | | ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Output must be checked back against a source document | ✅ Self-Reflection | | Fixed extract → summarize stages | ✅ Pipeline | | You need multiple independent reviewers to agree | ❌ Use [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) ([Essay Grading](https://pyagent.org/cookbook/education/essay-grader/index.md)) | | A second, different agent should review | ❌ Use [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) | ______________________________________________________________________ ## Cost Profile | Stage | Typical model | Avg cost | Volume (5k notes/day) | | ------------------------------ | ------------- | ------------ | --------------------- | | Extract | gpt-4o-mini | $0.0004 | $60/mo | | Draft + reflection (≤2 rounds) | claude-sonnet | $0.008 | $1,200/mo | | **Per note** | mix | **~$0.0084** | **~$1.26k/mo** | Cap `max_rounds` to bound cost; most notes converge in one reflection round. ______________________________________________________________________ ## See Also - [Pipeline pattern](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Self-Reflection pattern](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) - [Contract Review](https://pyagent.org/cookbook/legal-compliance/contract-review/index.md) — review-before-signoff on legal text - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent CV Screening Workflow in Python Screening is faster and fairer when distinct rubrics are scored independently rather than by one model forming a halo impression. This recipe anonymizes a CV with a guardrail, then uses **Fan-Out / Fan-In** to score skills, experience, and collaboration *in parallel*, and an aggregator merges them into one hire/no-hire recommendation with the rubric breakdown. **Patterns used:** [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Guardrails](https://pyagent.org/guides/guardrails/index.md) ______________________________________________________________________ ## Architecture ``` flowchart TD CV[CV] --> G[Guardrail\nstrip protected attributes] G --> S[Skills Reviewer] G --> E[Experience Reviewer] G --> C[Collaboration Reviewer] S --> A[Aggregator\nhire / no-hire] E --> A C --> A A --> R[Recommendation + breakdown] ``` ______________________________________________________________________ ## Implementation ``` pip install pyagent-patterns pyagent-providers ``` ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import FanOutFanIn from pyagent_patterns.guardrails import GuardrailChain, PIIGuard from pyagent_providers import AnthropicLLM, OpenAILLM fast_llm = OpenAILLM("gpt-4o-mini") smart_llm = AnthropicLLM("claude-sonnet-4-20250514") # ── Strip names, contact details, and other PII before any model sees the CV ──── anonymizer = GuardrailChain([PIIGuard(redact=True)]) # ── Three independent rubric reviewers run in parallel on the same CV ─────────── screener = FanOutFanIn( agents=[ Agent("skills", smart_llm, system_prompt="Score 0-10 on the required technical skills. Quote evidence from the CV."), Agent("experience", smart_llm, system_prompt="Score 0-10 on relevant experience and impact. Quote evidence."), Agent("collaboration", fast_llm, system_prompt="Score 0-10 on collaboration and ownership signals. Quote evidence."), ], aggregator=Agent( "panel", smart_llm, system_prompt=( "Combine the three rubric scores into one recommendation: STRONG HIRE / HIRE / NO HIRE. " "Show each rubric score, the overall, and the single biggest risk." ), ), ) SAMPLE_CV = """\ Senior Backend Engineer, 7 yrs. Built a payments platform handling 4k TPS (Python, Postgres, Kafka). Led a 5-person team; cut p99 latency 40%. Open-source: maintainer of a popular asyncio library. """ async def main(): safe_cv = anonymizer.check(SAMPLE_CV).sanitized_content or SAMPLE_CV result = await screener.run(f"Role: Staff Backend Engineer.\n\nCV:\n{safe_cv}") print(result.output) asyncio.run(main()) ``` ______________________________________________________________________ ## Expected Output ``` Skills: 9/10 — async Python + Kafka + Postgres match the stack; evidence: "4k TPS payments platform". Experience: 8/10 — 7 yrs, real scale and a latency win; slightly light on staff-level scope. Collaboration: 8/10 — led a 5-person team; OSS maintainer signals ownership. RECOMMENDATION: HIRE (overall 8.3/10) Biggest risk: limited evidence of cross-org/staff-level influence — probe in the interview. ``` Running rubrics in parallel keeps each score independent (no halo effect) and gives you an auditable breakdown rather than one opaque verdict. ______________________________________________________________________ ## Customization ### Weighted aggregation ``` screener.aggregator.system_prompt += " Weight skills 50%, experience 35%, collaboration 15%." ``` ### Add a structured scorecard for your ATS ``` from pyagent_patterns.orchestration import Pipeline to_json = Agent("scorecard", fast_llm, system_prompt='Output the result as JSON: {"recommendation":..,"scores":{..}}.') screener_with_json = Pipeline(stages=[screener, to_json]) ``` ### Stricter blind screening Add a `ContentGuard` that also removes school names and graduation years to reduce age/pedigree bias: ``` from pyagent_patterns.guardrails import ContentGuard import re anonymizer = GuardrailChain([ PIIGuard(redact=True), ContentGuard(deny_patterns=[re.compile(r"\b(19|20)\d{2}\b")]), # strip years ]) ``` ______________________________________________________________________ ## When to Use | Situation | Fit | | --------------------------------------------------- | ---------------------------------------------------------------------------------------- | | Independent rubrics scored in parallel, merged once | ✅ Fan-Out / Fan-In | | Protected attributes must be stripped first | ✅ Guardrails | | Reviewers should debate borderline candidates | ❌ Use [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) | | A fixed sequence of screening stages | ❌ Use [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) | ______________________________________________________________________ ## Cost Profile | Stage | Typical model | Avg cost | Volume (2k CVs/day) | | -------------------- | ------------- | ----------- | ------------------- | | 3 parallel reviewers | sonnet + mini | $0.007 | $420/mo | | Aggregator | claude-sonnet | $0.003 | $180/mo | | **Per CV** | mix | **~$0.010** | **~$600/mo** | Parallel fan-out keeps wall-clock to `max(reviewer latency)` rather than the sum. ______________________________________________________________________ ## See Also - [Fan-Out / Fan-In pattern](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) · [Guardrails guide](https://pyagent.org/guides/guardrails/index.md) - [Code Review](https://pyagent.org/cookbook/software-engineering/code-review/index.md) — parallel + iterative review of code - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Regulatory Compliance Checker in Python Compliance gap analysis splits naturally into "what does the regulation require?" and "what do our controls actually do?" — then a comparison. This recipe uses the **Hierarchical** pattern: a compliance director delegates an Obligations team and a Controls team, each with specialists, and synthesizes their outputs into a gap-analysis audit report. **Patterns used:** Hierarchical ______________________________________________________________________ ## Requirements - **Functional** — extract every legal obligation from a regulation, map obligations to existing controls, flag gaps where evidence doesn't support a control, and synthesize a single gap-analysis report. - **Non-functional** — the two sub-teams (Obligations, Controls) must be able to work without blocking on each other; only the final synthesis depends on both. - **Audit** — every flagged gap must trace back to the specific obligation and the specific control (or absence of one) that produced it. - **Not required** — no persistent memory across separate compliance checks (each regulation/scope is evaluated independently); no human approval gate in this recipe (a gap-analysis report is a draft for human review downstream, not an action taken automatically). ## Architecture decisions | Decision | Why | Why not the alternative | | -------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Hierarchical** for the two-team structure | Work decomposes into two genuinely distinct sub-teams (Obligations, Controls) that can each work internally in parallel, coordinated by one director. | **Orchestrator-Workers** would imply the team structure itself is discovered dynamically per-input; here it's fixed — every compliance check always needs an Obligations team and a Controls team. | | Two workers per team lead, not a flat 5-agent Supervisor | `requirement_extractor`+`scope_analyst` and `policy_mapper`+`evidence_checker` are each meaningfully coupled sub-tasks within their team — a flat Supervisor routing to 4 independent specialists would lose that grouping. | A flat structure works when specialists are independent; here Controls' output genuinely depends on Obligations' scope, which the hierarchy expresses naturally via the director's synthesis step. | | Uniform `fast` model for all 6 leaf workers, `smart` only for the director | Extraction/mapping/checking are pattern-matching tasks; synthesizing a coherent gap-analysis narrative across both teams' outputs needs stronger reasoning. | Using `smart` for every worker would multiply cost by ~7x for tasks that don't need it. | ## Four-pillar mapping | Requirement | Pillar | Capability | | --------------------------------------------------- | ------------- | ------------------------------------------ | | Two-team decomposition with a synthesizing director | Execution | `Hierarchical` pattern | | Track daily audit-run spend | Observability | `observability.cost_budget` | | Trace each team's internal calls | Observability | `observability.tracing` | | Version the check's scope/config as it evolves | Blueprint | `pyagent-blueprint diff` between revisions | ## Blueprint (declarative form) The real, verified file at `examples/cookbook/legal-compliance/compliance_checker/blueprint.yaml`, compiled against `PyAgentAdapter` as part of this repo's test suite: ``` api_version: pyagent/v1 metadata: name: compliance-checker version: 1.0.0 description: Director delegates obligations + controls teams; synthesizes gap-analysis report providers: fast: { model: gpt-4o-mini } smart: { model: claude-sonnet-4-20250514 } agents: compliance_director: { provider: smart, prompt: "Decompose, delegate, then synthesize gap-analysis report." } obligations_lead: { provider: fast, prompt: "Extract and structure all legal obligations." } controls_lead: { provider: fast, prompt: "Map obligations to controls; flag gaps." } requirement_extractor: { provider: fast, prompt: "Extract every MUST/SHALL requirement." } scope_analyst: { provider: fast, prompt: "Identify departments, systems, data in scope." } policy_mapper: { provider: fast, prompt: "Map obligations to existing policies." } evidence_checker: { provider: fast, prompt: "Assess whether audit evidence exists for each control." } workflows: check: pattern: hierarchical agents: manager: compliance_director teams: - { name: Obligations, lead: obligations_lead, workers: [requirement_extractor, scope_analyst] } - { name: Controls, lead: controls_lead, workers: [policy_mapper, evidence_checker] } observability: tracing: { enabled: true } cost_budget: { daily_usd: 100.0, alert_threshold: 0.8 } ``` ``` pyagent-blueprint validate compliance-checker.yaml pyagent-blueprint test compliance-checker.yaml ``` ## Production checklist Ran this exact blueprint through `PyAgentAdapter.compile()` and inspected the real diagnostics: - ✅ **The hierarchical check runs as declared** — `check` compiles and executes against the native pattern registry with no diagnostics on the workflow structure itself. - ⚠️ **`observability.cost_budget` is declared but not auto-enforced** — compiling this blueprint emits `BUDGET_UNSUPPORTED`: the $100/day budget is recorded in the spec but nothing stops a run from exceeding it. Wire real enforcement via `graph.wire_cost_tracker(tracker)` if you need a hard stop. - **No recovery policy is declared** — if a worker fails mid-run, this blueprint doesn't specify a retry/fallback; add one via the recovery block if that's a real requirement for your deployment. - **No human approval gate** — this composes cleanly with [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) as a follow-on step if the gap-analysis report needs sign-off before circulating outside the compliance team. ______________________________________________________________________ ## Architecture ``` flowchart TD Q[Regulation + Policies] --> D[Compliance Director\nDecompose + gap analysis] D --> OT[Obligations Lead] D --> CT[Controls Lead] OT --> O1[Requirement Extractor] OT --> O2[Scope Analyst] CT --> C1[Policy Mapper] CT --> C2[Evidence Checker] O1 --> D O2 --> D C1 --> D C2 --> D D --> R[Gap-Analysis Report] ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Hierarchical from pyagent_patterns.orchestration.hierarchical import Team from pyagent_providers import AnthropicLLM, OpenAILLM checker = Hierarchical( manager=Agent( "compliance_director", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "Decompose the review into obligations and controls subtasks. After both teams report, " "produce a gap-analysis report: each obligation, the mapped control (or NONE), a " "compliance status (Met/Partial/Gap), and a remediation action. Prioritize gaps by risk." ), ), teams=[ Team( name="Obligations", lead=Agent( "obligations_lead", OpenAILLM("gpt-4o-mini"), system_prompt="Consolidate the extracted obligations into a clean numbered list.", ), workers=[ Agent( "requirement_extractor", OpenAILLM("gpt-4o-mini"), system_prompt="Extract each discrete obligation ('the firm must...') from the regulation.", ), Agent( "scope_analyst", OpenAILLM("gpt-4o-mini"), system_prompt="For each obligation, note who/what it applies to and any thresholds.", ), ], ), Team( name="Controls", lead=Agent( "controls_lead", OpenAILLM("gpt-4o-mini"), system_prompt="Consolidate the control findings into one mapping table.", ), workers=[ Agent( "policy_mapper", OpenAILLM("gpt-4o-mini"), system_prompt="Map internal policies to obligations; mark any obligation with no policy.", ), Agent( "evidence_checker", OpenAILLM("gpt-4o-mini"), system_prompt="For each mapped control, note whether evidence of operation exists.", ), ], ), ], ) result = asyncio.run(checker.run( "Regulation: GDPR Art. 30 (records of processing) and Art. 33 (breach notification). " "Internal policies: data-inventory policy v2, incident-response runbook attached." )) print(result.output) print(f"Teams: {result.metadata['team_names']}") ``` ______________________________________________________________________ ## Expected output ``` GAP-ANALYSIS REPORT — GDPR Art. 30 & 33 Obligation 1 (records of processing) → Data-Inventory Policy v2 → Met. Obligation 2 (72-hour breach notification) → Incident-Response Runbook → Partial (runbook lacks the 72-hour clock + supervisory-authority template). Obligation 3 (processor records) → NONE → Gap (high risk). Remediation (by risk): add processor register; add 72-hour timer + notification template. Teams: ['Obligations', 'Controls'] ``` ______________________________________________________________________ ## Customization ### Add an evidence team ``` from pyagent_patterns.orchestration.hierarchical import Team checker.teams.append( Team(name="Evidence", lead=Agent("evidence_lead", OpenAILLM("gpt-4o-mini"), system_prompt="Consolidate control-evidence findings."), workers=[Agent("sampler", OpenAILLM("gpt-4o-mini"), system_prompt="Note what evidence would prove each control operates.")]), ) ``` ### Severity scoring ``` checker.manager.system_prompt += " Score each gap by risk (High/Medium/Low) and sort the remediation plan by it." ``` ### Multiple regulations at once Run `checker.run(...)` per regulation with `asyncio.gather` and merge the gap reports. ______________________________________________________________________ ## When to Use | Situation | Use Hierarchical? | | -------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | Work splits into teams (obligations vs controls) with sub-work | ✅ Yes | | You need one synthesized comparison/report | ✅ Yes | | A single reviewer iteratively critiques one document | ❌ Use [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) | | A flat pool of workers, no teams | ❌ Use [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) | ______________________________________________________________________ ## Cost Profile | Tier | Typical model | Avg cost | Volume (200 reviews/mo) | | --------------------- | ------------- | ----------- | ----------------------- | | Director | claude-sonnet | $0.009 | $1.80 → ×200 = $360… | | Leads ×2 + workers ×4 | gpt-4o-mini | $0.004 | $0.80 → ×200 = $160 | | **Per review** | mix | **~$0.013** | **~$2.6k/yr** | ______________________________________________________________________ ## See Also - [Hierarchical pattern](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) - [Contract Review](https://pyagent.org/cookbook/legal-compliance/contract-review/index.md) — clause-by-clause reflection on a single document - [Policy Briefing Pipeline](https://pyagent.org/cookbook/government/policy-briefing/index.md) — hierarchical analysis for policy - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Contract Review Assistant in Python A single model reviewing a contract will confidently miss the clause that matters. This recipe uses **Cross-Reflection**: a drafting "counsel" proposes redlines and risk notes, an independent reviewing "partner" critiques them, and they iterate until the review is solid — surfacing liability before a human ever opens the document. **Patterns used:** [Cross-Reflection](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Guardrails](https://pyagent.org/guides/guardrails/index.md) ______________________________________________________________________ ## Architecture ``` flowchart TD C[Clause] --> GD[Guardrail\nstrip client identifiers] GD --> G[Counsel\npropose redlines + risk] G --> R[Partner\ncritique redlines] R -->|revise| G R -->|approved| O[Reviewed Redlines + Flags] ``` ______________________________________________________________________ ## Implementation ``` pip install pyagent-patterns pyagent-providers ``` ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.resolution import CrossReflection from pyagent_patterns.guardrails import GuardrailChain, PIIGuard from pyagent_providers import AnthropicLLM smart_llm = AnthropicLLM("claude-sonnet-4-20250514") # Keep counterparty / client identifiers out of third-party model logs. confidentiality_guard = GuardrailChain([PIIGuard(redact=True)]) review = CrossReflection( generator=Agent( "counsel", smart_llm, system_prompt=( "You are reviewing counsel. For the clause, propose specific redlines and explain the " "risk each one mitigates. Be concrete: quote the language you would change." ), ), reviewer=Agent( "partner", smart_llm, system_prompt=( "You are the reviewing partner. Critique the proposed redlines: flag anything that " "increases our liability, weakens an indemnity, or misreads the clause. Reply 'APPROVED' " "only when the redlines are sound." ), ), max_rounds=2, ) CLAUSE = ( "Limitation of Liability. In no event shall either party's aggregate liability exceed the fees " "paid by Customer in the three (3) months preceding the claim. This cap applies to all claims, " "including those arising from gross negligence or breach of the confidentiality obligations." ) async def main(): safe_clause = confidentiality_guard.check(CLAUSE).sanitized_content or CLAUSE result = await review.run(safe_clause) print(result.output) # surface to a human for final sign-off asyncio.run(main()) ``` ______________________________________________________________________ ## Expected Output ``` REVIEWED REDLINES — Limitation of Liability 1. Carve out gross negligence, willful misconduct, and confidentiality breaches from the cap. Risk: as written, the 3-month cap shields the counterparty even for a data breach. (partner: agreed, highest priority) 2. Increase the cap to 12 months' fees (market standard for this contract value). 3. Make the cap mutual and add a separate, higher cap for IP-indemnity claims. FLAGS: the original cap is unusually low and applies to gross negligence — do not sign as-is. ``` The partner's critique is what catches the buried "including … gross negligence" — a second, adversarial reading that a single pass tends to gloss over. ______________________________________________________________________ ## Customization ### Review a whole contract clause-by-clause ``` async def review_contract(clauses: list[str]) -> list[str]: safe = [confidentiality_guard.check(c).sanitized_content or c for c in clauses] results = await asyncio.gather(*(review.run(c) for c in safe)) return [r.output for r in results] ``` ### Add a playbook check Prepend the firm's negotiation playbook to the counsel prompt so redlines follow house positions: ``` review.generator.system_prompt += "\n\nFollow our playbook: liability cap ≥ 12 months; mutual indemnities." ``` ### Gate sign-off behind a human Pair with [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) so any clause the partner flags as high-risk routes to an attorney — see the [Support Router](https://pyagent.org/cookbook/customer-support/support-router/index.md). ______________________________________________________________________ ## When to Use | Situation | Fit | | --------------------------------------------------- | --------------------------------------------------------------------------------------------------- | | A second, independent agent should review the first | ✅ Cross-Reflection | | Confidential identifiers must be stripped | ✅ Guardrails | | The same agent should critique its own work | ❌ Use [Self-Reflection](https://pyagent.org/packages/patterns/resolution/self-reflection/index.md) | | Many reviewers vote on a verdict | ❌ Use [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) | ______________________________________________________________________ ## Cost Profile | Stage | Typical model | Avg cost | Volume (500 contracts/mo, 20 clauses each) | | ----------------------------- | ------------- | ------------- | ------------------------------------------ | | Counsel + partner (≤2 rounds) | claude-sonnet | $0.010/clause | ~$100/mo | | **Per clause** | claude-sonnet | **~$0.010** | **~$100/mo** | Cross-Reflection roughly doubles per-clause cost vs a single pass — worth it for high-liability clauses; use a single pass for boilerplate. ______________________________________________________________________ ## See Also - [Cross-Reflection pattern](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) · [Guardrails guide](https://pyagent.org/guides/guardrails/index.md) - [Regulatory Compliance Checker](https://pyagent.org/cookbook/legal-compliance/compliance-checker/index.md) — hierarchical gap analysis against regulations - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Marketing Campaign Planner in Python A launch needs email, social, and blog working together — but drafting them one after another loses the parallelism and the cross-channel consistency. This recipe uses **Fan-Out / Fan-In**: channel specialists draft simultaneously, then an aggregator weaves them into one coordinated campaign plan with a shared timeline and message. **Patterns used:** [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) ______________________________________________________________________ ## Architecture ``` flowchart TD B[Launch Brief] --> E[Email Specialist] B --> S[Social Specialist] B --> G[Blog Specialist] E --> A[Aggregator\nmerge + timeline] S --> A G --> A A --> P[Coordinated Campaign Plan] ``` ______________________________________________________________________ ## Implementation ``` pip install pyagent-patterns pyagent-providers ``` ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import FanOutFanIn from pyagent_providers import AnthropicLLM, OpenAILLM fast_llm = OpenAILLM("gpt-4o-mini") smart_llm = AnthropicLLM("claude-sonnet-4-20250514") campaign = FanOutFanIn( agents=[ Agent("email", fast_llm, system_prompt="Draft a 3-email launch sequence: teaser, launch, follow-up. Subject + 2 lines each."), Agent("social", fast_llm, system_prompt="Draft a 1-week social calendar (LinkedIn + X): 5 posts with hooks and CTAs."), Agent("blog", fast_llm, system_prompt="Outline a launch blog post: title, 5 section headers, and the key takeaway."), ], aggregator=Agent( "planner", smart_llm, system_prompt=( "Merge the channel drafts into ONE coordinated campaign plan: a shared core message, a " "day-by-day timeline across channels, and the single primary call-to-action." ), ), ) async def main(): brief = "Launch: PyAgent 1.0 — multi-agent orchestration for Python. Audience: senior Python engineers." result = await campaign.run(brief) print(result.output) asyncio.run(main()) ``` ______________________________________________________________________ ## Expected Output ``` CAMPAIGN: PyAgent 1.0 Launch — core message: "18 named patterns, zero boilerplate" Primary CTA: pip install pyagent-all → quickstart Timeline Day 1 Blog goes live + teaser email + LinkedIn announcement post Day 2 X thread (pattern of the day) + email "what's new" Day 3 Social: benchmark chart post Day 5 Follow-up email (cookbook recipes) + LinkedIn case-study post Assets: 3 emails, 5 social posts, 1 blog outline — all on the shared message. ``` Because the channels are drafted in parallel and merged once, the plan stays consistent and the wall-clock is one channel's latency, not three. ______________________________________________________________________ ## Customization ### Add a brand-voice review ``` from pyagent_patterns.orchestration import Pipeline voice = Agent("brand_voice", fast_llm, system_prompt="Rewrite all copy to match our voice: confident, concrete, no hype.") campaign_on_brand = Pipeline(stages=[campaign, voice]) ``` ### Add a paid-ads channel ``` campaign.agents.append( Agent("ads", fast_llm, system_prompt="Draft 3 Google/LinkedIn ad variations with headlines + descriptions.") ) ``` ### Localize for multiple markets ``` async def localize(plan: str, locales: list[str]) -> dict: t = Agent("translator", fast_llm, system_prompt="Localize this campaign for the given market, adapting idioms.") out = await asyncio.gather(*(t.run(f"{plan}\n\nMarket: {loc}") for loc in locales)) return dict(zip(locales, (r.output for r in out))) ``` ______________________________________________________________________ ## When to Use | Situation | Fit | | -------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | Independent specialists drafted in parallel, merged once | ✅ Fan-Out / Fan-In | | Channels must stay consistent under one message | ✅ (aggregator enforces it) | | Which channels to use depends on the brief | ❌ Use [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) | | Drafts must iterate to a quality bar | ❌ Use [Evaluator-Optimizer](https://pyagent.org/packages/patterns/resolution/evaluator-optimizer/index.md) | ______________________________________________________________________ ## Cost Profile | Stage | Typical model | Avg cost | Volume (200 campaigns/mo) | | --------------------- | ------------- | ----------- | ------------------------- | | 3 channel specialists | gpt-4o-mini | $0.0012 | $0.24 | | Aggregator | claude-sonnet | $0.004 | $0.80 | | **Per campaign** | mix | **~$0.005** | **~$1/mo** | A great starter recipe — cheap, fast, and the parallel structure generalizes to any multi-channel brief. ______________________________________________________________________ ## See Also - [Fan-Out / Fan-In pattern](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) - [Product Launch Planner](https://pyagent.org/cookbook/ecommerce-retail/product-launch-planner/index.md) — orchestrator-driven launch planning - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Writers' Room in Python A writers' room works because each person holds a distinct role and reacts to the others across several passes. This recipe uses the **Role-Based** pattern: a showrunner, staff writer, script editor, and continuity supervisor each contribute from their role and update their view over multiple rounds, converging on a coherent episode outline. **Patterns used:** Role-Based ______________________________________________________________________ ## Architecture ``` flowchart LR P[Episode Premise] --> R((Writers' Room)) R --> SR[Showrunner\nvision + arc] R --> SW[Staff Writer\nbeats + scenes] R --> SE[Script Editor\npacing + dialogue] R --> CS[Continuity\ncanon + callbacks] SR <--> SW SW <--> SE SE <--> CS R --> O[Episode Outline] ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.structural import RoleBased from pyagent_providers import AnthropicLLM room = RoleBased( agents=[ Agent( "showrunner", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "You are the showrunner. Hold the season arc and tone. Set the episode's emotional " "throughline and make the final call. Update your view if a better idea surfaces." ), ), Agent( "staff_writer", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "You are the staff writer. Break the premise into act beats and key scenes. " "Propose concrete story turns that serve the showrunner's throughline." ), ), Agent( "script_editor", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt=( "You are the script editor. Tighten pacing, flag slow scenes, and sharpen the " "act breaks. Keep the episode to a 42-minute runtime." ), ), Agent( "continuity", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt=( "You are continuity supervisor. Guard canon: character knowledge, timeline, and " "established facts. Suggest callbacks and flag contradictions." ), ), ], rounds=2, ) result = asyncio.run(room.run( "Premise: our detective discovers her mentor faked evidence years ago. " "Episode 6 of 8. Must move the season's corruption arc forward." )) print(result.output) print(f"Roles: {result.metadata['roles']}, rounds: {result.metadata['rounds']}") ``` ______________________________________________________________________ ## Expected output ``` EPISODE 6 — "Chain of Custody" (cold open + 4 acts, ~42 min) Act 1: Detective finds the doctored file; chooses silence (showrunner: this is the moral pivot). Act 2: She tests the theory; staff-writer adds the evidence-locker set piece. Act 3: Mentor senses she knows (editor trimmed a redundant interrogation). Act 4: Cliffhanger — she pulls the original case file (continuity: callback to E2 partner death). Roles: ['showrunner', 'staff_writer', 'script_editor', 'continuity'], rounds: 2 ``` ______________________________________________________________________ ## Customization ### More revision rounds ``` room = RoleBased(agents=room._agents, rounds=3) # deeper iteration for a season finale ``` ### Add a network-notes role ``` room.agents.append( Agent("network_exec", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Give network notes: broad appeal, ad breaks, and standards-and-practices flags."), ) ``` ### Control tone ``` room.agents[0].system_prompt += " Target tone: grounded prestige drama, no camp." ``` ______________________________________________________________________ ## When to Use | Situation | Use Role-Based? | | ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------ | | Distinct perspectives must collaborate and react over rounds | ✅ Yes | | Each agent holds a stable role/voice | ✅ Yes | | Roles map to teams with their own sub-work | ❌ Use [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) | | You only need parallel opinions merged once | ❌ Use [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) | ______________________________________________________________________ ## Cost Profile | Driver | Typical model | Avg cost | Notes | | ----------------------- | ------------------ | ----------- | -------------------------- | | 4 roles × 2 rounds | sonnet + haiku mix | $0.018 | senior roles on sonnet | | **Per episode outline** | mix | **~$0.018** | scales with roles × rounds | Put the senior roles (showrunner, writer) on a stronger model and support roles on a cheaper one. ______________________________________________________________________ ## See Also - [Role-Based pattern](https://pyagent.org/packages/patterns/structural/role-based/index.md) - [Software Startup Simulation](https://pyagent.org/cookbook/software-engineering/startup-simulation/index.md) — role-play with PM/Architect/Engineer/QA - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Property Valuation Pipeline in Python A property valuation moves through clear levels of abstraction: pull the raw facts, run comparable analysis, then write the narrative. This recipe uses the **Layered** pattern, where each layer's agents process the previous layer's output at a coarser granularity — the same separation an appraiser uses between data collection, analysis, and reporting. **Patterns used:** Layered ______________________________________________________________________ ## Architecture ``` flowchart TD L[Listing + Address] --> G[Layer 1 — Gather] G --> A[Layer 2 — Analyze] A --> N[Layer 3 — Narrate] N --> R[Valuation Report] subgraph G[Gather] G1[Property Facts] G2[Local Market] end subgraph A[Analyze] A1[Comparables] A2[Adjustments] end subgraph N[Narrate] N1[Report Writer] end ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.structural import Layered from pyagent_patterns.structural.layered import Layer from pyagent_providers import GeminiLLM, AnthropicLLM valuation = Layered( layers=[ Layer( name="gather", agents=[ Agent( "property_facts", GeminiLLM("gemini-2.5-flash"), system_prompt="Extract property facts: beds, baths, sqft, lot, year built, condition.", ), Agent( "market_facts", GeminiLLM("gemini-2.5-flash"), system_prompt="Summarize the local market: median price/sqft, days on market, trend.", ), ], ), Layer( name="analyze", agents=[ Agent( "comps", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Select 3-5 comparable sales and explain why each is comparable.", ), Agent( "adjustments", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Apply $ adjustments to each comp for size, condition, and location.", ), ], ), Layer( name="narrate", agents=[ Agent( "report_writer", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "Write a valuation report: estimated value range, the adjusted comps that " "support it, key risks, and a confidence level." ), ), ], ), ], ) result = asyncio.run(valuation.run( "123 Maple St: 3 bed / 2 bath, 1,650 sqft, built 1998, good condition, " "quiet suburban street. Recent nearby sales attached." )) print(result.output) print(f"Layers: {result.metadata['layer_names']}") ``` ______________________________________________________________________ ## Expected output ``` VALUATION REPORT — 123 Maple St Estimated value: $452,000 – $471,000 (point estimate $462,000) Comps: 3 adjusted sales within 0.4 mi, +$8k condition / −$5k size adjustments. Risks: thin comp set; rising days-on-market suggests softening demand. Confidence: Medium-High. Layers: ['gather', 'analyze', 'narrate'] ``` Each layer hands a cleaner, more abstract artefact to the next — raw facts → adjusted comps → narrative — so no single agent has to do everything at once. ______________________________________________________________________ ## Customization ### Add a QA layer ``` from pyagent_patterns.structural.layered import Layer valuation.layers.append( Layer(name="qa", agents=[Agent("reviewer", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="Sanity-check the value range against the comps; flag unsupported adjustments.")]), ) ``` ### Swap in live data sources Replace the gather-layer prompts with tool-using [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) agents that pull MLS comps and tax records. ### Batch a portfolio ``` async def value_all(listings: list[str]) -> list[str]: out = await asyncio.gather(*(valuation.run(x) for x in listings)) return [r.output for r in out] ``` ______________________________________________________________________ ## When to Use | Situation | Use Layered? | | ------------------------------------------------- | ------------------------------------------------------------------------------------------------ | | Work moves through distinct levels of abstraction | ✅ Yes | | Each level has several agents working in parallel | ✅ Yes | | It's a single linear chain of agents | ❌ Use [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) | | A manager delegates to teams | ❌ Use [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) | ______________________________________________________________________ ## Cost Profile | Layer | Typical model | Avg cost | Volume (1k valuations/mo) | | ------------------ | ------------- | ----------- | ------------------------- | | Gather (2 agents) | gemini-flash | $0.0008 | $0.80 → $24/mo | | Analyze (2 agents) | claude-sonnet | $0.006 | $180/mo | | Narrate (1 agent) | claude-sonnet | $0.004 | $120/mo | | **Per valuation** | mix | **~$0.011** | **~$330/mo** | ______________________________________________________________________ ## See Also - [Layered pattern](https://pyagent.org/packages/patterns/structural/layered/index.md) - [Property data feeds a Pipeline?](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) — when the stages are strictly linear - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Literature Review Team in Python A literature review is team work: some people find and triage sources, others extract findings and write the synthesis. This recipe uses the **Hierarchical** pattern — a research lead delegates a Discovery team and a Synthesis team, each with specialists, and merges their output into a cited review with a clear gap statement. **Patterns used:** Hierarchical ______________________________________________________________________ ## Architecture ``` flowchart TD Q[Research Question] --> L[Research Lead\nDecompose + synthesize] L --> DT[Discovery Lead] L --> ST[Synthesis Lead] DT --> D1[Source Finder] DT --> D2[Relevance Triage] ST --> S1[Findings Extractor] ST --> S2[Citation Writer] D1 --> L D2 --> L S1 --> L S2 --> L L --> R[Literature Review] ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Hierarchical from pyagent_patterns.orchestration.hierarchical import Team from pyagent_providers import AnthropicLLM, OpenAILLM review_team = Hierarchical( manager=Agent( "research_lead", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "Decompose the question for the Discovery and Synthesis teams. After both report, " "write a literature review: themes, key findings with citations, points of consensus " "and disagreement, and a 'research gap' statement." ), ), teams=[ Team( name="Discovery", lead=Agent( "discovery_lead", OpenAILLM("gpt-4o-mini"), system_prompt="Consolidate candidate sources into a deduplicated, triaged shortlist.", ), workers=[ Agent( "source_finder", OpenAILLM("gpt-4o-mini"), system_prompt="Propose the most relevant papers/sources for the question with one-line reasons.", ), Agent( "relevance_triage", OpenAILLM("gpt-4o-mini"), system_prompt="Rate each source's relevance and recency; drop weak ones.", ), ], ), Team( name="Synthesis", lead=Agent( "synthesis_lead", OpenAILLM("gpt-4o-mini"), system_prompt="Organize extracted findings into themes for the lead.", ), workers=[ Agent( "findings_extractor", AnthropicLLM("claude-haiku-3-5-20241022"), system_prompt="Extract each source's key claim, method, and limitation.", ), Agent( "citation_writer", OpenAILLM("gpt-4o-mini"), system_prompt="Format every cited claim with an inline citation and a reference list.", ), ], ), ], ) result = asyncio.run(review_team.run( "Question: does retrieval-augmented generation reduce hallucination in clinical QA?" )) print(result.output) print(f"Teams: {result.metadata['team_names']}, workers: {result.metadata['total_workers']}") ``` ______________________________________________________________________ ## Expected output ``` LITERATURE REVIEW — RAG and hallucination in clinical QA Themes: grounding quality, retrieval recall, evaluation metrics. Findings: RAG cuts unsupported claims when retrieval recall is high [1][3]; gains vanish on rare conditions [2]; metric disagreement complicates comparison [4]. Consensus: retrieval quality dominates the effect. Gap: few studies test long-tail conditions or use clinician-graded faithfulness. Teams: ['Discovery', 'Synthesis'], workers: 4 ``` For an autonomous, tool-using alternative, see the [Research Assistant](https://pyagent.org/cookbook/research-analysis/research-assistant/index.md) (ReAct + Fan-Out). ______________________________________________________________________ ## Customization ### Live search tools Give the Discovery team a [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) agent that queries arXiv / Semantic Scholar instead of relying on the model's memory. ### Add a critique team ``` from pyagent_patterns.orchestration.hierarchical import Team review_team.teams.append( Team(name="Critique", lead=Agent("critique_lead", OpenAILLM("gpt-4o-mini"), system_prompt="Consolidate methodological critiques."), workers=[Agent("skeptic", OpenAILLM("gpt-4o-mini"), system_prompt="Challenge each finding's validity and bias.")]), ) ``` ### Citation format ``` review_team.manager.system_prompt += " Format references in APA 7th edition." ``` ______________________________________________________________________ ## When to Use | Situation | Use Hierarchical? | | ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | | Review work splits into find-then-synthesize teams | ✅ Yes | | You want one cited deliverable from many specialists | ✅ Yes | | A single agent should search-and-reason with tools | ❌ Use [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) | | Parallel analysts merged once, no hierarchy | ❌ Use [Fan-Out / Fan-In](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) | ______________________________________________________________________ ## Cost Profile | Tier | Typical model | Avg cost | Volume (500 reviews/mo) | | ----------------------------- | ------------- | ----------- | ----------------------- | | Lead (decompose + synthesize) | claude-sonnet | $0.009 | $4.50… ×500 | | Leads ×2 + workers ×4 | mini + haiku | $0.005 | $2.50 | | **Per review** | mix | **~$0.014** | **~$7/mo per 500** | ______________________________________________________________________ ## See Also - [Hierarchical pattern](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) - [Research Assistant](https://pyagent.org/cookbook/research-analysis/research-assistant/index.md) — autonomous ReAct + Fan-Out research - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Research Assistant in Python A multi-agent research system that parallelizes information gathering, debates findings to surface disagreements, synthesizes into a structured report, and cites sources — all within a controlled token budget. **Patterns used:** ReAct, FanOutFanIn, Debate, Pipeline, CompressMiddleware ______________________________________________________________________ ## Requirements - **Functional** — gather information from web, academic, and industry sources in parallel; surface disagreement between an optimistic and a skeptical read of the findings; synthesize a structured report under 500 words. - **Non-functional** — parallel gathering should be as fast as the slowest single source, not the sum of all three. - **Audit** — the final report's claims should be traceable to which source agent (web/academic/ industry) surfaced them, and the debate stage's verdict should show both sides considered. - **Not required** — no persistent memory across separate research sessions (each topic is researched independently). ## Architecture decisions | Decision | Why | Why not the alternative | | ---------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | **Fan-Out/Fan-In** for gathering | Three source types (web, academic, industry) are independent — none needs another's output to run. | A **Pipeline** here would force web→academic→industry sequencing with no benefit, tripling latency for zero quality gain. | | **Debate** for surfacing disagreement | The value is explicitly adversarial: an optimist and a skeptic force genuine counter-evidence into the record, judged by a third agent. | **Self-Reflection** would have one agent critique itself — it can't produce a genuinely opposing read the way two independently-prompted agents can. | | Three separate workflows (`gather`, `debate`, `synthesize`), chained by the caller | Each stage is independently testable and independently reusable — a caller could run just `gather` for a lighter-weight summary without the debate stage. | A single fused workflow would need a new composite pattern; three workflows reuse existing patterns and let each be validated on its own. | ## Four-pillar mapping | Requirement | Pillar | Capability | | -------------------------------------------------- | ------------- | --------------------------------------- | | Parallel source gathering | Execution | `FanOutFanIn` pattern | | Adversarial fact-checking | Execution | `Debate` pattern | | Stay within a token budget across all three stages | Context | `pyagent-compress` `CompressMiddleware` | | Track daily research spend | Observability | `observability.cost_budget` | | Trace each source/debate/synthesis call | Observability | `observability.tracing` | ## Blueprint (declarative form) The real, verified file at `examples/cookbook/research-analysis/research_assistant/blueprint.yaml`, compiled against `PyAgentAdapter` as part of this repo's test suite: ``` api_version: pyagent/v1 metadata: name: research-assistant version: 1.0.0 description: Parallel research, debate, and synthesis pipeline providers: fast: { model: gpt-4o-mini } smart: { model: claude-sonnet-4-20250514 } agents: web_agent: { provider: smart, prompt: "Web research specialist — gather facts." } academic_agent: { provider: smart, prompt: "Academic specialist — find papers." } industry_agent: { provider: fast, prompt: "Industry analyst — find news." } optimist: { provider: smart, prompt: "Argue positive interpretation with evidence." } sceptic: { provider: smart, prompt: "Challenge findings with counter-evidence." } judge: { provider: smart, prompt: "Weigh arguments and produce balanced verdict." } synthesizer: { provider: smart, prompt: "Write structured research report under 500 words." } workflows: gather: pattern: fan_out_fan_in agents: { agents: [web_agent, academic_agent, industry_agent], aggregator: synthesizer } debate: pattern: debate agents: { debaters: [optimist, sceptic], judge: judge } config: { rounds: 2 } synthesize: pattern: pipeline agents: { stages: [synthesizer] } observability: tracing: { enabled: true } cost_budget: { daily_usd: 100.0, alert_threshold: 0.8 } ``` ``` pyagent-blueprint validate research-assistant.yaml pyagent-blueprint test research-assistant.yaml ``` ## Production checklist Ran this exact blueprint through `PyAgentAdapter.compile()` and inspected the real diagnostics: - ✅ **All three workflows run as declared** — `gather`, `debate`, and `synthesize` each compile and execute against the native pattern registry with no diagnostics on workflow structure. - ⚠️ **`observability.cost_budget` is declared but not auto-enforced** — compiling emits `BUDGET_UNSUPPORTED`: the $100/day budget is recorded but not enforced. Wire real enforcement via `graph.wire_cost_tracker(tracker)`. - **No `context.compression` block is declared in this blueprint** — the `CompressMiddleware` token-budget behavior shown in the Python implementation isn't yet expressed as a `context:` block here, so it isn't captured by a compile-time diagnostic either. Add one if declared enforcement matters for your deployment. - **The three workflows are chained by the caller, not by the blueprint itself** — there's no single declared "run gather, then debate, then synthesize" sequence; that orchestration between workflows currently lives in whatever code invokes them. ______________________________________________________________________ ## Architecture ``` flowchart TD Q[Research Question] --> P1[Phase 1: Parallel Research] subgraph P1[Phase 1: Parallel Research — FanOut] WR[Web ReAct Agent\nweb_search tool] AR[Academic ReAct Agent\narxiv_search tool] IR[Industry ReAct Agent\nnews_search tool] end P1 --> P2[Phase 2: Compress + Debate] subgraph P2[Phase 2: Debate] OPT[Optimist Agent] SKP[Sceptic Agent] JDG[Judge Agent] end P2 --> P3[Phase 3: Synthesize] P3 --> RPT[Structured Report\nwith citations] ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import FanOutFanIn, Pipeline from pyagent_patterns.resolution import Debate from pyagent_patterns.advanced import ReAct from pyagent_compress import CompressMiddleware, TokenBudget from pyagent_providers import AnthropicLLM, OpenAILLM # ── LLMs ────────────────────────────────────────────────────────────────────── fast_llm = OpenAILLM("gpt-4o-mini") smart_llm = AnthropicLLM("claude-sonnet-4-20250514") judge_llm = AnthropicLLM("claude-sonnet-4-20250514") # ── Token budget ────────────────────────────────────────────────────────────── budget = TokenBudget(workflow_limit=80_000, per_agent_limit=15_000) middleware = CompressMiddleware(target_ratio=0.5, budget=budget) # ── Tool stubs (replace with real implementations) ─────────────────────────── def web_search(query: str) -> str: # In production: call Bing/Google Search API return f"[web results for '{query}': revenue +18% YoY, margins at 25%]" def arxiv_search(query: str) -> str: # In production: call arxiv.org API return f"[arxiv results for '{query}': 3 papers on scaling laws and efficiency]" def news_search(query: str) -> str: # In production: call NewsAPI or similar return f"[news for '{query}': new partnership announced, analyst upgrades to Buy]" # ── Phase 1: Parallel research via ReAct ───────────────────────────────────── web_researcher = middleware.wrap(ReAct( agent=Agent("web_agent", smart_llm, system_prompt=( "You are a web research specialist. Use the search tool to gather " "current facts, statistics, and news about the topic. " "Return findings as bullet points with source URLs." )), tools={"web_search": web_search}, max_steps=4, )) academic_researcher = middleware.wrap(ReAct( agent=Agent("academic_agent", smart_llm, system_prompt=( "You are an academic research specialist. Search for peer-reviewed " "papers and studies about the topic. Cite paper titles and authors." )), tools={"arxiv_search": arxiv_search}, max_steps=3, )) industry_researcher = middleware.wrap(ReAct( agent=Agent("industry_agent", fast_llm, system_prompt=( "You are an industry analyst. Find recent news, analyst reports, " "and market developments. Focus on practical business implications." )), tools={"news_search": news_search}, max_steps=3, )) # ── Phase 2: Fan-out to analysis agents, then debate ───────────────────────── analysis_fanout = FanOutFanIn( agents=[ middleware.wrap(Agent("market_analyst", fast_llm, system_prompt="Analyze market and competitive implications.")), middleware.wrap(Agent("tech_analyst", fast_llm, system_prompt="Analyze technical feasibility and innovation.")), middleware.wrap(Agent("risk_analyst", fast_llm, system_prompt="Identify risks, uncertainties, and failure modes.")), ], aggregator=Agent("pre_synthesis", fast_llm, system_prompt="Combine all analyses into a unified brief."), ) debate = Debate( debaters=[ Agent("optimist", fast_llm, system_prompt=( "You are an optimist. Argue for the most positive interpretation " "of the research findings. Cite specific evidence." )), Agent("sceptic", fast_llm, system_prompt=( "You are a sceptic. Challenge assumptions and point out gaps, " "risks, and counter-evidence in the research findings." )), ], judge=Agent("debate_judge", judge_llm, system_prompt=( "You are an impartial judge. Evaluate both sides and produce " "a balanced verdict with key uncertainties highlighted." )), rounds=2, ) # ── Phase 3: Final synthesis with citations ─────────────────────────────────── synthesizer = Agent( "synthesizer", smart_llm, system_prompt=( "You are a research writer. Synthesize all the research, analysis, and debate " "into a structured report with these sections:\n" "1. Executive Summary (3 bullets)\n" "2. Key Findings (with citations)\n" "3. Bull Case\n" "4. Bear Case\n" "5. Uncertainties\n" "6. Recommendation\n" "Be specific. Include numbers wherever available." ), ) # ── Main research function ──────────────────────────────────────────────────── async def research(question: str) -> dict: print(f"Researching: {question}") # Phase 1: gather in parallel web_task = web_researcher.run(question) academic_task = academic_researcher.run(question) industry_task = industry_researcher.run(question) web_r, academic_r, industry_r = await asyncio.gather( web_task, academic_task, industry_task ) # Combine gathered research combined_research = ( f"## Web Research\n{web_r.output}\n\n" f"## Academic Research\n{academic_r.output}\n\n" f"## Industry Findings\n{industry_r.output}" ) # Phase 2: analyze and debate analysis_r = await analysis_fanout.run(combined_research) debate_r = await debate.run( f"Based on this research:\n{analysis_r.output}\n\nDebate the implications." ) # Phase 3: final report final_r = await synthesizer.run( f"Research gathered:\n{combined_research}\n\n" f"Analysis:\n{analysis_r.output}\n\n" f"Debate verdict:\n{debate_r.output}" ) return { "report": final_r.output, "debate_verdict": debate_r.output, "budget_summary": budget.summary(), "tokens_used": budget.total_consumed, } if __name__ == "__main__": result = asyncio.run(research("Nvidia's position in the AI infrastructure market")) print("=" * 60) print(result["report"]) print("\n--- Token budget ---") print(result["budget_summary"]) ``` ______________________________________________________________________ ## Expected Output ``` Researching: Nvidia's position in the AI infrastructure market ============================================================ ## Executive Summary • Nvidia holds ~80% GPU market share for AI training workloads • H100/H200 demand exceeds supply by 3-4x; GB200 NVLink racks backlogged • Key risk: AMD MI300X gaining traction at hyperscalers, custom silicon (TPU/Trainium) ## Key Findings • Revenue: $44.1B (FY Q3 2025), up 94% YoY — driven by Data Center segment ($30.8B) • Margins: Gross margin 74.6%, operating margin 62.4% — best-in-class for hardware • CUDA ecosystem: 4M+ developers, 3,000+ GPU-optimized applications (arxiv: "CUDA Dominance in ML Workloads", Chen et al. 2024) • Announced partnerships with AWS, Azure, GCP for GB200 NVLink rack deployments ## Bull Case Strong moat: CUDA ecosystem is 10+ years deep and switching costs are very high... ## Bear Case AMD MI300X achieves 95% of H100 performance at 20% lower cost in benchmarks... ## Uncertainties • Custom silicon timeline (Apple, Google, Amazon) — 2-3 year horizon unclear • Export restrictions to China (~15% of historical revenue) • GB200 yield rates and supply ramp ## Recommendation Long-term hold. Near-term demand visibility strong through FY2026... --- Token budget --- Total consumed: 34,200 / 80,000 (42.8%) Remaining: 45,800 By agent: {web_agent: 8400, academic_agent: 5100, industry_agent: 3200, ...} ``` ______________________________________________________________________ ## Customization ### Add more research sources ``` def patent_search(query: str) -> str: # call Google Patents API ... def sec_filings_search(query: str) -> str: # call SEC EDGAR full-text search ... patent_researcher = middleware.wrap(ReAct( agent=Agent("patent_agent", fast_llm, system_prompt="Search patents filed in the last 2 years related to the topic."), tools={"patent_search": patent_search, "sec_search": sec_filings_search}, max_steps=3, )) ``` ### Structured output ``` import json final_synthesizer = Agent( "structured_writer", smart_llm, system_prompt=( "Return your report as a JSON object with keys: " "executive_summary (list of strings), key_findings (list), " "bull_case (string), bear_case (string), recommendation (string), " "confidence_score (int 1-10)." ), ) # Parse the output async def structured_research(question: str) -> dict: result = await research(question) try: return json.loads(result["report"]) except json.JSONDecodeError: return {"raw": result["report"]} ``` ### Adjustable depth ``` RESEARCH_DEPTH = "quick" # "quick" | "standard" | "deep" config = { "quick": {"max_steps": 2, "debate_rounds": 1, "target_ratio": 0.3}, "standard": {"max_steps": 4, "debate_rounds": 2, "target_ratio": 0.5}, "deep": {"max_steps": 6, "debate_rounds": 3, "target_ratio": 0.7}, }[RESEARCH_DEPTH] ``` ______________________________________________________________________ ## Cost Profile | Depth | Phase 1 | Phase 2 | Phase 3 | Total | | -------- | ------- | ------- | ------- | ------- | | Quick | ~$0.005 | ~$0.008 | ~$0.004 | ~$0.017 | | Standard | ~$0.012 | ~$0.018 | ~$0.008 | ~$0.038 | | Deep | ~$0.025 | ~$0.040 | ~$0.015 | ~$0.080 | Compression saves ~40-50% vs uncompressed. At 100 research queries/day, standard depth costs ~$115/month. ______________________________________________________________________ ## See Also - [ReAct pattern](https://pyagent.org/packages/patterns/advanced/react/index.md) - [Fan-Out / Fan-In pattern](https://pyagent.org/packages/patterns/orchestration/fan-out-fan-in/index.md) - [Debate pattern](https://pyagent.org/packages/patterns/resolution/debate/index.md) - [Compression Guide](https://pyagent.org/guides/compression/index.md) # How to Build a Multi-Agent Lead Qualification Workflow in Python Inbound leads arrive faster than reps can read them, and the cost of mis-routing a hot lead is a lost deal. This recipe uses a **Supervisor** to score each lead and route it to the right play — hot leads get a personalized outreach draft, warm leads enter a nurture sequence, cold leads get a polite hold — so every lead gets the right next step within seconds. **Patterns used:** [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) ______________________________________________________________________ ## Architecture ``` flowchart TD L[Inbound Lead] --> S[Supervisor\nscore hot / warm / cold] S -->|hot| A[AE\npersonalized outreach] S -->|warm| N[Nurture\n3-touch sequence] S -->|cold| C[Hold\npolite defer] A --> O[Routed Lead + Draft] N --> O C --> O ``` ______________________________________________________________________ ## Implementation ``` pip install pyagent-patterns pyagent-providers ``` ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Supervisor from pyagent_providers import AnthropicLLM, OpenAILLM fast_llm = OpenAILLM("gpt-4o-mini") smart_llm = AnthropicLLM("claude-sonnet-4-20250514") # ── Scorer: classifies the lead into a play ───────────────────────────────────── scorer = Agent( "scorer", fast_llm, system_prompt=( "Score this inbound lead as exactly one of: hot, warm, cold. " "Hot = strong fit + buying signal (title, company size, intent). " "Warm = good fit, weak signal. Cold = poor fit or no signal. " "Reply with ONLY the label." ), ) # ── Specialist plays per score ────────────────────────────────────────────────── ae = Agent( "account_exec", smart_llm, system_prompt=( "Draft a 4-sentence personalized outreach email for this hot lead. Reference their role and " "the trigger that makes now the right time. End with one specific call-to-action." ), ) nurture = Agent( "nurture", fast_llm, system_prompt=( "Add this warm lead to a 3-touch nurture sequence. Draft touch 1: a useful, no-ask message " "with a relevant resource. Outline touches 2 and 3 in one line each." ), ) cold_hold = Agent( "cold_hold", fast_llm, system_prompt="Write a brief, polite holding reply and tag the lead for a 90-day re-check.", ) crm = Supervisor( classifier=scorer, routes={"hot": ae, "warm": nurture, "cold": cold_hold}, default_route="warm", ) async def main(): lead = "VP Eng at a 500-person fintech, downloaded the multi-agent whitepaper twice this week." result = await crm.run(lead) print(f"Routed to: {result.metadata['route_key']}\n") print(result.output) asyncio.run(main()) ``` ______________________________________________________________________ ## Expected Output ``` Routed to: hot Subject: multi-agent orchestration for fintech-scale reliability Hi — saw your team is digging into multi-agent patterns (twice on the whitepaper this week!). At a 500-person fintech, the hard part is usually deterministic recovery and cost control across agents, which is exactly what PyAgent's router + recovery layers handle. Worth a 20-minute walkthrough on how teams your size cut model spend ~60%? — open to Thursday or Friday? ``` A warm lead would instead return touch 1 of a nurture sequence; a cold lead, a polite hold — all from the same routing call. ______________________________________________________________________ ## Customization ### Enrich the lead before scoring Prepend a [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) stage that pulls firmographics from your CRM so the scorer sees company size, tech stack, and prior touches: ``` from pyagent_patterns.orchestration import Pipeline enrich = Agent("enrich", fast_llm, system_prompt="Add firmographics (size, industry, stack) from CRM context.") qualified = Pipeline(stages=[enrich, scorer]) # feed the enriched lead into the supervisor ``` ### Add an SDR hand-off for hot leads ``` sdr_handoff = Agent("sdr", fast_llm, system_prompt="Write a Slack hand-off note + suggested meeting slots for the AE.") crm.routes["hot"] = Pipeline(stages=[ae, sdr_handoff]) ``` ### Weighted scoring with explicit criteria ``` scorer.system_prompt += " Weight: title 40%, company size 30%, intent signal 30%. Show the breakdown." ``` ______________________________________________________________________ ## When to Use | Situation | Fit | | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | Route each input to one of several mutually-exclusive plays | ✅ Supervisor | | Enrich → score as ordered stages | ✅ Pipeline | | Several scorers should vote on borderline leads | ❌ Use [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) | | The plan depends on the lead (variable subtasks) | ❌ Use [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) | ______________________________________________________________________ ## Cost Profile | Stage | Typical model | Avg cost | Volume (1k leads/day) | | -------------------- | ------------- | ------------ | ------------------------ | | Scorer | gpt-4o-mini | $0.0002 | $6/mo | | Hot play (AE draft) | claude-sonnet | $0.003 | applies to ~20% of leads | | Warm/cold plays | gpt-4o-mini | $0.0004 | majority | | **Blended per lead** | mix | **~$0.0009** | **~$27/mo** | Only hot leads hit the premium model, so cost scales with your hot-lead rate, not total volume. ______________________________________________________________________ ## See Also - [Supervisor pattern](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) · [Pipeline pattern](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) - [Support Router](https://pyagent.org/cookbook/customer-support/support-router/index.md) — the tiered classify-and-route flagship - [Marketing Campaign Planner](https://pyagent.org/cookbook/marketing-content/campaign-planner/index.md) - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Peer-Review Mesh in Python Real peer review isn't a chain — reviewers see each other's comments and reconcile. This recipe uses the **Topology** pattern in a **mesh** configuration: methodology, novelty, and clarity reviewers each critique the manuscript and exchange notes over several rounds, converging on a shared recommendation instead of three disconnected reports. **Patterns used:** Topology (mesh) ______________________________________________________________________ ## Architecture ``` flowchart TD M[Manuscript] --> R1[Methodology Reviewer] M --> R2[Novelty Reviewer] M --> R3[Clarity Reviewer] R1 <--> R2 R2 <--> R3 R1 <--> R3 R1 --> D[Consensus Assessment] R2 --> D R3 --> D ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.structural import Topology, TopologyType from pyagent_providers import AnthropicLLM, OpenAILLM review = Topology( agents=[ Agent( "methodology_reviewer", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "Review the manuscript's methods: study design, statistics, reproducibility, and " "threats to validity. When you see peers' notes, reconcile your view with theirs." ), ), Agent( "novelty_reviewer", OpenAILLM("gpt-4o"), system_prompt=( "Review the manuscript's novelty and contribution relative to prior work. " "Adjust your assessment when peers raise points you missed." ), ), Agent( "clarity_reviewer", OpenAILLM("gpt-4o-mini"), system_prompt=( "Review writing clarity, figures, and structure. Note where claims outrun evidence. " "Reconcile with peers, then help draft the consensus recommendation." ), ), ], topology=TopologyType.MESH, rounds=2, ) result = asyncio.run(review.run( "Manuscript: 'A Transformer for Protein Folding at Low Compute'. " "Abstract, methods, results, and discussion attached." )) print(result.output) print(f"Topology: {result.metadata['topology']}") ``` ______________________________________________________________________ ## Expected output ``` CONSENSUS REVIEW — "A Transformer for Protein Folding at Low Compute" Recommendation: Major revision. Methodology: ablations missing; baseline comparison not matched on compute (all reviewers agree). Novelty: contribution is incremental but the low-compute angle is valuable. Clarity: Figure 3 unreadable; claims in §5 overstated relative to results. Reviewers converged after round 2 on "major revision, resubmit with ablations". Topology: mesh ``` Because reviewers exchange notes (mesh), the final recommendation is reconciled rather than three conflicting scores a human editor must referee. ______________________________________________________________________ ## Customization ### Editor-hub (star) topology ``` review = Topology(agents=review._agents, topology=TopologyType.STAR, hub_index=0) # reviewer 0 = handling editor ``` ### Add a statistics reviewer ``` review.agents.append( Agent("stats_reviewer", OpenAILLM("gpt-4o"), system_prompt="Check the statistics: power, multiple comparisons, and appropriate tests."), ) ``` ### Deeper reconciliation ``` review = Topology(agents=review._agents, topology=TopologyType.MESH, rounds=3) ``` ______________________________________________________________________ ## When to Use | Situation | Use Topology (mesh)? | | ----------------------------------------------- | --------------------------------------------------------------------------------- | | Peers must see and reconcile each other's views | ✅ Yes — mesh | | A fixed review order (security → perf → style) | ✅ Use `TopologyType.CHAIN` | | One coordinator collects from spokes | ✅ Use `TopologyType.STAR` | | Agents argue opposing sides for a judge | ❌ Use [Debate](https://pyagent.org/packages/patterns/resolution/debate/index.md) | ______________________________________________________________________ ## Cost Profile | Driver | Typical model | Avg cost | Notes | | ---------------------- | ------------------- | ----------- | ------------------------------ | | 3 reviewers × 2 rounds | sonnet + gpt-4o mix | $0.024 | mesh = N×rounds calls | | **Per manuscript** | mix | **~$0.024** | scales with reviewers × rounds | Mesh is the most expensive topology (every agent runs each round) — keep `rounds` small (2–3). ______________________________________________________________________ ## See Also - [Topology pattern](https://pyagent.org/packages/patterns/structural/topology/index.md) - [Loan Origination Workflow](https://pyagent.org/cookbook/finance-trading/loan-origination/index.md) — a chain-topology state machine - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Fraud Investigation Assistant in Python A fraud alert needs investigation, not a one-shot verdict: pull the transactions, score anomalies, check sanctions, then reason about what it means. This recipe uses the **ReAct** pattern — the agent interleaves reasoning with tool calls, gathering evidence step by step until it can write a case file for a human investigator. **Patterns used:** ReAct (with tools) ______________________________________________________________________ ## Architecture ``` flowchart TD A[Fraud Alert] --> R[ReAct Agent] R -->|reason → act| T1[transaction_lookup] R -->|reason → act| T2[anomaly_score] R -->|reason → act| T3[sanctions_check] T1 --> R T2 --> R T3 --> R R --> C[Case File\nfor human investigator] ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.advanced import ReAct from pyagent_providers import OpenAILLM def transaction_lookup(account_id: str) -> str: """Return recent transactions for an account (replace with your ledger/API).""" ledger = { "ACC-8842": "12 txns in 24h; $9,900 ×3 to new payees; 2 logins from new country", } return ledger.get(account_id.strip(), f"No transactions found for {account_id}") def anomaly_score(pattern: str) -> str: """Score a described transaction pattern 0-100 for fraud likelihood.""" p = pattern.lower() score = 30 if "9,900" in pattern or "9900" in pattern: score += 35 # structuring just under $10k reporting threshold if "new payee" in p or "new payees" in p: score += 20 if "new country" in p or "new device" in p: score += 15 return f"anomaly_score={min(score, 100)} (drivers: structuring, new payees, geo)" def sanctions_check(payee: str) -> str: """Check a payee against a sanctions list (replace with OFAC/PEP screening).""" flagged = {"shell co ltd"} return "MATCH — on watchlist" if payee.strip().lower() in flagged else "no sanctions match" investigator = ReAct( agent=Agent( "fraud_analyst", OpenAILLM("gpt-4o"), system_prompt=( "You investigate fraud alerts. Reason step by step and use tools to gather evidence: " "look up transactions, score anomalies, and screen payees. When you have enough, write " "a case file: summary, evidence, risk level (Low/Medium/High), and recommended action." ), ), tools={ "transaction_lookup": transaction_lookup, "anomaly_score": anomaly_score, "sanctions_check": sanctions_check, }, max_steps=6, ) result = asyncio.run(investigator.run( "Alert: account ACC-8842 triggered a velocity rule. Investigate and recommend an action." )) print(result.output) print(f"Steps: {result.metadata['steps']}, tools used: {result.metadata['tools_used']}") ``` ______________________________________________________________________ ## Expected output ``` CASE FILE — ACC-8842 Summary: Three $9,900 transfers to new payees within 24h, plus logins from a new country. Evidence: transaction_lookup → velocity + structuring; anomaly_score=85; sanctions_check → no match. Risk: High — classic structuring below the $10k reporting threshold. Action: Freeze outbound transfers, file a SAR, escalate to a human investigator. Steps: 4, tools used: ['transaction_lookup', 'anomaly_score', 'sanctions_check'] ``` The agent decides *which* tools to call and *when*, based on what each step reveals — the core of ReAct. ______________________________________________________________________ ## Customization ### Add tools ``` def kyc_lookup(account_id: str) -> str: ... def device_history(account_id: str) -> str: ... investigator.tools.update({"kyc_lookup": kyc_lookup, "device_history": device_history}) ``` ### Bound the investigation ``` investigator = ReAct(agent=investigator._agent, tools=investigator._tools, max_steps=4) ``` ### Wrap with recovery Wrap the agent in [BoundedExecution](https://pyagent.org/guides/recovery/index.md) so a slow tool call can't hang the queue. ______________________________________________________________________ ## When to Use | Situation | Use ReAct? | | -------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The agent must gather evidence via tools before deciding | ✅ Yes | | Steps depend on intermediate results | ✅ Yes | | The flow is a fixed sequence of stages | ❌ Use [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) / [Topology](https://pyagent.org/packages/patterns/structural/topology/index.md) | | You only need to route to a specialist | ❌ Use [Supervisor](https://pyagent.org/packages/patterns/orchestration/supervisor/index.md) | ______________________________________________________________________ ## Cost Profile | Driver | Typical model | Avg cost | Notes | | --------------------- | ------------- | ---------- | ---------------------- | | Reason/act steps (≤6) | gpt-4o | $0.01 | cost scales with steps | | **Per investigation** | gpt-4o | **~$0.01** | cap with `max_steps` | Wrap the agent in [Recovery](https://pyagent.org/guides/recovery/index.md) for tool-call timeouts in production. ______________________________________________________________________ ## See Also - [ReAct pattern](https://pyagent.org/packages/patterns/advanced/react/index.md) - [Alert Triage](https://pyagent.org/cookbook/security/log-triage/index.md) — pipeline triage of security alerts - [Loan Origination Workflow](https://pyagent.org/cookbook/finance-trading/loan-origination/index.md) — staged financial workflow - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Security Alert Triage Workflow in Python A SOC drowns in alerts, most of them noise. This recipe runs each alert through a **Pipeline** that enriches it with context and correlates it against known attack patterns, then uses **Human-in-the-Loop** so confirmed true positives — and only those — page a human analyst with a ready-made case, while false positives are auto-closed with a reason. **Patterns used:** [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Human-in-the-Loop](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) ______________________________________________________________________ ## Architecture ``` flowchart TD A[Raw Alert] --> E[Enrich\nasset, owner, severity] E --> C[Correlate\nvs attack patterns] C --> T[Triage\nFP or escalate] T -->|false positive| X[Auto-close + reason] T -->|true positive| H[Human-in-the-Loop\nanalyst page] ``` ______________________________________________________________________ ## Implementation ``` pip install pyagent-patterns pyagent-providers ``` ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.orchestration import Pipeline from pyagent_patterns.advanced import HumanInTheLoop from pyagent_patterns.advanced.human_in_the_loop import HumanDecision from pyagent_patterns.composite import CompositePattern from pyagent_providers import AnthropicLLM, OpenAILLM fast_llm = OpenAILLM("gpt-4o-mini") smart_llm = AnthropicLLM("claude-sonnet-4-20250514") # ── Triage pipeline: enrich → correlate → classify ────────────────────────────── triage_pipeline = Pipeline(stages=[ Agent( "enricher", fast_llm, system_prompt=( "Enrich this alert with context: affected asset, business owner, data sensitivity, " "and a 1-5 severity. State assumptions explicitly. Pass everything forward." ), ), Agent( "correlator", smart_llm, system_prompt=( "Match the enriched alert against known attack patterns (MITRE ATT&CK style). " "Give a confidence 0-100 that this is a real attack, with the supporting signals." ), ), Agent( "classifier", smart_llm, system_prompt=( "Decide: FALSE_POSITIVE (confidence < 60) or ESCALATE. For ESCALATE, write a 3-line " "case: what happened, why it's likely real, and the recommended first response." ), ), ]) # ── Human escalation: only true positives reach an analyst ────────────────────── def page_analyst(output: str, metadata: dict) -> HumanDecision: """Create a case in your SOAR / ticketing system and page on-call.""" if "FALSE_POSITIVE" in output: return HumanDecision(approved=True, modified_output=f"Auto-closed: {output}") case_id = _create_soc_case(summary=output[:200], metadata=metadata) # your integration print(f"[PAGED] analyst on-call — case {case_id}") return HumanDecision(approved=True, modified_output=f"Escalated as case {case_id}\n{output}") soc = CompositePattern(patterns=[triage_pipeline]) soc_with_human = HumanInTheLoop( agent=Agent("case_writer", fast_llm, system_prompt="Format the triage result as a SOC case note."), review_fn=page_analyst, high_risk_keywords=["ransomware", "exfiltration", "domain admin", "lateral movement"], ) SAMPLE_ALERT = ( '{"rule":"impossible travel","user":"j.doe","from":"US","to":"RU","delta_min":12,' '"asset":"vpn-gw-01","mfa":"failed x3"}' ) async def main(): triaged = await soc.run(SAMPLE_ALERT) final = await soc_with_human.run(triaged.output) print(final.output) asyncio.run(main()) ``` ______________________________________________________________________ ## Expected Output ``` [PAGED] analyst on-call — case SOC-4471 Escalated as case SOC-4471 ESCALATE (confidence 88) What: impossible-travel sign-in to vpn-gw-01 for j.doe, US→RU in 12 min, 3 failed MFA pushes. Why real: geo-velocity + MFA fatigue pattern (ATT&CK T1110/T1621). First response: disable session, force re-auth, check VPN logs for successful auth from RU. ``` A clear false positive (e.g. a known corporate VPN egress) closes itself with a reason; only the high-confidence case pages a human — which is the whole point of triage. ______________________________________________________________________ ## Customization ### Add live enrichment with a ReAct tool agent Swap the static enricher for a [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) agent that queries your asset DB and threat-intel feeds — see the [Fraud Investigation Assistant](https://pyagent.org/cookbook/security/fraud-investigation/index.md). ### Tune the escalation threshold ``` # Raise the bar during a noisy migration; lower it during an active incident. classifier_prompt = "Decide FALSE_POSITIVE only if confidence < 75; otherwise ESCALATE." ``` ### Batch the overnight queue ``` async def triage_queue(alerts: list[str]) -> list[str]: results = await asyncio.gather(*(soc.run(a) for a in alerts)) return [r.output for r in results] ``` ______________________________________________________________________ ## When to Use | Situation | Fit | | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Fixed enrich → correlate → classify stages | ✅ Pipeline | | Only confirmed positives should page a human | ✅ Human-in-the-Loop | | The agent must call tools to gather evidence | ❌ Use [ReAct](https://pyagent.org/packages/patterns/advanced/react/index.md) ([Fraud Investigation](https://pyagent.org/cookbook/security/fraud-investigation/index.md)) | | You need several detectors to vote | ❌ Use [Voting](https://pyagent.org/packages/patterns/resolution/voting/index.md) | ______________________________________________________________________ ## Cost Profile | Stage | Typical model | Avg cost | Volume (50k alerts/day) | | -------------------- | ------------- | ------------ | ----------------------- | | Enrich | gpt-4o-mini | $0.0003 | $450/mo | | Correlate + classify | claude-sonnet | $0.006 | $9,000/mo | | **Per alert** | mix | **~$0.0063** | **~$9.5k/mo** | Run a cheap pre-filter (regex / allow-list) before the LLM pipeline to cut volume on known-benign rules. ______________________________________________________________________ ## See Also - [Pipeline pattern](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) · [Human-in-the-Loop pattern](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) - [Fraud Investigation Assistant](https://pyagent.org/cookbook/security/fraud-investigation/index.md) — tool-using ReAct investigation - [Incident Triage](https://pyagent.org/cookbook/devops-sre/incident-triage/index.md) — the same shape for ops incidents - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Code Review System in Python A production-grade multi-agent code review system that automatically guards inputs, performs iterative peer review, scans for security issues, and routes to human reviewers when needed. **Patterns used:** CrossReflection, Pipeline, HumanInTheLoop, GuardrailChain, RouterMiddleware ______________________________________________________________________ ## Requirements - **Functional** — iteratively review code for correctness/maintainability until a reviewer agent approves it; separately scan for security vulnerabilities; escalate low security scores to a human. - **Non-functional** — the review loop must terminate (bounded rounds), not iterate indefinitely on a disagreement between generator and reviewer. - **Audit** — every approval must show the specific review round and reviewer feedback that led to it; every escalation must show the security score that triggered it. - **Not required** — no automatic merge action — this recipe produces a review verdict and, for low-security-score cases, a human-readable escalation; it doesn't itself touch the repository. ## Architecture decisions | Decision | Why | Why not the alternative | | ---------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Cross-Reflection** for the review loop | The reviewer is a genuinely different agent/expertise from the code generator — an independent second opinion, not self-review. | **Self-Reflection** would have the code-writing agent critique its own work, losing the independent-perspective signal that catches blind spots. | | Security scan as its own **Pipeline**, not folded into the review loop | Security scanning is a single-pass check, not an iterative negotiation — it doesn't need Cross-Reflection's multi-round structure. | Folding it into the review loop would force every review round to re-run a security scan, wasting cost on an unchanging concern. | | **Human-in-the-Loop** gates only escalation, not every review | Bounded, iterative peer review runs at machine speed for the common case; only genuinely low security scores need a human. | Gating every review round on a human would eliminate the whole point of automating iterative review. | | Dedicated `security` provider tier, separate from `fast`/`smart` | Security scanning benefits from a specific model choice independent of the cost/quality tradeoff driving the other two tiers. | Reusing `smart` for security scanning would work, but conflates two independent provider decisions (review quality vs. security-scan model choice) into one knob. | ## Four-pillar mapping | Requirement | Pillar | Capability | | ------------------------------- | ------------- | --------------------------- | | Iterative review until approved | Execution | `CrossReflection` pattern | | Security vulnerability scan | Execution | `Pipeline` pattern | | Escalate low scores to a human | Execution | `HumanInTheLoop` pattern | | Track daily review spend | Observability | `observability.cost_budget` | | Trace each review round | Observability | `observability.tracing` | ## Blueprint (declarative form) The real, verified file at `examples/cookbook/software-engineering/code_review/blueprint.yaml`, compiled against `PyAgentAdapter` as part of this repo's test suite: ``` api_version: pyagent/v1 metadata: name: code-review version: 1.0.0 description: Iterative review + security scan with human escalation providers: fast: { model: gpt-4o-mini } smart: { model: claude-sonnet-4-20250514 } security: { model: gpt-4o } agents: code_agent: { provider: smart, prompt: "Review for correctness, maintainability, best practices." } review_agent: { provider: smart, prompt: "Review clarity, test gaps, design. Return APPROVED when ready." } security_agent: { provider: security, prompt: "Scan for vulns. Score 1-10. If < 8, escalate." } escalation_agent: { provider: fast, prompt: "Summarize security finding for human review." } workflows: review: pattern: cross_reflection agents: { generator: code_agent, reviewer: review_agent } config: { max_rounds: 3, stop_phrase: APPROVED } security: pattern: pipeline agents: { stages: [security_agent] } escalate: pattern: human_in_the_loop agents: { agent: escalation_agent } observability: tracing: { enabled: true } cost_budget: { daily_usd: 50.0, alert_threshold: 0.8 } ``` ``` pyagent-blueprint validate code-review.yaml pyagent-blueprint test code-review.yaml ``` ## Production checklist Ran this exact blueprint through `PyAgentAdapter.compile()` and inspected the real diagnostics: - ✅ **All three workflows run as declared** — `review`, `security`, and `escalate` each compile and execute against the native pattern registry with no diagnostics on workflow structure. - ⚠️ **`observability.cost_budget` is declared but not auto-enforced** — compiling emits `BUDGET_UNSUPPORTED`: the $50/day budget is recorded but not enforced. Wire real enforcement via `graph.wire_cost_tracker(tracker)`. - **The three workflows aren't sequenced by the blueprint itself** — deciding to run `security` and `escalate` only when the security score is below threshold is caller logic, not declared in the spec. If that branching needs to be reviewable/diffable, it currently isn't. - **The `GuardrailChain`/`RouterMiddleware` behavior shown in some Python variants of this recipe isn't represented in this blueprint** — the blueprint captures the three named patterns' structure, not every middleware layer a full production deployment might add. ______________________________________________________________________ ## Architecture ``` sequenceDiagram participant D as Developer participant G as Input Guardrails participant C as Code Agent participant R as Review Agent participant S as Security Agent participant H as Human (if needed) D->>G: Submit code G->>G: Length check, PII scan G->>C: Sanitized code C->>R: Initial review R->>C: Feedback (up to 3 rounds) C->>R: Revised code R-->>S: APPROVED S->>S: Vulnerability scan S-->>D: Final report Note over S,H: If security score < 8, escalate to human ``` ______________________________________________________________________ ## Implementation ```` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.resolution import CrossReflection from pyagent_patterns.orchestration import Pipeline from pyagent_patterns.advanced import HumanInTheLoop from pyagent_patterns.composite import CompositePattern from pyagent_patterns.guardrails import GuardrailChain, LengthGuard, PIIGuard, ContentGuard from pyagent_router.middleware import RouterMiddleware from pyagent_providers import AnthropicLLM, OpenAILLM # ── LLMs ────────────────────────────────────────────────────────────────────── fast_llm = OpenAILLM("gpt-4o-mini") smart_llm = AnthropicLLM("claude-sonnet-4-20250514") security_llm = OpenAILLM("gpt-4o") model_registry = { "gpt-4o-mini": fast_llm, "gpt-4o": security_llm, "claude-sonnet-4-20250514": smart_llm, } router = RouterMiddleware(model_registry=model_registry) # ── Guardrails ───────────────────────────────────────────────────────────────── input_guard = GuardrailChain([ LengthGuard(max_chars=50_000, truncate=False), # hard reject huge pastes PIIGuard(redact=True), # redact emails/tokens in comments ContentGuard(deny_patterns=[ # block embedded secrets __import__("re").compile(r"sk-[A-Za-z0-9]{20,}"), __import__("re").compile(r"ghp_[A-Za-z0-9]{36}"), ]), ]) # ── Stage 1: iterative code review via cross-reflection ─────────────────────── code_review = CrossReflection( generator=router.wrap( Agent("author", smart_llm, system_prompt=( "You are a senior Python engineer. Review and improve the submitted code. " "Focus on: correctness, edge cases, type hints, docstrings, and performance." )), ), reviewer=router.wrap( Agent("critic", smart_llm, system_prompt=( "You are a principal engineer doing code review. " "Point out specific issues with line references. " "Reply APPROVED when the code meets production standards." )), ), max_rounds=3, ) # ── Stage 2: security scan ──────────────────────────────────────────────────── security_pipeline = Pipeline(stages=[ Agent("security", security_llm, system_prompt=( "You are a security engineer. Scan the code for: SQL injection, " "XSS, insecure deserialization, hardcoded secrets, path traversal, " "and OWASP Top 10 issues. Score security 1-10. List every finding." )), ]) # ── Stage 3: human escalation for low security scores ──────────────────────── def needs_human_review(result) -> bool: """Escalate to human if security score < 8.""" import re match = re.search(r"[Ss]core[:\s]+(\d+)", result.output) if match: return int(match.group(1)) >= 8 return True # pass if no score found human_escalation = HumanInTheLoop( agent=Agent("prep", fast_llm, system_prompt="Summarize the security issues for the human reviewer."), review_fn=lambda output, meta: _queue_human_review(output), high_risk_keywords=["critical", "injection", "hardcoded"], ) full_pipeline = CompositePattern( patterns=[security_pipeline, human_escalation], quality_check=needs_human_review, ) # ── Main review function ─────────────────────────────────────────────────────── async def review_code(code: str) -> dict: # 1. Guardrail check check = input_guard.check(code) if not check.passed: return {"error": check.message, "blocked": True} safe_code = check.sanitized_content or code # 2. Iterative code review review_result = await code_review.run( f"Review and improve this code:\n\n```python\n{safe_code}\n```" ) # 3. Security scan + optional human escalation security_result = await full_pipeline.run( f"Security scan:\n\n```python\n{review_result.output}\n```" ) return { "improved_code": review_result.output, "security_report": security_result.output, "review_rounds": review_result.metadata.get("rounds", 0), "escalated_to_human": security_result.metadata.get("escalation_level", 0) > 0, } def _queue_human_review(summary: str): """Post security findings to the review queue and return a holding decision.""" import httpx, os r = httpx.post( os.environ["REVIEW_QUEUE_URL"] + "/reviews", json={"summary": summary[:500], "priority": "high", "source": "code-review-agent"}, headers={"Authorization": f"Bearer {os.environ['REVIEW_QUEUE_TOKEN']}"}, timeout=15.0, ) r.raise_for_status() ticket_id = r.json()["ticket_id"] print(f"[SECURITY REVIEW] ticket={ticket_id}") from pyagent_patterns.advanced.human_in_the_loop import HumanDecision return HumanDecision(approved=True, modified_output=f"[Ticket {ticket_id}]\n{summary}") # ── Run it ───────────────────────────────────────────────────────────────────── if __name__ == "__main__": code = ''' def get_user(user_id): query = "SELECT * FROM users WHERE id = " + user_id return db.execute(query) ''' result = asyncio.run(review_code(code)) print("=== Improved Code ===") print(result["improved_code"]) print("\n=== Security Report ===") print(result["security_report"]) print(f"\nReview rounds: {result['review_rounds']}") print(f"Escalated to human: {result['escalated_to_human']}") ```` ______________________________________________________________________ ## Expected Output Running the SQL injection example above: ``` === Improved Code === def get_user(user_id: int) -> dict | None: """Fetch a user by ID. Args: user_id: The integer user ID to look up. Returns: User dict or None if not found. """ query = "SELECT * FROM users WHERE id = ?" return db.execute(query, (user_id,)) === Security Report === Security Score: 3/10 CRITICAL — SQL Injection (original code): Line 2: String concatenation used in SQL query. Fix: Use parameterized queries (? placeholder). Applied in improved code. MEDIUM — Missing input validation: user_id is typed as `int` but callers may pass strings from HTTP params. Fix: Add isinstance(user_id, int) or use a Pydantic model at the boundary. Review rounds: 2 Escalated to human: True ← score was 3, below threshold of 8 ``` ______________________________________________________________________ ## Customization ### Change the review focus ``` # Security-focused review Agent("author", smart_llm, system_prompt="Review for security vulnerabilities only. OWASP Top 10.") # Performance-focused review Agent("author", smart_llm, system_prompt="Review for performance. Focus on O(n) complexity, DB queries, caching.") # Style-focused review Agent("author", smart_llm, system_prompt="Review for PEP 8, type hints, and Google docstring format.") ``` ### Adjust escalation threshold ``` # Always pass to human (security-critical systems) def always_human(result) -> bool: return False # never passes quality check → always escalates # Only escalate on critical findings def only_critical(result) -> bool: return "CRITICAL" not in result.output.upper() ``` ### Add language support ```` # Detect language and route to the right specialist def language_aware_prompt(code: str) -> str: if "def " in code or "import " in code: return f"Review this Python code:\n\n```python\n{code}\n```" elif "function " in code or "const " in code: return f"Review this JavaScript/TypeScript code:\n\n```js\n{code}\n```" return f"Review this code:\n\n```\n{code}\n```" ```` ______________________________________________________________________ ## Cost Profile | Code snippet | Review rounds | Models used | Approx cost | | ---------------- | ------------- | ---------------------- | -------------- | | 50-line function | 1–2 | gpt-4o-mini × 2 | $0.001 | | 200-line class | 2–3 | claude-sonnet × 3 | $0.008 | | 500-line module | 3 + security | claude-sonnet + gpt-4o | $0.025 | | Human escalation | — | + human time | $0.002 + human | ______________________________________________________________________ ## See Also - [CrossReflection pattern](https://pyagent.org/packages/patterns/resolution/cross-reflection/index.md) - [Guardrails Guide](https://pyagent.org/guides/guardrails/index.md) - [Routing Guide](https://pyagent.org/guides/router/index.md) - [HumanInTheLoop pattern](https://pyagent.org/packages/patterns/advanced/human-in-the-loop/index.md) # How to Simulate a Software Startup with Multi-Agent Role-Play in Python Inspired by MetaGPT's software-company simulation, this recipe uses the **Role-Based** pattern to turn a one-line idea into a buildable plan. A Product Manager, Architect, Engineer, and QA each hold their role and react to one another over rounds, producing a PRD, a system design, a code sketch, and a test plan — the standardized hand-offs a real team uses. **Patterns used:** Role-Based ______________________________________________________________________ ## Architecture ``` flowchart LR I[Product Idea] --> R((Software Company)) R --> PM[Product Manager\nPRD] R --> AR[Architect\nsystem design] R --> EN[Engineer\ncode sketch] R --> QA[QA\ntest plan] PM --> AR --> EN --> QA R --> O[Build-ready Plan] ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.structural import RoleBased from pyagent_providers import AnthropicLLM, OpenAILLM company = RoleBased( agents=[ Agent( "product_manager", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "You are the Product Manager. Write a tight PRD: problem, target user, 3-5 user " "stories, and success metrics. Keep scope to an MVP." ), ), Agent( "architect", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt=( "You are the Architect. From the PRD, define components, data model, key APIs, and " "the tech stack. Note the single riskiest technical decision." ), ), Agent( "engineer", OpenAILLM("gpt-4o"), system_prompt=( "You are the Engineer. From the design, sketch the core modules and a minimal code " "skeleton (function/class signatures, not full bodies). Flag unknowns for QA." ), ), Agent( "qa", OpenAILLM("gpt-4o-mini"), system_prompt=( "You are QA. From the PRD and code sketch, write a test plan: critical paths, edge " "cases, and acceptance criteria per user story." ), ), ], rounds=2, ) result = asyncio.run(company.run( "Idea: a CLI that summarizes a git repo's recent activity into a weekly stand-up note." )) print(result.output) print(f"Roles: {result.metadata['roles']}, rounds: {result.metadata['rounds']}") ``` ______________________________________________________________________ ## Expected output ``` BUILD-READY PLAN — "standup-cli" PRD: summarize commits + PRs into a weekly note; users = small dev teams; metric = adoption. Design: git-log parser → grouping → LLM summarizer; stack Python + Click; risk = noisy diffs. Code: modules collect.py / group.py / summarize.py with signatures sketched. QA: tests for empty repo, huge diffs, merge commits; acceptance per user story. Roles: ['product_manager', 'architect', 'engineer', 'qa'], rounds: 2 ``` ______________________________________________________________________ ## Customization ### Add a designer role ``` company.agents.insert(2, Agent("designer", AnthropicLLM("claude-sonnet-4-20250514"), system_prompt="You are the Designer. Define the core user flows and a minimal UI spec."), ) ``` ### Deeper iteration ``` company = RoleBased(agents=company._agents, rounds=3) ``` ### Emit structured artifacts Append a [Pipeline](https://pyagent.org/packages/patterns/orchestration/pipeline/index.md) stage that serialises the PRD, design, and test plan to JSON for your issue tracker. ______________________________________________________________________ ## When to Use | Situation | Use Role-Based? | | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | | A fixed cast of roles hands work down a chain and iterates | ✅ Yes | | You want standardized role outputs (PRD, design, tests) | ✅ Yes | | Roles map to whole teams with sub-workers | ❌ Use [Hierarchical](https://pyagent.org/packages/patterns/orchestration/hierarchical/index.md) | | You just need code reviewed and improved | ❌ Use [Code Review](https://pyagent.org/cookbook/software-engineering/code-review/index.md) | ______________________________________________________________________ ## Cost Profile | Driver | Typical model | Avg cost | Notes | | ------------------ | ------------------- | ---------- | ------------------------------- | | 4 roles × 2 rounds | sonnet + gpt-4o mix | $0.02 | senior roles on stronger models | | **Per simulation** | mix | **~$0.02** | scales with roles × rounds | ______________________________________________________________________ ## See Also - [Role-Based pattern](https://pyagent.org/packages/patterns/structural/role-based/index.md) - [Writers' Room](https://pyagent.org/cookbook/media-entertainment/writers-room/index.md) — the same pattern for creative work - [Code Review](https://pyagent.org/cookbook/software-engineering/code-review/index.md) — iterative review of real code - [Browse all recipes](https://pyagent.org/cookbook/index.md) # How to Build a Multi-Agent Trip Planner in Python Planning a trip means flights, lodging, activities, and budget all influencing each other — change the flight and the budget shifts, which changes the hotel. This recipe uses the **Swarm** pattern: specialist agents self-organize, each updating its plan as it sees neighbors' choices, until the itinerary settles into a consistent whole. **Patterns used:** Swarm ______________________________________________________________________ ## Architecture ``` flowchart TD T[Trip Request] --> S((Swarm)) S --> F[Flights Agent] S --> L[Lodging Agent] S --> A[Activities Agent] S --> B[Budget Agent] F <--> L L <--> A A <--> B B <--> F S --> I[Final Itinerary] ``` ______________________________________________________________________ ## Implementation ``` import asyncio from pyagent_patterns.base import Agent from pyagent_patterns.advanced import Swarm from pyagent_providers import GeminiLLM planner = Swarm( agents=[ Agent( "flights", GeminiLLM("gemini-2.5-flash"), system_prompt=( "You plan flights. Propose routes and times within budget. When neighbors change " "lodging or budget, adjust your picks to stay consistent." ), ), Agent( "lodging", GeminiLLM("gemini-2.5-flash"), system_prompt=( "You plan lodging near the chosen activities and airport. Update your choice when " "flight times or budget change." ), ), Agent( "activities", GeminiLLM("gemini-2.5-flash"), system_prompt=( "You plan day-by-day activities for the destination and dates. Keep them reachable " "from the lodging and within the remaining budget." ), ), Agent( "budget", GeminiLLM("gemini-2.5-flash"), system_prompt=( "You keep the trip within total budget. Flag overruns and push neighbors to trade " "off. Converge on a balanced allocation across flights, lodging, and activities." ), ), ], rounds=3, neighbor_count=2, aggregation="last", ) result = asyncio.run(planner.run( "7-day trip to Lisbon for two in May, total budget $3,500. " "Prefer central lodging, food + history, minimal early flights." )) print(result.output) print(f"Agents: {result.metadata['agents']}, rounds: {result.metadata['rounds']}") ``` ______________________________________________________________________ ## Expected output ``` LISBON ITINERARY (7 days, 2 people) — total ~$3,420 Flights: midday round-trip, $1,180 (budget pushed flights off the early slot). Lodging: central Baixa apartment, 6 nights, $1,140. Activities: Alfama food walk, Belém, Sintra day trip, ... $640. Budget: $460 buffer; all agents converged by round 3. Agents: ['flights', 'lodging', 'activities', 'budget'], rounds: 3 ``` No central planner dictates the trip — the agents negotiate to a consistent itinerary, the way a group of friends each owning one job would. ______________________________________________________________________ ## Customization ### Tune convergence ``` planner = Swarm(agents=planner._agents, rounds=4, neighbor_count=3) # more cross-talk → tighter consistency ``` ### Add a local-guide agent ``` planner.agents.append( Agent("local_guide", GeminiLLM("gemini-2.5-flash"), system_prompt="Add authentic local food and neighbourhood picks; avoid tourist traps."), ) ``` ### Hard budget cap ``` planner.agents[-1].system_prompt += " The total MUST stay under the stated budget; cut activities before lodging." ``` ______________________________________________________________________ ## When to Use | Situation | Use Swarm? | | --------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | Interdependent choices that must settle to consistency | ✅ Yes | | You want decentralized negotiation, no single coordinator | ✅ Yes | | A manager should assign and synthesize | ❌ Use [Orchestrator-Workers](https://pyagent.org/packages/patterns/orchestration/orchestrator-workers/index.md) | | Agents coordinate through shared named state | ❌ Use [Blackboard](https://pyagent.org/packages/patterns/structural/blackboard/index.md) | ______________________________________________________________________ ## Cost Profile | Driver | Typical model | Avg cost | Notes | | ------------------- | ------------- | ----------- | --------------------------- | | 4 agents × 3 rounds | gemini-flash | $0.0048 | N × rounds calls | | **Per itinerary** | gemini-flash | **~$0.005** | scales with agents × rounds | `neighbor_count` and `rounds` trade convergence quality against cost — start at 2 / 3. ______________________________________________________________________ ## See Also - [Swarm pattern](https://pyagent.org/packages/patterns/advanced/swarm/index.md) - [Emergent NPC World](https://pyagent.org/cookbook/gaming/npc-world/index.md) — decentralized coordination via a blackboard - [Browse all recipes](https://pyagent.org/cookbook/index.md)