Blog

AI agent evaluation is a controlled way to decide whether an agent change is safe enough to advance. Start with representative tasks and observable end states, then add trace, safety, reliability, and cost evidence only where the release decision needs it. A production framework is not a single benchmark score and it is not a vendor ranking.

For a team moving beyond a demo, the useful question is not “Did the agent produce a convincing answer?” It is “Did it complete the right work, use permitted actions, and remain inside a known operating envelope?” That distinction keeps an evaluation program tied to engineering decisions instead of a dashboard full of metrics.

What AI Agent Evaluation Measures

An agent run has more moving parts than a one-shot model answer. Anthropic's engineering guide to agent evaluations describes an evaluation in terms of a task, a trial, grading logic, and the resulting record. For an agent, that record can include the tool trajectory, intermediate state, final output, and the environment in which the task ran.

An evaluation harness is the repeatable wrapper around those pieces. It supplies a scenario, runs the agent under stated conditions, applies graders, and retains enough evidence to compare one version with another. The point is not to copy production perfectly. It is to make an important release question repeatable.

Use two distinct questions when defining a case:

• Outcome: Did the task finish in the intended state? A code task may require a test to pass; a support workflow may require the correct ticket fields and response.
• Process constraints: Did the agent stay within its approved tool, data, and policy boundaries while reaching that state?

That separation prevents a dangerous false positive: an agent can reach a correct-looking result through an unapproved action, or follow a tidy trajectory and still leave the system wrong. Teams working on multi-agent repository tasks should make the task boundaries explicit before delegating work across agents or contexts; the same principle applies to an agent swarm codebase workflow.

Quick Evaluation Matrix

The best AI agent evaluation metrics are decision-specific. Treat the following matrix as a starting scorecard, not as five universal KPIs that every pilot must optimize.

Evidence layer	Release question	Example check	What it does not prove
Outcome	Did the requested state change happen?	Tests pass; required record exists; task is complete	That the route was safe or efficient
Trajectory	Did the agent take an acceptable path?	Trace shows approved tools and critical steps	That a single trace generalizes
Safety	Did it respect non-negotiable constraints?	No restricted tool call, data exposure, or policy violation	Broader business usefulness
Reliability	Does it behave consistently across representative cases?	Pass rate and failure modes across a fixed suite	Production demand outside the suite
Cost and latency	Is the workflow viable to operate?	Runtime, tool calls, model usage, human correction time	Quality independent of those measures

AWS's practical guidance on evaluating agentic systems similarly frames evaluation as a layered operational exercise rather than a single quality number. Keep a metric only if someone can name the decision it changes: promote, hold, investigate, change a guardrail, or narrow the task.

Define Production Scenarios Before Metrics

A useful AI agent evaluation framework starts with a small set of scenarios that resemble the work the team is prepared to allow. Do not begin by asking which agent evaluation tools have the longest feature checklist. First write the cases that a reviewer would recognize as meaningful.

For each scenario, record:

1. Initial state. Pin the repository revision, fixture data, available credentials, and environment assumptions.
2. Task contract. State the goal, allowed tools, prohibited actions, and what the agent may ask a human to approve.
3. Expected end state. Prefer checks that can be independently observed: a test result, a generated artifact, a database state, or a reviewed diff.
4. Critical path expectations. Identify steps that must occur or boundaries the agent must not cross. This makes trace review targeted instead of theatrical.
5. Failure classification. Decide whether an unsupported claim, a timeout, a policy stop, or a wrong final state counts as a failure.

Representative does not mean “most difficult.” Begin with common work that has clear correctness signals, then add edge cases that have caused real review friction. For a coding-agent pilot, this might be a small bug with existing tests, a constrained documentation update, and a multi-file change with a known contract. It should not be an open-ended feature request with no accepted outcome.

This discipline also makes tool selection more honest. The best AI coding agents for a team are not established by a generic leaderboard alone; fit depends on the tasks, permissions, and review process the team can actually evaluate.

Measure Outcomes, Trajectories, Safety, Reliability, and Cost

Start with an outcome grader whenever a deterministic check exists. Unit tests, schema validation, exact record assertions, and validated build outputs are usually easier to audit than a model opinion. Anthropic notes that code-based graders work especially well when the environment and expected result are stable. They are not sufficient for every qualitative task, but they make a durable backbone for agent regression testing.

Then add a trace grader for the part that the final state hides. A trace review may check whether the agent read an authoritative source before editing, used only allowlisted tools, requested confirmation for a protected action, or stopped when an expected dependency was absent. Make the rule concrete. “Good reasoning” is not a reviewable criterion; “did not invoke a write-capable tool before approval” is.

Safety checks should be hard gates, not averages. If a run accesses a prohibited resource or bypasses a required human approval, it should be visible as a distinct stop condition even if the final output is otherwise useful. This is why a high success rate cannot compensate for a serious policy failure.

Reliability belongs to a fixed, versioned scenario set. Record the agent version, prompt or instruction revision, enabled tools, permissions, model configuration, environment revision, and grader revision beside each run. Without that provenance, a changed pass rate has no stable meaning. A baseline is not a promise of future quality; it is the comparison point that lets a team detect a behavioral change.

Finally, include operating cost in the decision. Count elapsed time, tool calls, model usage where available, retries, and human correction or review time. A workflow that passes a narrow suite but needs constant intervention may still be the wrong production choice. Conversely, lower latency is not a quality result unless it preserves the outcome and safety evidence.

Build an Offline-to-Online Evaluation Loop

Offline evaluation and production monitoring answer different questions. An offline suite asks whether a known set of scenarios still behaves acceptably before release. Production monitoring observes what happens after release, where inputs, integrations, and user behavior vary. Anthropic explicitly recommends using both: monitoring complements controlled evaluations; it does not retroactively prove that a release was safe.

Use a simple loop:

1. Build a capability set for new behavior and a regression set for behavior that must not change.
2. Run both before changing a model, prompt, tool, retrieval source, or permission boundary.
3. Compare results against the stored baseline, including failed traces rather than only aggregate pass rates.
4. Release narrowly with monitored signals, alert thresholds, and an explicit rollback or pause owner.
5. Turn recurring production failures into sanitized, versioned offline scenarios after human review.

Structured repository context can improve the evidence an agent sees before it acts, but it does not replace this loop. A code knowledge graph can help an agent locate relevant modules and tests; the source, grader, and reviewer still decide whether the change is acceptable.

Run a Pilot and Freeze a Regression Set

Keep the first pilot small enough that a technical lead can read every failure. A practical sequence is:

1. Select three to ten representative scenarios with explicit end states and hard safety rules.
2. Run the current workflow to create a baseline, preserving the environment and grader configuration.
3. Change one meaningful variable at a time, such as the prompt, tool policy, model, or retrieval layer.
4. Review failures by category: bad task definition, wrong source selection, tool-use error, policy stop, grader defect, or genuine capability gap.
5. Freeze the accepted scenarios as a regression set before expanding scope.

Do not silently replace failing cases with easier ones. If a scenario is invalid, document why and replace it with a representative alternative; otherwise it belongs in the regression history. This produces a decision trail that a future reviewer can inspect instead of a claim that the agent was “tested.”

Before freezing a set, hold a short calibration review with the people who will approve releases. Read a small sample of passes and failures together. Confirm that the task contract still describes the work, the grader is checking the intended state, and each hard stop means what the team thinks it means. If reviewers disagree about a case, record the disagreement as a task-definition or grader issue rather than quietly reclassifying the run.

The regression set should remain narrow enough to explain. Keep the baseline result, failed traces, and acceptance decision beside the scenario revision. When a new task type is added, give it the same contract and review treatment before allowing its result to change the aggregate release decision. This preserves the link between a score and the concrete work it represents.

Treat every new scenario as a hypothesis about a release decision, not a tally target that a narrow prompt can merely optimize.

FAQ

Which AI agent evaluation metrics should a team track?

Track the smallest set that supports a release decision: a final-state success check, explicit safety-constraint pass or fail, targeted trajectory evidence, reliability across a versioned scenario set, and operating cost or latency. Add a metric only when its threshold changes what the team will do. There is no defensible universal composite score because a production agent's task, permissions, and harm boundaries differ.

Conclusion

Production AI agent evaluation is a release discipline: define representative scenarios, make the desired state and non-negotiable constraints observable, preserve the run evidence, and freeze accepted behavior into a regression set. Promote a change only when that evidence supports the specific operating decision in front of the team. Name the reviewer, rollback owner, and evidence record before the expanded release begins.

For teams evaluating open source AI agent projects in 2026, the most interesting projects are no longer only chat interfaces. My opening inference from RisingRepo's snapshot generated on 2026-07-25 at 17:19 UTC is that the stronger pattern was infrastructure: desktop execution, shared browser state, model routing, human-agent workspaces, reusable skills, vertical workflows, and edge inference.

These eight repositories are worth watching because they target different failure points in an agent workflow. Their GitHub momentum is a discovery signal, not proof of quality. Seven had recognized open-source licenses when I checked on July 26, 2026; ESP32-AI had public source but no detected license, so teams should not assume reuse rights.

Quick Verdict

RisingRepo marked its 2026-07-25 17:19 UTC report as a "High Signal Day." For readers comparing open source AI agent projects in 2026, the useful signal is not that one repository won a popularity contest. It is that several projects from different layers rose together.

Project	Agent-stack role	RisingRepo signal	Verified fit	Main caveat
OpenWorker	Desktop execution	+18	Local-first agent with connectors, schedules, and approval gates	Open beta; Windows build was not code-signed
Buzz	Human-agent workspace	+14	Self-hosted rooms, identities, workflows, and signed events	Broad architecture; parts of workflow approval were still in progress
OmniRoute	Model routing	+13	One local endpoint with routing and fallback	Provider, free-tier, and savings figures are maintainer-reported and volatile
Skills	Engineering process	+12	Small, composable skills for coding agents	Process skills do not supply repository facts by themselves
AI Job Search	Vertical workflow	+9	Local application, CV, review, and interview workflow	Outcome numbers are one maintainer's self-reported case
video-shotcraft	Creative skill	+7	Remotion-based product-video workflow	Very new; third-party asset and Remotion licensing still matter
ESP32-AI	Edge inference	+7	Tiny language model experiment on an ESP32-S3	No detected license; not a general-purpose agent
ego-lite	Browser execution	+7	Separate agent browser Spaces with migrated login state	macOS only; performance claims are vendor-reported

The star totals were also substantial at the 2026-07-26 02:15 UTC GitHub check: from 824 for the two-day-old ESP32-AI repository to 188,326 for Matt Pocock's Skills. Those totals are volatile. I would use them to decide what to inspect, not what to deploy.

For an editorial team, this makes the snapshot useful as a reporting queue. OpenWorker and ego-lite support an article about controlled execution; Buzz supports a collaboration and auditability angle; OmniRoute supports a routing and cost-governance angle; Skills and video-shotcraft support the rise of packaged expertise; AI Job Search supports vertical workflow design; and ESP32-AI supports an edge-inference experiment. Each angle has a different evidence burden, so a single star-growth ranking would hide the distinctions that matter to readers. Graphify's AI Coding hub provides the broader engineering context, while the best AI coding agents in 2026 page owns the evergreen product-ranking intent.

Why This Snapshot Is High Signal

I map the eight selected repositories to seven distinct layers: reusable skills, desktop execution, collaboration state, model routing, authenticated browser work, vertical applications, and edge inference. That editorial distribution matters more than any single star count.

The inference I draw is that AI agent infrastructure is becoming more modular. Instead of building one agent that owns every interface and workflow, developers are packaging narrower capabilities around an existing harness. The model remains important, but the engineering pressure has moved toward access, state, permissions, context, and repeatable execution.

RisingRepo's addedStars field cannot establish reliability, security, or long-term maintenance. The captured snapshot does not provide enough methodological detail to treat the values as audited lifetime growth or a universal velocity window. They can still show where attention clustered inside that report. On this date, it clustered around making agents act in real environments rather than making chat responses look better.

Eight Projects and the Problems They Solve

OpenWorker

OpenWorker tries to turn a desktop agent into a coworker that delivers files and completed actions. It was created on July 20 and had 5,054 stars and 669 forks at the July 26 check, after a +18 RisingRepo signal.

The commit-pinned README describes a local Python agent server, macOS and Windows desktop packages, 25+ connectors, scheduled runs, approval gates, multiple model providers, and Ollama support. That combination targets a practical gap: an agent can reason about a task, but it still needs controlled access to files, calendars, messages, and business tools.

The project is MIT-licensed and unusually complete for its age, but it was still labeled open beta. The Windows build was not yet code-signed. I would test one bounded workflow and inspect its approval and credential paths before connecting a production account.

This is the practical boundary for local AI agents: the control plane can stay on the user's machine while selected model calls still leave it. “Local” should describe the verified data path and execution surface, not imply that every dependency runs offline.

Buzz

Buzz addresses coordination rather than individual task execution. It had 12,005 stars, 961 forks, and a +14 RisingRepo signal. The repository is Apache-2.0 licensed and was pushed again during the verification window.

Buzz models people, agents, workflows, reviews, and Git activity as signed events on a Nostr relay. The useful idea is not "Slack with an agent." It is that an agent gets an identity, room membership, and an audit trail similar to a teammate. That makes project memory and responsibility more visible.

The cost is system scope. Buzz includes a desktop client, relay, database, search, workflows, Git integration, and agent tools. Its README also said workflow approval infrastructure existed while some glue was still drying. Teams should evaluate the operational burden, not only the collaboration model.

OmniRoute

OmniRoute sits between coding tools and model providers. It had 30,067 stars, 3,917 forks, and a +13 RisingRepo signal. The MIT-licensed project exposes an OpenAI-compatible local endpoint with routing, fallback, and provider management.

This layer becomes useful when a team already uses several models or subscriptions and wants one policy surface for quota, cost, and availability. It solves a routing problem, not a repository-understanding problem.

The README reported 290 providers, 516 models, large free-tier totals, and token savings from compression. Those are maintainer-reported, fast-changing figures rather than independent benchmarks. Provider terms and free allowances can change independently of the code, so verify the current catalog and each provider's terms before treating the headline totals as capacity.

Skills

Matt Pocock's Skills had the largest star total in this group: 188,326 stars, 16,181 forks, and +12 in the RisingRepo snapshot. The MIT-licensed repository packages engineering practices as small, editable, composable agent skills.

The README supports two distribution models: copy skills through skills.sh so a team can edit them, or install a managed Claude Code plugin. Agent Skills-compatible harnesses such as Codex can use the copied form. The important pattern is that workflow knowledge is becoming a distributable dependency.

That does not remove the need for repository context. A testing or design skill can enforce a process, but it cannot know module boundaries or historical decisions that were never supplied. Graphify's Matt Pocock Skills code graph is useful when you want to inspect the repository itself, while the Skills directory page owns installation-oriented details.

AI Job Search

AI Job Search shows what happens when an agent workflow becomes a vertical application. It had 26,832 stars, 8,783 forks, and a +9 RisingRepo signal. The MIT-licensed repository covers profile setup, job discovery, fit evaluation, tailored CV and cover-letter drafting, independent review, and interview preparation.

The core workflow is described as country-agnostic, while several portal integrations target Denmark. The maintainer reports 69 tailored applications, 20 first interviews, and one signed contract. That is useful first-person evidence, but it is not a controlled success-rate benchmark and should not be generalized.

The project is valuable as a workflow design example: it keeps source materials local, separates drafting from review, and records outcomes. You can inspect its structure through the existing AI Job Search code graph.

video-shotcraft

video-shotcraft packages motion-design knowledge as an installable skill for Claude Code and Codex. The seven-day-old Apache-2.0 repository had 1,749 stars, 150 forks, and a +7 RisingRepo signal.

Its README lists 106 shot recipe cards, 162 styles, 161 motion previews, Remotion implementations, and a 36.2-second product-video template. This is a concrete example of an agent skill carrying domain constraints, references, and reusable implementation patterns rather than a single prompt.

The repository's license does not erase downstream rights checks. Its README calls out audio attribution, replacement of demonstration screenshots, and Remotion's separate license conditions. A team should verify every asset and runtime license before publishing commercial output.

ESP32-AI

ESP32-AI is the outlier in this list. It is not a general-purpose agent. It is an edge-inference experiment that reports a 28.9-million-parameter TinyStories model running on an ESP32-S3 at roughly nine tokens per second.

The two-day-old repository had 824 stars, 93 forks, and a +7 RisingRepo signal. Its README explains how most parameters stay in flash while a smaller computation core uses faster memory. It also states the model's boundary clearly: it generates simple stories but does not reliably answer questions, follow instructions, write code, or know facts.

GitHub exposed no license metadata when I checked. Public source is not the same as permission to reuse, modify, or distribute it. Treat the architecture as research material unless the maintainer adds explicit terms or grants permission.

ego-lite

ego-lite focuses on authenticated browser work. It had 3,626 stars, 178 forks, and a +7 RisingRepo signal. The repository is MIT-licensed, while the browser itself is described as a separate free download.

The product gives agents separate browser Spaces and can migrate Chrome logins, cookies, extensions, and bookmarks. That targets a common automation failure: the agent needs an authenticated browser, but the user still needs control of their own tabs.

The current download is macOS-only. Claims about being up to 2.5 times faster, using fewer tokens, or producing the strongest snapshots come from the project's own benchmark and marketing material. Test the pages, authentication boundaries, and tasks that matter to your team.

What These Projects Say About the Agent Stack

Three patterns connect these otherwise different repositories.

My first editorial inference is that capabilities are becoming packages. Skills and video-shotcraft distribute engineering or creative judgment as files an existing agent can load. AI Job Search applies the same idea to a complete vertical workflow. These agent skill repositories make process knowledge portable, but they still depend on the receiving harness and project context.

My second editorial inference is that execution is moving closer to the user's environment. OpenWorker acts from the desktop, ego-lite carries browser state, and Buzz keeps human and agent actions in one signed workspace. These systems still call external models in many cases, but the control plane, credentials, approvals, and artifacts increasingly live near the user. Graphify's browser automation MCP server guide covers adjacent browser-control approaches without turning this snapshot into an evergreen tool comparison.

Third, the stack is separating into specialized infrastructure. OmniRoute handles model access. Buzz handles shared state and coordination. ego-lite handles browser execution. ESP32-AI explores a much smaller edge boundary. This is an editorial inference from one snapshot, not a claim that the market has settled on a standard architecture.

What Teams Should Verify Before Adopting One

A fast-rising repository deserves inspection, not automatic installation.

Use these checks before giving a project real data or credentials:

1. License: Confirm that the code, bundled assets, model weights, and downloaded binaries have terms that fit your use.
2. Maturity: Check repository age, release history, recent commits, issue patterns, and whether critical components are still marked beta or roadmap.
3. Data path: Identify what stays local, what reaches model providers, and where credentials or browser state are stored.
4. Approval boundary: Test which actions require confirmation and what unattended runs can do.
5. Claim provenance: Separate independent results from maintainer outcomes, vendor benchmarks, and future features.
6. Integration surface: Verify the exact operating systems, agents, APIs, providers, and file formats your workflow needs.
7. Maintenance cost: Estimate how often skills, routing rules, browser state, indexes, or workflow definitions must change.

The right project is the one that reduces a known execution failure without creating a larger review or maintenance burden.

Apply the gates in order. A missing or incompatible license can end the evaluation before a team spends time benchmarking. If the terms fit, test the data and approval boundaries with disposable accounts and non-sensitive fixtures. Only then measure workflow quality, integration effort, and maintenance cost. This sequence keeps a polished demo or a large star count from outranking the conditions that determine whether a tool can be used safely.

FAQ

What is the best AI agent in 2026?

There is no universal best agent in this snapshot. OpenWorker targets desktop work, Buzz targets collaboration, OmniRoute targets model routing, Skills targets engineering process, and ego-lite targets browser execution. Choose the layer that matches the failure mode you need to reduce, then validate it on a bounded task.

Are all eight projects open source?

Seven repositories had MIT or Apache-2.0 licenses when checked on July 26, 2026. ESP32-AI had public source but no detected license. ego-lite also distinguishes its MIT-licensed repository contents from a separate free browser download, so binary terms should be verified independently.

How should teams evaluate a fast-rising GitHub project?

Treat growth as a discovery signal. Verify license, commit and release activity, platform support, data flow, approval controls, claim provenance, integration fit, and maintenance cost. Run one real task with limited permissions before expanding access.

Conclusion

My concluding inference from the July 25 RisingRepo snapshot is that an agent stack is taking shape across several repositories at once. Skills, execution environments, state, routing, and vertical workflows are becoming separate design choices.

I would follow the project that addresses a concrete bottleneck, not the one with the largest star count. Then I would verify its license, data path, maintenance model, and failure behavior before trusting how polished the demo looks.

The smallest truthful way to share context between Pi and Claude Code is to commit neutral repository guidance in AGENTS.md, import it from Claude Code’s CLAUDE.md, keep tool-specific procedures in native Skills, and expose current code relationships through a read-only model-independent layer. Do not copy sessions or agent summaries into a universal memory and assume the tools now understand the repository the same way.

This pattern follows the tools’ documented loading behavior. Pi prefers AGENTS.md over CLAUDE.md in each directory. Claude Code reads CLAUDE.md and documents @AGENTS.md as a compatibility import. The shared file therefore remains canonical, while Claude-specific instructions can follow the import.

The article was checked against Pi v0.82.0 and current Claude Code documentation on July 24, 2026. A shared graph or index is an external team layer, not native shared memory in either tool.

Pi stores task conversations as JSONL session trees and can compact older turns, but session history is not organization-wide shared memory. Claude Code has separate CLAUDE.md, Skills, and auto-memory surfaces; tool-specific rules must remain distinct from repository facts.

Before Sharing Repository Knowledge

To share context between Pi and Claude Code safely, classify each item as neutral guidance, tool-specific procedure, task state, or revision-bound repository evidence.

Separate information by authority and lifecycle before choosing files.

Share stable, tool-neutral guidance:

• repository purpose and major boundaries;
• approved build, test, lint, and formatting commands;
• generated-code and migration rules;
• security and data-handling constraints;
• ownership and escalation paths;
• links to authoritative architecture records.

Do not put these items in a universal file:

• tool permission syntax;
• provider-specific commands or plugins;
• one session’s private reasoning;
• temporary branch state;
• unrestricted credentials;
• volatile lists of callers, dependencies, and tests.

The last category is especially important. A static Markdown file can describe the intention behind a boundary. It should not pretend to maintain a current call graph.

Create an inventory of existing AGENTS.md, CLAUDE.md, rules, Skills, prompt templates, and repository indexes. Resolve contradictions before adding imports. Otherwise sharing simply distributes existing drift.

Define a Model-Independent Context Layer

Start the model-independent code context layer with one neutral instruction file and one revision-aware query contract.

Commit a root AGENTS.md containing only neutral guidance. Pi’s resource loader walks from ancestor directories toward the current directory and selects the first recognized context candidate per directory, preferring AGENTS.md. Nested directories can add narrower AGENTS.md files as needed.

Create a root CLAUDE.md like this:

@AGENTS.md

# Claude Code-specific guidance

- Use the project permission rules for read-only analysis.
- Invoke the repository-analysis Skill for cross-module impact work.

Claude Code resolves the import; Pi selects AGENTS.md first and does not need to interpret Claude’s import syntax. Pi’s CLI @file argument only adds a file to one user message. It is not evidence that Pi supports @import inside context files.

For dynamic model-independent code context, define a tool-neutral query schema:

repo + commit + question
→ modules + relationships + tests + sources + unknowns + index age

Both agents should receive the same relationship semantics and provenance. Claude Code can use an MCP or CLI adapter. Pi needs a CLI or extension/package adapter because built-in MCP is not documented.

Search comparisons such as Pi coding agent vs Claude Code should not change this contract. The shared layer is useful precisely because tool choice can change while the repository evidence format remains stable.

Separate Tool Preferences From System Facts

The practical Pi coding agent vs Claude Code distinction belongs in native tool behavior and permissions, not in the shared repository-fact contract.

Put tool behavior in tool-native locations.

For Pi:

• use Pi context for harness-specific defaults;
• use Skills for reusable procedures;
• use Prompt Templates for lightweight slash-command prompts;
• use extensions/packages for new tools and adapters.

For Claude Code:

• keep stable persistent instructions in CLAUDE.md and rules;
• use Skills for on-demand procedures;
• use MCP/plugins for external tools;
• enforce filesystem and command behavior with permission settings, not prose alone.

If one Agent Skill genuinely works in both harnesses, make one directory canonical and add it to Pi’s skills configuration. Test the Skill in both tools. Claude’s allowed-tools is a pre-approval mechanism; Pi labels the field experimental and does not have the same permission-popup layer. Shared frontmatter does not imply shared enforcement.

System facts belong elsewhere. Current definitions, imports, calls, registrations, schemas, event flows, generated sources, and test mappings must come from current source or a revision-bound index. Store file and symbol citations beside each material relationship.

Reuse Graph Queries Across Both Agents

Design a small set of task-shaped queries:

• locate the authoritative definition and entry point;
• trace direct, indirect, and unresolved callers;
• map change impact across modules and contracts;
• find relevant tests and coverage gaps;
• retrieve architecture decisions connected to a component.

Return bounded results with indexed_commit, source locations, extraction method, confidence or edge type, and failed coverage. The adapter should reject a task when its checkout does not match the indexed commit.

A shared query does not require identical runtime integration. Claude Code may call repo_graph.impact through MCP. Pi may run repo-graph impact --json. Normalize both responses into the same evidence object before the Skill interprets them.

Keep the service read-only for task agents. Index updates run separately and produce reviewable artifacts. Require both tools to verify high-impact edges in their checkouts and record mismatches. This prevents a stale graph from becoming a cross-tool source of synchronized error.

Define a portable handoff file for tasks that genuinely move from one harness to the other. Keep it small and factual:

task: AUTH-214
repository: platform
base_commit: 8f31c2a
inspected_paths: []
verified_relationships: []
tests_run: []
decisions: []
unknowns: []

The handoff should not contain secrets, unrestricted tool output, or a claim that hidden reasoning was transferred. The receiving agent rechecks the base commit, opens cited paths, and confirms material relationships. If the branch moved, it refreshes the evidence packet before continuing.

Version shared files and adapters together in a small compatibility matrix. Record which Pi release, Claude Code release, Skill hash, and query schema were tested. A Markdown instruction may remain readable after a tool update while its permission frontmatter or adapter command changes meaning. Run fixture tasks after upgrades: one normal query, one denied repository, one revision mismatch, and one deliberately unresolved edge. Treat a silent fallback to broad shell search as a failed integration, not a successful shared-context run.

Keep one human-readable migration note beside the matrix. It should explain removed commands, changed permission assumptions, renamed Skills, and the rollback path. This prevents a tool upgrade from becoming an invisible context fork across developer machines.

Review it together.

Roll the pattern out in read-only mode first. Choose one repository and two analysis tasks, record which files each harness loads, and compare the evidence objects returned by both adapters. Fix differences in path normalization, symbol identity, permission denial, and revision handling before allowing either tool to edit.

After the pilot, assign owners to the shared file, each native Skill, and the repository index. Define a response time for a stale or incorrect relationship. Without ownership, the shared layer becomes another abandoned document that both agents load confidently.

Keep an escape hatch. If the adapter fails or the index is behind, the agent should state the limitation and either stop or use a documented bounded source-search fallback. It should never silently switch to an unrestricted repository scan and report the result as graph-backed context.

Conflict, Permission, and Drift Risks

The cross-tool system can fail in four places.

Instruction conflict: a nested file or duplicate Skill overrides or competes with shared guidance. Record which files and Skill hashes each run loaded.

Fact conflict: an agent summary or graph edge disagrees with current source. Preserve both claims, verify at the task commit, and update the derived layer after review.

Permission mismatch: one harness pre-approves a tool that the other treats differently. Apply OS credentials, repository access, and tool-native permission controls independently.

Revision drift: the graph, session, and checkout refer to different commits. Make revision mismatch a hard stop for material impact analysis.

Resolve conflicts by provenance and authority, not length or confidence. Current source supports implementation facts. Approved policy and architecture records explain intended constraints. Humans decide security, ownership, destructive operations, and material unresolved architecture questions.

FAQ

What should stay tool-specific?

Keep permissions, plugin or extension setup, provider configuration, native commands, and harness-specific procedures in Pi or Claude Code locations. Share only neutral repository guidance and a common evidence schema.

How should conflicting summaries be handled?

Record each summary’s task, commit, cited paths, and creation time. Verify the disputed claim in current source. Update or retire the derived summary; do not choose whichever agent sounds more certain.

Who approves shared context updates?

Code owners approve structural or architecture changes, security owners approve sensitive rules, and Skill maintainers approve workflow changes. Automated index refreshes can update observed edges, but schema and manual-fact changes need review.

install pi-coding agent

Use the installation method in the current first-party earendil-works/pi documentation and pin the version for reproducible team setups. Installation alone does not enable Claude-style MCP or import semantics; configure the shared adapters explicitly.

Conclusion

Share a small committed instruction core, not an undifferentiated memory dump. Let Pi read AGENTS.md; let Claude Code import it from CLAUDE.md; keep native procedures and permissions separate; and give both tools one revision-bound evidence contract for current code relationships. This setup is simple enough to audit and honest about what the tools do not share.

Pi Claude Code context engineering works best when you separate four things: always-relevant instructions, on-demand procedures, conversation state, and current repository facts. Pi and Claude Code implement the first three with different files and runtime features. Neither tool should be the sole source of truth for dynamic module, dependency, call-chain, or test relationships.

This is a layer-design problem, not a contest over who can load more text. Good context is the minimum material that helps an agent act correctly at the current repository revision. It has an owner, scope, source, refresh trigger, and verification rule.

The product facts below were checked on July 24, 2026 against Pi v0.82.0 and current Claude Code documentation. Pi refers to the earendil-works/pi terminal coding harness.

Quick Verdict

Pi Claude Code context engineering works best when stable instructions, on-demand procedures, session state, and revision-bound repository facts remain distinct.

Use tool-native context for tool-native behavior:

• Pi context files, Skills, Prompt Templates, extensions, packages, and sessions should control how Pi operates.
• Claude Code CLAUDE.md, rules, Skills, auto memory, plugins/MCP, subagents, and sessions should control how Claude Code operates.

Use committed neutral instructions for conventions both tools must follow. Use source code and a model-independent context layer for repository facts that both tools need to query. Keep each task’s plan, branch, and intermediate reasoning local.

The important distinction is stable versus volatile. Formatting rules and approved commands may remain stable for months. A symbol’s callers can change in the next commit. Loading volatile facts into a permanent instruction file turns yesterday’s observation into tomorrow’s false authority.

Pi Skills and Prompt Templates

Pi loads the first recognized context file in each directory, preferring AGENTS.md and falling back to CLAUDE.md. It walks from ancestors toward the current directory and also supports global context. These files are suitable for stable project conventions, commands, and constraints.

Pi skills follow the Agent Skills standard. Pi scans Skill names and descriptions, then reads the full SKILL.md when a Skill is selected. Skills are appropriate for reusable multi-step procedures, specialized references, output schemas, and task-specific checks.

Prompt Templates are different. They are Markdown snippets invoked as filename-based slash commands. A template can expand a common prompt, but it does not become a Skill package with the same discovery and supporting-file behavior.

Sessions are JSONL trees. Compaction summarizes older turns so work can continue under context limits. That session record belongs to one conversation; it is not shared team repository memory.

Searches for Pi-coding-agent GitHub should resolve to the current first-party earendil-works/pi repository. Historical package and repository names can appear in third-party content, so pin links and versions when documenting extensions or Skills.

Use Pi layers this way:

• AGENTS.md: stable cross-tool conventions and commands;
• Pi-specific global or directory context: harness behavior that should always apply;
• Skill: an on-demand repository-analysis or review workflow;
• Prompt Template: a lightweight reusable prompt;
• session: task conversation and recoverable local state;
• extension/package: new runtime capability or adapter.

Claude Code CLAUDE.md, Skills, and Memory

The Pi-coding-agent GitHub source remains the authority for Pi behavior even when a cross-tool comparison is organized around Claude Code's context surfaces.

Claude Code reads CLAUDE.md as human-written persistent instructions. It supports nested files, rules, and @path imports. Anthropic’s documented compatibility pattern lets CLAUDE.md import AGENTS.md, which is useful when a repository also serves tools that read AGENTS.md directly.

Claude Code Skills are on-demand procedures. Their descriptions are available for selection while full bodies load when used. Legacy custom commands still work but are merged into the Skills model; “recipe” is therefore an editorial label rather than an official feature name.

Claude auto memory is Claude-written, per repository, and local to a machine. It can retain useful observations across local sessions and worktrees. It is not an organization-wide source of truth and can drift from current code. Treat it as a lead that needs verification.

Claude Code also exposes plugins, MCP, subagents, code-intelligence integrations, and fine-grained permission rules. These can improve retrieval and control. They do not make a material code claim true without evidence.

Keep CLAUDE.md concise and stable. Put specialized procedures in Skills so they load on demand. Put current structural relationships in source or an external index. Keep permission enforcement in actual permission rules rather than prose alone.

What Rules Cannot Express About Code

Rules can name architectural intent: “the API package must not depend on storage adapters,” or “generated clients are never edited directly.” They should not attempt to enumerate every current caller and test.

Large repositories contain relationships that are expensive and error-prone to maintain as prose:

• symbols defined and referenced across packages;
• registration and dependency-injection paths;
• schemas that generate clients or migrations;
• events and queues connecting services;
• tests covering contracts through indirect behavior;
• architecture decisions constraining a dependency;
• unresolved dynamic edges and failed parses.

A CLAUDE.md line that says “all payments are in packages/billing” becomes dangerous after a migration. A Pi Skill that hard-codes the same assumption has the same problem. Procedures should ask the current code or index where the path lives and require source verification.

Shared Repository Knowledge Layer

Build one tool-neutral evidence contract:

repository
indexed_commit
query
entities
relationships
source_locations
derivation
coverage_gaps
generated_at

The layer can be structured files, a language-server-backed index, or a code knowledge graph. Give Claude Code an MCP or CLI adapter. Give Pi a CLI or extension/package adapter because Pi does not document built-in MCP. Return the same schema to both tools.

The code knowledge graph or index proposed here is a shared external layer, operated by the team, for model-independent structural queries with revision-aware provenance.

Keep retrieval task-shaped. A request for impact analysis should return likely definitions, callers, schemas, tests, owners, and unknown edges—not a dump of the repository. Bind results to the task commit and reject them when the revisions differ.

Ordinary agents should query read-only. A separate reviewed job updates the index. Every relationship should cite current source, and agents should verify high-impact edges before planning edits. The shared layer reduces repeated discovery; it does not merge the tools’ private reasoning.

Add a small handoff object when work moves between harnesses. Record the task, repository commit, evidence queries, inspected paths, decisions, rejected alternatives, tests, and unresolved questions. Keep it declarative; do not attempt to serialize hidden reasoning. The receiving tool should re-open material files and mark any conclusion it cannot reproduce. This makes a Pi-to-Claude or Claude-to-Pi handoff auditable without pretending that a session transcript is portable memory.

Expire the handoff when its base commit or acceptance criteria changes, and keep replacement history for review.

Make every replacement explicit and reversible.

Treat handoff generation as a reviewed workflow. A convenience command may collect paths and test output, but a human or independent reviewer should approve material decisions before another tool inherits them.

Limits and Setup Risks

Cross-tool context fails in predictable ways:

• Pi and Claude Code can interpret the same prose differently.
• A Skill frontmatter field may have different permission semantics.
• A session summary can preserve a conclusion after its source changes.
• An index can miss reflection, generated code, or runtime configuration.
• Broad shared context can expose sensitive relationships.
• Duplicate Skills can resolve differently in each harness.

Test every shared asset in both tools. Record the loaded context files and Skill versions. Restrict the repository adapter by identity and path. Include “unknown” and “revision mismatch” fixtures. Require human review for architecture, security, data contracts, and destructive actions.

The smallest safe system is usually better: one neutral committed file, one tool-specific instruction layer per harness, a few well-tested Skills, and a read-only evidence adapter.

FAQ

Which context should be tool-specific?

Keep permission behavior, native commands, provider adapters, extension setup, and harness-specific workflow in tool-native files or Skills. Shared files should contain only neutral repository conventions that both tools can interpret safely.

Who maintains shared repo knowledge?

Assign a repository-context owner and an automated extraction service. Code owners review material schema or manual-fact changes. Task agents consume the layer read-only and report mismatches.

How should teams compare outputs?

Pin the same commit and task, require path-and-symbol evidence, score supported relationships and honest unknowns, and compare tool/file reads separately from correctness. Do not judge by prose confidence.

pi coding agent extensions

Pi extensions add runtime capabilities such as tools, commands, or integrations. Treat each extension as code: review its source, permissions, update path, and behavior before using it as a repository-context adapter.

Conclusion

Context engineering is lifecycle engineering. Put stable rules where they always load, procedures where they load on demand, task conversation in sessions, and volatile repository facts in a source-attributed layer. Pi and Claude Code can then share evidence without pretending to share one mind.

There is no defensible universal winner for Pi vs Claude Code large codebase context without running both tools against the same repository revision, task, model conditions, permissions, and evidence rubric. Claude Code documents more built-in large-repository context controls. Pi offers a deliberately minimal, extensible harness. Feature presence is not proof of more accurate architecture, call-chain, impact, or test discovery.

Use this comparison to design a fair test, not to import benchmark conclusions from unrelated tasks. The current Pi project is the terminal coding harness published as @earendil-works/pi-coding-agent from the earendil-works/pi repository. This article was checked against Pi v0.82.0 and current Claude Code documentation on July 24, 2026.

Quick Verdict

For Pi vs Claude Code large codebase work, choose the harness only after running both against the same revision, permissions, context budget, and evidence rubric.

Choose Claude Code when you want official, built-in surfaces for nested CLAUDE.md, path-scoped rules, Skills, auto memory, code-intelligence plugins, MCP/plugins, subagents, and fine-grained permissions. Those surfaces can reduce setup and help control context in a large repository.

Choose Pi when you want a small terminal harness whose behavior you can extend through TypeScript Extensions, Skills, Prompt Templates, Themes, and Packages. Pi’s built-in tools are read, bash, edit, write, grep, find, and ls. It intentionally leaves features such as MCP, subagents, permission popups, plan mode, to-dos, and background Bash to extensions or external workflows.

Neither choice eliminates repository discovery. Pi repository context and Claude Code context both depend on what the tool loads, which paths it inspects, what its underlying model infers, and how the team verifies results. The correct verdict is conditional: run a controlled evaluation and publish the evidence, not a brand score.

Repository Context Criteria

Compare Pi repository context and Claude Code context with an identical fixture rather than treating either tool's feature list as proof of repository understanding.

Evaluate the same engineering jobs in both tools:

Criterion	Evidence required
Architecture location	modules, entry points, ownership, and source citations
Call-chain discovery	direct, indirect, generated, and unresolved paths
Change impact	affected contracts, consumers, configuration, and data paths
Test mapping	relevant unit, integration, contract, migration, and end-to-end tests
Context recovery	what survives a fresh, resumed, or compacted session
Objectivity	unsupported claims, missing evidence, and explicit unknowns
Efficiency	tool/file reads, input/cache tokens where available, and time to defensible map

Pin the same repository commit. Use the same underlying model and reasoning level where the harnesses permit it. Give both the same neutral task and committed instructions. Restrict the run to read-only analysis: Pi can allow only read,grep,find,ls; Claude Code can use plan mode or permission rules that prevent edits.

Require both outputs to cite repository-relative paths and symbols. Score an honest “unknown” above a confident unsupported edge.

Run more than one task. A single architecture question can favor the tool whose default search happens to match that repository. Use at least three bounded cases: locating an unfamiliar feature, tracing a change through an indirect contract, and mapping the tests required for a risky modification. Reset the checkout between runs and preserve raw tool logs so reviewers can distinguish agent reasoning from harness behavior.

Separate correctness from operating cost. Record wall time, file reads, shell or tool calls, and available token/usage data, but do not collapse them into one synthetic score. A fast answer with a missing consumer is not efficient engineering. A complete answer that reads the entire repository may be too expensive to repeat. Report the trade-off and its confidence interval if multiple runs are available.

Also control human assistance. Decide in advance whether the evaluator may clarify a task, approve tools, or point the agent toward a module. Apply the same rule to both harnesses and log every intervention. Otherwise the comparison measures how much guidance one run received rather than how each tool recovered repository context.

Repeat borderline cases to expose run-to-run variance.

Publish the repository fixture and scoring instructions when confidentiality allows. Reproducibility matters more than a polished comparison table.

Search demand around Pi coding agent vscode can confuse the comparison. Pi is documented as a terminal coding agent; community or editor integrations should be evaluated separately and not treated as a built-in large-repository capability. Test the official harness configuration you will actually deploy.

Architecture and Module Location

A Pi coding agent VSCode integration, if a team adopts one, is an additional interface layer; the first-party Pi harness documented here remains terminal-oriented and extensible.

Pi walks ancestor directories to the current directory and loads the first existing context-file candidate per directory, preferring AGENTS.md over CLAUDE.md. It also supports global context. That provides stable instructions, but it does not create a module index. The agent still uses reads, search tools, shell commands, Skills, or extensions to locate architecture.

Claude Code loads CLAUDE.md context, supports nested instruction files and path-scoped rules, and documents large-codebase scoping. It can use code-intelligence plugins for definitions and references when a supported language server is installed. It also recommends excluding generated or vendor code and moving specialized procedures into on-demand Skills.

Claude Code also documents importing AGENTS.md from CLAUDE.md, so cross-tool guidance can remain neutral while Claude-specific rules stay distinct from repository facts.

These controls can improve search discipline. They do not prove that Claude Code locates the correct architecture more often. A badly scoped CLAUDE.md can mislead. A language server can omit runtime registration. A Pi extension can provide a stronger project-specific index. The evaluation must inspect cited evidence and missed paths.

For both tools, a small revision-bound module map is useful. Include packages, deployable units, generated areas, owners, decisions, and coverage gaps. Let the agent verify it rather than loading an enormous prose summary.

Call-Chain, Impact, and Test Discovery

Direct text search finds literal names. Real impact can pass through aliases, dependency injection, reflection, configuration, generated clients, schemas, events, queues, and other repositories. Neither harness turns Grep into a complete call graph.

Run a fixture task with known difficulties:

1. one direct caller;
2. one aliased import;
3. one registration or dynamic edge;
4. one generated artifact with an authoritative source;
5. one cross-module contract;
6. one relevant test with a misleading neighbor;
7. one intentionally unresolved path.

Score definitions found, paths supported, false edges, omitted edges, and tests justified. If you expose an external code graph, give both tools the same read-only query contract. Claude Code may access it through MCP or CLI; Pi needs a CLI or extension because built-in MCP is not documented.

That code knowledge graph is a shared external layer for model-independent structural queries with revision-aware provenance.

Context Recovery and Failure Modes

Pi stores sessions as JSONL trees and uses manual or automatic compaction to summarize older turns. Session data is conversation state, not a shared organization memory. Its context files and Skills can reload procedures, while extensions may add other persistence.

Claude Code supports session resume, compaction, CLAUDE.md, Skills, and per-repository machine-local auto memory. Auto memory can help one local repository workflow, but it is not a cross-machine source of truth and should not replace current source evidence.

Test three recovery states:

• a genuinely fresh session with only committed context;
• a resumed session after the initial repository analysis;
• a compacted session after enough work to force summarization.

Ask each tool to restate the affected modules, evidence, unknowns, and tests. Then compare the answer with current source. Recovery quality is not how much prose survives; it is whether material claims remain supported and whether stale conclusions are rejected after the revision changes.

Common failure modes include reading popular directories instead of authoritative paths, treating session summaries as facts, trusting an external index built for another commit, and assuming permission features enforce semantic correctness.

FAQ

What evidence should teams compare?

Compare source-cited module locations, call paths, impact surfaces, test mappings, unsupported claims, explicit unknowns, tool/file reads, and recovery after fresh, resumed, and compacted states. Preserve the prompts and repository commit.

When should tests be manually verified?

Always verify material contract, security, migration, and cross-module tests before publishing a winner. Also inspect tests when an agent selects only a nearby unit test but claims broader behavior coverage.

Can both tools share one context layer?

Yes. Use committed neutral instructions plus a read-only, model-independent repository index with revision and provenance. Give each harness its own adapter and permission controls, then require current-source verification.

pi coding agent

Here, “Pi coding agent” means the current earendil-works/pi terminal harness and package, not an unrelated product named Pi. Pin the version when reproducing the comparison.

Conclusion

Claude Code has a broader documented set of large-repository controls; Pi has a smaller extensible core. That difference is useful context, not a verdict. A trustworthy comparison pins the task and revision, equalizes the evidence contract, restricts both runs to analysis, and scores supported repository conclusions. If you have not run that test, publish a protocol—not a winner.

Multica multi-agent context stays consistent only when every shared item has a source, version, owner, scope, and expiry rule. Workspace context, Skills, Project Resources, issue history, provider sessions, and external repository facts move on different clocks. Do not merge them into one “memory.” Detect version skew, show conflicts to the next agent, and require human review for architecture, security, ownership, and data-contract decisions.

Multica coordinates tasks across agents and providers, but coordination does not semantically reconcile what each agent believes. The product’s documented behaviors make several concrete drift cases predictable: running tasks keep an older Skill copy, resource preparation can fail without blocking a task, and provider-session resume continues one conversation rather than merging knowledge among agents.

This guide was verified against Multica v0.4.10 on July 24, 2026. The governance controls below are team recommendations. Multica does not document an automatic context-consistency engine or a native code knowledge graph.

Why Agent Teams Drift Apart

Multica multi-agent context stays consistent only when every shared item carries an owner, source, version, audience, and expiry rule.

Parallel agents rarely receive identical context. One task may start before a Skill update. Another may run from a newer branch. A third may lack the Project Resource section because a best-effort fetch failed. Even when the prompts match, agents can inspect different files and produce incompatible summaries.

An agent context conflict becomes dangerous when the system hides the difference. Typical examples include:

• one agent says a package owns authentication while current routing points elsewhere;
• two attached Skills use the same concept but prescribe different commands;
• an issue comment records a decision that never reached workspace context;
• a resumed session relies on a file removed in a later commit;
• a graph packet describes the base branch while the task runs on a feature branch;
• a provider-specific permission assumption does not transfer to another provider.

The goal is not perfect agreement. Independent agents should be allowed to disagree. The goal is traceable disagreement: reviewers can see which source, version, and scope produced each conclusion.

Sources of Conflicting Context

An agent context conflict is observable when the task records which context surfaces, revisions, and Skill versions each run actually received.

Multica’s actual context surfaces have distinct failure modes.

Workspace context is shared guidance injected into runs. It can drift from repository policy or become overly broad. Give it a named owner and use it only for stable organization guidance.

Workspace Skills are copied into provider-native locations at task start. Editing a Skill affects newly created tasks; existing runs keep the old copy. Record a Skill version or content hash in every task so review can reproduce what the agent received.

Repository Skills travel with the checkout. If a workspace Skill collides with a repository Skill directory, Multica writes the workspace copy into a collision-free sibling. That avoids overwriting files, but the provider may discover both copies. Directory safety is not semantic reconciliation.

Project Resources supply typed repository or directory pointers. Their preparation is best-effort, so one task can start without the project section or manifest. Treat presence, resource revision, and preparation errors as explicit task metadata.

Provider sessions can resume supported task conversations. Resume preserves that conversation’s state; it does not reconcile summaries between agents or update old conclusions for a new commit.

Searches such as Multica Codex reflect the platform’s multi-provider surface, but provider choice introduces another boundary. Different tools load context files, Skills, MCP configuration, and permissions differently. A shared Markdown file does not guarantee identical enforcement.

Version, Source, and Expiry Rules

The Multica Codex pairing still requires explicit provenance: provider choice does not reconcile stale summaries, divergent Skills, or mismatched repository revisions.

Use a small envelope for every shared context item:

id
type
source
source_revision
content_hash
owner
audience
created_at
expires_at or refresh_trigger
authority
verification_status

The fields do not make a fact correct. They make the fact inspectable.

Choose refresh rules by context type:

• stable workspace conventions: review on policy change and quarterly ownership check;
• Skills: review on provider changes, command changes, permission changes, or failed fixture tests;
• repository facts: rebuild on commit movement relevant to indexed paths;
• task summaries: expire when the branch or acceptance criteria change;
• security and data contracts: require explicit owner approval rather than passive expiry.

Define authority explicitly. Current source and executable behavior support implementation claims. Approved policy controls workflow and security requirements. Architecture records explain intentional constraints. Agent summaries remain interpretations, even when several agents repeat them.

Record timezone and clock source for volatile operational events so ordering can be reproduced across local daemons and the managed service.

Avoid copying entire provider transcripts into the consistency layer. Preserve evidence, decisions, and unresolved questions in a compact task record. Transcripts are useful for debugging, but they carry noise, sensitive data, and model-specific state that should not become shared authority.

Review Workflows for Shared Memory

Run consistency checks at three points.

Before assignment: verify required resources, base revision, Skill hash, and provider adapter. If the task depends on missing context, block it rather than allowing a best-effort run to improvise.

During handoff: require the agent to list evidence inspected, context mismatches, unresolved claims, and any instruction collision. Persist this in the issue, not only the provider session.

Before merge: compare the patch against the task’s recorded context envelope. Re-run affected tests and re-query high-impact repository relationships if the base moved.

A useful conflict workflow is:

1. Preserve both claims and their provenance.
2. Classify the conflict as instruction, source fact, task decision, permission, or ownership.
3. Check revision and scope before judging content.
4. Verify source-fact conflicts in current code or executable behavior.
5. Ask the named owner to resolve policy or architecture conflicts.
6. Record the resolution, affected context IDs, and replacement version.

Do not use voting among agents as an authority mechanism. Agreement can come from a shared stale input. Also avoid “latest timestamp wins”; a newer agent summary may rely on an older repository checkout.

Risks and Escalation Paths

Consistency controls can become bureaucracy if every low-risk observation requires approval. Use risk tiers.

• Low risk: navigation hints and non-authoritative search leads can be refreshed automatically and verified by the consumer.
• Medium risk: module ownership, test mappings, and workflow procedures require source citations and named maintainers.
• High risk: security boundaries, destructive operations, data contracts, compliance rules, and cross-repository write authority require human approval.

Permissions also need independent controls. Workspace roles, agent assignability, personal access tokens, daemon tokens, and underlying tool permissions are not interchangeable. In the current source, Multica’s Claude daemon handler auto-approves tool-control requests for autonomous execution. Contain the daemon with a least-privilege operating-system identity, scoped credentials, and restricted workdir. Do not assume the task queue enforces the coding tool’s filesystem boundary.

When a conflict cannot be resolved before execution, reduce the task to read-only analysis, preserve unknowns, and escalate. Stopping is a valid consistency outcome.

FAQ

Who decides which memory wins?

The authority owner decides after reviewing source, revision, and scope. Current code resolves implementation facts; policy owners resolve organization rules; architecture owners resolve intentional boundaries. An agent summary does not win merely because it is newer.

How should old Skills be retired?

Mark the Skill deprecated, detach it from agents, replace references, and wait for or explicitly stop tasks that still carry the old copy. Archive its hash and migration note so reviews can reproduce historical runs.

What conflicts require human review?

Escalate security, permissions, data contracts, destructive commands, architecture ownership, cross-repository writes, and any material claim that remains unresolved after source verification. Minor navigation hints can be corrected by the consuming agent with evidence.

Is ChatGPT a multi-agent system?

That label depends on the implementation behind a product experience and is outside this article’s scope. The relevant point is operational: in a Multica team, several assigned agents are distinct task actors, and their context must be versioned even if some use the same model provider.

Conclusion

Consistent context is not identical prose in every prompt. It is a controlled set of facts, instructions, and task decisions whose versions and authority are visible. Record the context envelope, detect skew before execution, preserve conflicting claims, and escalate material disagreements. That gives Multica teams a reviewable consistency model without inventing a native shared-memory feature.

Multica agent context can be shared through workspace guidance, attached Skills, Project Resources, and issue history. Those native surfaces coordinate work, but they are not one semantic memory of the repository. Keep stable instructions in the appropriate Multica surface, put current code relationships in a read-only external layer, and keep each agent’s plan, branch, credentials, and unverified conclusions isolated.

The useful distinction is between shared policy, shared pointers, shared facts, and task-local state. Collapsing them into “memory” makes conflicts hard to diagnose. A stale Skill is not the same failure as a stale dependency edge. An issue comment is not the same authority as source code. A resumed provider session is not evidence that another agent sees the same conclusions.

This article reflects Multica v0.4.10 and the public documentation checked on July 24, 2026. The proposed repository layer below is external to Multica; the product does not document a built-in code graph or automatic cross-agent summary reconciliation.

Before Sharing Context Across Agents

Multica agent context should be separated by authority and lifecycle before it is shared among planning, coding, testing, and review roles.

Start by naming the job of each context surface. A useful inventory has five columns: content, owner, consumer, refresh trigger, and authority.

Multica provides several native shared surfaces:

• Workspace context is guidance injected into agent runs. It is appropriate for stable team conventions, escalation rules, and common constraints.
• Workspace Skills are Agent Skills-compatible packages attached to selected agents and copied into provider-native discovery locations when a task starts.
• Repository Skills remain in the checked-out repository and can travel with its revision.
• Project Resources provide typed pointers to repositories or directories and a structured .multica/project/resources.json manifest.
• Issue comments, status, and history preserve collaboration state around a work item.

These mechanisms are more precise than a vague shared coding agent memory promise. They also have different lifecycles. An edited workspace Skill reaches newly created tasks, not a running task that already received its copy. A resource fetch can fail without blocking the task. A provider session belongs to one task or conversation rather than becoming shared memory for every agent.

Before adding another layer, define what must remain local: branch state, intermediate reasoning, temporary patches, destructive credentials, personal notes, and uncertain hypotheses. Sharing all of that creates more conflict, not more understanding.

Define a Read-Only Repository Knowledge Layer

A trustworthy shared coding agent memory layer is read-only to task agents and binds every structural claim to source evidence and a repository revision.

Use the repository itself as primary evidence, then generate a bounded, queryable view of relationships that agents repeatedly need. The layer can be structured files, a search index, or a code knowledge graph. Its storage choice matters less than provenance and access control.

Include facts such as:

• module and package boundaries;
• symbol definitions, direct references, registrations, and event flows;
• generated-code sources and build commands;
• schema, API, queue, and database relationships;
• test-to-feature and test-to-symbol mappings;
• architecture records, owners, and deprecations;
• indexed commit, excluded paths, failed parses, and extractor version.

Every material result should cite a source path and revision. An agent should be able to ask, “Why does the layer say service A consumes event B?” and receive the source location or derivation that supports the edge.

The official Multica GitHub repository and product docs do not expose a native graph ingestion or query API. Treat this integration as your architecture. Expose a read-only query command through the mechanism supported by the underlying provider. Multica’s MCP field is provider-specific; some providers will need a CLI, Skill wrapper, or other adapter instead.

Do not let task agents mutate the index they query. Index writes should run in a separate identity, produce an audit record, and require review for schema or manual-fact changes.

Separate Planning, Coding, Testing, and Review Needs

The current Multica GitHub repository documents the coordination implementation; it does not turn one role's task summary into shared repository truth for every other role.

Different roles need different slices of the same repository truth.

Role	Shared input	State that stays local	Required output
Planner	module map, decisions, likely dependencies	alternative plans and uncertainty	bounded plan with cited evidence
Coder	approved plan, definitions, callers, write scope	worktree changes and experiments	patch plus changed-contract notes
Tester	behavior contracts, test index, risk paths	test fixtures and temporary diagnostics	results, gaps, and reproduction steps
Reviewer	base revision, patch, evidence trail, test results	independent review notes until submitted	supported findings and merge decision

The shared layer should answer repository questions, not dictate every role’s procedure. Put role workflow in Skills and workspace context. Put structural claims in the source-attributed layer. Put work-item decisions in the issue. This separation makes a disagreement traceable: agents can disagree about a plan while still consulting the same definition and callers.

Project Resources help each role reach the same repository family, but the checkout revision can still differ. Record the resource identifier and actual commit in every task result. If two agents analyze different commits, their summaries should not be merged as if they describe one state.

Reduce Duplicate Scanning

Do not solve repeated search by dumping a giant repository summary into every prompt. Build a small retrieval contract.

1. Pin the task’s repository and commit.
2. Classify the question: locate, call path, change impact, test coverage, or design constraint.
3. Retrieve a bounded set of paths and relationships with provenance.
4. Ask the agent to verify high-impact edges in its checkout.
5. Persist only reusable, reviewed facts; keep task conclusions in the issue.
6. Measure file reads, retrieval misses, review corrections, and evidence age.

This pattern reduces duplicate scanning while preserving freshness checks. A graph hit is a shortlist, not a verdict. A graph miss is also useful when it records coverage gaps instead of silently implying that no dependency exists.

Track whether the packet actually helps: time to the first evidence-backed plan, number of repeated file reads, missed relationships found in review, and the age of cited facts. A smaller packet is successful only when correctness holds.

Compare those measures by task class rather than averaging unrelated work. A navigation task and a cross-service migration need different evidence depth. Record why an agent widened retrieval so a reviewer can distinguish justified exploration from an ineffective query.

Workspace Skills can standardize this query-and-verify procedure for multiple agents. The Skill should request an indexed revision, cite sources, report unknowns, and stop when the task revision differs. The Skill itself remains workflow memory; it does not become the repository model.

Conflict, Permission, and Drift Risks

Shared context increases the blast radius of a bad fact. A wrong module owner can misroute many tasks. A stale security note can normalize unsafe behavior. A broad graph query can reveal relationships from a repository the assigned agent should not access.

Control the risks explicitly:

• Conflict: define precedence by source type. Current source and approved policy outrank agent summaries. A human resolves architecture or security conflicts.
• Permission: filter facts before retrieval. Multica agent visibility and assignability do not make the returned data read-only or authorized.
• Drift: attach version, source, and expiry metadata. Reject packets built for the wrong commit.
• Provenance: keep file and symbol citations beside relationship claims.
• Supply chain: inspect third-party Skills before attachment; Multica does not sign or sandbox their content.

The underlying coding tool also retains its own execution permissions. Multica’s current Claude daemon handler auto-approves tool-control requests during autonomous execution, so daemon workdirs, credentials, and operating-system identity must carry the real containment. A task board is not a sandbox.

FAQ

What context should agents not share?

Do not share secrets, unrestricted credentials, private reasoning, unreviewed hypotheses, temporary patches, or one task’s write authority. Share the minimum reviewed facts and policies needed for the next role.

Who can update shared memory?

Assign owners per surface. Team leads may own workspace context, Skill maintainers own procedures, issue owners own task decisions, and an index service plus reviewers own repository facts. Ordinary coding agents should query the fact layer read-only.

How should conflicting summaries be resolved?

Compare each summary’s repository revision, cited source, and observation time. Verify the disputed relationship in current source. Escalate architecture, security, ownership, and data-contract conflicts to a human rather than choosing the longer or newer-looking summary.

What is a multi-agent system?

In this workflow, it is a set of agents with distinct roles coordinated around shared tasks and controlled information exchange. Multica supplies coordination surfaces; your repository layer supplies auditable structural evidence.

Conclusion

Use Multica’s native surfaces for shared guidance, reusable procedures, resource pointers, and issue state. Add a read-only, revision-bound layer for current repository relationships. Keep task reasoning and write state isolated. The result is not one magical memory: it is a set of owned context layers whose conflicts can be located, verified, and corrected.