TeamGym

TeamGym

A gym for team decision-making by AI agents with distributed information

Anonymous authors · Under review

AI agents increasingly work in teams, but they are still trained and tested alone. TeamGym turns real legal, finance, medical and software tasks into team decisions in which the information a decision needs is split across members and buried in each member's own material. Splitting that information cuts success by 22.9 points; burying it cuts another 19.9.

How TeamGym works

Real decisions often depend on facts that are spread across people and buried in each person's documents. TeamGym builds such problems from real sources and marks the facts a decision cannot do without, the pivotal information units (PIUs). How each problem is built ↓

Figure 1 · How TeamGym works

A team decision episode, and the policies TeamGym trains

Turn on sound to hear the narration (voice by ElevenLabs).

Findings

Distributing pivotal information costs 22.9 points; retrieving it costs 19.9 more

Same problems, same PIUs, three conditions: one agent holds every PIU; the PIUs are split across a team; each PIU is buried in its holder's source material. Splitting the PIUs costs 22.9 points (the communication gap); burying them costs 19.9 more (the retrieval gap).

Figure 2 · Three evaluation conditions

The same PIUs, distributed, then embedded in source material

The drop appears for every model in every domain. Each row shows one model working alone with every PIU, as a team handed the PIUs, and as a team that must retrieve them.

Figure 3 · Every model

Every model drops when it has to work as a team

Single-Agent All-PIUMulti-Agent PIU-OnlyMulti-Agent Full-Context
Episode success (%). Each domain pools its problem instances (paper Fig. 6); “All domains” averages over each model's 12 source corpora (Appendix Table 30). Hover a dot for its value and standard error.

Which communication structure works best depends on who holds what

No structure is best everywhere. Across 11 problem families and 8 models, the interaction between PIU distribution and communication structure is significant in both multi-agent conditions (p < .001).

Figure 4 · Distribution and structure

Who holds which PIU, and how does the team talk?

PIU distribution
Communication structure
Change in each structure's advantage over coordinator-selected speaker (pp ± SE)

Colored cells: the 95% CI excludes zero.

Coordination emerges from outcome rewards alone

We train policies initialized from Qwen2.5-7B-Instruct with GRPO, rewarding only a GPT-5.5-judged score of the final decision. They learn to query every participant with short queries and to let more participants speak, though neither behavior is rewarded.

Coordinator-selected speaker. CUAD-derived validation, Full-Context. Two-stage: 12.14% → 16.84% → 23.33%. Joint two-policy: 11.97% → 22.38%.
Current-speaker selection. FinDVer validation, PIU-Only, every 10 updates. Greedy evaluation after training: 21.85% → 41.08% (paper Table 6).
CUAD validation (n = 36)BaseTwo-stageJoint
Every participant queried (%)80.666.7100.0
Mean words per query14.048.012.5
Question copied in first query (%)0.088.60.0
Later queries reusing reply terms (%)3.242.913.5
Communication diagnosticValidation baseTrainedFinLongDocQA test baseTrained
Every participant speaks (%)61.786.762.582.5
Mean participant coverage (%)65.289.464.490.0
Mean unique speakers2.283.252.583.60
Self-handoff rate (%)14.210.617.616.2

Examples

Four episodes that illustrate the findings. Each plays on its own; use the arrows to step through it yourself. The dialogue is schematic; the numbers come from the paper.

A narrow question leaves PIUs unsaid

Participants without a PIU can talk the team out of the answer

Correct synthesis helps more than error uptake hurts

Reaching every participant is not hearing every PIU

How each problem is built

Each problem starts from a real source and its reference decision. Candidate units are located using the source's annotations and the reasoning behind that decision, then checked: together they must be sufficient, and dropping any one must break the decision. The units that pass are the PIUs, which are then allocated across participants.

Figure 5 · Building a problem instance

Which facts can the decision not do without?

Problem instances are built from real source corpora in four domains.

Try it

Lead a team of AI participants on a short task, or watch one work. Connect with a TeamGym access code or your own API key, then choose who holds which PIU and how the team talks.