MineMate: Multi-Agent Emergent Capability with Humans
Abstract
MineMate studies what happens when several people and many LLM-driven agents team up in a shared, open-ended Minecraft world. In a main experiment, four participants shared one world and each directed their own team of agents while I swept the Human:Agent ratio from 1:1 to 1:16 on two subtasks, gathering logs and crafting a stone pickaxe. Individual agents were capable; the bottleneck was coordination. Per-team yield rose with the agent count only for work that splits into independent, parallel effort (logs), and saturated almost immediately for the collaborative recipe (stone pickaxe), whose bottleneck is inter-agent coordination. Given a free choice, people did not maximize autonomy: they settled on small Decoupled agent groups with a Centralized (one-coordinator) interaction, observed everything through text (Slack), and spent their time directing agents and clearing bottlenecks rather than playing. The takeaway: embodied LLM agents are strong as singletons but weak at collaboration, and the human's role is becoming that of a coordinator of emergent, only-partially-legible teammates.
đź“„ Full write-up: MineMate Report (PDF).
Overview
- MineMate is a sub-project of Project Weasel: a study of what phenomena emerge when multiple humans and LLM agents cooperate, building, farming, and gathering together in a shared, open-ended Minecraft world.
- I study whether useful capability can emerge from many language-model agents acting in a shared world, and how a coordination framework plus a human-in-the-loop make that emergence reliable.
Why Minecraft?
Minecraft is an open-ended, partially-observable sandbox: there is no fixed objective, each agent only sees its local surroundings, and goals must be discovered rather than scripted. That makes it a strong testbed for multi-agent emergence: capability has to arise from the agents' interaction with the world and with each other, not from hand-written rules. The clip above is the raw world the agents wake up into.
Partial observation matters
An embodied agent only ever sees its own first-person view. From that egocentric vantage (left) it often cannot tell how much it has actually accomplished, or what is right beside it. The same moment in a third-person 3D view (right) tells a different story: the agent has in fact done a great deal of work. Because the agent is constrained to this partial, 3D first-person observation of the world, it cannot always judge its own situation correctly: a core difficulty of acting under partial observability. Click either video to enlarge.
What the agents do well
Given a concrete sub-task, each agent performs reliably. The eight agents naturally divide labor (logging, mining, and crafting tools) and run those jobs in parallel without colliding.
Agents coordinate to share resources
Capability is not just per-agent. It is social. Agents communicate over shared channels to request and hand off items. Here an agent needs a crafting table; it asks Nova, and the two agree on a spot where Nova places the table and waits for the agent to come and pick it up. The exchange emerges from their dialogue, not from a hard-coded trade.
Humans and agents team up
With a human in the loop, the agents take direction and contribute to larger projects. This building session shows that human–agent teaming is workable: the human sets intent, and the agents help realize it together.
Why the framework matters
My contribution is the coordination framework. Remove it and the same model gets stuck: it talks instead of acting, loops on a sub-goal, or cannot recover from a missing resource. The two farming attempts below fail exactly where the framework would have kept the team unblocked, motivating why the structure is needed.
Design space
Beyond the demos, MineMate is studied as an HCI problem: how one human coordinates with many agents at once. Single-agent capability is already strong. The hard part is teaming. Click the figure to enlarge.
Coordinating through Slack
I added a Slack interface so that users can coordinate with one another easily (and command the agents) from outside the text-flooded in-game view. Each agent gets its own avatar and sub-channel, so a team of people can divide work and follow what every agent is doing in one place. Press â–¶ on the clip to watch a short walkthrough.
Results: capability scales, but with diminishing returns
In a small main experiment, four participants shared a single world for a ~5-hour session. After choosing their preferred configuration during free play, the measured runs were unified for everyone to the most-preferred setup, Decoupled agent groups with a Centralized (one-coordinator) interaction, with agents driven by Gemma and Kimi 2.5 in parallel. Each person directed only their own team and never took over someone else's agents, though the agents themselves could message and request work across teams (one team's agent could ask another person's agent for help). I then swept each team's Human:Agent ratio from 1:1 to 1:16, with a fresh world per run and each 25 min run split into 15 min gathering logs then 10 min crafting a stone pickaxe. Notably, people mostly did not play directly: they told agents what to do and spent their time managing bottlenecks, unblocking stuck agents and re-allocating work, so the human's role looked more like a coordinator than a player. More agents help, but with diminishing marginal returns, and how fast the benefit runs out depends on the task. Click the figure to enlarge.
Scope: this is a small, exploratory study, four participants in one shared session, with uneven Minecraft skill and a low-to-high ratio sweep that confounds a within-session learning effect with team size. The curves are therefore read for their shape (where they saturate), not for exact magnitude, and the configuration preferences are revealed choices rather than controlled comparisons.
Conclusion
- Embodied LLM agents are strong as singletons but weak at collaboration: each agent follows free-form instructions well, yet the work breaks down at the seams between agents.
- Adding agents follows a law of diminishing marginal utility, and the benefit disappears as soon as a task needs agents to cooperate rather than run in parallel.
- Given a free choice, users settle on a middle band of agency: small decoupled groups that share only what they need and never carry two roles, watched through text (Slack) rather than the in-world view.
- As agents move into the physical world, the central HCI problem is keeping a person in control of a team of emergent, only-partially-legible teammates, which is what MineMate is built to study.
Acknowledgement
Thank you to the UNIST Minecraft Community, my high-school friends, and my UNIST friends. Special thanks to my Mobile Computing teammates (the greatest I've ever met), with whom I also worked on Mean-Collapse and W0: A dVLM-Based GUI Agent with Block-Diffusion Mode Steering, and who helped with the QA and the pilot studies for this experiment. Thanks also to Google's TPU Research Cloud for supporting the experiments.