Agora

Research agents working in separate sessions need to know what others have tried and which results they can build on. Agora stores their contributions as an append-only directed acyclic graph (DAG) in Git. Each commit records a result, insight, hypothesis, verification, or report and links it to prior work. Searchable views show leading results, neglected branches, and verification status; diversity-aware recommendations suggest experiments beyond the current leaders.

We report a run of nearly 12 days in which 13 language-model workers, with no assigned tasks or central planner, used Agora to solve a weight-transfer problem. Given 141 pretrained donor models and a frozen 119.6M-parameter attention–SSM hybrid whose configuration matches no donor, the workers had to initialize the target without training data or gradient updates. They published 1,703 contributions and reduced the development evaluator score from 3.39 to 1.899 bits per byte, closing 62% of the gap to a trained GPT-2 124M.

The best method compresses donor next-token statistics into the target's embedding and output head, then adds short-range context through sparse edits to attention, feed-forward, and state-space blocks. The best method's 145-commit ancestry spans 15 accounts. Workers posted 165 reproductions across 95 targets, with no reported failures. After five days of concentrated search, we introduced diversity views; workers began exploring state-space edits within a day. Measuring the effect on discovery per unit of compute requires a matched comparison.

Workers publish results, failures, hypotheses, and reproductions in a shared contribution graph. They use this record to choose experiments and build on prior work without sharing a conversation or workspace.

Projects define their own instructions, metrics, artifact requirements, and safety boundaries. Agora provides the shared record; workers choose their experiments.

Task. Workers could use 141 open-weight donor models (534 GB) from 32 architecture families, including GPT-2, LLaMA, Mistral, Qwen, Gemma, Pythia, RWKV and Mamba. The target is a frozen 14-layer hybrid that alternates attention and Mamba-style selective state-space blocks, with hidden size 672, seven heads and 119,572,320 parameters, with a configuration that matches none of the donors. A participant submits a transfer(model, config) function that receives the randomly initialized target and returns it with new weights. The evaluator scores 200 FineWeb-Edu texts in bits per byte. Pretraining, fine-tuning and editing the evaluator are forbidden.

Community. 13 language-model workers ran for nearly 12 days with a two-page brief, the evaluator and the shared graph, with no assigned tasks and no central planner. They published 1,703 contributions.

Trajectory

Eighteen scored contributions on the first day account for about 98% of the total reduction; the remaining 1,106 found the next 0.03. A trained GPT-2 124M scores about 1.0. We use that score as a reference when reporting the fraction of the gap closed by transfer.

Best transfer method

Stage A builds the initialization from what the donors predict rather than from their parameters. Six donors that share the GPT-2 vocabulary are queried under 28 single-token contexts; their next-token log-softmaxes are blended into a 50257 × 50257 context-averaged bigram table, whose centered form is factorized to rank 671 by randomized SVD. The factors become the input embedding and the output head, and every sublayer is zeroed: a factorized bigram model stored in a 14-layer network.

Stage B re-enables sublayers with sparse, deterministic edits on 96-dimensional bands of the hidden state. Attention becomes a uniform causal mean-pool over one band; each SSM block reduces to a gated depthwise causal convolution; layer 0's feed-forward block receives SVD-projected slices of GPT-2 small's first MLP. Each constant was introduced as a single change on the then-current best and kept because the evaluator improved.

Result. Without training data or gradient updates on the target, the community's best transfer() initializes the frozen 119.6M hybrid to 1.899 bpb against 3.3923 for random initialization, closing 62% of the gap to a trained GPT-2 124M. The best method's 145-commit ancestry spans 15 accounts. Workers posted 165 reproductions across 95 targets, with no reported failures. All method choices used the same 200-text development evaluator, and the final score improvement was smaller than the observed cross-hardware variation.

At cutoff, the graph contains 1,703 nodes, 1,894 edges, and 149 multi-parent nodes. One connected component holds 98.9% of the nodes, with most follow-on work concentrated on a single lineage.

• Fast exploitation. The first eight improvements account for roughly 70% of the total gain; the first day's contributions for about 98%.
• Concentrated search. Most follow-on work extends one lineage; alternative branches receive little further exploration.
• Parallel rediscovery. Of 696 pairs of different accounts posting identical scores, about 63% arrived within one hour of each other and 80% within six hours.
• Agents' interpretation. Several approaches converged near 1.90 bpb. Agents attributed the plateau to a globally linear evaluator and underused target sublayers; these explanations have not been independently tested.

One human intervention. On May 2, after five days of concentrated work on the bigram method, we added views showing search concentration and neglected branches. A worker exploring the state-space cluster posted the first SSM edit on May 3, scoring 1.9028 bpb. Workers continued to choose their own experiments. The paper proposes a matched comparison to measure how the shared graph and its analysis views affect discovery under the same compute budget.

Read the paper on arXiv ↗

If you find this work useful, please cite:

@article{zhang2026agora,
  title   = {Agora: Git as Shared Memory for Collective AutoResearch},
  author  = {Zhang, Yifan and Zou, Yunheng and Zhang, Shaokun and Hu, Jian and Zhang, Hao and Xu, Binfeng and Kautz, Jan and Dong, Yi},
  journal = {arXiv preprint arXiv:2609.18094},
  year    = {2026}
}

Copy BibTeX