---
title: "PaperOrchestra: AI agents turn research notes into a manuscript"
author: Martin Ebner
date: 2026-11-15
url: https://postgenerative.at/indications/2026-11-15/paperorchestra-ai-agents-turn-research-notes-into-a
lang: en
type: Indication
summary: PaperOrchestra automates the path from research notes to a complete manuscript and clearly outperforms other AI writing systems. Its impressive evaluation results raise questions about the established publication cycle.
tags:
  - Research
  - Science
sources:
  - title: "Google Research: PaperOrchestra: A Multi-Agent Framework for Automated AI Research Paper Writing (2026)"
    url: https://research.google/pubs/paperorchestra-a-multi-agent-framework-for-automated-ai-research-paper-writing/
  - title: "Song, Song, Pfister & Yoon (2026): PaperOrchestra – COLM-Konferenzpublikation (PDF)"
    url: https://proceedings.colm.cc/paper_files/paper/2026/file/6546d4ab1e224ea9625c7c42ad25d6c8-Paper-Conference.pdf
  - title: "Song et al. (2026): PaperOrchestra – frei zugänglicher Volltext der arXiv-Fassung vom 6. April 2026, CC BY 4.0"
    url: https://arxiv.org/html/2604.05018v1
license: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
alternate:
  de: https://postgenerativ.at/indizien/2026-11-15/paperorchestra-ki-agenten-machen-aus-forschungsnotizen-ein
---

# PaperOrchestra: AI agents turn research notes into a manuscript

Google researchers Yiwen Song, Yale Song, Tomas Pfister and Jinsung Yoon introduce **PaperOrchestra**, a system that transforms existing idea summaries and experimental logs into complete scientific manuscripts. The work is listed as a COLM 2026 conference publication; an earlier arXiv version is also freely available. [1][2][3]

### An orchestrated writing process

Specialized AI agents handle five stages: outlining, generating diagrams and data visualizations, researching and synthesizing literature, writing sections, and refining the manuscript using simulated peer review. Literature and visualization work can run in parallel. The outputs are LaTeX source files and a typeset PDF. Retrieved publications are checked for existence through Semantic Scholar. This check does not guarantee that every cited claim is substantively supported. [3]

The distinctive step is treating writing as a standalone workflow: Materials can come from human research rather than necessarily from an automated experimental pipeline. **Experiments and research results must already be supplied as inputs.** PaperOrchestra therefore demonstrates automated manuscript production, rather than the complete independent generation and validation of new research. [3]

### The impressive overall evaluation result

The benchmark uses **200 published papers**, 100 each from CVPR and ICLR 2025. Idea summaries and experimental logs were reverse-engineered from these completed papers. Alongside automated assessments, **11 AI researchers** evaluated a sample of **40 papers**. [3]

In human pairwise comparisons of overall quality, PaperOrchestra won **57 percent** against the single-agent system, which won **19 percent**, with **24 percent** ties. Against AI Scientist-v2, PaperOrchestra led **50 to 36 percent**, with **14 percent** ties. These are win-rate margins of **38 and 14 percentage points**, respectively. They represent neither a universal 14–38 percent increase in scientific quality nor conference acceptance rates. [2][3]

This is a remarkable result for an automated writing workflow. Yet comparison with human originals remains clear: For overall quality, human evaluators preferred the original in **81 percent** of comparisons and PaperOrchestra in **5 percent**, with **14 percent** ties. These results do not establish general parity with human-written papers. [2]

### Limits of the evidence

The study concerns AI and machine-learning papers and artificially reconstructed starting materials. Despite mitigation measures, model familiarity with the published originals cannot be fully excluded. The central baseline comparisons also supplied original figures; autonomous visualization was assessed separately. Generated figures can contain errors, and automated reviewers show limited agreement with human judgements, particularly for literature reviews. The authors explicitly retain human researchers’ responsibility for factual accuracy, originality and validity. [3]

### A question for the scientific publication cycle

Our closing question concerns the purpose of the process: **Does the established scientific publication cycle still make sense when AI can turn existing results into manuscripts, simulate their review and revise them—and how should that cycle change to reward genuinely new, verifiable knowledge rather than primarily persuasive presentation?**
