This system lets Foundation Models, like Large Language Models (LLMs), do research on their own. We developed it with the Foerster Lab at the University of Oxford and Jeff Clune and Cong Lu at the University of British Columbia. The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery.
- Run a fully AI-driven system for automated scientific discovery, applied to machine learning research.
- The AI Scientist handles the whole research process: coming up with new ideas, writing code, running experiments, summarizing and visualizing results, and even creating a complete scientific paper
- The AI Scientist explores different areas of machine learning research and makes new discoveries in popular topics like diffusion models, transformers, and grokking.
Overview of The AI Scientist
The AI Scientist is a fully automated system for generating research papers from start to finish, thanks to recent improvements in foundation models. Starting with a basic codebase, like an open-source project from GitHub, it can handle idea generation, literature search, planning and running experiments, creating figures, writing and reviewing the paper, and more. It can also keep improving by using feedback and previous ideas, similar to how the human scientific community works.
The AI Scientist has 4 main processes :
| Idea Generation | Experimental Iteration | Paper Write-up | Automated Paper Reviewing |
| Given a starting template, The AI Scientist first “brainstorms” a diverse set of novel research directions. It uses a starting code “template” of an existing topic to explore further. The template includes a LaTeX folder with style files and section headers for writing the paper. The AI Scientist also searches Semantic Scholar to ensure its idea is novel. | Given an idea and a template, the second phase of The AI Scientist first executes the proposed experiments and then obtains and produces plots to visualize its results. It makes a note describing what each plot contains, enabling the saved figures and experimental notes to provide all the information required to write up the paper. | Finally, The AI Scientist produces a concise and informative write-up of its progress in the style of a standard machine learning conference proceeding in LaTeX. It uses Semantic Scholar to autonomously find relevant papers to cite. | A key aspect of this work is the development of an automated LLM-powered reviewer, capable of evaluating generated papers with near-human accuracy. The generated reviews can be used to either improve the project or as feedback to future generations for open-ended ideation. This enables a continuous feedback loop, allowing The AI Scientist to iteratively improve its research output. |
Website Link : https://sakana.ai/blog/
