What is Agent Laboratory?

Agent Laboratory is an autonomous research framework developed by AMD and Johns Hopkins University, based on large language models (LLMs). It accelerates scientific discovery, reduces costs, and improves research quality. The framework takes research ideas provided by humans and processes them through three stages: literature review, experimentation, and report writing, producing comprehensive research outputs including code repositories and research reports. Agent Laboratory allows users to provide feedback and guidance at each stage, enhancing the overall quality of the research.

Key Features of Agent Laboratory

• Literature Review: Automatically collects and organizes literature related to the research topic, providing references for subsequent research stages.
• Experiment Design and Execution: Based on the literature review and research objectives, it formulates detailed experimental plans and automatically executes experiments.
• Code Generation: Automatically generates machine learning code for experiments, supporting multiple LLM backends such as gpt-4o, o1-mini, and o1-preview.
• Result Interpretation: Analyzes and interprets experimental results, providing a foundation for writing research reports.
• Report Writing: Generates structured research reports, including sections such as abstract, introduction, background, related work, methods, experimental setup, results, and discussion.
• User Interaction: Supports autonomous and co-pilot modes, allowing users to provide feedback and guidance at each stage to improve research quality.

Technical Principles of Agent Laboratory

• Based on Large Language Models (LLMs): Uses pre-trained LLMs such as gpt-4o, o1-mini, and o1-preview to generate natural language text, including literature reviews, experimental plans, code, and research reports.
• Autonomous Agent System: Utilizes multiple specialized agents (e.g., PhD agent, Postdoc agent, ML Engineer agent, and Professor agent) to collaborate on tasks such as literature retrieval, experiment design, code writing, result interpretation, and report writing.
• Modular Tools: The mle-solver module automatically generates and optimizes machine learning code, while the paper-solver module generates and optimizes research reports, ensuring the quality of experiments and reports.
• Iterative Improvement Mechanism: Agents engage in self-reflection at each stage, generating improvement measures based on experimental results or error signals, and iteratively optimizing the quality of code and reports.
• User Interaction and Feedback: Supports autonomous and co-pilot modes, allowing users to provide feedback and guidance at each stage, with agents adjusting and optimizing based on feedback to improve overall research quality.

Project Links for Agent Laboratory

• Project Website: https://agentlaboratory.github.io/
• GitHub Repository: https://github.com/SamuelSchmidgall/AgentLaboratory
• arXiv Technical Paper: https://arxiv.org/pdf/2501.04227

Application Scenarios of Agent Laboratory

• Scientific Literature Review: Quickly collects and organizes relevant literature, generating literature review reports to provide background information for research.
• Experiment Design and Execution: Formulates detailed experimental plans, automatically generates experimental code, executes experiments, and monitors results, improving research efficiency.
• Code Generation and Optimization: Generates high-quality machine learning code, supporting multiple programming languages and frameworks, and optimizes code performance through an iterative improvement mechanism.
• Result Interpretation and Report Writing: Analyzes experimental results and generates structured research reports, ensuring clarity and logical consistency.
• Multi-Domain Research Support: Applicable to various fields such as machine learning, biomedicine, materials science, and social sciences, accelerating the research process.