BitsAI-CR: ByteDance's Automated Code Review Tool

What is BitsAI-CR?

BitsAI-CR is an automated code review tool developed by ByteDance, leveraging a large language model (LLM) to enhance the efficiency and accuracy of code reviews. It employs a two-stage process: RuleChecker identifies potential issues based on 219 predefined rules, and ReviewFilter validates these issues to ensure accuracy. The tool introduces an "Outdated Rate" metric to measure developers' acceptance of review suggestions, enabling continuous optimization of review rules through a data flywheel mechanism.

Key Features of BitsAI-CR

• High-Precision Code Review Comment Generation: BitsAI-CR generates high-precision code review comments through a two-stage process.
• Code Review Rule Classification: A comprehensive classification system of review rules based on code style guidelines and practical review experience provides foundational support for the system.
• Integration and Automation: BitsAI-CR can be easily integrated into existing code review processes, automatically identifying potential issues, confirming review categories, locating problematic code lines, and providing relevant comments. After developers modify the code based on review suggestions, BitsAI-CR re-evaluates the code and marks the original comments as "Outdated," while providing a "LGTM" (Looks Good To Me) approval.

Technical Principles of BitsAI-CR

• Two-Stage Review Pipeline:
• RuleChecker: Detects potential issues in code based on a comprehensive classification of 219 review rules using a fine-tuned LLM.
• ReviewFilter: Validates the detected issues using another fine-tuned LLM, further improving review accuracy.
• Data Flywheel Mechanism: Optimizes large-scale industrial scenarios through real feedback.
• Outdated Rate Metric: Automatically measures the percentage of code lines modified after being flagged by BitsAI-CR, assessing whether developers accept and implement review suggestions.
• Dynamic Adjustment of Review Rules: Dynamically adjusts review rules based on the Outdated Rate and accuracy measurements, removing rules that generate low-value comments.
• Model Training and Optimization: Based on ByteDance's LLM (Doubao-Pro-32K-0828), fine-tuned using Low-Rank Adaptation (LoRA) technology to ensure a balance between security and performance. The model is trained on historical review data to optimize review accuracy.

Application Scenarios of BitsAI-CR

• Automated Code Review: BitsAI-CR automatically identifies potential issues in code through a two-stage review pipeline (RuleChecker and ReviewFilter), generating high-precision review comments and locating problematic code lines.
• Continuous Optimization and Feedback Mechanism: BitsAI-CR introduces an "Outdated Rate" metric, dynamically adjusting review rules by automatically measuring developers' acceptance and modification of review suggestions.
• Multi-Language Support and Rule Classification: BitsAI-CR supports multiple programming languages (e.g., Go) based on a comprehensive classification system of review rules, with detailed review rules tailored to the characteristics and common issues of different languages.
• Integration and Automation Process: BitsAI-CR seamlessly integrates into existing code review processes, automatically handling the entire process of code submission, review, modification, and approval.