ChatCite: LLM Agent with Human Workflow Guidance for Comparative Literature Summary

Background and Motivation

• Literature review is a key stage in the research process
• Enables understanding of the research problem and comparison of existing work
• Literature summarization is time-consuming and challenging

---

Limitations of Existing Approaches

• Previous LLM-based studies focused on the full pipeline: retrieval, screening, and summarization
• Summarization often relied on simple Chain-of-Thought techniques
• Lacked sufficient ability for comparative summarization

---

Proposed Method: ChatCite

• ChatCite is an LLM agent designed to mimic human literature summarization workflow
• Extracts key elements from relevant papers
• Generates summaries using a Reflective Incremental Mechanism
• Focuses solely on the summarization step as an independent task

---

Evaluation and Performance

• Introduces G-Score, an automatic LLM-based evaluation metric aligned with human assessment criteria
• ChatCite outperformed baseline models across multiple evaluation dimensions
• Generated summaries can be directly used to draft literature reviews

---

Source Code

• Source code is publicly available: https://github.com/miaisamelia/ChatCite
• Datasets: https://github.com/NudtRwG/NudtRwG-Dataset

---

Question

• ref가 많으면 context가 점점 길어지는 거 아닌지? 그럼 어떻게 처리할건지
• Reflective Evaluator (반성적 평가기) 평가 기준은?
•