LitLLM: A Toolkit for Scientific Literature Review

Background and Motivation

• Literature review is essential for understanding research context, limitations, and building on existing work
• Manual review is tedious and time-consuming
• Automatic literature review generation using LLMs is an attractive solution

---

Problems in Existing LLM-based Methods

• Many current LLM-based approaches suffer from hallucination and factual errors
• Lack awareness of recent research not included in their training data
• These issues limit the reliability and usefulness of generated reviews

---

Proposed Solution

• Introduce a toolkit based on Retrieval-Augmented Generation (RAG) principles
• Uses specialized prompting and instruction techniques to guide LLMs
• Goal: reduce hallucination and improve relevance to up-to-date research

---

System Workflow

1. Generate keywords from a user-provided abstract using a general-purpose LLM
2. Perform a web search to retrieve related papers
3. Allow users to refine search by adding papers or keywords manually
4. Re-rank retrieved papers based on similarity to the abstract
5. Generate the related work section using the abstract and re-ranked papers

---

Impact and Results

• Significant reduction in time and effort required for literature review
• Enables a more efficient and customizable alternative to manual review

---

Access to Toolkit

• Project page with demo and toolkit available at: https://litllm.github.io

LitLLMs: LLMs for Literature Review

• https://github.com/LitLLM/litllms-for-literature-review-tmlr

• https://litllm.onrender.com/