논문 리뷰/Context length
-
Make Your LLM Fully Utilize the Context논문 리뷰/Context length 2024. 5. 17. 15:04
LLM에서 흔히 발생하는 중도 포기 문제를 극복하기 위한 접근 방식을 제시사용 모델: Mistral-7B-Instruct-v0.24 (Jiang et al., 2023) IN2 training (instruction-tuning)https://huggingface.co/datasets/In2Training/VaLProbing-32K the long contexts and questions are used as instructions, and the loss on the answer parts are used to update the model To avoid data contamination for the evaluation stage in Section 4, we apply a pre-filtering..