Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

Adds Interlm2 support to AutoAWQ

A simple program of the MCT Self-Refine (MCTSr) algorithm, welcome to adapt or modify it

Implementation of the Quiet-STAR paper (https://arxiv.org/pdf/2403.09629.pdf)

Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance. Accepted to ACL 2024.

This is our own implementation of 'Layer Selective Rank Reduction'

This project aims to implements quiet_star algoithm

Code for Quiet-STaR

根据Qwen2（Qwen1.5）模型生成qwen2 MoE模型的工具

Tools for merging pretrained large language models.

Go ahead and axolotl questions

🤗 Transformers: Add GemMoE support