Qingru Zhang

AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning

Qingru Zhang, Minshuo Chen, Alexander Bukharin · arXiv (Cornell University) · 2023 · 31 citations · Full text

Llm Fine-tuning, Engineering, Machine Learning +27

AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methods

Zhiming Zhou, Qingru Zhang, Guansong Lu et al. · arXiv (Cornell University) · 2018 · 29 citations · Full text

Artificial Intelligence, Engineering, Machine Learning +11

MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation

Simiao Zuo, Qingru Zhang, Liang Chen et al. · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · 2022 · 25 citations · Full text

Artificial Intelligence, Llm Fine-tuning, Engineering +19

AdaShift: Decorrelation and Convergence of Adaptive Learning Rate\n Methods

Zhiming Zhou, Qingru Zhang, Guansong Lu et al. · arXiv (Cornell University) · 2018 · 18 citations · Full text

Less is More: Task-aware Layer-wise Distillation for Language Model Compression

Liang Chen, Simiao Zuo, Qingru Zhang et al. · arXiv (Cornell University) · 2022 · 14 citations · Full text

Artificial Intelligence, Llm Fine-tuning, Engineering +17