QLoRA: Fine-Tuning Your LLMs With a Single GPUby@aibites

QLoRA: Fine-Tuning Your LLMs With a Single GPU

tldt arrow
Read on Terminal Reader
Read this story w/o Javascript

Too Long; Didn't Read

To fine-tune a LLAMA 65 billion parameter model, we need 780 GB of GPU memory. That is about 16 A40 GPUs. The solution lies with QLoRA where Q stands for Quantisation.

Company Mentioned

Mention Thumbnail
featured image - QLoRA: Fine-Tuning Your LLMs With a Single GPU
Shrinivasan Sankar HackerNoon profile picture

@aibites

Shrinivasan Sankar

I am an AI Reseach Engineer. I was formerly a researcher @Oxford VGG before founding the AI Bites YouTube channel.


Receive Stories from @aibites


Credibility

react to story with heart
Shrinivasan Sankar HackerNoon profile picture
by Shrinivasan Sankar @aibites.I am an AI Reseach Engineer. I was formerly a researcher @Oxford VGG before founding the AI Bites YouTube channel.
Read my stories

RELATED STORIES

L O A D I N G
. . . comments & more!