TLDR: As an open-source enthusiast > Iâm amazed that weâre capable of running something as powerful as Llama 3 locally & that WE control the data. Let me show you how.
Step 1: Install Docker Desktop on your machine (if you havenât already)
https://www.docker.com/products/docker-desktop/
Why use Docker? Instead of wasting time writing 3 separate articles that handle the nuances of each different OS (MacOS, Linux, or Windows) > we make use of one source of truth (universal environment) as defined in a Dockerfile to get things situated and running.
^ This is a simplified explanation. I may revisit this when I have a better answer.
Step 2: In your terminal, paste and enter this command to get Ollama installed on your system:
docker pull ollama/ollama
You should see something similar to this after
Step 3: Now that itâs installed, letâs run it by entering this:
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama Boom! You know can communicate with one of the smartest LLMs on the planet OFFLINE on your computer. Welcome to the club. Perhaps youâll ask a more informative input than just âhey!â
Bonus Points!
If youâve made it this far, but are feeling dissatisfied, you probably miss how intuitive the experience is when chatting with ChatGPT in itâs website form.
Letâs make something similar.
Shoutout to the contributors of WebUI. Theyâve helped make a beautiful interface so that those of us who arenât terminal lovers can live in peace.
In the same terminal window or the environment of your choice (perhaps VSCode?), enter this:
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
After a minute, youâll be welcomed to this:
Select the Ollama model above and chat away!