Ditch ChatGPT: Run A Fast LLM On Your Computer Insteadby@timeo
527 reads
527 reads

Ditch ChatGPT: Run A Fast LLM On Your Computer Instead

by Timeo WilliamsMay 17th, 2024
Read on Terminal Reader
Read this story w/o Javascript
tldt arrow

Too Long; Didn't Read

Docker Desktop is a tool that allows you to run apps in a Docker container. It can be used to run applications such as Slack’s ChatGPT. This article explains how to use Ollama to run Slack on a Linux or Windows machine.
featured image - Ditch ChatGPT: Run A Fast LLM On Your Computer Instead
Timeo Williams HackerNoon profile picture

TLDR: As an open-source enthusiast > I’m amazed that we’re capable of running something as powerful as Llama 3 locally & that WE control the data. Let me show you how.

Step 1: Install Docker Desktop on your machine (if you haven’t already)

Why use Docker? Instead of wasting time writing 3 separate articles that handle the nuances of each different OS (MacOS, Linux, or Windows) > we make use of one source of truth (universal environment) as defined in a Dockerfile to get things situated and running.

^ This is a simplified explanation. I may revisit this when I have a better answer.

Step 2: In your terminal, paste and enter this command to get Ollama installed on your system:

docker pull ollama/ollama

You should see something similar to this after

Pull the latest Ollama Image

Step 3: Now that it’s installed, let’s run it by entering this:

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama Run Ollama container locallyBoom! You know can communicate with one of the smartest LLMs on the planet OFFLINE on your computer. Welcome to the club. Perhaps you’ll ask a more informative input than just ‘hey!’

Bonus Points!

If you’ve made it this far, but are feeling dissatisfied, you probably miss how intuitive the experience is when chatting with ChatGPT in it’s website form.

Let’s make something similar.

Shoutout to the contributors of WebUI. They’ve helped make a beautiful interface so that those of us who aren’t terminal lovers can live in peace.

In the same terminal window or the environment of your choice (perhaps VSCode?), enter this:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always

After a minute, you’ll be welcomed to this:

Open WebUI Interface

Select the Ollama model above and chat away!