Ollama

Ollama lets you run Llama 3, Mistral, Gemma, and other open-source LLMs locally. Simple CLI, REST API, and model library. No GPU required for smaller models.