Ollama Complete Guide

Ollama has emerged as one of the most widely adopted tools for running large language models locally. Rather than relyin

Ollama has emerged as one of the most widely adopted tools for running large language models locally. Rather than relying on remote APIs and cloud infrastructure, it lets developers and power users execute models such as Llama, Qwen, DeepSeek, and Gemma directly on personal computers or private servers. Built on top of the llama.cpp inference engine, Ollama abstracts away the complexity of CUDA configuration, manual compilation, and dependency management. The result is a streamlined, single-command workflow that turns model downloading, quantized loading, and inference into an accessible experience on macOS, Linux, and Windows.
Ollama is an open-source framework for local large-model execution and management. Its core value proposition is simplicity: users can pull a model, load it with an appropriate quantization level, and start chatting or generating text without touching low-level build scripts. Under the hood, Ollama handles model distribution, runtime optimization, and server orchestration, while exposing a straightforward command-line interface and a REST API for programmatic access.

Key Features

  • One-click execution: A single command triggers model retrieval, quantized loading, and interactive conversation.
  • Built-in model library: Models are organized by parameter count and quantization grade, using tags such as qwen2.5:7b or llama3.1:8b.
  • REST API: Ollama listens on port 11434 by default and provides endpoints compatible with the OpenAI API format, making it easy to integrate with existing applications.
  • Modelfile customization: Similar in spirit to a Dockerfile, a Modelfile lets users define base models, system prompts, temperature, context length, and other inference parameters.
  • Multimodal support: Vision-language models such as LLaVA and Llama 3.2 Vision can be run locally for image understanding tasks.

Core Concepts

Before diving into commands, it helps to understand three foundational ideas.

Model
Ollama distributes models in the GGUF format. A single model name can map to multiple variants that differ in parameter size and quantization precision. For example, deepseek-r1:7b refers to a specific 7-billion-parameter version of DeepSeek-R1.

Modelfile
A Modelfile is a plain-text definition file that describes how a model should behave. Common directives include:

  • FROM: specifies the base model.
  • SYSTEM: sets the system prompt or persona.
  • PARAMETER: adjusts inference settings such as temperature, top-p, and context window.

Service process
The ollama serve command runs a background daemon. Models are loaded into memory or GPU memory only when needed and can be unloaded after a period of inactivity. The keep_alive setting controls how long a loaded model remains resident.

Installation and Setup

On Linux, installation can be completed with a one-line script:

curl -fsSL https://ollama.com/install.sh | sh

macOS and Windows users can download official installers from the Ollama website. Once installed, the ollama command becomes available in the terminal. To start the background service, run:

ollama serve

In most desktop environments, the service starts automatically after installation, so this step is often optional.

Running Your First Model

To pull and converse with a model in one step:

ollama run deepseek-r1:7b

If the model is not already present locally, Ollama downloads it automatically before launching an interactive chat session. Other frequently used commands include:

  • ollama pull qwen2.5:7b — download a model without starting a chat.
  • ollama list — display all models stored on the local machine.
  • ollama ps — show models currently loaded in memory.
  • ollama show qwen2.5:7b — print metadata and configuration details for a model.
  • ollama rm qwen2.5:7b — remove a model from local storage.

Customizing Models with a Modelfile

One of Ollama’s most powerful capabilities is model customization through a Modelfile. Create a file named Modelfile with contents similar to the following:

FROM qwen2.5:7b

SYSTEM You are a concise coding assistant. Answer in plain English and provide runnable code snippets when helpful.

PARAMETER temperature 0.7
PARAMETER num_ctx 4096

Then register the custom model:

ollama create my-coding-assistant -f Modelfile

Afterward, invoke it just like any other model:

ollama run my-coding-assistant

This approach allows teams to standardize system prompts and generation parameters across projects without modifying application code