Ollama API

Ollama provides a lightweight, open-source runtime for deploying large language models on local hardware. Rather than requiring developers to configure CUDA, manage model files, or implement a separate inference server, Ollama packages those tasks behind a consistent command-line interface and REST API.

Its value extends beyond running pretrained models. With Modelfile, developers can define reusable models that combine a base model, system instructions, inference parameters, and conversation templates. These custom models can then be accessed through the same API as any other local model, making Ollama practical for prototypes, internal tools, retrieval-augmented generation, and domain-specific assistants.
Ollama follows a client-server architecture. The Ollama service loads model weights, prepares tensors, executes inference, and returns generated output. Both the CLI and HTTP clients communicate with this service, which normally listens on localhost:11434.

The main native API endpoints are:

Requests and responses use JSON, so the API integrates naturally with Python, Go, Node.js, and other backend technologies. For example, a basic generation request looks like this:

curl http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3",
    "prompt": "Explain the difference between a process and a thread.",
    "stream": false
  }'

With "stream": false, Ollama waits until generation is complete and returns one JSON object. With "stream": true, it sends incremental chunks as they become available. This is especially useful for long responses because it reduces the time users wait for the first visible token.

The /api/chat endpoint works with a messages array rather than a raw prompt:

{
  "model": "llama3",
  "messages": [
    {
      "role": "user",
      "content": "Write a Python function that validates an email address."
    }
  ],
  "stream": false
}

Ollama does not automatically maintain a persistent conversation for the application. To preserve context, the client must include previous messages in each new request. This design keeps the API stateless and makes conversation storage straightforward to control.

Creating custom models with Modelfile

A Modelfile describes how a custom Ollama model should be assembled. It is similar in spirit to a Dockerfile, but it creates a model configuration rather than a container image.

A typical Modelfile uses the following instructions:

• FROM selects the base model.
• SYSTEM establishes the model’s role, instructions, and behavioral boundaries.
• PARAMETER configures inference settings such as temperature, context size, and output limits.
• TEMPLATE optionally defines how chat messages are formatted before inference.

For example, the following configuration creates a deterministic code-review assistant:

FROM llama3

SYSTEM """
You are a senior Python code reviewer.
Review submissions for correctness, security, readability, and PEP 8 compliance.
Identify concrete problems before suggesting improvements.
When you propose a rewrite, preserve the original behavior unless a change is necessary.
"""

PARAMETER temperature 0.2
PARAMETER top_p 0.9
PARAMETER num_predict 768
PARAMETER num_ctx 8192

Build it with:

ollama create code-reviewer -f Modelfile

The resulting model appears in the local model list and can be used anywhere the base model could be used:

ollama run code-reviewer

curl http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d '{
    "model": "code-reviewer",
    "prompt": "Review this function: def add(a,b): return a+b",
    "stream": false
  }'

This form of customization does not retrain or alter the model’s weights. Instead, it combines prompt engineering with runtime configuration. That makes it fast, inexpensive, and easy to revise. If deeper domain adaptation is required, the base model can first be fine-tuned through a separate training pipeline and then referenced by FROM.

Inference parameters can also be overridden per request. This makes it possible to keep a stable default Modelfile while adjusting behavior for individual workloads:

{
  "model": "code-reviewer",
  "prompt": "Review the following code...",
  "stream": false,
  "options": {
    "temperature": 0,
    "num_predict": 512
  }
}

Calling a custom model from Python

The native API is straightforward to call with the requests library. The following function sends code to the custom model and returns its complete response:

import requests

OLLAMA_URL = "http://localhost:11434"


def review_code(code_snippet: str) -> str:
    payload = {
        "model": "code-reviewer",
        "prompt": (
            "Review the following Python code. Identify correctness issues, "
            "security risks, and opportunities for clearer formatting.\n\n"
            f"{code_snippet}"
        ),
        "stream": False,
    }

    response = requests.post(
        f"{OLLAMA_URL}/api/generate",
        json=payload,
        timeout=120,
    )
    response.raise_for_status()
    return response.json().get("response", "")

Example usage:

sample = """
def calculate_total(items):
    total = 0
    for item in items:
        total += item["price"]
    return total
"""

print(review_code(sample))

For conversational applications, a small chat client can send an ordered message history:

import requests


def chat(messages: list[dict[str, str]]) -> str:
    payload = {
        "model": "code-reviewer",
        "messages": messages,
        "stream": False,
    }

    response = requests.post(
        "http://localhost:11434/api/chat",
        json=payload,
        timeout=120,
    )
    response.raise_for_status()
    return response.json()["message"]["content"]

The caller can then append both the user message and the assistant’s answer to messages before making the next request.

Handling streamed output

Streaming is preferable when users should see output progressively