Running an LLM inside your Obsidian vault sounds like a technical project for developers. The actual setup takes under an hour, and the result is AI assistance on your notes that never leaves your machine.

For operators who work with sensitive client data, proprietary research, or anything they’d rather not send to a cloud API, local LLM integration with Obsidian is the practical answer — not a theoretical one.

Quick Answer: Ollama runs local LLMs (Llama 3, Gemma 3, Mistral, Qwen3) on your machine with a local API server. The Obsidian Smart Connections or Local GPT plugin connects Ollama’s API to your vault. Setup: install Ollama, pull a model, install the plugin, point it at localhost:11434. Total time: 30–60 minutes depending on model download speed.

Why run LLMs locally inside Obsidian

The cloud API path (Claude, GPT-4o, Gemini) is faster and more capable for most tasks. Local LLMs make sense when:

  • Privacy is non-negotiable. Notes never leave your machine. No training data, no API logs, no third-party access.
  • You work offline. Local models work without internet. Useful for travel or restricted network environments.
  • You want zero ongoing API cost. After the initial hardware and model setup, there’s no per-token cost.
  • You process notes in bulk. Summarizing 500 notes in a session would cost meaningfully on cloud APIs. Locally it’s free.

The trade-off: local models are slower and less capable than frontier cloud models. For note summarization, concept extraction, and Q&A over your vault, the quality gap is acceptable. For complex reasoning or creative generation, cloud APIs are still significantly better.

What you need

Hardware: A machine with at least 8GB RAM for small models (7B parameters). 16GB RAM for comfortable use of 8B–13B models. Apple Silicon Macs (M1/M2/M3) run Ollama efficiently — the unified memory architecture handles LLM inference well.

Ollama: An open-source runtime that manages model downloads, serves a local API, and handles inference. macOS, Linux, and Windows supported.

A model: Llama 3.2 3B is a good starting point — fast, capable enough for vault tasks, runs on 8GB RAM. Llama 3.1 8B or Gemma 3 9B for higher quality on 16GB+ RAM.

Obsidian plugin: Local GPT (searches by that name in the Obsidian plugin browser) or Smart Connections (more feature-rich, supports semantic search across the vault).

Step-by-step setup

Step 1 — Install Ollama

Download from the Ollama website and install. On macOS, it runs as a menu bar app. On Linux, it runs as a systemd service.

After installation, open a terminal and verify:

ollama --version

Step 2 — Pull a model

ollama pull llama3.2:3b

This downloads the model (2–4GB depending on the model). Pull time depends on your connection. After the download, the model is stored locally.

To test the model in the terminal:

ollama run llama3.2:3b "Summarize the concept of a Zettelkasten in 2 sentences."

If you get a coherent response, the model is working.

Step 3 — Confirm the API server

Ollama automatically starts an API server at http://localhost:11434. Verify with:

curl http://localhost:11434/api/tags

You should see a JSON response listing your downloaded models.

Step 4 — Install the Obsidian plugin

In Obsidian: Settings → Community plugins → Browse → search “Local GPT” → Install → Enable.

In the Local GPT plugin settings:

  • Model provider: Ollama
  • API endpoint: http://localhost:11434
  • Model: llama3.2:3b (or whatever you pulled)

Step 5 — Test in your vault

Open any note. Use the command palette (Cmd+P on macOS) and search “Local GPT.” Select “Ask about current note.” The plugin sends the note content to your local Ollama instance and returns a response.

If it works, the integration is complete.

Practical use cases in Obsidian

Summarize long notes: Open a meeting transcript or research note and ask Local GPT to summarize the key points. Useful for processing reference material you’ve accumulated but haven’t reviewed.

Extract action items: Ask the model to list all action items or commitments mentioned in a note. Useful for processing meeting notes or project documentation.

Generate related questions: Ask the model “what questions does this note leave unanswered?” to identify gaps in your research or thinking.

Rephrase for clarity: Select a section and ask Local GPT to rephrase it more clearly. Useful for notes you’ll share or use as blog drafts.

Cross-note Q&A (Smart Connections): The Smart Connections plugin with local embedding models can answer questions across your entire vault — “what have I written about OpenRouter pricing?” — by semantically searching your notes and passing relevant results to the LLM.

Performance and quality expectations

A 3B model on Apple Silicon M1 responds in 5–15 seconds for a typical note-length input. An 8B model takes 15–40 seconds. This is slower than cloud APIs but acceptable for non-time-sensitive tasks.

Quality is noticeably lower than Claude or GPT-4o. For summarization and extraction tasks on your own notes, the quality is sufficient. For tasks requiring deep reasoning, generation of new ideas, or handling of ambiguous instructions, cloud APIs are better.

For day-to-day research and writing workflows that involve cloud APIs alongside Obsidian, the Obsidian for AI research guide covers how both local and cloud AI fit a unified knowledge workflow. The Obsidian tool page covers recommended plugins and setup decisions.

Frequently Asked Questions

Does Ollama work on Windows? Yes — Ollama has native Windows support as of late 2024. Setup is similar to macOS. GPU acceleration on Windows uses CUDA (NVIDIA) or DirectML.

Which models work best for Obsidian vault tasks? For summarization and extraction: Llama 3.2 3B (fast) or Llama 3.1 8B (higher quality). For code-heavy notes or technical documentation: Qwen2.5-Coder 7B. For multilingual vaults: Qwen3 8B.

Can local models access the internet to look up current information? No — local models are static. They have a training cutoff and cannot retrieve live information. For tasks requiring current data, use a cloud API with search capability.

Is Local GPT the best Obsidian plugin for Ollama? Local GPT is the most straightforward. Smart Connections is more powerful if you want semantic search across your entire vault — it builds a local embedding index of all your notes. Smart Connections supports Ollama embedding models alongside its cloud embedding options.

What happens to my notes during a local LLM query? The note content is sent to the Ollama API running on localhost. It never leaves your machine. Ollama does not have telemetry that sends note content externally.

Build your Obsidian knowledge system

If you’re using Obsidian as the foundation for an AI-assisted research and content workflow, the Obsidian tool overview covers setup decisions, plugin recommendations, and use cases by operator type.