Fully Local RAG in n8n: Private Embeddings, No Cloud APIs (2026)

You followed a "local RAG" tutorial, got Ollama running, and felt safe. Then you opened your network monitor and noticed the embeddings step was still calling api.openai.com, and your vectors were sitting in a hosted Pinecone cluster. Every line of every client contract had already left the building. A fully local RAG in n8n pipeline closes that leak by running embeddings, the vector store, and the answering model all on hardware you control.

For a solopreneur handling client contracts, HR records, or anything under GDPR, this distinction is not academic. The embedding step reads every sentence of every document. A hosted embeddings API or a cloud vector DB holds a searchable copy of that text indefinitely. One misconfigured Base URL and the leak is invisible in your workflow but very visible to anyone watching the wire.

I run Ollama on a 16GB Mac and n8n self-hosted on a small Hetzner VPS. The VPS handles n8n and even Qdrant or Postgres fine, but it cannot load an LLM with any real capacity. The honest deployment note: either co-locate n8n with Ollama on the machine that has the RAM, or keep Ollama on your workstation and point n8n at it over your LAN.

What Does Fully Local RAG in n8n Actually Mean?#

A fully local RAG pipeline in n8n means every node that touches your text points at a service on your own hardware, not a third-party API. Data can leave at exactly three points: embedding documents, embedding the question, and generating the answer. All three must point at localhost or an internal address.

n8n has native nodes for all three layers. The Embeddings Ollama node generates embeddings by calling an Ollama instance you control, with no remote endpoint hard-coded. The vector store nodes (Qdrant and PGVector) connect to whatever URL you configure. The Ollama Chat Model node uses the same Ollama credential. Point all three at internal addresses and the pipeline is genuinely local.

The n8n Community edition runs on your own infrastructure with no license fee, so the self-hosted path costs nothing beyond your hardware.

Where Do Most Local RAG Builds Still Leak to the Cloud?#

Most "local RAG" builds fix only the answering layer. They swap GPT-4 for a local Ollama chat model and call it done, while the embedding step still calls OpenAI or Cohere and the vectors still land in a hosted Pinecone cluster. The full text of every document you index has already left twice.

The table below maps each pipeline stage to the cloud service that usually handles it and the local n8n node that closes the leak.

Run this check before trusting the pipeline with real client data.

How Do You Run Private Embeddings with the Embeddings Ollama Node?#

Private embeddings in n8n work by pointing the Embeddings Ollama node at a local Ollama instance via the Ollama credential. The credential takes an instance Base URL, which defaults to http://localhost:11434 unless your OLLAMA_HOST environment variable overrides it. On IPv6 machines, use http://127.0.0.1:11434 instead of localhost to avoid resolution issues.

The Ollama embeddings docs recommend three models for semantic search and RAG: embeddinggemma, qwen3-embedding, and all-minilm. For parameter size reference, Ollama's embedding models blog post lists mxbai-embed-large at 334M parameters, nomic-embed-text at 137M parameters, and all-minilm at 23M parameters. Bigger means richer vectors and more RAM. Embedding vector dimensions typically range from 384 to 1024, per Ollama's docs.

The Embeddings Ollama node makes no external call beyond the Base URL set in the credential. If that URL is a local address, the text being embedded never leaves the machine.

Which Self-Hosted Vector Store Should You Use, Qdrant or PGVector?#

For a fully local build, the right vector store depends on what you already run. Pick PGVector if you already have Postgres: it adds SQL metadata filtering. Pick Qdrant for a purpose-built database that runs in Docker on port 6333. Both cost $0 in license fees.

The Simple (In-Memory) Vector Store is also fully local, but n8n's docs are explicit: data is lost when n8n restarts and the store may be purged under memory pressure. Use it only to prototype wiring, never for documents you need to keep.

Both are open-source and cost $0 in license fees.

Both the Qdrant Vector Store node and the PGVector node support four modes: Get Many, Insert Documents, Retrieve (as vector store for chain), and Retrieve (as tool for AI Agent). The mode you choose depends on whether you use a Question and Answer Chain or an AI Agent for the answering step.

How Do You Wire the Full Pipeline in n8n?#

Wiring a fully local RAG pipeline in n8n takes two separate workflows: one for ingestion (loading and indexing documents) and one for querying (answering questions). Both share the same Embeddings Ollama node pointing at the same local model. That shared model is the key constraint that keeps retrieval coherent.

The same Embeddings Ollama model must be used to index and to query.

The ingestion workflow#

1. Bring your document into n8n (from disk, HTTP, or a trigger node).
2. Connect a Default Data Loader sub-node. In Simple mode, it uses the Recursive Character Text Splitter with a default chunk size of 1000 and overlap of 200.
3. Feed the chunks into the Embeddings Ollama node with your chosen local embedding model.
4. Connect the Qdrant Vector Store or PGVector Vector Store node in Insert Documents mode.

The query workflow#

1. Take the user's question as input.
2. Pass it to the same Embeddings Ollama node with the identical model used during indexing.
3. Connect the vector store node in Retrieve (as vector store for chain) mode.
4. Feed the retrieved chunks into the Ollama Chat Model node via a Question and Answer Chain or an AI Agent node set to use the vector store as a tool.

The Ollama Chat Model common-issues page covers all four deployment shapes (both on one host, Ollama in Docker, n8n in Docker, both in Docker) with the correct Base URL for each: http://localhost:11434, http://host.docker.internal:11434, or the Docker container name.

What Do You Give Up by Going Fully Local?#

Going fully local is a deliberate trade, not a free lunch. You get full data privacy and $0 per-query API cost, but you carry the hardware and accept the reasoning ceiling of whatever local model you run. On hard multi-step reasoning across many documents, a frontier cloud API still outperforms a small local model.

The hardware floor is real. A small VPS handles n8n and even Qdrant or Postgres comfortably, but it cannot run Ollama with any meaningful model size. You need the RAM on the machine hosting Ollama, not the machine hosting n8n.

There is also a switching cost that catches people off guard. Change the embedding model and every vector currently in your store is stale. You must re-embed the entire document corpus from scratch because vectors from two different models are not comparable, as Ollama's embeddings docs make clear.

The trade is real: full privacy costs you reasoning depth and RAM.

When is a cloud API the honest fallback? When the reasoning task is complex enough that a local model produces consistently wrong answers and your data is not under a hard privacy constraint. The choice is privacy-first versus raw answer quality, and both are legitimate.

How Do You Verify Nothing Leaves Your Server?#

Verification is two steps: audit every credential, then confirm the pipeline works with the network completely blocked. Open every Ollama credential in n8n and confirm the Base URL is a local address. Check every vector store credential and confirm the Qdrant URL or Postgres host is internal, never a cloud cluster address.

Then pull the cable (or block egress via your firewall). Ask the bot a real question about a document you indexed. If it returns the correct snippet with no network access, the pipeline is genuinely local. If it times out or errors, something in the chain is still phoning home.

Do this every time you make a credential change or add a new node to the pipeline. It takes 30 seconds and it is the only test that actually proves privacy.

Where to Go from Here#

If you have not wired Ollama into n8n yet, start with connecting Ollama to n8n for a local AI agent to get the credential and Base URL right first. Then compare PGVector, Chroma, and Qdrant for self-hosted RAG to decide whether to migrate stores as your corpus grows.