Self-Hosted n8n RAG Pipeline: The Private Stack

You want a chatbot that answers from your own documents, but the documents are contracts, patient notes, or an internal wiki that cannot be shipped to a third-party API. That single constraint rules out most of the easy RAG tutorials. The fix is a fully self-hosted stack: n8n as the orchestrator, Ollama serving a local model, and pgvector storing embeddings inside Postgres, so every step runs on hardware you own.

This is an architecture guide. It walks the whole node graph from ingest to answer, shows where each piece runs, and gives you the modeled cost so you can decide whether the privacy is worth the operations work. For the server build itself, the VPS setup guide goes deeper on hosting.

What is a self-hosted n8n RAG pipeline?#

A self-hosted n8n RAG pipeline runs the entire retrieval-augmented flow on hardware you control: n8n orchestrates, Ollama serves a local model, and pgvector stores the embeddings inside Postgres. No document text or query ever leaves your network, and no per-token API bill arrives.

Retrieval-augmented generation is a simple loop. You index your documents once by splitting them into chunks and storing a vector for each chunk. Then, at query time, you embed the question, find the closest chunks by vector similarity, and pass them to a language model as context so its answer is grounded in your data rather than its training set.

A managed pipeline outsources three of those jobs: the embedding API, the hosted vector database, and the model endpoint. Each hop sends your text to someone else's servers. The self-hosted version keeps all three in-house. n8n already ships native nodes for every piece, so you are wiring existing building blocks, not writing glue code. The deeper mechanics of the local approach live in the fully local RAG walkthrough.

What does the architecture look like?#

Four pieces, each on your own box. n8n is the orchestrator, the Ollama Chat Model node writes answers, the Embeddings Ollama node turns text into vectors, and the PGVector Store node reads and writes those vectors in Postgres. Data flows ingest, embed, store, retrieve, then generate, all inside one private network.

Picture the stack as five stages laid over four nodes. n8n handles orchestration and chunking. The Embeddings Ollama node calls your local Ollama server to convert text into vectors. The PGVector Store node writes and reads those vectors. The Ollama Chat Model node generates the final answer. Postgres, with the pgvector extension enabled, is the one piece that is pure infrastructure.

Here is the same stack as a component table, with the role and cost of each layer:

Every credential points inward. The two Ollama nodes share one Ollama credential whose Base URL defaults to http://localhost:11434, and the PGVector node uses a standard Postgres credential. Nothing in this graph holds an external API key, which is the whole point. Set the self-hosted shape next to the managed one you might be replacing and the trade becomes clear.

Same five stages, two homes. Self-hosting keeps the data and the bill on your side; managed trades that for zero operations.

How do you wire the ingest and retrieval flow?#

Two flows share the same embedding model. The ingest flow loads a file, splits it into chunks, sends each chunk through the Embeddings Ollama node, and writes the vectors with PGVector Insert Documents. The retrieval flow embeds the user question, runs PGVector Get Many for the closest chunks, and hands them to the Ollama Chat Model.

Start with ingest. A trigger or manual run feeds a document loader, which splits each file into chunks of a few hundred tokens. Each chunk goes to the Embeddings Ollama node. Pick nomic-embed-text for 768 dimensions of quality, or all-minilm for 384 dimensions when you want smaller, faster vectors. The PGVector node in Insert Documents mode writes every chunk and its vector into a table.

Retrieval reuses the exact same embedding model, and that matters. Query vectors and document vectors have to live in the same space or similarity search returns noise. The PGVector node in Get Many mode takes the embedded question and returns the closest chunks by distance. Those chunks become the context for the Ollama Chat Model node, which writes the grounded reply. You wire this once in n8n, and the build order is short.

Build it once: enable the extension, wire the ingest branch, then the retrieval branch, and hand chunks to the local model.

If you have not connected the model layer yet, the Ollama-to-n8n guide covers the credential and the first working call.

What does it cost and where does it run?#

The recurring software bill is 0 dollars. n8n Community edition, Ollama, pgvector, and Postgres are all free, and local inference means no per-token API charge. What you pay for is compute. A 7B model needs roughly 8 GB of RAM, so this runs on a modest VPS or a spare workstation.

Split the bill into two lines. The software line is 0 dollars: n8n Community edition runs free under the Sustainable Use License, Ollama and pgvector are open source, and Postgres has been free for decades. The inference line, which is where a managed stack quietly runs up a tab, is also 0 dollars here because the model runs on your own silicon and charges nothing per token.

That leaves compute as the only real cost, and it is a fixed one. A 7B model at 4-bit quantization needs on the order of 8 GB of RAM to run, and something closer to 16 GB makes it comfortable. On a CPU-only box you might see single-digit tokens per second, so a small GPU or an Apple Silicon machine is the difference between a demo and a tool. Those figures are modeled rules of thumb, not a benchmark from this site, but they set the right expectation before you provision a machine.

The software and API bill is zero. The one real cost is compute, and it is fixed, not per query.

The honest catch is where a managed stack still earns its fee. It scales elastically, it needs no patching, and it answers faster on cold hardware. You are trading a variable per-token bill for a fixed compute bill plus your own operations time. For a private document chatbot, that trade usually favors self-hosting, which the private document chat build demonstrates end to end.

When is self-hosting worth it over a managed stack?#

Self-host when privacy or cost control outweighs convenience. If your documents cannot leave your network, or you want a flat compute bill instead of per-token fees, the local stack wins. A managed stack wins when you value zero ops, elastic scale, and the fastest possible answers over data residency.

The decision is not about which stack is better in the abstract. It is about which constraint binds you. If data residency is a hard requirement, the choice is already made, because a managed embedding API sees every document you index. If your volume is spiky and small, a managed free tier may cost less attention than running a server. Most private-data use cases sit firmly on the self-host side. Run through a short readiness check before you commit.

If most of these hold, the private stack earns its keep and the rest is provisioning and monitoring.

If most of those hold, the stack is a good fit, and the remaining work is provisioning and monitoring. One networking gotcha to expect: when n8n and Ollama run in separate Docker containers, localhost inside the n8n container is not the host, so point the Ollama credential at host.docker.internal or your host IP, and swap localhost for 127.0.0.1 if a local instance refuses to connect.

Put it together and the whole private pipeline is one graph: a document enters on one branch, a question enters on the other, and both stay on your machine from ingest to answer.

One graph, two branches. A document is chunked, embedded, and stored; a question is embedded, matched, and answered, all on your machine.