Self-hosting an AnythingLLM alternative: Docker or an open-source AI Operating System

What self-hosting AnythingLLM involves

This page compares the two self-hosting setups the way each project's own instructions describe them: what you install, what it asks of the machine, and where each setup stops. If you want the product-level story first, the overview of AnythingLLM alternatives covers it. The rest of this page stays on the plumbing.

AnythingLLM is an all-in-one AI app from Mintplex Labs, built to run locally by default: you connect a local or hosted model, upload documents, and chat with them or run agents inside a workspace. Its anything-llm repository is MIT licensed, and the code behind every claim on this page sits in it.

AnythingLLM self-hosts as one Docker image, mintplexlabs/anythingllm, published for amd64 and arm64 machines. The image bundles the app's three working parts, which the technical overview names: a React frontend, a NodeJS server, and a document collector. The unit of work is the workspace. You drop documents in, the built-in LanceDB vector store embeds them, and your chats and agent runs answer from what you uploaded.

The project's Docker guide, checked October 2026, asks for Docker and a model you can reach, and suggests at least 2 GB of memory and 10 GB of disk on a cloud box. A laptop clears that bar, which is the point of a local-first design.

The steps look like this:

1. Pull mintplexlabs/anythingllm on any machine that runs Docker.
2. Create a storage folder and an empty .env file on the host, then run the container with port 3001 published, the storage folder mounted at /app/server/storage, and the .env mounted in. The documented command also adds the SYS_ADMIN capability. The storage mount is the part to get right: it keeps your documents, vectors, and settings alive when you re-pull the image for an update.
3. Open http://localhost:3001 and the app is ready.
4. Connect a model: point the app at a local runtime such as Ollama, or paste an API key for a hosted provider. The documented catch is that localhost inside the container does not reach the host machine. The guide works around it with host.docker.internal, which works out of the box on Docker for Windows and Mac; on Linux you add an --add-host flag to the run command or point at the 172.17.0.1 gateway address.
5. Update by re-pulling the image, with the storage mount carrying your data across. If the box serves other people, the guide shows how to set the frontend's VITE_API_BASE to the machine's real address so browsers can find the server.

Two details matter when you weigh this against a company platform. Multi-user support is a Docker-version feature: user accounts and permissioning exist only in the server build, while the desktop app stays single-user. And the permissioning its docs describe is per user, not per tool or per call: agents run inside workspace chats and can browse the web or call tools through MCP, but the documented model defines no approval step for a tool call and no review step where an agent's work lands as a diff. A chat is a request to that one server; a Kortix session is a task on its own machine, on its own branch.

If one box is not the shape you want, the repo lists one-click deploy templates for AWS, GCP, DigitalOcean, Railway, and more, plus a bare-metal guide for a production instance without Docker. One more default to know: the app sends anonymous usage telemetry until you set DISABLE_TELEMETRY or flip the privacy switch inside the app.

What self-hosting Kortix involves

Kortix is the open-source AI Operating System. Your agents, skills, company memory, and connectors live in one git repo you own, and each session runs on its own isolated machine. AnythingLLM gives one person a place to work with documents; Kortix gives a team a place to run work.

Kortix self-hosts a company-level agent system. A project is a git repo: agents are markdown files, skills are markdown plus scripts, memory is plain files, and kortix.yaml declares the machine image that sessions boot, the connectors they may use, the triggers that start work, and the names and grants of secrets. Secret values never appear in the repo, and connector credentials are brokered server-side, so they never enter the machine a session runs on.

Kortix documents three ways to install (checked October 2026):

1. The one-shot bootstrap, Linux only: one command installs Docker, installs the kortix CLI, and starts the whole stack.
2. The manual path: install the CLI, add DNS records for your domain and for api., and open ports 80 and 443 so the bundled Caddy proxy can issue TLS certificates. Then run kortix self-host init with your domain and kortix self-host start. Status, logs, and a doctor command cover health checks from there.
3. Evaluation mode: a Cloudflare tunnel with no domain at all. The URL changes on every restart, so treat it as a trial run rather than production.

Once the stack is up, kortix self-host configure asks for the sandbox provider key and, optionally, a managed git token. Daytona is the default sandbox provider, with Platinum and E2B also supported. The first run asks six things and no model key: you create your account in the dashboard and connect your own model key in the model picker, because self-hosted instances run on your keys by default. GitHub connects afterward, in the dashboard under Settings → Git.

The stack itself is one Docker Compose file: frontend, API, LLM gateway, and the Supabase distribution. Sessions do not run on it. Each session boots on the sandbox provider as its own isolated machine, on its own branch. The API service has a 640 MiB memory limit by default; the guide keeps that on an 8 GB host and raises it to 1 GiB on a 16 GB one.

Day-to-day operations stay short. Every instance updates itself, and the update command takes a --tag flag when you want to pin an exact version. Backups are on you: there is no separate backup system, so you copy three things, the Postgres data directory, the file storage directory, and the instance .env that holds secrets and signing keys. Miss one and a restore comes back as half a company.

What that box gives you is the layer a document-chat app has no room for. Connectors reach Slack, docs, tickets, CRM, and code, each governed by a per-call policy of Allow, Ask, or Block. Approval gates exist too, and they stay off until you set them. Per-resource permissions cover people and agents, alongside roles, groups, and an audit trail.

Work lands as a change request a person reviews, and merge is default-deny for agents. Slack and Microsoft Teams are live channels. You pick any model, on your own keys, chosen per agent, per session, or per message. Self-hosting is free, and the same install also runs in a VPC or on your own network.

All of it is public code: Kortix on GitHub carries 20,000+ GitHub stars, and the Compose file, the manifest schema, and the change-request gate are all in the repo you would be hosting.

Side by side: what you need and what you get

The requirements are closer than the outcomes: a machine you control and a model you can reach, for either route. What the install then becomes is the difference.

The feature-level version of this table, with each cell traced to a source, lives in the Kortix vs AnythingLLM comparison.

Which setup suits you

If you are one person with a folder of documents and a local model, the AnythingLLM container does that job well. One image, one port, your files on your own disk, and an MIT license that asks nothing for running it.

The picture changes when the system you host has to behave like a member of a team: acting across Slack, tickets, and code, remembering the business between tasks, running several jobs at once without touching each other, and handing finished work to a person for review. Those are platform jobs, and a single chat container has no place to put them. If what you want is a self-hosted system where agents do a company's work, that is the setup Kortix is built for, and it is the one this site recommends.

From AnythingLLM to Kortix

If you land on the Kortix side, you do not start from zero. The migration guide for leaving AnythingLLM covers what carries over and what changes shape. The two setups also coexist fine: some teams keep the document container for private reading and run company work on the platform.

You can try Kortix on the managed cloud first and self-host later, or go straight to your own hardware. Self-hosting is free either way.