|
|
|
|
@ -38,7 +38,8 @@ Two containers do all the work:
|
|
|
|
|
- **`mcp-codebase-index`** (port 8001) — a private search index for each of your code |
|
|
|
|
projects, built with [open-codebase-index](https://github.com/Helweg/open-codebase-index). |
|
|
|
|
It lets the AI find things in a large codebase quickly (keyword search, "where is this |
|
|
|
|
defined", "who calls this") instead of reading files one by one. Every project gets its |
|
|
|
|
defined", "who calls this", and — with the bundled ollama embeddings — "find this by |
|
|
|
|
meaning") instead of reading files one by one. Every project gets its |
|
|
|
|
own address: `http://localhost:8001/<project>/mcp`. |
|
|
|
|
|
|
|
|
|
## Quick start (once per machine) |
|
|
|
|
@ -164,44 +165,56 @@ tools everywhere, add them once to your plugin's global settings file:
|
|
|
|
|
|
|
|
|
|
## What the code index actually searches |
|
|
|
|
|
|
|
|
|
By default the index runs in **structural mode**: keyword (BM25) search, symbol |
|
|
|
|
definitions, call graphs, branch awareness — entirely **offline**, no AI provider needed. |
|
|
|
|
This is usually plenty and it works out of the box. |
|
|
|
|
The index runs in **hybrid mode**: keyword (BM25) search, symbol definitions, call |
|
|
|
|
graphs, branch awareness — plus **semantic search** ("find code by meaning") via local |
|
|
|
|
[ollama](https://ollama.com) embeddings. The `ollama` container ships with this stack |
|
|
|
|
but stays dormant (it lives behind the `embeddings` profile), so everything already |
|
|
|
|
works fully offline without it: while ollama is off, semantic ranking just falls back |
|
|
|
|
to keyword ranking. |
|
|
|
|
|
|
|
|
|
To add **semantic search** ("find code by meaning"), plug in an embedding provider: |
|
|
|
|
|
|
|
|
|
1. Start local Ollama embeddings: `docker compose --profile embeddings up -d ollama`, then |
|
|
|
|
`docker compose exec ollama ollama pull nomic-embed-text`. |
|
|
|
|
2. In `codebase-index.config.json` switch `"mode": "structural"` to `"hybrid"` and add: |
|
|
|
|
|
|
|
|
|
```json |
|
|
|
|
{ |
|
|
|
|
"embeddingProvider": "custom", |
|
|
|
|
"customProvider": { |
|
|
|
|
"baseUrl": "http://ollama:11434/v1", |
|
|
|
|
"model": "nomic-embed-text", |
|
|
|
|
"dimensions": 768 |
|
|
|
|
} |
|
|
|
|
} |
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
3. Rebuild each project's index: |
|
|
|
|
Enable semantic search (once per machine): |
|
|
|
|
|
|
|
|
|
```bash |
|
|
|
|
docker compose exec mcp-codebase-index ocbi index --project ~/src/git/my-project --config /etc/codebase-index/config.json --force |
|
|
|
|
docker compose --profile embeddings up -d # start the ollama container |
|
|
|
|
docker compose exec ollama ollama pull nomic-embed-text # one-time, ~274 MB, kept in a volume |
|
|
|
|
# build vectors for your projects (safe to repeat): |
|
|
|
|
docker compose exec mcp-codebase-index ocbi index --project ~/src/git/my-project \ |
|
|
|
|
--config /etc/codebase-index/config.json --force |
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
(Or ask the AI to run the `index_codebase` / `index_status` tools — rebuilding and |
|
|
|
|
refreshing indexes is something the assistants can do themselves through MCP.) |
|
|
|
|
Projects registered later with `./add-project.sh` pick embeddings up automatically. |
|
|
|
|
Check a project's mode: `docker compose exec mcp-codebase-index ocbi status --project |
|
|
|
|
~/src/git/my-project --config /etc/codebase-index/config.json` — semantic search is |
|
|
|
|
live when it reports `Provider: ollama` and the index shows embedded chunks. (You can |
|
|
|
|
also just ask the AI to run `index_status` / `codebase_search`; semantic hits carry |
|
|
|
|
similarity scores near 1.0.) |
|
|
|
|
|
|
|
|
|
Gotchas: |
|
|
|
|
|
|
|
|
|
- Plain `docker compose up -d` / `down` / `ps` do **not** manage the ollama container. |
|
|
|
|
Pass `--profile embeddings` whenever a command should include it; |
|
|
|
|
`docker compose --profile embeddings down` stops everything. |
|
|
|
|
- If indexing logs `INDEXING WARNING: N chunks failed to embed`, ollama was unreachable |
|
|
|
|
or too slow (CPU). Start it, make sure the model is pulled, re-run the index — failed |
|
|
|
|
chunks are simply rebuilt on the next run. Search keeps working keyword-only meanwhile. |
|
|
|
|
- Batch sizes for the index container are pre-tuned small (`OLLAMA_MAX_BATCH_TOKENS` / |
|
|
|
|
`OLLAMA_MAX_BATCH_ITEMS` env in `docker-compose.yml`) so CPU-only ollama stays under |
|
|
|
|
the embedding API's 30s timeout. Raise them if you have a GPU. |
|
|
|
|
- Embeddings don't have to be local: any OpenAI-compatible endpoint works. Switch |
|
|
|
|
`embeddingProvider` to `"custom"` in `codebase-index.config.json` and set |
|
|
|
|
`customProvider: { "baseUrl", "model", "apiKey" }`. The chat model provider (z.ai, |
|
|
|
|
OpenRouter, cortecs, …) is configured in your AI tool and is completely independent |
|
|
|
|
of this stack — the index never talks to it. |
|
|
|
|
|
|
|
|
|
## Everyday commands |
|
|
|
|
|
|
|
|
|
```bash |
|
|
|
|
docker compose ps # status |
|
|
|
|
docker compose --profile embeddings ps # status, including ollama |
|
|
|
|
docker compose logs -f mcp # shared-tools gateway logs |
|
|
|
|
docker compose logs -f mcp-codebase-index # index gateway logs (per-project lines prefixed) |
|
|
|
|
docker compose restart mcp mcp-codebase-index # after editing supergateway.config.jsonc / codebase-index.config.json |
|
|
|
|
docker compose down # stop everything (index data survives, it lives in the projects) |
|
|
|
|
docker compose --profile embeddings down # stop everything incl. ollama (index data survives, it lives in the projects) |
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
## Adding or removing shared tools (port 8000) |
|
|
|
|
@ -262,6 +275,12 @@ docker compose ps # both services "Up"
|
|
|
|
|
which refuses `network: host` builds with "Pass `--allow=network.host`..." unless the |
|
|
|
|
host really needs it — on machines with working bridge networking leave `BUILD_NETWORK` |
|
|
|
|
unset (or set it to `default`). If it still fails, check your firewall. |
|
|
|
|
- **Indexing logs `INDEXING WARNING: N chunks failed to embed` (no semantic scores)** — |
|
|
|
|
the ollama embeddings container isn't running or the model isn't pulled yet. Fix with |
|
|
|
|
`docker compose --profile embeddings up -d` + |
|
|
|
|
`docker compose exec ollama ollama pull nomic-embed-text`, then re-run the index. |
|
|
|
|
Search still works meanwhile, just keyword-ranked (see "What the code index actually |
|
|
|
|
searches"). |
|
|
|
|
- **`brave-search` tools fail** — missing/invalid `BRAVE_API_KEY` in `.env`. |
|
|
|
|
- **Index seems stale** — re-run the project's index: |
|
|
|
|
`./add-project.sh ~/src/git/my-project` (safe to repeat), or ask the AI to run |
|
|
|
|
|
