# 🧠 SecondBrain **A local-first, privacy-scrubbing personal memory you can point at everything you produce.** *Notes Β· code Β· agent transcripts Β· event logs β†’ one portable file β†’ instant hybrid recall.* *No cloud. No API keys. No data egress. Secrets scrubbed before anything is stored.*
--- SecondBrain is a tiny (~500-line, single-file) knowledge engine. You feed it the things you already generate β€” documents, source code, AI-agent chat transcripts, structured event logs β€” and it gives you back one command: ```bash brain recall "what did we decide about the pricing model" --hybrid ``` Everything runs on your own machine. Embeddings are computed locally by [Ollama](https://ollama.com) (`nomic-embed-text`), and the entire brain is a **single portable `brain.db`** (SQLite + [`sqlite-vec`](https://github.com/asg017/sqlite-vec)) you can copy to a USB stick. There is no server to run, no account to create, and **no credential ever leaves your box** β€” because SecondBrain redacts them *before* embedding. ## Why it exists Vector-RAG demos are easy; a memory you'd actually trust with your real life is not. SecondBrain is opinionated about the three things that usually break: | Problem | SecondBrain's answer | |---|---| | **Secrets leak into your index** (and then into an LLM's context) | A mandatory scrub pass strips PEM keys, JWTs, `sk-ant-…`/`sk-…`/`ghp_…`/AWS keys, and any `password: / token= / bearer …` pattern **before** a chunk is stored. IPs, hostnames and emails are kept as operational knowledge (toggle with `SCRUB_KEEP_PII=0`). | | **Dense-only search misses exact strings** (invoice numbers, IBANs, container names, file paths) | Hybrid recall fuses dense KNN with FTS5/BM25 lexical search via **reciprocal-rank-fusion** β€” meaning *and* identifiers both hit. | | **Cloud RAG = data egress + lock-in + cost** | 100% local. Ollama for embeddings, SQLite for storage. Portable, free, offline-capable, redeployable on any device in minutes. | ## The senses SecondBrain ingests from pluggable "senses" β€” each strictly **read-only**: - **`ingest-files `** β€” code, docs and config (60+ text extensions; skips `node_modules`, `.git`, binaries, and files > 400 KB). - **`ingest-transcripts `** β€” `*.jsonl` AI-agent sessions (Claude Code–style event logs); tool spam is collapsed to signal. - **`ingest-events --db `** β€” any SQLite with an `events(id, ts, sense, subject, payload)` table (e.g. email, chat, calendar, finance). Turns real activity into recallable memory. Add your own sense in ~10 lines: chunk β†’ `scrub()` β†’ `_stage()`. That's the whole contract. ## Quickstart ```bash # 0. prerequisites: python3, and Ollama running (https://ollama.com) ollama pull nomic-embed-text # 1. install git clone https://git.atlascorporation.nl/atlas/secondbrain cd secondbrain bash deploy/install.sh # venv + deps + schema . .venv/bin/activate # 2. feed it something python3 brain.py ingest-files ~/notes python3 brain.py ingest-files ~/code/my-project # 3. embed (local, free) python3 brain.py embed # 4. recall python3 brain.py recall "how does the auth flow work" --hybrid python3 brain.py stats ``` Output: ``` #1 score=0.72 via=both rrf=0.0312 [file/my-project] auth.py def login(user, pw): # verifies against argon2 hash, issues a signed … ``` ## Configuration All via environment (or a `.env` next to `brain.py`). Copy `.env.example` β†’ `.env`: | Var | Default | Meaning | |---|---|---| | `BRAIN_HOME` | `~/.secondbrain` | where `brain.db` lives | | `BRAIN_DB` | `$BRAIN_HOME/brain.db` | explicit db path (overrides `BRAIN_HOME`) | | `OLLAMA_URL` | `http://localhost:11434` | any reachable Ollama β€” including a **GPU box on your LAN/tailnet** | | `EMBED_MODEL` | `nomic-embed-text` | embedding model | | `EMBED_DIM` | `768` | must match the model | | `SCRUB_KEEP_PII` | `1` | `0` also strips emails (secrets always scrubbed) | > **Tip β€” offload embeddings to a GPU:** point `OLLAMA_URL` at another machine's Ollama > (`http://100.x.y.z:11434`). SecondBrain will embed there and store locally. This is how you > keep a laptop's memory current using a desktop GPU, with zero extra infrastructure. ## Deploy anywhere **systemd (continuous refresh):** set your sources in `.env` (`SB_FILE_DIRS`, `SB_TRANSCRIPTS`, `SB_EVENTS_DB`), then: ```bash sudo cp deploy/secondbrain-refresh.* /etc/systemd/system/ sudo systemctl enable --now secondbrain-refresh.timer # ingest+embed every 15 min ``` **Docker:** ```bash docker build -t secondbrain -f deploy/Dockerfile . docker run --rm -v sbdata:/data -e OLLAMA_URL=http://host.docker.internal:11434 \ secondbrain recall "quarterly numbers" --hybrid ``` ## Use it as a library ```python import brain for hit in brain.brain_recall_hybrid("open invoices for June", k=8): print(hit["score"], hit["ref"], hit["text"][:120]) ``` `brain_recall_hybrid()` degrades gracefully to dense-only if the lexical index is empty, so it's a safe drop-in for `brain_recall()`. ## How it works ``` sources ──▢ scrub() ──▢ chunk ──▢ chunks table ──┬─▢ FTS5 (BM25, via triggers) (senses) (redact) (≀1600 ch) └─▢ vec_chunks (nomic 768-dim, Ollama) β”‚ recall("q") ──▢ embed query ──▢ dense KNN ┐ β”‚ β””β–Ά FTS5 lexical ─────────────┴─ RRF fuse ─┴─▢ ranked hits ``` See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for the full design, the scrub guarantees, and the reciprocal-rank-fusion math. ## Privacy & security - **Scrub-before-store** is not optional and runs on every chunk from every sense. `rescrub` re-applies a hardened scrub to an existing db in place. - The database **never leaves your machine** unless you copy it. `.gitignore` blocks `*.db`, `.env`, `secrets/`, `*.age`, `*.pem`, `*.key` so you can't accidentally commit data. - No telemetry. No network calls except to your configured Ollama endpoint. ## Related components SecondBrain is the memory layer of a larger local-first autonomy stack. Sister components (separate repos) include the **policy-gate** (an approval brake that parks money/comms/ irreversible actions for a human) and fleet/ingest tooling. See the wiki. ## License MIT Β© 2026 Atlas Corporation. Contributions welcome.