secondbrain/README.md

🧠 SecondBrain

A local-first, privacy-scrubbing personal memory you can point at everything you produce.

Notes · code · agent transcripts · event logs → one portable file → instant hybrid recall. No cloud. No API keys. No data egress. Secrets scrubbed before anything is stored.

An Atlas Corporation product · © 2026 Atlas Corporation · Proprietary

---

SecondBrain is a tiny (~500-line, single-file) knowledge engine. You feed it the things you already generate — documents, source code, AI-agent chat transcripts, structured event logs — and it gives you back one command:

brain recall "what did we decide about the pricing model" --hybrid

Everything runs on your own machine. Embeddings are computed locally by Ollama (nomic-embed-text), and the entire brain is a single portable brain.db (SQLite + sqlite-vec) you can copy to a USB stick. There is no server to run, no account to create, and no credential ever leaves your box — because SecondBrain redacts them before embedding.

Why it exists

Vector-RAG demos are easy; a memory you'd actually trust with your real life is not. SecondBrain is opinionated about the three things that usually break:

Problem	SecondBrain's answer
Secrets leak into your index (and then into an LLM's context)	A mandatory scrub pass strips PEM keys, JWTs, sk-ant-…/sk-…/ghp_…/AWS keys, and any password: / token= / bearer … pattern before a chunk is stored. IPs, hostnames and emails are kept as operational knowledge (toggle with SCRUB_KEEP_PII=0).
Dense-only search misses exact strings (invoice numbers, IBANs, container names, file paths)	Hybrid recall fuses dense KNN with FTS5/BM25 lexical search via reciprocal-rank-fusion — meaning and identifiers both hit.
Cloud RAG = data egress + lock-in + cost	100% local. Ollama for embeddings, SQLite for storage. Portable, free, offline-capable, redeployable on any device in minutes.

The senses

SecondBrain ingests from pluggable "senses" — each strictly read-only:

• ingest-files <dir> — code, docs and config (60+ text extensions; skips node_modules, .git, binaries, and files > 400 KB).
• ingest-transcripts <dir> — *.jsonl AI-agent sessions (Claude Code–style event logs); tool spam is collapsed to signal.
• ingest-events --db <sqlite> — any SQLite with an events(id, ts, sense, subject, payload) table (e.g. email, chat, calendar, finance). Turns real activity into recallable memory.

Add your own sense in ~10 lines: chunk → scrub() → _stage(). That's the whole contract.

Quickstart

# 0. prerequisites: python3, and Ollama running (https://ollama.com)
ollama pull nomic-embed-text

# 1. install
git clone https://git.atlascorporation.nl/atlas/secondbrain
cd secondbrain
bash deploy/install.sh          # venv + deps + schema
. .venv/bin/activate

# 2. feed it something
python3 brain.py ingest-files ~/notes
python3 brain.py ingest-files ~/code/my-project

# 3. embed (local, free)
python3 brain.py embed

# 4. recall
python3 brain.py recall "how does the auth flow work" --hybrid
python3 brain.py stats

Output:

#1  score=0.72  via=both rrf=0.0312  [file/my-project]  auth.py
   def login(user, pw):  # verifies against argon2 hash, issues a signed …

Setup wizard — connect your apps

brain.py is the engine; the setup wizard is the product experience. Instead of manually pointing at directories, run:

python3 setup.py            # pick apps → sign in → test → sync
python3 setup.py sync       # re-sync everything (put on a cron/timer)
python3 setup.py list       # available connectors

The wizard walks you through connecting the apps that hold your life, stores each app's login in an encrypted vault (connectors/vault.py — Fernet/PBKDF2, ~/.secondbrain/vault.enc, chmod 600), tests the connection, and syncs. App credentials never enter brain.db — connectors pull data and only scrubbed text reaches the brain.

Connectors today: Files & folders · Email (IMAP) · Nextcloud / WebDAV. On the roadmap: Google & Microsoft (OAuth) · WhatsApp (Evolution) · Calendar (CalDAV) · Paperless-ngx · Odoo · Obsidian · Git/GitHub. A connector is ~40 lines — subclass Connector, declare its fields, implement test() + sync(); feed() runs it through the engine's scrub.

Architecture note: the engine stays pure and connector-agnostic. Connectors + vault + wizard are an optional layer on top — SecondBrain still works as a bare brain.py library/CLI without them.

Configuration

All via environment (or a .env next to brain.py). Copy .env.example → .env:

Var	Default	Meaning
BRAIN_HOME	~/.secondbrain	where brain.db lives
BRAIN_DB	$BRAIN_HOME/brain.db	explicit db path (overrides BRAIN_HOME)
OLLAMA_URL	http://localhost:11434	any reachable Ollama — including a GPU box on your LAN/tailnet
EMBED_MODEL	nomic-embed-text	embedding model
EMBED_DIM	768	must match the model
SCRUB_KEEP_PII	1	0 also strips emails (secrets always scrubbed)

Tip — offload embeddings to a GPU: point OLLAMA_URL at another machine's Ollama (http://100.x.y.z:11434). SecondBrain will embed there and store locally. This is how you keep a laptop's memory current using a desktop GPU, with zero extra infrastructure.

Deploy anywhere

systemd (continuous refresh): set your sources in .env (SB_FILE_DIRS, SB_TRANSCRIPTS, SB_EVENTS_DB), then:

sudo cp deploy/secondbrain-refresh.* /etc/systemd/system/
sudo systemctl enable --now secondbrain-refresh.timer   # ingest+embed every 15 min

Docker:

docker build -t secondbrain -f deploy/Dockerfile .
docker run --rm -v sbdata:/data -e OLLAMA_URL=http://host.docker.internal:11434 \
  secondbrain recall "quarterly numbers" --hybrid

Plug into any AI (MCP server)

Expose your brain as tools to any MCP host — Claude Desktop, Open WebUI, LibreChat, Cursor, etc. — so the assistant gains local, private, long-term memory:

pip install "mcp[cli]"
python3 mcp_server.py           # stdio  — Claude Desktop, Cursor
python3 mcp_server.py --http    # Streamable HTTP (MCP_HOST/MCP_PORT) — Open WebUI, LibreChat, web UIs

Add to your MCP host's config:

{ "mcpServers": { "secondbrain": {
    "command": "python3", "args": ["/opt/secondbrain/mcp_server.py"],
    "env": { "BRAIN_HOME": "/opt/secondbrain/data", "OLLAMA_URL": "http://localhost:11434" } } } }

Tools exposed: recall (hybrid search of your memory), remember (store a note, scrubbed), ingest_path (index a folder), stats. This is how SecondBrain becomes a first-class connector in a Claude-style product surface — the same way MCP servers extend Claude.

Use it as a library

import brain
for hit in brain.brain_recall_hybrid("open invoices for June", k=8):
    print(hit["score"], hit["ref"], hit["text"][:120])

brain_recall_hybrid() degrades gracefully to dense-only if the lexical index is empty, so it's a safe drop-in for brain_recall().

How it works

  sources ──▶ scrub() ──▶ chunk ──▶ chunks table ──┬─▶ FTS5 (BM25, via triggers)
 (senses)   (redact)   (≤1600 ch)                   └─▶ vec_chunks (nomic 768-dim, Ollama)
                                                          │
   recall("q") ──▶ embed query ──▶ dense KNN ┐            │
                └▶ FTS5 lexical ─────────────┴─ RRF fuse ─┴─▶ ranked hits

See docs/ARCHITECTURE.md for the full design, the scrub guarantees, and the reciprocal-rank-fusion math.

Privacy & security

• Scrub-before-store is not optional and runs on every chunk from every sense. rescrub re-applies a hardened scrub to an existing db in place.
• The database never leaves your machine unless you copy it. .gitignore blocks *.db, .env, secrets/, *.age, *.pem, *.key so you can't accidentally commit data.
• No telemetry. No network calls except to your configured Ollama endpoint.

Related components

SecondBrain is the memory layer of a larger local-first autonomy stack. Sister components (separate repos) include the policy-gate (an approval brake that parks money/comms/ irreversible actions for a human) and fleet/ingest tooling. See the wiki.

License & ownership

© 2026 Atlas Corporation. All Rights Reserved.

SecondBrain is proprietary, confidential software owned by Atlas Corporation. It is provided for reference and for use only under a written agreement with Atlas Corporation. No right to use, copy, modify, redistribute, or create derivative works is granted absent such an agreement — see LICENSE for the full terms. "Atlas Corporation" and its marks are the property of Atlas Corporation.