add Connectors & Setup Wizard page

atlas 2026-07-01 19:50:21 +00:00
parent 4a81ca3cd7
commit 7381f0d24b

59
Connectors.md Normal file

@ -0,0 +1,59 @@
# Connectors & Setup Wizard
`brain.py` is the engine. **Connectors + the setup wizard** are the product layer that turns it
into a real "second brain" — plugging into the apps where your life actually lives, instead of
manually pointing at directories.
```
your apps ──▶ Connector ──▶ scrub() + _stage() ──▶ brain.db ──▶ recall
(mail, cloud, (auth+pull) (SAME engine path) (hybrid)
files, chat…) │
└── app credentials ──▶ encrypted vault (NEVER in brain.db)
```
## Three-layer design (why it's built this way)
1. **Engine** (`brain.py`) — pure, connector-agnostic, unchanged. Still usable as a bare library.
2. **Connectors** — one per app. Each authenticates, pulls data, and feeds it through the engine's
**same scrub + dedup path** — so secrets inside emails/files are redacted before storage too.
3. **Vault** (`connectors/vault.py`) — app logins are encrypted at rest (Fernet + PBKDF2, 390k
iters, `~/.secondbrain/vault.enc`, chmod 600). **`brain.db` never sees a credential.**
This preserves SecondBrain's core promise — local-first, secrets-scrubbed — while adding real
onboarding. (Verified: a `DB_PASSWORD=…` inside a synced file is scrubbed out of `brain.db`.)
## The setup wizard
```bash
python3 setup.py # pick apps → sign in → test connection → sync
python3 setup.py sync # re-sync all connected apps (cron/systemd timer)
python3 setup.py list # list available connectors
```
Set `BRAIN_VAULT_PASS` to run the `sync` non-interactively from a timer.
## Connectors today
| Connector | Auth | Ingests as |
|---|---|---|
| **Files & folders** | path | files |
| **Email (IMAP)** | host + app password | events |
| **Nextcloud / WebDAV** | URL + app password | files |
## Roadmap connectors
- **Google & Microsoft** (Gmail/Drive/Calendar, Outlook/OneDrive) — OAuth; needs the wizard's
local-web variant for the redirect + a registered app client id/secret.
- **WhatsApp** (via Evolution API), **Calendar** (CalDAV), **Paperless-ngx**, **Odoo** (CRM/ERP),
**Obsidian** (vault folder), **Git/GitHub** (repos + issues).
## Writing a connector (~40 lines)
```python
from .base import Connector, Field, register, feed
@register
class MyAppConnector(Connector):
id = "myapp"; name = "My App"; blurb = "Pull stuff from My App."
fields = [Field("host","Host"), Field("token","API token", secret=True)]
def test(self, cfg): ... # verify creds, raise on failure
def sync(self, cfg): # pull -> feed() -> engine scrub+stage
items = [{"text": "...", "ref": "...", "ts": "...", "project": "myapp"}]
return feed(items, "events")
```
Drop the file in `connectors/`; the wizard auto-discovers it. `feed()` runs everything through the
engine's credential scrub — you never have to think about redaction.