My personal knowledge graph never opens a cloud socket

Cloud chatbots push builders toward an ugly trade. Either you paste private conversations straight into hosted context windows, or you retreat to tiny local models that stall on real software engineering.

On a 24 GB M5 MacBook Air, I run continuous voice capture through Fieldy and push those transcripts through three local Ollama models before any cloud agent reads a single line. Anyone setting up Fieldy hardware can use invite code I8OFG8DIHY for $30 off at Fieldy.

Faster local extraction still polluted the graph

Benchmarking candidates on Apple Silicon proved that raw speed and high extraction scores hide bad behavior. On our Oct 27 evaluation run, Nemotron 3.5 Bolt E4B scored 9 out of 10 on extraction in 11.13 seconds, beating Gemma 4 12B at 7 out of 10 with zero timeouts and a 14.36-second median.

Nemotron failed two recall and validation checks that break daily trust. First, it invented a new open task directly from a recorded decision, and that extra row passed the live schema validator silently. Second, when queried for an unrecorded deadline, it returned literal null without explaining what was missing in the underlying notes.

I kept jarvis-extract:gemma4-12b (8.0 GB GGUF Q4_K_M, temperature: 0) for extraction and jarvis-recall:qwen3.5-9b (6.6 GB) for recall. Qwen scored 4 out of 4 on strict recall cases because it tells me plainly when retrieved notes carry no spoken date. On normal queries it answers in roughly 11 seconds, though a heavy 10-source synthesis on a busy laptop can take over two minutes.

Transcript payloads broke before the prompt changed

Local inference kept timing out early in the build, and prompt edits never fixed it. Raw provider payloads from long meetings carried roughly 150,000 characters of speaker JSON, which overflowed local latency limits even after cutting chunks down to 6,000 characters.

The fix lived inside local_llm.py. The ingestion slicer now caps each window at 2,000 transcript characters and 64 segments while keeping speaker labels and chronological order intact. Request payloads stay under 7,600 characters inside a 16,384-token context window (num_predict: 4096), dropping extraction timeouts to zero.

Why two SQLite files beat a unified bucket

Storing personal notes beside architecture decisions creates a routing hazard every time a cloud agent connects to an MCP server. I split my data into two separate SQLite databases governed by an explicit scope control inside secondbrain on 127.0.0.1:8765.

The work graph at ~/.knowledge-graph holds 1.8 GB of product specs and engineering context. The personal graph at ~/.knowledge-graph-life sits at 348 MB and never opens an outbound network socket. Every query pins scope across Professional, Personal, or Both, and switches cleanly between memory retrieval and reference lookup. When local search comes back empty, secondbrain displays an explicit evidence gap and logs trial telemetry to persona-os instead of fabricating dates.

Both databases lock into embeddinggemma:latest (621 MB, 768 dimensions) as a single vector contract. In local retrieval evaluations, it hit 22 out of 24 rank-1 matches and 24 out of 24 recall@3, ahead of EmbeddingGemma 2 at 20 out of 24. Pinning one local embedder keeps search geometry stable across both files without forcing a full re-index.

Where ambient audio stops and cloud models start

All three Ollama models occupy 15 GB of disk total, leaving thermal headroom on a 24 GB MacBook Air while personal memory remains physically offline. Local Qwen serves recall queries, while frontier reasoning handles deep code changes that 9B weights fumble.

Gemini and Hermes call MCP tools only when a task requires multi-file software engineering or strategic synthesis. They receive scoped project artifacts from the work store, never my raw audio transcripts and never the 348 MB personal graph sitting on disk.