What gbrain is actually doing in the background

01Overview

gbrain is a markdown-vault-backed knowledge database with a self-driving maintenance daemon. Your notes live as plain .md files in an iCloud folder — that's the truth, and it stays human-editable. gbrain continuously mirrors them into Postgres, chunks and embeds them for semantic search, runs LLM passes to extract links, facts and ideas, and rolls those up into higher-level pages.

It then exposes the whole thing back to Claude Code as roughly 150 MCP tools, so any agent session can query your brain. The "consolidation" question has a specific answer: a pipeline that runs source note → atom → concept. That pipeline exists and works. It is currently stalled.

                    ┌──────────────────────────────────────┐
                    │  Postgres  (localhost:5432/gbrain)   │
                    │  pages · chunks · embeddings ·       │
                    │  minion_jobs (the queue)             │
                    └───▲──────────▲──────────────▲────────┘
       enqueues only    │          │ leases+runs  │ reads/writes
                        │          │              │
              ┌─────────┴───┐  ┌───┴──────┐  ┌────┴─────────┐
              │  autopilot  │  │  worker  │  │  serve-http  │
              │  scheduler  │  │ executor │  │   gateway    │
              └─────────────┘  └──────────┘  └──────▲───────┘
                                                    │ HTTP / OAuth 2.1
                                            Tailscale 100.120.101.14:8787

The three daemons are deliberately not one process. Autopilot writes job rows; the worker claims and runs them. Kill either and the other keeps functioning — jobs simply pile up, or stop arriving. That isolation is what kept your scheduler healthy while the executor died silently.

02Where everything lives

Brain/
├── media/      816 files   ← YouTube / article transcripts + summaries (dominant)
├── life/        87 files   ← diary/ and events/
├── projects/    38 files   ← product ideas, app concepts
├── notes/       37 files   ← thinking notes by topic
├── writing/     37 files   ← your essays (mostly Chinese)
└── wiki/         2 files   ← analysis / reference

03Input — how things get in

Five doors. Two are in active use.

The vault — effectively 100% of your data

Drop a .md file anywhere under Brain/. The sync phase picks it up on the next cycle — incremental and hash-based, so unchanged files cost nothing. Frontmatter drives typing, and directory prefixes auto-type too: media/ → source, life/diary/ → diary, projects/ → project.

gbrain capture — the inbox door

gbrain capture "some thought"              # inline
gbrain capture --file ~/Downloads/x.md     # a file
pbpaste | gbrain capture --stdin           # piped

Lands in inbox/; the cycle types and files it from there.

Agent writes

Via MCP — put_page, put_raw_data, add_link, add_tag, add_timeline_entry, file_upload. This is how an agent files something on your behalf mid-conversation.

Code indexing unused

gbrain sync --strategy code indexes a repo's symbols so you can ask code-def, code-callers, code-callees across it. You have one source and it's the vault.

Integrations — "senses"

04The maintenance cycle — 21 phases

This is the heart of it. gbrain dream runs it once; autopilot runs it forever. The order is semantic, and stated as such in the source: fix the files first, then index them, then reason over them. Numbering here is load-bearing — each phase depends on fresh state from the one above.

The global / per-source split

Phases scoped global — embed, orphans, purge, grade_takes, calibration_profile, synthesize_concepts, skillopt, resolve_symbol_edges — run once in a separate autopilot-global-maintenance job. Everything else runs per-source. That split exists because running the global phases concurrently per source once blew RSS from 4 GB to 10 GB.

Locking is a Postgres row with a 30-minute TTL, refreshed between phases. Read-only phases skip it.

05The consolidation pipeline

This is the direct answer to "what does gbrain do to consolidate my notes." It is not merge many notes into fewer notes. It is: distil each source into its irreducible ideas, then cluster those ideas across all sources into concepts you actually hold.

media/*.md  (816 transcripts)
     │
     │  extract_atoms — Haiku reads each source, pulls out distinct ideas
     ▼
atoms/<date>/<slug>.md  (19 so far)
     │   type: atom
     │   frontmatter: lesson, concepts[], atom_type, source_slug,
     │                source_quote, virality_score, emotional_register
     │
     │  synthesize_concepts — dedup → tier → Sonnet narrative, voice-gated
     ▼
concept pages  (0 so far)

A real atom from your brain

---
type: atom
title: Revenue = views × conversion rate
lesson: Once format-market fit is proven, app growth stops being a product
        problem and becomes a pure capital and distribution problem.
concepts: [growth-equation, distribution-economics, unit-economics]
atom_type: framework
source_slug: media/0-to-300kmo-in-45-days-with-my-ai-app-just-copy-me
source_quote: "Revenue is equivalent to the number of views we get times
               the conversion rate..."
virality_score: 60
emotional_register: practical
---

The gbrain-everything schema pack — creator, investor and engineer lenses stacked on one brain — is what enables both phases. Other packs skip them entirely.

06Daemon 1 — autopilot, the scheduler

Runs

~/.gbrain/autopilot-run.sh → gbrain autopilot --no-worker --repo <vault>

launchd

RunAtLoad · KeepAlive · ThrottleInterval 60

Logs

stdout → autopilot.log · stderr → autopilot.err

Status

PID 24092, up since 12:25 — healthy

A while(true) loop that decides what work is due and writes job rows. With --no-worker it executes nothing itself.

 1. refresh ~/.gbrain/autopilot.lock mtime      single-instance guard
 2. DB health probe → reconnect if needed       classified: recoverable vs fatal
 3. self-upgrade check                          no-op; mode=notify
 4. --no-worker peer-liveness probe             is any worker holding a lease?
 5. per-source freshness check                  → submit `sync` if stale
 6. extract-atoms auto-drain gate               → submit drain if backlog > 25
 7. compute brain score + remediation plan
 8. route: sleep | targeted jobs | full cycle fan-out
 9. re-read health → pick next interval
10. nightly quality probes                      both disabled
11. sleep

Step 8 — the routing decision

Autopilot doesn't blindly submit a full cycle every tick. It computes a remediation plan from engine.getHealth() plus onboard checks — embed staleness, link coverage, timeline coverage, takes count — then routes:

Your brain scores 47, so it takes the last branch on every single tick. Fan-out then dispatches one autopilot-cycle per stale source (capped at fanoutMax, clamped to worker concurrency minus a reserved slot) plus exactly one autopilot-global-maintenance.

Step 9 — adaptive interval

interval = score >= 90 ? base * 2
         : score <  70 ? max(base / 2, 60)
         : base

Base is 300s. At score 47 you get 150s — which is exactly what the log shows.

Anti-thrash machinery

Why --no-worker

Your run-script sets it deliberately, and the comment explains why: autopilot's built-in ChildWorkerSupervisor pipes the child worker's stdout, the worker emits a ~1 KB handler-registration line at startup, the pipe fills, and the worker blocks on write forever — observed as a worker pinned at 0.2s CPU while the queue backs up. Running the worker as its own launchd service, writing straight to a file, removes the pipe.

The trade-off is that autopilot can no longer restart a dead worker. It compensates with the step-4 probe:

SELECT count(*) FROM minion_jobs
 WHERE status='active' AND lock_until > now() - interval '2 minutes'

Zero for N consecutive ticks fires one loud warning. That warning has fired. See §11.

07Daemon 2 — worker, the executor

Runs

~/.gbrain/worker-run.sh → gbrain jobs work --concurrency 1 --max-rss 8192 --repo <vault>

launchd

RunAtLoad · KeepAlive · no throttle

Logs

stdout + stderr → worker.log

Status

PID 42553, up since 13:23 — HUNG

MinionWorker — a BullMQ-shaped job runner. Polls Postgres, claims a job with a lease, runs its handler, renews the lease while running, reports success or failure. 39 handlers registered:

sync · embed · lint · lint-fix · import · extract · extract_facts · extract-ner
extract-takes-from-pages · extract-timeline-from-meetings · extract-conversation-facts
extract-atoms-drain · backlinks · facts-absorb · chronicle_extract · enrich
synthesize · patterns · consolidate · resolve_symbol_edges · recompute_emotional_weight
autopilot-cycle · autopilot-global-maintenance · embed-backfill · embed-catch-up
contextual_reindex_per_chunk · reindex · orphans · purge · integrity · integrity-auto
repair-jsonb · unify-types · skillopt · sync-retry-failed · ingest_capture
subagent · subagent_aggregator · shell

Defaults it runs with

Per-job lifecycle

claim (SELECT … FOR UPDATE SKIP LOCKED, filtered to registered handler names)
  → set status=active, lock_until=now()+lease
  → start lock-renewal timer            heartbeat; proves the worker is alive
  → start AbortController               timeout / shutdown / lock-loss
  → run handler
  → success → status=done
     failure → exponential backoff retry up to max_attempts → then `dead`

Two failure classes are handled differently, deliberately:

• Job defect — handler threw. Burn an attempt, back off, eventually dead-letter.
• Infrastructure — lock-renewal-failed, lock-lost, a Postgres blip. Don't burn an attempt; let the stall detector requeue it cleanly.

The shell handler registers in guarded mode — it refuses to execute anything unless GBRAIN_ALLOW_SHELL_JOBS=1. A good default.

Two environment traps its run-script works around

1. Don't cd into the vault. ~/Library/Mobile Documents is TCC-protected; a launchd agent has no TTY to answer a consent prompt, so bun blocks at startup just touching cwd. The script cds to ~/.gbrain and passes --repo instead.
2. PATH and credentials. launchd's PATH lacks /opt/homebrew/bin, so claude resolves to NOT_FOUND and every LLM-backed job fails with "all provider calls failed this batch". The script pins GBRAIN_CLAUDE_CLI_BIN absolutely and fixes PATH, so neither alone is load-bearing.

Hold onto trap 1. It is the thread that unravels in §11.

08Daemon 3 — serve-http, the gateway

Runs

gbrain serve --http --bind 100.120.101.14 --port 8787

Status

PID 32199, up since 11:37 — healthy

An MCP server over HTTP with OAuth 2.1, bound to your Tailscale IP only. The run-script comment is explicit about why:

Bound to the Tailscale interface ONLY — never 0.0.0.0. The Air reaches it over the tailnet; the coffee shop does not.

Port:      8787
Bind:      100.120.101.14        ← tailnet only
Engine:    postgres
Clients:   4 registered
DCR:       disabled              ← no dynamic client registration
Skills:    published             ← serves the skill pack to connecting agents
Token TTL: 3600s

/mcp     MCP endpoint
/admin   admin UI (bootstrap token from $GBRAIN_ADMIN_BOOTSTRAP_TOKEN)
/health  liveness

It does no scheduled work. Request and response only: a remote agent connects, authenticates, calls tools, gets answers — the same ~150 MCP tools the local stdio server exposes.

Note there are two MCP servers running. This Claude Code session talks to a separate gbrain serve over stdio, launched from your terminal. serve-http exists for other machines — your Air, your phone, a cloud agent.

Security posture is sound: tailnet-bound, OAuth with 1-hour tokens, DCR off so a stranger can't self-register a client, skills published deliberately.

09Operations — what you can drive

CLI — the useful subset

# Read / search
gbrain query "what do I think about pricing"   hybrid: keyword + vector + RRF
gbrain search "pricing"                         keyword only (fast)
gbrain get <slug>
gbrain list --type atom -n 20

# Write
gbrain capture "thought"                        → inbox/
gbrain put <slug> < file.md
gbrain link <from> <to> --link-type supports

# Think
gbrain brainstorm "<question>"                  bisociation: hybrid search + far-set + judge
gbrain lsd "<question>"                         inverted judge — rewards far-from-obvious
gbrain salience --days 30                       what's been emotionally / activity-hot
gbrain anomalies --since 2026-07-01             statistical outliers vs cohort
gbrain orphans

# Operate
gbrain dream                                    run one full cycle now, foreground
gbrain dream --phase extract_atoms --drain      drain one phase's backlog
gbrain sync --all
gbrain jobs list / stats / retry <id> / cancel <id>
gbrain health / stats / doctor
gbrain export --dir ./out/                      DB → markdown round-trip
gbrain publish <page.md> [--password]           shareable HTML, strips private data

MCP tools — roughly 150, grouped

The operation you never invoke

retrieval-reflex, currently active, has two halves:

1. A deterministic pointer layer — zero-LLM, on by default. Every conversation turn is scanned for salient resolvable entities; if one matches a brain page, a compact pointer (name → slug → one-line summary) is injected into the agent's context. The agent then knows the page exists and can open it.
2. A policy skill — prose telling the host agent when to look something up and what to pull.

Without it, an agent can discuss a person you have a rich page on for ten messages and never open it. Related: the volunteer_context and volunteer_chronicle MCP ops, and gbrain watch — pipe conversation turns in, volunteered pages stream out.

10Output — what it produces

New pages written back to the vault

DB-only structures — queryable, not files

• Chunks and embeddings — 5,884 chunks, 100% embedded
• Links — typed edges with provenance
• Facts — extracted assertions with an audit trail. Never deleted, only marked consolidated_into.
• Takes — synthesized opinions, gradeable, with a resolution scorecard
• Calibration profile — 2–4 narrative statements about how your judgment skews, across seven domains: concept_themes, deal_success, founder_evaluation, market_call, architecture_calls, effort_estimates, risk_assessment
• Chronicle — day-level timeline of what happened

Edits to your existing files

lint --fix and backlinks --fix modify your markdown in place every cycle. Worth knowing plainly: the daemon has write access to your iCloud vault.

11Root cause — why nothing has run since 13:23

$ ps -o pid,stat,%cpu,time,etime -p 42553
  PID STAT %CPU     TIME  ELAPSED
42553 S     0.0  0:00.39  08:05:29     ← 0.39s CPU across 8 hours

$ sample 42553
1553/1553 samples, main thread:
  …
  openat$NOCANCEL       (in libsystem_kernel.dylib) + 76
    __openat_nocancel   (in libsystem_kernel.dylib) + 8

[gbrain phase] sync.resolve_repo
[gbrain phase] sync.load_active_pack
[gbrain phase] sync.validate_repo_state
[gbrain phase] sync.discover_git_root      ← never reached sync.detect_head

The diagnosis

Main thread hung means single-threaded death: no lease renewal, no polling, no stall detector, no RSS watchdog — every safety mechanism lives on that same blocked thread. The process looks alive to launchd, which sees a running PID, and burns no CPU, so nothing restarts it.

The cause is the exact hazard your own worker-run.sh documents:

~/Library/Mobile Documents is TCC-protected; a launchd agent has no TTY to answer a permission prompt, so bun blocks at startup touching cwd.

The fix was applied to cwd but not to --repo — and --repo is the iCloud vault. So the first phase that actually walks the filesystem hits the same wall, one layer deeper. From your terminal, git and file ops on the vault complete in 0.03s, because terminal-launched processes inherit the terminal's TCC grant. The launchd agent has none.

Why nothing alerted you

12Current state

92 waiting, 2 active (stalled), 482m since last completion
⚠  WEDGED QUEUE 'default': live-lock
Last successful sync: 2026-08-09T05:20Z   vault is 8h stale in the DB

Jobs, last 24 hours

Secondary findings

13Next actions

1. Grant Full Disk Access to the bun binary. System Settings → Privacy & Security → Full Disk Access → add /Users/eugenechan/.bun/bin/bun — the real binary, not the gbrain script. This is the one step that needs you.
2. Kick the worker and clear the backlog.

   launchctl kickstart -k gui/$(id -u)/com.gbrain.worker
   gbrain jobs prune            # 92 waiting, mostly duplicate autopilot-cycles
   gbrain jobs retry 30 33 36   # the dead extract-atoms-drain jobs

3. Drain the atom backlog in the foreground, where you can watch it fail: gbrain dream --phase extract_atoms --drain. The 300s timeout on job #36 suggests the drain window needs raising for an 816-page corpus.
4. Export atoms to disk so the pipeline's output is visible in iCloud: gbrain export --restore-only --type atom.
5. Build a watchdog that reaches you. A daily launchd job running gbrain jobs stats --json and gbrain health --json, pushing a notification when the queue is wedged, last-completion age exceeds an hour, or dead jobs appear. Same shape as your existing simulator-cleanup and tmux-cleanup jobs.
6. Switch models.tier.subagent to an Anthropic API model to get prompt caching back.
7. Fix the graph, once the pipeline is running. 956 orphans and 0% link coverage means query works but traverse_graph, find_trajectory and find_experts have almost nothing to walk.
8. Only then consider a new sense — calendar-to-brain or email-to-brain. Adding input to a stalled pipeline just makes the backlog worse.