OpenWorker source analysis: how an “AI coworker that ships finished work” is built

Contents

1. Project overview: not a coding assistant, a “digital coworker”
2. Panorama: four layers + three languages + one event stream
3. Startup: three entry points and a supervised sidecar
4. Input capture and routing: desktop, Slack @mentions, unattended
5. Context assembly: persona + memory + skills + per-turn ephemeral context
6. The agent loop (the heart): TurnEngine concurrency, interrupt, durable resume
7. Tools and permissions: aisuite tools + five modes + standing rules
8. Memory: explicit SQLite facts, not automatic compaction
9. Sub-agents and multi-agent: persona is the agent + read-only explorer + self-wake
10. Ecosystem: persona marketplace, skills, MCP, 40 connectors, a dozen-plus models
11. Features: inbox, scheduled automation, model switching, PDF/vision fallback, audit
12. Summary: the three most distinctive designs and the trade-offs

---

Chapter 1 · Project overview: not a coding assistant, a “digital coworker”

What is OpenWorker? The first sentence of the README: an open-source AI coworker that delivers finished work, not chat. You tell it the outcome you want—“prepare a customer brief,” “put my calendar in order,” “where is this release in Jira and GitHub”—it breaks that into steps, works across the desktop, files, and connected apps, confirms with you before doing anything consequential (send a message, change a calendar, run a command), and finally hands you the thing, done.

Who builds it? Andrew Ng’s team. The README says “built on aisuite” (README.md:107), but note: aisuite is used only at the tool layer (ai.tool / ai.toolkits—see imports in about 21 files such as subagent.py). The provider layer is home-grown: providers/registry.py natively implements OpenAI / Anthropic / Gemini providers, plus _openai_compat (an OpenAIProvider pointed at each vendor’s OpenAI-compatible endpoint); router.py’s ProviderRouter routes on the provider: prefix. Inside providers/, aisuite appears only in comments, marked “later swap.” pyproject.toml pins aisuite to a specific commit (aisuite @ git+…@1b4bbf30).

Positioning and selling points (especially sharp against the eight coding agents):

• Local-first: the agent loop, conversations, connector tokens, model keys—all live in a local keystore on your machine (README.md:59-61). The only cloud piece is a small service that “brokers the OAuth handshake”; you can also skip login entirely and run connectors with credentials you type in by hand.
• Model-agnostic (BYO key): bound to nobody. OpenAI / Anthropic / Google Gemini / Inkling / GLM(Z.ai) / DeepSeek / Kimi / Qwen / MiniMax / Mistral / Grok(xAI), plus open-weight models on Together/Fireworks, plus Ollama fully local (README.md:53-55). There is a “curated list” of which ones have been verified to do tool-calling work.
• Ships finished artifacts: documents, tables, reports, web pages land as files you can open and share.
• Works from Slack: @OpenWorker in a channel, a session opens on the desktop, it uses your tools, the answer comes back as a thread reply.
• Can run on a schedule: morning brief, weekly report, watching a channel—recurring work is scheduled as automation, and when it finishes it lands in the app with a full verbatim transcript.
• Asks before it acts: writes, sends, and shell commands all need approval. Unattended runs park “things that need a person” in an inbox, rather than improvising.

🧠 One sentence: the eight coding agents compete on “who writes code better”; OpenWorker changed the question—“how do you let a model finish cross-app everyday work unattended, like a coworker, without causing a disaster?” The weight of the answer sits entirely on inbox, approval governance, connectors, and scheduling.

The package-name mismatch: the repo is called openworker, the Python package is coworker, the CLI command is again openworker (pyproject.toml [project.scripts]). Outwardly the product is OpenWorker; in the code it is coworker everywhere—this lecture cites source as coworker/… and the product as OpenWorker.

---

Chapter 2 · Panorama: four layers + three languages + one event stream

The README draws its own layered diagram (README.md:32-41). Split out, it is four layers, three languages:

flowchart TB
    subgraph L1["① 桌面壳 · Rust/Tauri（surfaces/gui/src-tauri）"]
        SHELL["原生窗口 + 自动更新 + 监管 Python 服务"]
    end
    subgraph L2["② 界面 · React（surfaces/gui/src）"]
        GUI["会话视图 · 工具卡 · 审批卡 · 收件箱 · 设置"]
        STT["Rust STT 侧车（stt/）：语音输入"]
    end
    subgraph L3["③ 本地 agent 服务 · Python（coworker/）"]
        MGR["SessionManager 编排"]
        ENG["TurnEngine 主循环"]
        PERM["权限引擎"]
        INBOX["收件箱（跨会话人类注意力队列）"]
        SCHED["调度器（定时自动化）"]
    end
    subgraph L4["④ 你的东西（全用你的密钥，在你机器上）"]
        FILES["文件 & 终端"]
        CONN["40 个连接器 + MCP"]
        MODEL["任意 provider 的模型"]
    end
    SHELL --> GUI --> MGR
    STT --> GUI
    MGR --> ENG --> PERM
    ENG --> INBOX
    MGR --> SCHED
    ENG -->|工具| FILES
    ENG -->|工具| CONN
    ENG -->|自研 provider 层| MODEL

• ① Rust/Tauri shell: native window, signing/notarization, auto-update, and supervises the Python service as a sidecar (surfaces/gui/src-tauri).
• ② React UI + Rust STT: sessions, tool cards, approval cards, inbox, settings live here; stt/ (Rust) is the speech-input sidecar.
• ③ Python agent service: the actual brain. Inside the coworker/ package, SessionManager (server/manager.py, 3762 lines) orchestrates sessions, TurnEngine (engine.py) is the agent loop, and the permission engine, inbox, and scheduler orbit it. It comes up with FastAPI + uvicorn (server/run.py).
• ④ Your stuff: files/terminal, connectors/MCP, models—all with your keys, on your machine.

The contract that runs through the stack: one event stream. TurnEngine does not return a result; it yields a sequence of Event (events.py): TURN_START → ASSISTANT_DELTA/REASONING_DELTA (streaming text/thinking) → TOOL_PROPOSED → PERMISSION_REQUIRED → TOOL_STARTED → TOOL_FINISHED → ITERATION_END → TURN_END (or ERROR/INTERRUPTED). GUI, TUI, and the Slack connector are all consumers of that same event stream—the UI just draws the events.

🧠 One sentence: welding the UI (Rust/React) to the brain (Python) with a single event stream is the root of “the same session is visible on the desktop, answers in Slack, and still runs unattended.” Anyone can consume the events; the events themselves do not care.

---

Chapter 3 · Startup: three entry points and a supervised sidecar

pyproject.toml declares three CLI entry points:

Desktop-mode startup is the interesting one: the Tauri shell pulls up the Python service as a sidecar and sets COWORKER_EXIT_WITH_PARENT=1—the moment the parent (GUI) dies, the service kills itself (server/run.py:_exit_when_orphaned). There is a pit they have already stepped in: under PyInstaller onefile packaging, the service is a grandchild of the GUI (a bootloader in between), so getppid() points at the bootloader, not the GUI, and you cannot detect this by “re-adopting the parent”—the GUI must pass its own PID explicitly in COWORKER_PARENT_PID; POSIX polls liveness with os.kill(pid, 0), Windows blocks on the process handle. The comment records that this bug used to leak a pair of service processes on every App exit.

Auth: a standalone openworker-server generates a one-shot token on each start and writes it to <state-dir>/sidecar-8765.token (user-readable only); API calls carry an X-OpenWorker-Token header. The desktop App uses an in-memory startup token that never hits disk (README.md:85-88).

Unified state: TUI, GUI, and the service share the same global store—state_dir()/coworker.db (SQLite memory) + ConversationStore (cli.py: SQLiteMemoryStore(data_dir/"coworker.db") + ConversationStore(data_dir)). A session you open in the terminal and a session you open on the desktop land in the same place.

flowchart LR
    U["用户"] -->|双击 App| TAURI["Tauri 壳"]
    TAURI -->|拉起侧车 + 传 PID| SRV["openworker-server
FastAPI+uvicorn"]
    U -->|终端| TUI["openworker (TUI)"]
    SRV --> STORE["统一全局库
coworker.db + Conversations"]
    TUI --> STORE
    TAURI -. 一死 .-> KILL["孤儿守卫：服务自杀"]
    SRV -. 监视父 PID .-> KILL

---

Chapter 4 · Input capture and routing: desktop, Slack @mentions, unattended

OpenWorker’s “input” sources are far more numerous than a coding agent’s—not just you typing in a box, but Slack/Telegram messages, scheduled triggers, self-wake. The routing key is that a session is born from exactly one persona, and after that every input is fed to that session’s TurnEngine.

The input paths:

• Desktop/TUI typing: directly engine.run(user_input, source=…). source is a display-only sidecar (metadata that came with a connector message); it travels with the persisted user message, but is stripped before the model is called (engine.py:_outbound_messages, see Chapter 5).
• Steering mid-session: engine.queue_steering(text) (engine.py:150)—does not interrupt the current turn; after this batch of tools finishes / the model finishes talking, the steering is injected as a new user message (_inject_steering, engine.py:866).
• Slack @mention → spawn a session (§31): this is the coworker-attribute’s punch line. When @OpenWorker is mentioned in a channel that has no subscribed session, the router spawns a coworker session that “owns this thread” and replies into it (mentions.py). The clever bit is the thread_target string ("slack:C0123:1700….000100"), one string, three jobs: ① the thread→session dedup lookup key; ② the delivery target for send_message; ③ the target for §25 standing authorization rules. And every time get_engine rebuilds the engine it re-pushes permissions.task_rules from that record—so the pre-authorization “reply in this thread without asking” survives a service restart.

flowchart TD
    MENT["Slack 频道里 @OpenWorker"] --> HAS{"该线程已有
订阅会话？"}
    HAS -->|有| ROUTE["路由到已有会话"]
    HAS -->|没有| SPAWN["生成'拥有此线程'的会话"]
    SPAWN --> TARGET["thread_target 一串三用"]
    TARGET --> T1["① 去重查找键"]
    TARGET --> T2["② send_message 投递目标"]
    TARGET --> T3["③ §25 标准授权 target
（重启后从 mentions 重推 task_rules）"]
    ROUTE --> ENG["喂给会话的 TurnEngine"]
    SPAWN --> ENG

• Unattended: a session can be set to “unattended.” Then the things it needs to ask a person do not pop inline; they go into a cross-session inbox (see Chapters 6 and 11). That is the routing switch for “it can run while nobody is there, and when it needs a person it parks the ask and waits for you.”

Attended vs unattended differs by a single visibility: VIS_INLINE (inline, answer in the input box, re-send on reconnect) vs VIS_INBOX (into the cross-session queue)—but underneath it is the same persistent, waitable record that any UI can answer (inbox.py:38-39).

---

Chapter 5 · Context assembly: persona + memory + skills + per-turn ephemeral context

Assembly all lives in agent.py’s build_engine (agent.py:109)—it turns a persona into a TurnEngine.

The static system prompt is concatenated in this order (agent.py:243-266):

persona.system_prompt
+ narrative guidance (before each tool batch, write one sentence of “what I am doing,” as live progress for the user, agent.py:78)
+ environment_context(workspace)
+ AGENTS.md (workspace conventions, if present)
+ memory guide + already-remembered facts (if a memory store is attached)
+ skill catalog (progressive disclosure: give the list first, load full text on demand with load_skill)

Tools are packed by family (this is the design that replaced “if-else on agent name,” agents/base.py:39):

Per-turn ephemeral context—this is the trick for stable injection across providers. Mid-stream system messages are unreliable across providers, so dynamic context is appended to the tail of the last user message, assembled only at send time, never persisted (engine.py:_outbound_messages:963-981). Two producers (agent.py:context_provider:297): ① plan/discuss mode reminders (mode can flip mid-session, so look it up each turn rather than baking it into the static prompt); ② a live directory list (an unowned Cowork session can pick up a new folder mid-session).

Sanitization before the model is called (engine.py:_outbound_messages:878)—self.messages is the only provider feed, but it has to be stripped clean first:

• Strip display-only sidecars: source (connector cards), _display (privacy-filter counts), ts (timestamps), reasoning (thinking text); drop entire notice messages (error / interrupt / model-switch markers).
• PDF attachments are re-adapted to the current model: models with native PDF support get the real document; everyone else gets a local extract-text / rasterize-pages fallback (pdf_support.py).
• Images likewise: a model without vision gets a visible placeholder [image attachment — not viewable by this model], not a payload it would reject.
• The key: these are re-judged on every call, so a mid-session model switch (Chapter 11) always does the right thing.

🧠 One sentence: history is “canonical OpenAI shape”; capability adaptation happens at send time, against the current model—that design makes “switch models whenever” and “attachments upgrade/downgrade with model capability” free byproducts, not special-case code.

---

Chapter 6 · The agent loop (the heart): TurnEngine concurrency, interrupt, durable resume

TurnEngine (engine.py:52) is where the project’s craft shows most clearly. It is an async loop, but it wraps blocking provider/tool calls in asyncio.to_thread so the loop (and the UI consuming events) never stalls. One user turn spans many “model ↔ tool” iterations until the model stops asking for tools, hits a gate, or is interrupted. Default iteration cap 150 (config.py:31; the engine dataclass default is 12, explorer sub-agent 10).

flowchart TD
    START["run(user_input)"] --> APPEND["追加用户消息 + TURN_START"]
    APPEND --> LOOP{"未达迭代上限?"}
    LOOP -->|否, 已到上限| MAXOUT["TURN_END: max_iterations"]
    LOOP -->|是| STREAM["_astream: 线程+队列桥接
流式吐 ASSISTANT_DELTA/REASONING_DELTA"]
    STREAM --> TOOLS{"模型要工具?"}
    TOOLS -->|否| STEER{"有插话?"}
    STEER -->|有| INJECT["注入 steering, 继续"] --> LOOP
    STEER -->|没有| DONE["TURN_END: completed"]
    TOOLS -->|要| AUTH["逐个鉴权（串行，审批是交互的）"]
    AUTH --> SPLIT{"按风险分流"}
    SPLIT -->|low risk 只读/搜索| PAR["asyncio.gather 并发跑"]
    SPLIT -->|写/shell/未标注| SER["严格按序一个个跑"]
    PAR --> REC["记录结果 + TOOL_FINISHED"]
    SER --> REC
    REC --> CANCEL{"被中断?"}
    CANCEL -->|是| INT["每个待跑工具补一个错误结果
INTERRUPTED（不留孤儿）"]
    CANCEL -->|否| LOOP

Four hard details:

1. Concurrent low-risk / serial high-risk (engine.py:_handle_tool_calls:437): when the model asks for several tools in one turn, read-only / search (metadata marked risk_level="low" and requires_approval=False) run concurrently (asyncio.gather); write / shell / anything unlabeled run strictly one-by-one in call order (_parallel_safe:517).

2. Interruptible from any state, no orphans (engine.py:request_interrupt:120): Stop can halt from any state—during streaming (the producer thread drops the stream between two chunks), during a tool (the interrupt hook kills a running shell command), while waiting for approval / a question / a plan (await resolves as “interrupted”), between iterations (loop checkpoints). Every still-unanswered tool_call is backfilled with a tool-error result—because hosted chat templates reject orphan tool_calls, and durable resume would otherwise pop them again.

3. Streaming bridge (engine.py:_astream:388): the provider’s blocking stream generator runs on a thread and stuffs chunks into an asyncio.Queue via loop.call_soon_threadsafe; the main loop races the queue against the Stop event, so a stuck stream (dry wait before the first token, a dead connection) cannot hold the whole turn.

4. Durable resume (engine.py:resume:250)—this is the foundation of “coworker”: a turn that hung at a prompt and was persisted can continue after a restart / after the engine was evicted. How: re-process the still-unanswered tool_calls in the last assistant message (_unanswered_trailing_tool_calls:268)—those prompt callbacks find already-resolved inbox items and return immediately, without popping again; already-answered calls are skipped, so nothing double-runs. Then the model loop runs again to close the turn.

There is also retry() (re-run after a provider error, standing on “the tail is an error notice,” engine.py:237) and switch_model() (mid-session model switch, Chapter 11).

🧠 One sentence: this engine treats “interrupt” and “resume” as first-class citizens—stop at any moment, leave no orphans when you stop, and after a restart continue from that “waiting for an answer” inbox item. Without those three, “unattended AI coworker” is just a slogan.

---

Chapter 7 · Tools and permissions: aisuite tools + five modes + standing rules

The tool substrate is aisuite. You can see it directly in tools/subagent.py: import aisuite as ai, tools wrapped with ai.tool(fn, metadata=ai.ToolMetadata(category=…, risk_level=…, requires_approval=…)), toolkits with ai.toolkits.files(root=ws) / ai.toolkits.git(root=ws) (subagent.py:60-64,129-137). OpenWorker stacks its own implementations on top of aisuite tools (windowed read_file, its own grep, and so on) and attaches ToolMetadata—that is where risk level comes from; the agent loop’s “concurrent / serial” split and the permission engine’s verdict both read it.

The permission engine’s five modes (permissions.py:37):

Verdict flow (permissions.py:evaluate:120):

flowchart TD
    CALL["工具调用"] --> RISK["classify: WRITE_LOCAL / EXEC / EXTERNAL / 低危"]
    RISK --> RO{"只读模式且有后果?"}
    RO -->|是| DENY1["拒：只读模式"]
    RO -->|否| WPATH{"是写且指定路径?"}
    WPATH -->|路径不在可写根内| DENY2["拒：不在可写目录"]
    WPATH -->|否/在根内| LOW{"无后果?"}
    LOW -->|是| ALLOW1["放行：低风险"]
    LOW -->|否| AUTO{"auto 模式?"}
    AUTO -->|是| ALLOW2["放行：全权限"]
    AUTO -->|否| ALLOWLIST["查命令白名单 / 会话放行 / §25 标准规则 / custom"]
    ALLOWLIST -->|命中| ALLOW3["放行 + 记审计"]
    ALLOWLIST -->|未命中| ASK["needs_user：弹审批"]

Several safety designs deserve their own sentence:

• Write-path constraint: if a write specifies path, it must land inside a writable root (_under_writable_root:204). Roots are mutable and re-read on every check, so adding or removing a folder at runtime takes effect immediately.
• Shell-whitelist injection defense (permissions.py:_command_allowed:216): the whitelist is “run without asking,” so prefix matching is unsafe—git status must not also allow git status && rm -rf ~. The move: any command that contains a shell operator (; & | > < ` $( ( newline) is refused the whitelist outright, then shlex.split argv is matched against whitelist entries as an exact token prefix (git status matches git status -s, but not git statusfoo or bare git).
• §25 standing rules (permissions.py:standing_rule_candidate:62): task-scoped {tool: {allowed targets}}, only for external risk (connectors), never for exec / local write—shell always asks. Why can connectors? Because they bind an exact target (for example “reply to this Slack thread”); only that target precision makes “auto-allow a connector tool” safe. Rules are seeded from automation tasks and re-read on every check, so clicking “always allow” mid-run also covers the next call of this run.
• Out-of-band approval (engine.py:_authorize:526): the engine only decides; it hands needs_user to an injected approver. Attended, the approver pops an inline card; unattended, the approver is inbox_approver (inbox.py:348), which turns the request into an inbox item and suspends waiting for a person. Approval results are once / always_tool / always_command / deny.
• _display privacy sidecar (engine.py:_record_result:642): the _display key in a tool result is metadata for the user to see, that the agent must never see (for example “the privacy filter hid N Gmail hits”—a count the model could use to probe). It is lifted onto the message sidecar, stripped from every provider feed, but persisted for the GUI’s tool card. The user can see an audit of “hid N rows / stripped N field values by the privacy filter,” but only category and count, never content.
• Independent send_file approval surface (§34) (agent.py:173): handing a deliverable into chat has its own approval surface—a thread’s standing authorization for send_message does not cover uploading a file.

🧠 One sentence: its obsession with “permissions” is coworker-grade: shell always asks (and injection is blocked for you), connectors can skip asking on an exact target, and the agent is not even allowed to know “how many the privacy filter hid.” This is not a coding assistant’s “may I edit this file,” it is “what rules should a coworker who can send messages on your behalf keep.”

---

Chapter 8 · Memory: explicit SQLite facts, not automatic compaction

Unlike Claude Code’s “automatic context compaction (microcompact),” OpenWorker has no automatic context-window compaction. Its “memory” is explicit, persistent SQLite facts:

• Store: SQLiteMemoryStore (memory/sqlite_store.py:13), two scopes Scope.GLOBAL / Scope.WORKSPACE, add/get/list/update/delete.
• Tools: remember / memory_update / memory_forget (agent.py:250-260 assembles memory_tools).
• Guidance on when to remember (agent.py:_MEMORY_GUIDANCE:62)—without it the model either never calls remember, or stores a pile of noise the repo already records: store only “user corrections and preferences (with reasons) + project background you cannot infer from code”; do not store what the repo already records (code structure, git history, AGENTS.md); use absolute dates, not “yesterday”; look up known memories first, and memory_update rather than stuffing near-duplicates; memories reflect the state at write time—verify a file / flag / URL still exists before citing it.

This “when to remember” guidance is almost homologous with the memory spec described in this repo’s Claude Code deep dive—both constrain the model’s memory behavior into disciplined persistent facts.

Then what about long-session context? Two things: ① a turn is bounded by max_iterations (default 150); ② history is persisted and reloadable by ConversationStore (conversations.py). There is no automatic summary-and-drop—this is a deliberate trade-off: rather let a person manage memory (explicit remember) than let the model drop context automatically. For a “coworker that ships finished work,” predictability matters more than saving tokens.

---

Chapter 9 · Sub-agents and multi-agent: persona *is* the agent + read-only explorer + self-wake

OpenWorker’s “multi-agent” has three layers:

① Persona is the agent. agents.get_agent delegates to the persona registry (agents/registry.py:19)—a persona is an agent. An Agent (agents/base.py:28) = name + title + system prompt + whether a workspace is required + tool factory + family (code/knowledge) + messaging/connectors flags. Core personas:

② Read-only explorer sub-agent (tools/subagent.py)—for the code family. A broad question (“where is retry logic handled?”) would burn the main session’s context on dozens of file reads. The explore tool opens a child TurnEngine, same workspace, read-only tools, fresh context, and returns only the final report to the caller; intermediate file reads never enter the main context. The child engine is forced into PLAN mode (the permission engine hard-blocks write/shell no matter what the sub-agent thinks), no approver (so it never needs an approval round-trip)—that is why explore can carry low-risk metadata, so multiple explores in one turn qualify for concurrent execution (Chapter 6). No recursion (the child registry has no explore tool).

flowchart LR
    MAIN["主会话（code persona）
上下文：要改的正事"] -->|explore 宽泛调研| CHILD["子 TurnEngine
全新上下文 · PLAN 只读 · 无审批"]
    CHILD --> SEARCH["grep / read_file / git_log …"]
    SEARCH --> REPORT["只回最终报告（path:line + 关键片段）"]
    REPORT --> MAIN
    MAIN -->|多个 explore 一起请求| PAR["因低危 → 并发跑"]

③ self-wake (selfwake.py, packed for the knowledge family at agent.py:240): a knowledge-family session can suspend itself + schedule a resume (on a timer / on completion / on an event). Each scheduler tick’s extra_tick resumes due wakes (automation/scheduler.py:85). That lets a coworker “sleep for now, wake me when X is done / when the time comes, and I’ll continue.”

---

Chapter 10 · Ecosystem: persona marketplace, skills, MCP, 40 connectors, a dozen-plus models

Persona marketplace (with a consent gate) (personas/registry.py): personas split into built-in (cowork/code/chat + markdown ops) and third-party. Third-party can be installed from a directory or a git repo (install_from_dir:340 / install_from_git:380); install snapshots the manifest into a managed area (definition stable, independent of the source directory), and they are disabled + not shown by default, enabled only after the user authorizes the declared capabilities (_enabled[id]=False)—never auto-enabled. A fresh install enables only cowork; everything else is opt-in from Settings. The manifest is markdown with YAML frontmatter (PersonaManifest); to_agent() becomes a runtime Agent.

Skills: Anthropic-format loadable capabilities that any agent can pull in (unlike personas—a persona is a top-level surface). Progressive disclosure: the system prompt first gets a skill-catalog list (skill_catalog_text); the load_skill tool loads full text on demand (agent.py:262-266). Skill directories come from state_dir/skills + workspace/.coworker/skills.

MCP (mcp/: client + config + oauth + tools): a built-in MCP client with an async layer, stdio + streamable-http, with OAuth (mcp/oauth.py). Any MCP-reachable tool can be plugged in, controlled per tool.

40 connectors (connectors/descriptors.py’s DESCRIPTORS): GitHub, Slack, Jira, Notion, Linear, HubSpot, Outlook, monday.com, Gmail, Google Calendar, Telegram, Asana, Confluence…. Plus terminal and local files. Connector tokens live in the local keystore; the cloud only brokers the OAuth handshake. integration_tools.py (4892 lines) is the largest single file—the center of gravity of the integration tools.

A dozen-plus models (providers/): ProviderRouter routes on the model’s provider: prefix (agent.py:214). Native providers: OpenAI, Anthropic (native Claude Messages API), Gemini. Plus a crowd via _openai_compat (OpenAI-compatible endpoints): DeepSeek, GLM(Z.ai), Kimi, Qwen, MiniMax, Grok(xAI), Mistral, plus Ollama fully local. providers/matrix.py has a curated “verified for tool calling” model list (anthropic:claude-opus-4-8, gemini:gemini-3.1-pro-preview, xai:grok-4.3…); capabilities.py records each model’s vision/pdf capability (Chapter 5’s adaptation looks this up).

flowchart TB
    AGENT["一个 persona 的 TurnEngine"] --> SKILL["技能：Anthropic 格式 · 渐进披露 · load_skill"]
    AGENT --> MCP["MCP 客户端：stdio + http + OAuth"]
    AGENT --> CONN["40 个连接器：GitHub/Slack/Jira/Notion/Gmail/GCal…"]
    AGENT --> PROV["ProviderRouter：按 provider: 前缀路由"]
    PROV --> NATIVE["原生：OpenAI · Anthropic · Gemini"]
    PROV --> COMPAT["OpenAI 兼容端点：DeepSeek/GLM/Kimi/Qwen/MiniMax/Grok/Mistral"]
    PROV --> LOCAL["Ollama：全本地"]
    MARKET["persona 市场"] -->|目录/git 安装 · 快照| GATE["默认禁用 + 待同意"]
    GATE -->|用户授权声明能力| AGENT

---

Chapter 11 · Features: inbox, scheduled automation, model switching, PDF/vision fallback, audit

① Inbox—the coworker’s defining primitive (inbox.py). A cross-session “human-attention queue”: while you are working in one session (or nobody is there and the session is running unattended), the inbox holds things other agents need you to do: approvals, questions, notices, directory grants, plan approvals. The state machine is an anti-race contract: each item pending → resolved, resolved once, idempotent, first answer wins—safe to answer from any UI (in-app, Slack, the input box after a restore) (inbox.py:295). The idempotency key is (session_id, tool_call_id) (inbox.py:133): when durable resume re-pops the same prompt, it reuses the existing (possibly already resolved) item rather than popping a new one—that is exactly why Chapter 6’s durable resume can exist.

stateDiagram-v2
    [*] --> pending: agent 需要人（审批/提问/授权/计划）
    pending --> resolved: 任意界面首次回答（谁先答谁赢）
    resolved --> [*]: 唤醒挂起的 agent（inbox_approver）
    note right of pending
        幂等键 (session_id, tool_call_id)
        重启后重弹 → 复用同一项，不双跑
        VIS_INLINE 内联 / VIS_INBOX 跨会话
    end note

② Scheduled automation (scheduler) (automation/scheduler.py:23): a scheduling loop that runs inside the always-on service. Policy: run-once-catch-up (due tasks missed while down are run once on boot, then resume) + skip-on-overlap (if the previous run is still going, do not stack another, _running_ids guards). Each due task is spawned, not awaited (_tick:76)—because a run may hang on an inbox pending-approval, and one stuck automation must never block the scheduling loop, other due tasks, or self-wake resumes. next_run is computed with croniter.

flowchart LR
    TICK["每 30s tick（首次=补跑）"] --> DUE["store.due() 到期任务"]
    DUE --> SPAWN["只 spawn 不 await（重叠跳过）"]
    SPAWN --> RUN["跑一个自动化会话"]
    RUN --> SUSPEND{"要审批?"}
    SUSPEND -->|是| PARK["挂进收件箱，不堵调度环"]
    SUSPEND -->|否| DELIVER["产出成品 + 逐字记录落应用"]
    TICK --> WAKE["extra_tick：恢复到期 self-wake"]

③ Mid-session model switch (engine.py:switch_model:180): history is canonical OpenAI shape, converted per provider per call, so switching models is just changing a field + a “switched here” notice. If history has images and the new model has no vision, the notice appends a fallback warning (those images go out as placeholders, see Chapter 5).

④ PDF/vision capability fallback (Chapter 5): re-judged on every call against the current model; native gets the real thing, otherwise local fallback—the companion of “switch models whenever.”

⑤ Audit trail (engine.py:_audit:690): every tool call is audited at multiple stages (proposed / auto_allowed / started / finished / approval_requested / approval_resolved / filtered……). §25 invariant: every auto-allowed call must cite its rule.

⑥ Local-first and secrets (secrets.py): model keys and connector tokens all in the local keystore; works without login (hand-filled credentials).

⑦ Speech input: stt/ (Rust) speech-to-text sidecar. ⑧ Self-update: the README says the App self-updates, so fixes reach installs quickly.

---

Chapter 12 · Summary: the three most distinctive designs and the trade-offs

Three things (if you remember only three designs)

1. Inbox + durable resume = the foundation of an “unattended AI coworker.” A cross-session human-attention queue, pending→resolved resolved once, idempotent (by session_id+tool_call_id), first answer wins, answerable from any UI; attended inline, unattended into the inbox. A turn can suspend at a prompt and persist, and after a service restart continue from that “waiting for an answer,” without double-running. Other agents’ loops stop at “I am done”; OpenWorker’s can stop at “I need you to nod”—and that “stop” survives a restart.

2. Provider-agnostic (home-grown provider layer; aisuite only at the tool layer). History is OpenAI-shaped, converted per provider per call; a mid-session model switch is just changing a field; PDF/vision capability is re-judged each turn against the current model (native gets the real thing, otherwise local extract-text / rasterize / placeholder). “Switch models whenever” and “attachments upgrade/downgrade with the model” are free byproducts. The _display privacy sidecar is even sharper—the agent is not even allowed to know “how many the privacy filter hid.”

3. Coworker-grade approval-governance closed loop. Scheduler (catch-up once + skip-on-overlap + spawn-don’t-await) + self-wake (timer / completion / event self-wake) + §25 standing rules (external risk only, exact target; shell always asks, and even && injection is blocked for you) + @mention as thread authorization (thread_target one string three jobs, survives restart). The four together: do unattended work on a schedule, park asks safely in the inbox, never overreach into a shell.

Trade-offs and “who is it for”

Dimension	OpenWorker’s choice	Cost / who it is for
Goal	an everyday coworker that ships finished work (cross-app)	not optimized for writing code; a pure coding scene is less fluent than opencode / Claude Code
Deploy	local-first desktop App (Tauri+Python+Rust)	heavy install (needs Python+Node+Rust to run from source); but data only leaves through models/integrations you chose
Models	provider-agnostic (including local Ollama)	home-grown provider layer (native + OpenAI-compatible endpoints); aisuite only owns the tool layer
Permissions	coworker-grade governance (inbox / approval / standing rules / privacy sidecar)	high complexity; the payoff is “you dare let it send messages unattended on your behalf”
Context	explicit SQLite memory, no automatic compaction	long sessions rely on bounded iteration + persistence; predictability over saving tokens
Memory	global / workspace dual scope + when-to-remember guidance	needs the model to remember with discipline

One-line selection: want a “digital coworker” that can take Slack, run on a schedule, asks before it acts, hands you a finished artifact, and can stay fully local—OpenWorker is currently the most complete open-source thought-through of that job. Want a pure coding terminal agent, go back to opencode / Claude Code / grok-build. It is closest to the Manus family (OpenManus/Suna) (both general-task), but OpenWorker’s center of gravity is desktop-local + approval governance + connectors + scheduling, and the Manus family’s is cloud sandbox + platformization—one is “a coworker that lives on your computer,” the other is “a task worker in the cloud.”

🔍 Source pointers (walk back to the code)

Want to see	Go here
Agent loop (concurrency / interrupt / durable resume)	coworker/engine.py:52,156,294,437,120,250
Permission verdict + shell injection defense + §25	coworker/permissions.py:120,216,62
Engine assembly (tools packed by family)	coworker/agent.py:109,217,228
Inbox state machine + out-of-band approval	coworker/inbox.py:61,295,133,348
Scheduler	coworker/automation/scheduler.py:23,63,76
Read-only explorer sub-agent	coworker/tools/subagent.py:42,86
Persona marketplace (consent gate)	coworker/personas/registry.py:340,:363 (:123 is builder registration)
@mention → thread authorization (one string, three jobs)	coworker/mentions.py
Provider routing + capability matrix	coworker/providers/router.py · matrix.py · capabilities.py
Startup sidecar + orphan guard	coworker/server/run.py · cli.py

---

This analysis is written against an actual checkout of andrewyng/openworker commit db93d75 (2026-07-25); every conclusion is marked file:line and can be walked back to source. Same frame as the other analyses in this series.