# OpenAI4S source analysis · a Code-as-Action scientific-research agent

> **Subject**: [PKU-YuanGroup/OpenAI4S](https://github.com/PKU-YuanGroup/OpenAI4S) (Open AI for Scientist)  
> **Baseline commit**: `85e9fa0` (`Feat/retrosynthesis workflow triage #70`, `85e9fa062c354b2ea9a40a668f03669a5452e56d`)  
> **License**: MIT  
> **Product in one sentence**: an **open-source hybrid scientific agent** that recreates the Claude Science experience with Volcengine Ark / Doubao’s ¥9.9 plan — native JSON tools as the orchestration and permission control plane; a persistent Python/R kernel as the science execution plane; core zero third-party deps (pure stdlib).  
> **Audience**: PMs and senior engineers who have already read at least one analysis in this series (Claude Code / Open Design / Codex / OpenManus) and want to take “scientific-research Agent” from slogan down to checkable source.  
> **Companion tutorial**: [从零构建OpenAI4S-开发全流程教程.md](./从零构建OpenAI4S-开发全流程教程.md) · [HTML](./从零构建OpenAI4S-开发全流程教程.html)  
> **Local baseline path**: `参考项目/OpenAI4S @ 85e9fa0`

---

## Conventions before reading

| Convention | Notes |
|---|---|
| Citation format | load-bearing claims try to land on `file:line`, or at least `file` + symbol name, so you can check source |
| Terms | **control plane** = provider-native JSON Tool; **science plane** = Python/R Cell; **Host** = the orchestration envelope behind the in-kernel `host` singleton |
| Completion signal | non-science turns → Engine-owned `finalize_response`; science Python Cell → the only internal completion signal `host.submit_output(...)` |
| HTML | companion site: `OpenAI4S-解析.html` (generated by `build-html.py`) |

## Contents

**Part I · What it is**  
1. [One-sentence positioning: JSON control plane + Python/R science plane](#ch1)  
2. [Repo map and stack](#ch2)  
3. [Relation to Claude Science / CodeAct / ReAct](#ch3)  

**Part II · Dual-plane hybrid engine**  
4. [AgentEngine outer loop](#ch4)  
5. [The route_action iron law](#ch5)  
6. [finalize_response vs host.submit_output](#ch6)  
7. [Action Ledger and Store SQLite overview](#ch7)  

**Part III · Host · Kernel · Tools**  
8. [host singleton and the host_dispatch orchestration envelope](#ch8)  
9. [LazyKernel, sandbox, and environment allowlist](#ch9)  
10. [tools/ class catalog and read-only parallel waves](#ch10)  
11. [llm/: ark\|openai\|anthropic\|gemini over urllib](#ch11)  

**Part IV · Web workbench**  
12. [serve → gateway.SessionRunner](#ch12)  
13. [WebSocket `/api/v1/ws` event surface](#ch13)  
14. [The full path of one user prompt (core sequence)](#ch14)  
15. [Notebook read-only default, Artifacts, Branch/Revert](#ch15)  

**Part V · Skills · Compute · Security**  
16. [Skills: 34 code recipes, not JSON schema](#ch16)  
17. [BYOC and `openai4s_compute_provider`](#ch17)  
18. [Permissions, approval persist, biosecurity, egress](#ch18)  

**Part VI · Kernel deep-dive and developer taste** (one of this lecture’s centers of gravity)  
19. [Kernel protocol: dup2, frame caps, and “wrong is worse than absent”](#ch19)  
20. [Ten development choices: reverse-engineering how they built it, and taste](#ch20)  

**Part VII · Side-by-side and summary**  
21. [Comparison table: Claude Code \| Open Design \| Codex \| OpenManus \| OpenAI4S](#ch21)  
22. [Five distinctive design decisions and honest boundaries](#ch22)  
23. [Source tour index](#ch23)  

---

# Part I · What it is

<h2 id="ch1">Chapter 1 · One-sentence positioning: JSON control plane + Python/R science plane</h2>

### 1.1 It is not another ReAct coding assistant

OpenAI4S (Open AI for Scientist) nails itself to a very narrow, very hard product seat:

> **Native JSON Tool Call owns orchestration and permissions; a persistent Python/R Code-as-Action kernel owns scientific execution.**

The official Chinese README slogan is “💸 9.9 元豆包 API 复刻 Claude Science” (`README_zh.md:7-10`). Engineering-wise it is not “a scientific-research skin on Claude Code,” nor “yet another ReAct with a shell tool.” It deliberately splits into **two action channels that never compete in the same step** (`docs/architecture.md:3-20`):

| Plane | Action unit | Good for | Not good for |
|---|---|---|---|
| **JSON control plane** | an ordered native-tool batch, or a sole `FinalizeAction` | permissions, metadata, external services, workflow control, session fork/revert | 100k-row DataFrame analysis, simulation loops, figure-drawing pipelines |
| **Python/R science plane** | **exactly one** complete fenced Cell | compute, exploration, analysis, simulation, long tasks; Python can mid-cell sync `host.*` RPC | splitting “read files / search the web / change permissions” into a dozen tool_use round-trips |

The README comparison table is blunt (`README_zh.md:52-68`): the same job “find matching files → sort → read CSV → plot,” ReAct may take ~14 round-trips; OpenAI4S presses it into **one code cell**, large objects stay in kernel memory, context only keeps a line `"<DataFrame 100000×20>"`.

### 1.2 PM view: what experience it is selling

| Selling point | Engineering landing |
|---|---|
| ¥9.9 Doubao / Volcengine Ark | `llm/capabilities.py:206-221` built-in `ark` provider, `wire="openai"`, default `doubao-seed-2.0-pro` |
| Claude Science–grade research workbench | static WebUI (no bundler) + Notebook projection + versioned Artifacts + approval cards |
| Open source, self-hostable | MIT; daemon default-binds `127.0.0.1:8760` |
| Core does not drag a science stack | `pyproject.toml:9` `dependencies = []`; numpy/pandas go through optional `science` extra |

> 🧠 **One sentence**: others optimize “how to make bash / editor tools smarter”; OpenAI4S optimizes “how to let the model finish writing a piece of real scientific code, and leave the state in the kernel.”

```mermaid
flowchart LR
    U["科学家 / 用户提示"] --> CP["① JSON 控制平面<br/>permissions · metadata · web · MCP · remote"]
    U --> SP["② Python/R 科学平面<br/>persistent kernel · host RPC · artifacts"]
    CP -->|"route 铁律：native > finalize > one cell"| ENG["AgentEngine 外循环"]
    SP --> ENG
    ENG --> DONE["完成信号<br/>finalize_response 或 host.submit_output"]
```

---

<h2 id="ch2">Chapter 2 · Repo map and stack</h2>

### 2.1 The project’s shape in numbers (baseline 85e9fa0)

| Metric | Magnitude (approx.) |
|---|---|
| In-package Python | `openai4s/` ≈ **255** `.py`, about **114k** LOC |
| Tests | `tests/` ≈ **286** `test_*.py` (offline gate by default) |
| WebUI | `server/webui/app.js` ≈ **9.7k** lines; no bundler |
| Gateway | `server/gateway.py` ≈ **12.9k** lines (HTTP + hand-written WS composition facade) |
| Built-in Skills | **34** `skills/*/SKILL.md` |
| Control tools | `TOOL_TYPES` **70** `Tool` subclasses (`tools/registry.py:118-189`) |
| Core deps | **0** (`pyproject.toml:9`) |

```
OpenAI4S/                          # commit 85e9fa0 · MIT
├── openai4s/                      # 产品主体（stdlib）
│   ├── agent/                     # 外循环：engine · actions · finalize · ledger · loop
│   ├── kernel/                    # 科学平面：manager · worker · lazy · r_kernel · sandbox 对接
│   ├── host/ + host_dispatch.py   # Host 能力服务 + 共享编排信封
│   ├── sdk/host.py                # 注入到 Python 内核的 host 单例门面
│   ├── tools/                     # 70 个 native Tool 子类 + registry
│   ├── llm/                       # urllib 传输 + openai/responses/anthropic/gemini 线
│   ├── server/                    # gateway · agent_run · branching · webui/
│   ├── security/                  # sandbox · permissions · biosecurity · injection
│   ├── skills_loader/             # SKILL.md 发现与渐进披露
│   ├── store.py + storage/        # 单连接 SQLite + 仓储
│   ├── compute/                   # BYOC 主机侧
│   ├── egress.py                  # 出站域名 allowlist
│   └── …
├── openai4s_compute_provider/     # 远端 GPU 上跑的 stdlib 沙箱 SDK
├── openai4s_worker_runtime/       # worker 运行时附属包
├── skills/                        # 34 份代码食谱（非 JSON tool schema）
├── envs/                          # conda 环境配方
├── workflows/                     # 科学工作流 benchmark 清单
├── harness/                       # 脚本化场景 / golden（在生产 import 图之外）
├── docs/                          # architecture · security · compute · webapp…
├── pyproject.toml                 # dependencies=[] ；science/chemistry extras
├── setup.sh / start.sh
└── tests/                         # 离线默认；external/network/live_llm 需显式 -m
```

### 2.2 Stack choice: why “pure stdlib”

| Layer | Choice | Source anchor |
|---|---|---|
| Package / run | `uv` + `setuptools` | `pyproject.toml`; `./setup.sh`, `./start.sh` |
| Agent core | pure Python stdlib | `agent/engine.py` no third-party import |
| LLM client | `urllib.request` | `llm/transport.py:27-28` |
| HTTP server | `http.server` | `server/gateway.py` module-header comment |
| WebSocket | **hand-written** frame encode/decode | top of `gateway.py`: `GET /api/v1/ws` |
| Frontend | static `app.js` / `index.html` / `style.css` | no build step (`CLAUDE.md:100`) |
| Persist | SQLite single connection | `store.py` |
| Optional science stack | `uv sync --extra science` | `pyproject.toml:36-41` (numpy/pandas/matplotlib/sklearn) |
| Optional chemistry | `--extra chemistry` (RDKit) | `pyproject.toml:45-47` |

The hard constraint is written in `CLAUDE.md:90` (verification correction 2026-08-04: the original draft wrote :11, but the quoted sentence "Never add a hard third-party import to the core" really lives at :90; :11 is a related but different sentence "Core is zero-dependency by design") and `pyproject.toml:9`: **Never add a hard third-party import to the core.** Science libraries may appear in agent *cells* (the kernel inherits the venv), but the engine itself must `try/except ImportError` guard every in-tree use.

### 2.3 Two entries, one engine

```mermaid
flowchart TB
    subgraph CLI["CLI 路径"]
        RUN["openai4s run '…'"] --> AGENT["agent/loop.py · Agent.run"]
        AGENT --> LK["LazyKernel"]
        AGENT --> LAE["LocalActionExecutor"]
    end
    subgraph WEB["Web 路径"]
        SERVE["openai4s serve / start.sh"] --> GW["server/gateway.py"]
        GW --> SR["SessionRunner._loop"]
        SR --> WAE["WebActionExecutor"]
        SR --> WES["WebEventSink"]
    end
    AGENT --> ENG["AgentEngine.run while"]
    SR --> ENG
    ENG --> RA["route_action"]
```

CLI uses `Agent.run` (`agent/loop.py:352-447`) to assemble `LazyKernel` + `LocalActionExecutor` + `ChatModel`; Web uses `SessionRunner._loop` (`gateway.py:6337-6447`) to assemble the same `AgentEngine`, only swapping executor and event projection to Web versions. This is the most important architectural symmetry in the lecture: **one engine, two thin adapters** (`actions.py:3-6` calls itself CoreCoder-style).

---

<h2 id="ch3">Chapter 3 · Relation to Claude Science / CodeAct / ReAct</h2>

### 3.1 Three comparisons

| Paradigm | Typical form | How OpenAI4S lines up |
|---|---|---|
| **ReAct** | Thought → Action(tool) → Observation, tools atomic, no mid-exec callback | control plane borrows its “structured tools”; science plane **refuses** to split analysis into atomic tools |
| **CodeAct** | the model mainly produces executable code | science plane is Code-as-Action; the outer loop still keeps a JSON control channel |
| **Claude Science (closed-source)** | research workbench + strong model + closed-source runtime | product experience is the counterpart; implementation uses an open dual plane + ¥9.9 Ark as substitute |

### 3.2 Key claim: why “no registered shell tool”

`tools/registry.py:115-117` and `:192` are hardcoded:

- deliberately **NO shell tool**;
- `_FORBIDDEN_CONTROL_NAMES = frozenset({"bash", "submit_output"})`;
- `register_tool` hitting them raises `ValueError` immediately (`:211`).

Shell can only happen inside the Python kernel: `host.bash(...)` (`sdk/host.py:854-865`). This is not a style preference; it is the load-bearing wall of plane separation — once shell becomes a JSON tool, the model falls back to ReAct-style “one command per step,” and 100k-row analysis is shredded again.

### 3.3 The key difference vs “pure tool_use”: mid-flight Host RPC

`docs/architecture.md:56`:

> This inner RPC loop does not exist in a `tool_use` architecture — there, actions are atomic and never call back into the host mid-execution.

During Python Cell execution, `host.llm` / `host.delegate` / `host.compute` ride a `host_call → host_ack → host_response` channel independent of stdout; the Cell blocks, Host serves, the Cell resumes. That is the **structural** advantage of Code-as-Action over ReAct, not a prompt trick.

```mermaid
sequenceDiagram
    participant M as Model
    participant E as AgentEngine
    participant K as Python Kernel
    participant H as HostDispatcher
    M->>E: assistant reply (fenced python)
    E->>K: execute one CodeCell
    K->>H: host_call(llm / compute / …)
    H-->>K: host_response
    K->>H: host.submit_output(...)
    H-->>E: completion signal
    E-->>M: stop_reason=submitted
```

---

# Part II · Dual-plane hybrid engine

<h2 id="ch4">Chapter 4 · AgentEngine outer loop</h2>

### 4.1 The state-machine body is only ~143 lines

`AgentEngine` (`agent/engine.py:34-143`) is a provider-neutral outer loop. It does **not** import a concrete kernel, dispatcher, store, or server — those are ports, injected by CLI / Web adapters (`docs/architecture.md:58-59`).

The core `run` while (`engine.py:60-108`) can be read as:

```
state ← RunState(messages, max_turns)
emit RunStarted
while turn < max_turns:
    if cancelled → finish("cancelled")
    if completion.completion() → finish("submitted")   # 例如已有 submit_output
    prepare context → model.complete(stream deltas)
    append assistant message
    action ← route_action(content, tool_calls)
    outcome ← executor.execute(action, reply, state)
    append history_messages; turn++
    if outcome.stop_reason → finish(that)
    if outcome.completion or completion port → finish("submitted")
finish("max_turns")
```

Default `max_turns=32` (`engine.py:47`); Web sessions often take `cfg.max_turns` (gateway-side default commonly 12, explore can raise it, `gateway.py:6349-6351`).

### 4.2 The benefit of ports

| Port | Role | Default implementation |
|---|---|---|
| `ModelPort` | call the model | CLI/Web `ChatModel` |
| `ActionExecutor` | execute the routed action | `LocalActionExecutor` / `WebActionExecutor` |
| `ContextPolicy` | compact context | `CompactionPolicy` / `PassthroughContext` |
| `EventSink` | project events | Transcript / `WebEventSink` / Null |
| `CancellationPort` | Stop | `NeverCancelled` / `EventCancellation` |
| `CompletionPort` | detect `submit_output` | `CompletionSignal(dispatcher.last_output)` |

> 🧠 **Product meaning**: tests can verify “cancel / max turns / finalize / routing priority” without starting the daemon or spawning a kernel. That is an engineering-maturity signal, not over-abstraction.

```mermaid
stateDiagram-v2
    [*] --> Running: RunStarted
    Running --> ModelCall: TurnStarted
    ModelCall --> Routing: ReplyReceived
    Routing --> Executing: ActionRouted
    Executing --> Running: OutcomeProduced (continue)
    Executing --> Submitted: stop_reason / completion
    Running --> Cancelled: cancellation
    Running --> MaxTurns: turn == max_turns
    Submitted --> [*]
    Cancelled --> [*]
    MaxTurns --> [*]
```

---

<h2 id="ch5">Chapter 5 · The route_action iron law</h2>

### 5.1 Single-channel decision

`route_action` (`agent/actions.py:165-185`) is the **only** action-decision point shared by both loops:

```python
# 伪代码忠实于 actions.py:176-185
calls = normalize(tool_calls)
if len(calls) == 1 and calls[0].name == "finalize_response":
    return FinalizeAction(calls[0])
if calls:
    return NativeToolBatch(calls)          # 任意原生调用压过代码
return extract_action(content)             # 第一个完整 python/r fence
```

The iron law can be memorized as three sentences:

1. **If there are structured native calls → never run code** (control plane cannot compete with science plane in the same turn);
2. **Sole and named `finalize_response` → FinalizeAction** (mixed with other tools does not count as complete);
3. **Otherwise → the first complete python/r Cell in document order** (one cell per step; an unclosed fence is not executable).

The `FINALIZE_RESPONSE_NAME` literal lives on the routing boundary (`actions.py:40-43`), deliberately not importing the registry — so the actions module does not pick up tool-registration side effects.

### 5.2 Action-type algebra

| Type | Defined at | Meaning |
|---|---|---|
| `CodeCell` | `actions.py:46-51` | `language ∈ {python,r}` + code |
| `NativeToolCall` | `:55-71` | lossless keep of raw_arguments / parse_error / provider_meta |
| `NativeToolBatch` | `:74-78` | ordered tuple |
| `FinalizeAction` | `:81-90` | Engine-owned terminal-state declaration |
| `Action` | `:93` | union of the three above |

`extract_action` (`:146-162`) only recognizes:

- Python: `info ∈ {"", "python", "py"}` (bare ``` defaults to python);
- R: must be explicit `` ```r ``.

### 5.3 Hiding a “pure-completion Cell”

`is_completion_only_cell` (`actions.py:96-143`) uses AST: if a Python Cell **as a whole** is only `host.submit_output(...)` and the arguments have no nested compute, the Web Notebook **hides** that cell — completion is a real RPC, but not scientific analysis, and should not pollute the read-only Notebook (`docs/architecture.md` / CLAUDE.md user-visible completion-projection convention).

```mermaid
flowchart TD
    R["Model reply"] --> Q1{"有 native tool_calls?"}
    Q1 -->|否| EX["extract_action → 至多一个 Cell"]
    Q1 -->|是| Q2{"恰好 1 个且名=finalize_response?"}
    Q2 -->|是| F["FinalizeAction"]
    Q2 -->|否| B["NativeToolBatch（整批）"]
    EX --> C["CodeCell | None"]
    B --> NOTE["代码 fence 即使存在也被忽略"]
```

---

<h2 id="ch6">Chapter 6 · finalize_response vs host.submit_output</h2>

This is the chapter in the whole analysis most easily misled by a “tool-list screenshot.”

### 6.1 finalize_response: Engine-owned, not a Tool registry entry

`agent/finalize.py:1-12` opens by saying:

> `finalize_response` is deliberately **not** a control-plane `Tool` and is **never** registered in `openai4s.tools.registry`.

What it does:

| Mechanism | Location | Behavior |
|---|---|---|
| Closed schema | `finalize.py:35-83` | requires `summary` + `completion_bullets`; optional findings/metrics/artifacts… |
| Spec given to the model | `finalize_response_tool_spec()` `:92-111` | metadata-only `ToolSpec` |
| Stitched into the tool catalog | `with_finalize_response()` `:114-123` | append the spec; if the catalog already has the same name → **raise** (prevent plugins impersonating terminal state) |
| Routing | `route_action` | becomes `FinalizeAction` only when it is the **sole** native call |

A science Python Cell **cannot** use it as a substitute for internal completion: `description` writes plainly *does not replace host.submit_output* (`:106-107`).

### 6.2 host.submit_output: the only in-Cell completion signal

`sdk/host.py:803-827`:

```text
host.submit_output(output, completion_bullets, output_schema=None)
→ HostDispatcher CompletionService
→ dispatcher.last_output
→ AgentEngine CompletionPort 侦测到 → stop_reason="submitted"
```

Constraint: `completion_bullets` must be 1–4 “completed-action” phrases; optional `output_schema` validate failure is a soft-fail `{"error":...}` so the model can retry.

### 6.3 Comparison table (must remember)

| | `finalize_response` | `host.submit_output` |
|---|---|---|
| Who owns | Engine (`finalize.py`) | in-kernel Host RPC |
| In `TOOL_TYPES`? | **no** | **no** (and registration forbidden) |
| When to use | wrap-up of chat / pure-tool turns | wrap-up of a science Python Cell |
| Can share a turn with other tools | **no** (must be sole) | N/A (called in code) |
| R Cell | not emitted from inside R | R has **no** in-Cell completion signal |
| Ordinary prose / max turns | **not** completion | **not** completion |

```mermaid
flowchart LR
    subgraph NonSci["非科学回合"]
        T["Native tools…"] --> F["sole finalize_response"]
        F --> CR["CompletionRecord"]
    end
    subgraph Sci["科学 Python Cell"]
        CELL["```python …"] --> SO["host.submit_output"]
        SO --> CR2["CompletionRecord 同源结构"]
    end
    CR --> UI["Gateway 投影：output + bullets + artifact delta"]
    CR2 --> UI
```

---

<h2 id="ch7">Chapter 7 · Action Ledger and Store SQLite overview</h2>

### 7.1 Ledger: book first, then execute

The module docs of `agent/ledger.py` (`:1-12`) state the design goal:

- storage repositories **do not understand** agent semantics;
- `RuntimeActionLedger` translates typed `AgentEngine` events into immutable groups/events;
- Native declarations and results are **atomically reduced**; on crash, half-open batches are filled with canonical error/cancel results, **never** feeding a half-open tool batch back to the LLM.

CLI constructs the ledger in `Agent.run` (`loop.py:405-409, 463+`); Web in `SessionRunner._loop` passes `action_ledger` into `WebEventSink` (`gateway.py:6343-6377`).

The engine is **ledger-first** (`docs/architecture.md:59-64`): open an append-only action group → execute → close with tool result / Cell milestone → terminal state is appended, not reverse-engineered from the UI transcript.

### 7.2 Store: one connection per process as the source of truth

`store.py` holds the only SQLite connection and schema/migration. Core tables visible at baseline include (excerpt, `store.py:114+` and `storage/`):

| Table | Duty |
|---|---|
| `frames` | session / turn frames (tree depth, model, tokens) |
| `messages` | chat-message projection |
| `execution_log` | execution log |
| `artifacts` / `artifact_versions` | versioned artifacts |
| `lineage_edges` | data lineage |
| `host_call_log` | Host RPC audit |
| `permission_rules` / `permission_requests` | persistent approval |
| `plans` | plan mode |
| `annotations` / `annotation_admissions` | image annotations and client admission |
| `compute_jobs` / `compute_job_events` | remote-compute jobs |
| `session_branches` / `session_checkpoints` | branches and checkpoints (`storage/snapshots.py`) |
| `kernel_generations` | kernel generations (`storage/kernels.py`) |
| `recovery_journal` | recovery journal |
| `settings` / `connectors` / `memories` / `shares` | config, connectors, memory, shares |

The Agent side exposes SQL **read-only** via `host.query` (CLAUDE.md / architecture); write paths always go through services and repositories.

```mermaid
erDiagram
    frames ||--o{ messages : contains
    frames ||--o{ artifacts : produces
    artifacts ||--o{ artifact_versions : versions
    artifact_versions ||--o{ lineage_edges : lineage
    frames ||--o{ permission_requests : asks
    frames ||--o{ session_checkpoints : checkpoints
    session_branches ||--o{ session_checkpoints : head
    frames ||--o{ kernel_generations : generations
    frames ||--o{ compute_jobs : remote
```

---

# Part III · Host · Kernel · Tools

<h2 id="ch8">Chapter 8 · host singleton and the host_dispatch orchestration envelope</h2>

### 8.1 `host` inside the kernel

`sdk/host.py` (about 1177 lines) is the compatibility facade injected into the Python worker. The architecture doc lists the capability surface (`docs/architecture.md:77-88`) including:

- network: `web_search` / `web_fetch` / `web_download`
- filesystem: workspace-jailed read/write/edit/grep/glob/list
- model and sub-agents: `llm` / `delegate` / `collect`
- science API: `science.*`
- remote compute: `compute.*` / `fold`
- artifacts: `save_artifact` / `artifacts` / `view_image`
- skills / environment / MCP / read-only SQL
- **completion**: `submit_output`
- **Shell**: `bash` (in-kernel only)

### 8.2 HostDispatcher: a shared orchestration envelope, not a god-class implementation bucket

`HostDispatcher` (`host_dispatch.py:646+`) comments its positioning clearly:

> Backs control tools and worker host.* RPC. One instance per session.

`__call__(method, args)` (`:996`) is the unified entry. Capability implementations split into `openai4s/host/*` services: files, llm, completion, data, delegation, remote_science, progress, skills, mcp, endpoints, credentials… (module import region `:26-51`).

**Soft-fail contract**: a handler may return a single-key `{"error": msg}`; the worker turns it into `RuntimeError`. Uncaught exceptions are likewise pressed into an error dict by the manager onto the wire.

Control-plane native Tool `execute()` and in-kernel `host.*` **share the same dispatcher** — permissions, egress, injection screening, activity step, audit log are shared (`docs/architecture.md:142-147`).

```mermaid
flowchart TB
    subgraph Surfaces["调用表面"]
        NT["Native Tool.execute"]
        HC["kernel host.* RPC"]
    end
    NT --> HD["HostDispatcher.__call__"]
    HC --> HD
    HD --> PERM["permissions / approval"]
    HD --> AUD["audit / replay / injection"]
    HD --> SVC["host/* services"]
    SVC --> STORE[(SQLite Store)]
    SVC --> LLM["llm.chat"]
    SVC --> NET["webtools + egress"]
```

---

<h2 id="ch9">Chapter 9 · LazyKernel, sandbox, and environment allowlist</h2>

### 9.1 LazyKernel: tool turns do not spawn for nothing

`LazyKernel` (`kernel/lazy.py:15-22`):

> Create a worker only when code first needs an interpreter. Control-tool and structured-finalization turns can carry this object without spawning a process.

How `Agent.run` assembles it (`loop.py:385-389`): factory + skill bootstrap + foreground publish. Attributes `spawned` / `generation` / `execute` / `shutdown` all revolve around “thread-safe one-shot ownership.”

The R kernel is a sibling channel: `r_kernel.py` + `r_worker.R`, the same manager protocol; frames ride fd3/fd4, so print does not pollute the protocol (CLAUDE.md).

### 9.2 OS sandbox: Seatbelt / bubblewrap

`security/sandbox.py` header comment (`:7-8`):

- macOS → `sandbox-exec` (Seatbelt)
- Linux → `bwrap` (bubblewrap)

Modes: `OPENAI4S_KERNEL_SANDBOX=auto|enforce|off`. `auto` **visibly degrades** if self-check fails; `enforce` failure shuts down. Write limits on workspace/private temp; default denies raw outbound network (architecture / CLAUDE).

### 9.3 Subprocess environment allowlist

At spawn, the worker environment is **rebuilt from a strict allowlist**, not `os.environ.copy()` — so daemon provider/API/cloud secrets and loader-injected vars do not leak into Python/R and their children (`docs/architecture.md:111-113`, `kernel/environment.py`).

### 9.4 host.bash one-shot token

CLAUDE.md emphasizes: `host.bash` still executes locally in the kernel, but subprocess start needs a one-shot Host token bound to command hash, cwd, active worker generation, challenge; **Host authorizes/audits, never executes shell in the daemon process**.

```mermaid
flowchart LR
    CELL["Python Cell"] --> BASH["host.bash(cmd)"]
    BASH --> TOK["one-shot Host token"]
    TOK --> AUTH["BashAuthorizationService"]
    AUTH -->|ok| SUB["subprocess inside worker"]
    AUTH -->|deny| ERR["soft error"]
    SUB --> SANDBOX["Seatbelt / bwrap wrapper"]
```

### 9.5 Hard discipline of the kernel protocol (taste you only get from worker comments)

The file header of `kernel/worker.py` is not a decorative docstring; it is a **protocol design specification**. Read it as “what the developers are afraid of”:

| Mechanism | Afraid of | Practice |
|---|---|---|
| **dup2 track-switch** | C extensions / noisy `print` polluting the protocol line | real protocol stdin/stdout moved to high, non-inheritable fds, published on `sys._openai4s_protocol_*`; `dup2(2,1)` aliases fd1 to stderr (`worker.py:86-119`) |
| **Two locks** | frame interleave / reading someone else’s response | `_PROTOCOL_WRITE_LOCK` (write frames) + `_HOST_CALL_LOCK` (whole host_call transaction) (`worker.py:12-15`) |
| **15MB host_call cap** | one RPC dragging down the daemon | `_HOST_CALL_WIRE_CAP = 15_000_000` |
| **stdout chunked 64KB** | `print("x"*2e8)` becoming one 200MB JSON line | `_MAX_CHUNK_CHARS = 64_000` (`worker.py:47-51`) |
| **Frame byte cap derived from UTF-8 worst case** | CJK / emoji bursting a “char cap” through a “byte cap” | `_JSON_WORST_BYTES_PER_CHAR = 12` (surrogate pairs); the comment writes plainly: a six-byte derivation was slapped by tests (`worker.py:52-77`) |
| **SIGINT discipline** | user `raise KeyboardInterrupt` confused with a real signal | one-shot handler + `_in_user_code` + `_sigint_delivered` |
| **crash recovery** | fd identity drifted while blocked on read | check `(st_dev, st_ino)` before and after each read; budget one `os.dup(reserve)` rebuild |

```mermaid
flowchart LR
    subgraph Worker["worker.py 进程"]
      CODE["用户 Cell / C 扩展"] -->|fd1 已 alias| STDERR["stderr 捕获"]
      HOST["host.* SDK"] -->|高位 fd| PROTO["protocol JSON lines"]
      HOST --> LOCK["_HOST_CALL_LOCK"]
      LOCK --> CALL["host_call 帧"]
    end
    subgraph Manager["manager.py"]
      CALL --> READ["单 reader 循环"]
      READ --> DISP["HostDispatcher"]
      DISP --> RESP["host_response"]
      RESP --> LOCK
    end
```

**Taste observation**: they are not afraid of making the protocol “hard to read” — they are afraid of **silent confusion**. Prefer a comment full of “why this number is 12 not 6” over letting one CJK output drop a whole `error_lineno`/`usage` frame.

### 9.6 Artifact environment fingerprint: bind to kernel generation, not the daemon

CLAUDE.md writes a heavy sentence (paraphrased):

> Artifact environment provenance comes from **the kernel generation that produced the file**, not a zero-arg freeze of the daemon process. They once stamped R Cell artifacts with a Python package list — **wrong provenance is worse than none, because it will be believed**.

This epistemology recurs through the whole codebase:

| Bad smell | OpenAI4S’s correction |
|---|---|
| Silent success / false green | harness: a scenario declared to fail, if it succeeds, is judged a failure |
| Fake provenance | record “why missing” rather than borrow the daemon’s package list |
| stub shapes entering the contract | `stubbed_backend` marker pauses the schema recorder |
| Replay args after approval restart | only record the decision; require Continue/replan |

### 9.7 R channel: “same protocol, different completion semantics”

R reuses the same `Kernel` manager and frame contract, but:

- protocol rides fd3/fd4, noise goes to stderr (shell redirect equivalent to Python’s dup2);
- **no** mid-cell `host` RPC, and no in-Cell `submit_output`;
- completion narrative still leans Python finalize / a subsequent Python cell.

**Taste**: not “two parallel complete runtimes,” but “one science-plane main channel + one statistics/plotting sibling channel.” The product honestly labels the difference rather than pretending symmetry.

---

<h2 id="ch10">Chapter 10 · tools/ class catalog and read-only parallel waves</h2>

### 10.1 TOOL_TYPES: 70 named Tool subclasses

`tools/registry.py:118-189` is the only composition root. Roughly by domain:

| Domain | Representative tools |
|---|---|
| Files | ListDirectory / ReadTextFile / WriteFile / Glob / ContentSearch / Edit |
| Environment | EnvList / EnvUse / EnvCreate |
| Web | WebSearch / WebDownload / WebFetch |
| Science DBs | ScienceListDatabases / ScienceSearch (behind them, normalized UniProt/PDB/…) |
| Skills | Search / Load / Status / History / Rollback |
| Artifacts | List / Metadata / Versions / Save / Restore |
| Session control | Query / Frames / Lineage / Todos / Plan / Review / Checkpoint / Fork / RevertPreview / Permissions |
| Delegation | Delegate / ListChildren / Collect / Stop / SendMessage |
| Background exec | Submit / List / Peek / Interrupt |
| MCP | ListServers/Tools/Resources/Prompts + Call |
| Network grant | RequestNetworkAccess |
| Remote compute | RemoteGPU / Register / Submit / Status / Result / Cancel / Close |
| Dynamic tools | Define / List / Promote / Versions / Activate / Rollback |

**Absent**: `bash`, `submit_output`, `finalize_response`.

### 10.2 Read-only parallel waves

`agent/control.py`:

- `tool_parallel_policy` (`:152+`) identifies read-only scheduling;
- `_execute_read_only_waves` (`:167+`) splits waves by resource-key conflict;
- the first mutating / unknown call is a **barrier**, serial after that;
- results are written back to history in the provider’s **original order** (`docs/architecture.md:149-153`).

That lowers latency for “read several files / search several DBs in a row” without breaking canonical tool-group order.

### 10.3 Science DBs do not inflate tool count

Only two control tools, `science_list_dbs` + `science_search`; a connector service behind them normalizes several public databases, attaching a provenance envelope (timestamp + request + SHA-256) (later architecture). In-Cell counterpart is `host.science.*`.

---

<h2 id="ch11">Chapter 11 · llm/: ark \| openai \| anthropic \| gemini over urllib</h2>

### 11.1 Four wires, one transport

`SUPPORTED_WIRES = frozenset({"openai", "responses", "anthropic", "gemini"})` (`llm/capabilities.py:24`).

Transport layer `llm/transport.py` is pure stdlib: `urllib.request` + bounded retry (only safely replayable status codes; respects `Retry-After`; cancellable; total-budget cap).

Provider adapter files:

| File | Wire |
|---|---|
| `llm/providers/openai.py` | OpenAI Chat compatible (**ark rides this wire**) |
| `llm/providers/responses.py` | OpenAI Responses |
| `llm/providers/anthropic.py` | Anthropic Messages |
| `llm/providers/gemini.py` | Gemini `generateContent` |

### 11.2 ark / Doubao is first-class

`capabilities.py:206-221`:

```text
ark:
  wire=openai
  base=https://ark.cn-beijing.volces.com/api/plan/v3
  default_model=doubao-seed-2.0-pro
  context_window=262_144
  tool_calling=True, parallel_tool_calls=True, vision=True, streaming=True
```

`llm/catalog.py` presets several Ark models (doubao-seed series, glm, kimi, deepseek, minimax…). That is the engineering base of “¥9.9 recreation,” not marketing copy spinning.

### 11.3 Config layering

CLAUDE.md: each api_key / base_url / model resolves as  
`per-provider vars → OPENAI4S_LLM_* → provider default`.  
The daemon can start with no key, then configure in UI Customize → Models or `.env`.

`register_provider` (`llm/registry.py:47-60`) only allows hanging onto an existing wire — **cannot** dynamically load arbitrary transport code.

```mermaid
flowchart TB
    CFG["Config / UI model profile"] --> RES["llm/resolve"]
    RES --> CAP["ProviderCapabilities"]
    CAP --> WIRE{"wire"}
    WIRE -->|openai| OA["providers/openai.py"]
    WIRE -->|responses| RP["providers/responses.py"]
    WIRE -->|anthropic| AN["providers/anthropic.py"]
    WIRE -->|gemini| GE["providers/gemini.py"]
    OA --> TR["transport.post_json · urllib"]
    RP --> TR
    AN --> TR
    GE --> TR
```

---

# Part IV · Web workbench

<h2 id="ch12">Chapter 12 · serve → gateway.SessionRunner</h2>

### 12.1 Process and bind

`openai4s serve` (`start.sh` thin wrap) pulls up a singleton daemon (pidfile); default `OPENAI4S_HOST=127.0.0.1`, `OPENAI4S_PORT=8760`. Docs are explicit: do not naked-run `0.0.0.0` on a trusted network; use an SSH tunnel.

`gateway.py` is a stdlib HTTP/WebSocket **composition adapter** (CLAUDE.md: surgical edits; whole-file rewrite forbidden).

### 12.2 SessionRunner._loop: the assembly site of a Web turn

What `SessionRunner._loop` (`gateway.py:6337-6447`) does:

1. take `max_turns` (explore can raise it);
2. construct `WebEventSink(emit, rid, assistant_visible, add_usage, …)`;
3. non-plan mode: take `tool_catalog` from the dispatcher, stitch the terminal-state spec with `with_finalize_response(...)`;
4. `AgentEngine(ChatModel(...), WebActionExecutor(...), CompactionPolicy, …)`;
5. `WebActionExecutor` injects: `execute_cell` → `_execute_and_log`, native wrapper → `_invoke_control_with_artifacts`, plan/explore hooks.

Symmetric with CLI `Agent.run`; only events sink to WebSocket, Cell execution goes through the session FIFO coordinator.

### 12.3 POST /frames/{id}/message

Route match (`gateway.py:9654+`):

- parse `input_data.request` (compatible with top-level `request`);
- handle `annotation_ids` + **client-generated** `annotation_reservation_id`;
- when `wait:false`, return **202** + `execution_id`, the turn runs in the background (fire-and-forget MessageJob).

Frontend contract in the next chapter’s sequence.

---

<h2 id="ch13">Chapter 13 · WebSocket `/api/v1/ws` event surface</h2>

### 13.1 Channel duty

`gateway.py` file header (near `:11`):

> WebSocket `GET /api/v1/ws`（`view_session` / `ping`；`text_reset` / `text_chunk` / …）

`WSHub` (`:650+`) maintains:

- the set of subscribed connections;
- **per-frame live-turn buffer** (reconnect can replay);
- monotonic `seq` + process `epoch` (against a false “you’ve caught up” after daemon restart).

### 13.2 High-frequency event types (product-visible)

| Event | Meaning |
|---|---|
| `text_reset` / `text_chunk` | streaming assistant body |
| `artifact_created` / `artifact_ref_problems` | artifact panel (new artifact / citation problems) |
| `permission_resolved` | human-approval result |
| `kernel_status` / `plan_progress` / `execution_queue` / `branch_activation_state` | kernel status / plan progress / execution queue / branch activation |

> ⚠️ **Verification correction (2026-08-04)**: the original table listed `cell` / `artifacts` / `permissions` as WS event `type`s, which is **wrong**. `cell` is a live-buffer `scope` value (`gateway.py:1057`); `artifacts` / `permissions` are **field names** of the projection dict; none of the three is a WS event `type`. Real event types are the three rows above (enumerated in `gateway.py` source).
| `frame_update` | frame-level state |
| `step` / `step_update` | Host activity timeline |

Turn-scoped type set is `_TURN_SCOPED_TYPES` (about `:954`): `text_reset`, `text_chunk`, `frame_update`.

---

<h2 id="ch14">Chapter 14 · The full path of one user prompt (the most important)</h2>

This is the workbench’s “main artery.” The following steps can all be checked against baseline source.

### 14.1 Frontend `send()` (`openai4s/server/webui/app.js:5574+`)

Key order (comments write why it must be this order):

1. handle plan / explore / `/skillname` instruction rewrite;
2. if no session → `POST /frames` create a frame and `sub(id)`;
3. optimistic insert of a user bubble;
4. **if there are image annotations**: generate `admissionId = "resv-" + hex` with CSPRNG, **first** `rememberAdmission`, then send the request (`:5640-5656`);
5. **`sub(S.currentId)` must be before POST** (`:5663-5669`) — otherwise the first-turn `text_chunk` is dropped outside the subscribe set;
6. `POST /frames/{id}/message`, body:

```json
{
  "input_data": { "request": "<payload>" },
  "plan": false,
  "explore": false,
  "annotation_ids": [...],
  "annotation_reservation_id": "resv-...",
  "wait": false
}
```

(`app.js:5676`)

7. associate the bubble with the 202 `execution_id`; reconcile admission by the `annotations` field.

### 14.2 Server catch → SessionRunner → AgentEngine

```mermaid
sequenceDiagram
    autonumber
    participant UI as app.js send()
    participant API as gateway HTTP
    participant Hub as WSHub
    participant SR as SessionRunner
    participant ENG as AgentEngine
    participant EX as WebActionExecutor
    participant K as LazyKernel/Worker
    participant HD as HostDispatcher

    UI->>UI: sub(frameId) 先订阅
    UI->>API: POST /frames/{id}/message wait:false
    API-->>UI: 202 execution_id
    API->>SR: 后台 MessageJob / run turn
    SR->>ENG: _loop 组装 Engine+Executor+Sink
    loop 每回合
        ENG->>ENG: model.complete → route_action
        alt NativeToolBatch
            ENG->>EX: 控制工具（可只读并行波）
            EX->>HD: 同源 HostDispatcher
            HD-->>Hub: step / permissions / artifacts…
        else FinalizeAction
            ENG->>ENG: CompletionRecord → stop
        else CodeCell
            ENG->>K: 执行一格
            K->>HD: host_call mid-cell
            HD-->>K: host_response
            K->>HD: host.submit_output?
        end
        ENG-->>Hub: text_chunk / cell / frame_update
        Hub-->>UI: WS 推送（含 seq）
    end
    ENG-->>SR: EngineResult
    SR-->>Hub: 终态投影（bullets + artifact delta）
```

### 14.3 Why `wait:false` + `sub()` first is a product-level detail

A blocking POST makes the browser feel like a “stuck form”; 202 + WS feels like a research workbench. But async brings a classic race: the thread has started `broadcast(text_chunk)` while the client is not yet in `conn.subs`. Those two comment lines in `send()` (`:5663-5669`) are not fluff; they are the tombstone of a production incident.

Likewise, `admissionId` must be **generated on the client first, then sent** — because the mechanism must cover the case of “never receiving a 202”; a server-minted id is useless to the browser in that case (`:5640-5646`, server echo at `gateway.py:9680-9685`).

---

<h2 id="ch15">Chapter 15 · Notebook read-only default, Artifacts, Branch/Revert</h2>

### 15.1 Notebook default read-only

`config.py:364-368`:

> read-only Notebook by default; set `OPENAI4S_NOTEBOOK_REPL=1` to re-enable the in-Notebook developer REPL.

`notebook_repl` defaults `False`. `kernel_routes.py` in several places `if not runner.cfg.notebook_repl` directly refuses interactive write paths. Product semantics: Notebook is a **read-only projection of Agent execution**, not a second Jupyter main loop (the Jupyter KernelSpec bridge is also an optional bypass, without tool batch / finalize, see CLAUDE.md).

### 15.2 Artifact versions

- tables: `artifacts` + `artifact_versions`;
- Cell file writes / `host.save_artifact` / native tools declaring `writes_files=True` (Web-boundary wrap) all generate versions;
- environment provenance binds to **kernel generation**, avoiding stamping R artifacts with a Python package list (that CLAUDE.md sentence *wrong rather than absent*);
- completion projection carries “the artifact-version delta actually produced this turn.”

### 15.3 Branch / Revert

`server/session_branching.py`:

- checkpoints are immutable;
- `fork` materializes an isolated workspace (`:116+`);
- `preview_revert` / `revert_and_continue`: first record the current checkpoint, then safely restore, **append** revert, do not rewrite history (`:9-10, :240+`);
- forking from a cell only when that cell carries a cursor checkpoint, otherwise **409** (CLAUDE.md) — without a checkpoint, state cannot be rebuilt.

```mermaid
gitGraph
    commit id: "checkpoint A"
    commit id: "checkpoint B"
    branch experiment
    checkout experiment
    commit id: "fork workspace"
    commit id: "new cells"
    checkout main
    commit id: "continue"
    commit id: "revert→B (append-only)"
```

---

# Part V · Skills · Compute · Security

<h2 id="ch16">Chapter 16 · Skills: 34 code recipes, not JSON schema</h2>

### 16.1 Model

`skills_loader/loader.py:1-8`:

1. **Discovery** — scan `skills/<name>/SKILL.md` (+ optional `kernel.py`);  
2. **Progressive disclosure** — the system prompt only lists name + one-line summary; full text is pulled via `host.search_skills` / `load_skill`;  
3. **Sidecar gate** — `kernel.py` compile-check first.

Baseline `find skills -name SKILL.md` = **34**. Directory coverage: protein folding / docking / single-cell / literature / figures / remote compute-provider recipes / **retrosynthesis_planning** (the theme of this baseline PR), etc.

### 16.2 The essential difference vs Tool

| | Native Tool | Skill |
|---|---|---|
| Form | Python `Tool` subclass + JSON schema | Markdown recipe + optional sidecar |
| Registration | `TOOL_TYPES` | filesystem discovery |
| When it enters context | schema into the tool catalogue | default only an index; full text on demand |
| Failure mode | schema / permission | if the model does not load → the skill never runs (UI hard-injects with `/skill` instruction, `app.js:5590-5601`) |

bundled is read-only; user skills live in `<data_dir>/user-skills`; on name collision, bundled wins.

---

<h2 id="ch17">Chapter 17 · BYOC and `openai4s_compute_provider`</h2>

### 17.1 Division of labor

| Package | Runs where | Duty |
|---|---|---|
| `openai4s/compute/` | local daemon | registry, job orchestration, Host attach |
| `openai4s_compute_provider/` | **remote machine** | stdlib-only sandbox SDK; oneshot/repl |
| `skills/remote-compute-*` | skill tree | concrete provider shims (e.g. nvidia; **note `remote-compute-ssh` has only SKILL.md + README, no `provider.py`, a pure recipe not a shim, not a good shim example**) |

`__main__.py` emphasizes two-phase secret scrub: a generic scrub **before** importing `provider.py`, then the resident prologue erases again by the provider-declared `secret_env_prefixes`; credentials are read from stdin/fd-3, **not into env vars**.

### 17.2 Control-plane entry

Remote* Tools in the registry (Submit/Status/Result/Cancel/Close…) share approval and audit with `host.compute.*` / `host.fold`. `host.fold` (single-sequence folding class) follows a strict no-fabrication policy (CLAUDE.md).

Addressing forms in product docs commonly look like `ssh:` / `byoc:` style endpoints — the local machine only orchestrates; heavy compute is on your own GPU.

```mermaid
flowchart LR
    AGENT["Agent / Cell"] --> HOST["host.compute / Remote* tools"]
    HOST --> MGR["openai4s/compute manager"]
    MGR -->|SSH/BYOC| REM["remote GPU"]
    REM --> PROV["openai4s_compute_provider __main__"]
    PROV --> SHIM["skills/.../provider.py"]
```

---

<h2 id="ch18">Chapter 18 · Permissions, approval persist, biosecurity, egress</h2>

### 18.1 Multi-layer defense (independent layers, not substitutes)

| Layer | Module | Points |
|---|---|---|
| OS sandbox | `security/sandbox.py` | Seatbelt/bwrap; auto/enforce/off |
| Environment allowlist | `kernel/environment.py` | against secrets leaking into the worker |
| Permissions / approval | `openai4s/permissions.py` (771 lines, opencode-style allow/deny/ask gate; **note `security/permissions.py` is a different file**, only managing data-dir file permission bits, unrelated to approval) + `storage/permissions.py` Store tables | persistent rules; unattended default deny |

> ⚠️ **Verification correction (2026-08-04)**: the original draft wrote `security/permissions.py`, a **wrong attribution**. The approval broker is top-level `openai4s/permissions.py`; the same-named `security/permissions.py` (116 lines) tightens SQLite plaintext-credential default 0644, not approval logic.
| Code gate | `security/classifier.py` + loop `_pre_exec_gate` | refuse to run unsafe Cells |
| Biosecurity | `security/biosecurity.py` | trajectory screener; BLOCK stops the Cell |
| Injection screen | `security/injection.py` | untrusted output |
| Outbound | `egress.py` | suffix-domain allowlist when `OPENAI4S_EGRESS=allowlist` |
| bash token | `host/bash.py` | generation-bound one-shot |

### 18.2 Approval-restart semantics (an easy-to-get-wrong product detail)

`docs/architecture.md:121-128`:

- what is persisted is the **decision**, not the Python call stack;
- in-process can restore the exact blocked gate;
- after daemon restart: surface the request from SQLite; record a no-arg `permission_resolution` ledger marker; declare the old op was not executed; require an explicit continue;
- restart-only `once` grants expire in 15 minutes, exact-match conversation/tool/target, atomically consumed.

Test default posture (CLAUDE.md): `OPENAI4S_UNATTENDED_APPROVAL=deny`.

### 18.3 Honest boundary of egress

`egress.py:9-19` calls itself a **best-effort** fence on the host-tool boundary (`web_*` and static `bash` checks); default mode `off`. After opening `allowlist`, science DBs / package indexes / data warehouses are reachable; other domains need `request_network_access` widened via approval. It is **not** a substitute for the OS sandbox.

> ⚠️ **Verification correction (2026-08-04)**: the docstring at `egress.py:9-10` writes “openai4s does not ship an OS-level sandbox (Seatbelt/bubblewrap)”, **directly conflicting with `security/sandbox.py` (863 lines, real Seatbelt/bwrap implementation)** — that docstring is **stale**. The original draft still attributed the rewritten sentence to `egress.py:9-19`; a reader following the citation would read the opposite. Here we explicitly mark the egress docstring stale, rather than patching it for the author.

---

# Part VI · Kernel deep-dive and developer taste

> This chapter is one of the lecture’s centers of gravity. OpenAI4S’s authors (PKU–YuanKong joint lab, open source) publicly narrative-match Claude Science, but the real signature in source is not “yet another Agent while” — it is: **allergy to “false truth,” paranoia about dependency boundaries, and surgical discipline on facade files**. First read the kernel protocol through, then reverse-engineer ten build choices.

---

<h2 id="ch19">Chapter 19 · Kernel protocol: dup2, frame caps, and “wrong is worse than absent”</h2>

### 19.1 Technical chain (from Cell to Host and back)

```mermaid
sequenceDiagram
    participant M as 模型
    participant Eng as AgentEngine
    participant Ex as Web/Local Executor
    participant Km as Kernel.manager
    participant W as worker.py
    participant H as HostDispatcher
    participant Store as SQLite Store

    M->>Eng: assistant + tool_calls 或 ```python
    Eng->>Ex: route_action → CodeCell
    Ex->>Km: execute(code)
    Km->>W: {"type":"execute",...}
    Note over W: stdout→捕获；协议在高位 fd
    W->>Km: host_call(method,args)  [持锁]
    Km->>H: dispatcher(method,args)
    H->>Store: audit / permission / ledger
    H-->>Km: data | {"error":...}
    Km->>W: host_response
    W-->>Km: result(stdout, usage, error_lineno)
    Km-->>Ex: observation
    Ex-->>Eng: history + maybe CompletionSignal
```

Key invariants (CLAUDE.md + `manager.py` / `worker.py`):

1. **Single reader**: only `Kernel` reads protocol outbound; supervisor/watchdog forbidden from stealing frames.  
2. **Single in-flight host_call**: the whole RPC one lock; concurrency would interleave.  
3. **generation monotonic**: respawn must bump; old bash tokens / leases invalidate.  
4. **Soft-fail contract**: Host may return `{"error": msg}`, worker turns `RuntimeError` — the Cell sees an exception, the daemon does not crash.  
5. **Do not autoclose matplotlib in the worker**: the gateway must `savefig` first to capture Artifacts (CLAUDE.md explicit ban).

### 19.2 Why this kernel “has heavy taste”

Most agents make “run code” into:

```
tool Bash → subprocess → 文本结果 → 塞回上下文
```

OpenAI4S makes “run code” into:

```
持久 REPL + 协议分家 + 中途同步 Host RPC + 环境指纹 + 账本
```

The former optimizes “will the model use tools”; the latter optimizes “will scientific compute still be defensible a week later.” **Taste differs in problem definition, not loop syntax.**

### 19.3 Build thought-chain (reverse-engineered: what they believed first)

Restore a developer inner monologue in dependency order:

1. **If scientific state cannot persist, Code-as-Action is a joke** → do persistent kernel first, not pile 70 Tools first.  
2. **If print can smash the protocol, all observability is fake** → do dup2 track-switch first, then talk notebook.  
   > Verification correction (2026-08-04): the original draft wrote “dup2 / fd3·fd4,” mixing two mechanisms. Python `worker.py` uses `os.dup()` to get **kernel-allocated anonymous high fds** + `dup2(2,1)` aliasing fd1 to stderr, **no fd3/fd4 constants in the whole file**; `fd3/fd4` is **R-channel exclusive** (`r_kernel.py:12-13` “protocol OUT rides fd 3 / protocol IN rides fd 4”). Should be “Python rides anonymous high fds; R rides fixed fd3/fd4.” 
3. **If mid-cell cannot callback Host, research-loop count will explode** → inner-loop RPC and outer-loop tool_use must coexist.  
4. **If control and science compete in the same turn, audit and compute pollute each other** → `route_action` iron law.  
5. **If UI bubbles are the source of truth, restart will lie** → Ledger-first; bubbles are projections (`completions.py`).  
6. **If provenance can be “guessed”** → prefer recording the reason for absence.  
7. **If the core takes heavy deps** → boundaries blur, audit gets harder, the ¥9.9 narrative also cannot stand → stdlib hard constraint.  
8. **If facade files are whole-page rewritten** → compatibility/routing/transport contracts silently evaporate → surgical-edit convention written into CLAUDE.md / PR template.

```mermaid
flowchart TB
    P1["① 科学要持久状态"] --> P2["② 协议必须抗污染"]
    P2 --> P3["③ 需要 mid-cell Host RPC"]
    P3 --> P4["④ 控制/科学分平面"]
    P4 --> P5["⑤ 账本先于 UI"]
    P5 --> P6["⑥ 假真相不可接受"]
    P6 --> P7["⑦ 核心零依赖"]
    P7 --> P8["⑧ 巨石门面只许外科改"]
```

---

<h2 id="ch20">Chapter 20 · Ten development choices: reverse-engineering how they built it, and taste</h2>

> Form aligned with this series’ CodexMonitor “ten chapters of taste”: each choice gets **claim → source evidence → what was given up → taste tag**.

### Choice 1: counterpart the experience, do not copy the implementation — “Claude Science open-source recreation” is product narrative, not a fork

Claim: independently recreate with public architectural ideas (Code-as-Action, persistent kernel, host RPC, security layers), MIT open source; models can ride Ark ¥9.9.  
Evidence: README Acknowledgement; `docs/architecture.md` dual-plane figure; `ark` first-class in `llm/capabilities.py`.  
Given up: line-by-line closed-source compatibility, binding a single frontier model.  
Taste: **a vertical-domain open substitute**, trading closed flywheels for price and auditability.

### Choice 2: stdlib is a hard constraint, not a style preference

Claim: engine / urllib LLM / http.server+hand-written WS zero third-party; science libs only `try/except ImportError`.  
Evidence: `pyproject.toml` `dependencies = []`; CLAUDE.md “Never add a hard third-party import.”  
Given up: FastAPI, websockets libraries, httpx, ORM.  
Taste: **dependency discipline = trust-boundary discipline**. One fewer package, one fewer supply-chain and audit blind spot.

### Choice 3: the two planes do not compete — ReAct “everything is a tool” is explicitly refused

Claim: at most one action channel per turn; native tools first; sole `finalize_response` a separate case.  
Evidence: `actions.py:route_action`; forbidding registration of `bash`/`submit_output` (`registry.py:_FORBIDDEN_CONTROL_NAMES`).  
Given up: the “unity” of stuffing shell, submit, and scientific compute all into a JSON tool table.  
Taste: **unity yields to auditability**. Scientific compute needs a language; orchestration needs a schema.

### Choice 4: the completion signal is a product contract, not a model habit

Claim: chat/tool complete ≠ science complete; UI complete is a projection.  
Evidence: `agent/finalize.py`; `host.submit_output`; `server/completions.py`; a pure-submit Cell does not enter Notebook display.  
Given up: “the model said goodbye, so we’re done.”  
Taste: **downgrade “done” from natural language to a structured fact** — a research product must be able to answer “what metric did you actually submit.”

### Choice 5: kernel protocol designed for “worst character,” not ASCII fantasy

Claim: caps are derived from measurement; CJK/emoji surrogate pairs pull byte budget to 12×.  
Evidence: `worker.py` long comments on `_MAX_FRAME_BYTES` and test-driven correction.  
Given up: made-up constants, tacit “output will not be too big.”  
Taste: **write failure history in comments** — lab engineering culture, not tutorial engineering culture.

### Choice 6: false green is more shameful than red

Claim: a scenario that should fail, if it succeeds, benchmark/harness judges failure; stubs must not pollute the response-schema contract.  
Evidence: CLAUDE.md harness/workflows section; `stubbed_backend` marker; `capture_response_schemas.py --check`.  
Given up: the lazy score of “no exception = pass.”  
Taste: **treat the ability to refuse as half the product** — a science agent’s value is often “dare to say no.”

### Choice 7: monolith facades may exist, but whole-page rewrite is forbidden

Claim: `gateway.py` / `host_dispatch.py` / `store.py` / `app.js` / `worker.py` / `manager.py` / `sdk/host.py` (verification correction 2026-08-04: CLAUDE.md:98 lists 7 facade files; the original draft missed `sdk/host.py`) are stacked compatibility and routing contracts; new algorithms go into service/repo/tool.  
Evidence: CLAUDE.md “Edit the compatibility/composition facades surgically”; PR template checklist.  
Given up: a clean “ideal layering in one shot.”  
Taste: **admit historical debt, manage it with discipline, rather than manufacturing silent regressions with a big-bang rewrite**.

### Choice 8: Skills are recipes, not a plugin-SDK fantasy

Claim: `SKILL.md` + optional `kernel.py`; progressive disclosure; bundled wins name collisions; user skills cannot impersonate trust.  
Evidence: `skills_loader/loader.py`; 34 bundled skills; Customize CRUD into `user-skills`.  
Given up: making every scientific workflow a JSON tool (explosion) or dynamically importing arbitrary code as first-class.  
Taste: **put the extension surface on “code recipes”**; the trust surface still lives in Host and sandbox.

### Choice 9: security is stacked layers, default posture is paranoid

Claim: loopback, env allowlist, Seatbelt/bwrap, approval (headless deny), biosecurity, injection screen, egress (default off) are independent of each other.  
Evidence: `docs/security.md`; `OPENAI4S_UNATTENDED_APPROVAL=deny` test default; approval restart does not replay args.  
Given up: a single-layer “sandbox is omnipotent” narrative; the DX pleasure of headless auto-allow.  
Taste: **each layer only promises what it can promise**; egress calling itself best-effort is actually honesty.

### Choice 10: Web is a zero-build workbench, not a frontend division

Claim: `webui/` static, served from the working tree; edit is hot; no bundler.  
Evidence: `server/webui/app.js` nearly 10k lines still hand-written; CLAUDE.md “no build step.”  
Given up: React/Vite engineering aesthetics and component ecology.  
Taste: **product center of gravity is runtime and evidence, not frontend stack** — one language (Python) + static JS presses the ops surface, serving “a scientist one-clicks start on their machine.”

### Taste in one sentence

> **Hold the auditable boundary with pure stdlib, split orchestration from compute with two planes, fight silent confusion with a kernel protocol, fight false truth with ledger and provenance — engineering-wise paranoid-conservative, product-wise ambitious about the definition of scientific completion, architecture-wise prefer a monolith facade plus surgical discipline over a big rewrite that buys false cleanliness.**

---

# Part VII · Side-by-side and summary

<h2 id="ch21">Chapter 21 · Comparison table</h2>

| Dimension | Claude Code | Open Design | OpenAI Codex | OpenManus / Suna | **OpenAI4S** | Claude Science (closed-source) |
|---|---|---|---|---|---|---|
| Product seat | terminal coding Agent | design-Agent **host** (does not first-party the loop) | cloud+local coding Agent | general-task / browser Agent | **scientific Code-as-Action** | closed-source research workbench |
| Main loop | first-party tool loop | delegates to an external CLI | first-party | first-party ReAct variant | **dual-plane AgentEngine** | closed-source |
| Tool philosophy | Shell/editor first-class | filesystem skills fed to CLI | sandbox+tools | the more tools the better | **no shell Tool; bash in-kernel only** | unknown |
| Science kernel | no persistent research kernel | none | none | none | **persistent Python/R + mid-cell host RPC** | yes (closed-source) |
| Completion signal | session/task convention | CLI exit/artifact files | task-complete events | often a final message | **finalize ⊕ submit_output** | unknown |
| Frontend | TUI | Next.js heavy frontend | multi-end | Web | **zero-build static workbench** | proprietary UI |
| Core deps | Node ecology | large monorepo | Rust/TS etc. | Python heavy deps | **stdlib only** | cloud service |
| Extension | MCP/hooks/skills | 26 CLI + content library | MCP/plugins | tools/browser | **SKILL.md recipes + BYOC** | closed |
| Price narrative | Pro subscription | self-host + bring-your-own CLI | Plus/API | bring-your-own key | **¥9.9 Ark/Doubao** | high-price closed-source |

```mermaid
quadrantChart
    title Agent 产品地图（示意）
    x-axis 通用任务 --> 垂直领域
    y-axis 委托他人循环 --> 自研循环
    quadrant-1 垂直自研
    quadrant-2 通用自研
    quadrant-3 通用宿主
    quadrant-4 垂直宿主
    Claude-Code: [0.25, 0.8]
    Codex: [0.3, 0.75]
    OpenManus: [0.35, 0.7]
    Open-Design: [0.55, 0.2]
    OpenAI4S: [0.85, 0.78]
    Claude-Science: [0.9, 0.85]
```

---

<h2 id="ch22">Chapter 22 · Five distinctive design decisions and honest boundaries</h2>

### 22.1 Five distinctive decisions (condensed; expansion in Chapter 20)

1. **The two planes do not compete** — the `route_action` iron law splits control from science; an explicit refusal of ReAct “everything is a tool.”  
2. **Dual-track completion signals that do not impersonate each other** — `finalize_response` never enters the registry; `submit_output` / `bash` forbidden as control tools.  
3. **Mid-flight Host RPC** — “in-Cell synchronous callback” that a tool_use architecture cannot do.  
4. **Ledger-first + projection split** — the UI transcript is not truth; Action Ledger / Store is.  
5. **Pure-stdlib core** — trade dependency discipline for an auditable boundary and “science stack optional.”

### 22.2 Honest boundaries

| Boundary | Notes |
|---|---|
| Not a line-by-line clone of Claude Science | experience counterpart; model quality, lab integration, closed-source data flywheel cannot be equated |
| Notebook default read-only | need a REPL, explicitly turn on `OPENAI4S_NOTEBOOK_REPL` |
| egress default off | opening allowlist is the strong constraint; still an application-layer fence |
| R has no in-Cell completion | R is an analysis channel; completion narrative still leans Python/`finalize` |
| Gateway / app.js monolith | composition facades are huge; the convention is surgical edit; rewrite cost is high |
| Offline tests ≠ the whole gate | harness, response schema capture, browser smoke, Linux CI branches are independent gates |

### 22.3 When to pick it

- what you want is a **scientific-analysis Agent**, not another coding assistant;  
- you accept “the model writes code, the kernel holds state”;  
- you want the core auditable, few deps, attachable to cheap domestic models;  
- you need BYOC to put heavy compute on your own GPU.

When not to pick: pure software-engineering repo walks, needing a heavyweight browser operator, or a “zero-code, only click tools” ops Agent — those look more like the Claude Code / Manus track.

---

<h2 id="ch23">Chapter 23 · Source tour index</h2>

### 23.1 In reading order (suggested one afternoon)

| Order | Path | What to look at |
|---|---|---|
| 1 | `docs/architecture.md` | official dual-loop narrative |
| 2 | `agent/actions.py` | `route_action` iron law |
| 3 | `agent/engine.py` | `AgentEngine.run` while |
| 4 | `agent/finalize.py` | why finalize is not a Tool |
| 5 | `agent/loop.py` | CLI `Agent.run` assembly |
| 6 | `server/gateway.py` → `SessionRunner._loop` | Web assembly |
| 7 | `server/webui/app.js` → `send()` | 202 + sub first + admission |
| 8 | `host_dispatch.py` + `sdk/host.py` | orchestration envelope and kernel facade |
| 9 | `kernel/lazy.py` + `manager.py` + `worker.py` | science-plane protocol (**read the worker file-header comments first**) |
| 9b | this lecture Chapters 19–20 | kernel taste and ten choices |
| 10 | `tools/registry.py` | 70 tools and forbidden items |
| 11 | `llm/capabilities.py` + `transport.py` | ark and urllib |
| 12 | `skills_loader/loader.py` + `skills/*/SKILL.md` | recipe model |
| 13 | `security/sandbox.py` + `openai4s/egress.py` (verification correction: top-level, not `security/`) + `openai4s/permissions.py` (approval, not `security/permissions.py`) | security layers |
| 14 | `store.py` / `storage/snapshots.py` | persist and branching |
| 15 | `openai4s_compute_provider/` | BYOC remote SDK |

### 23.2 Symbol quick lookup

| Symbol | File |
|---|---|
| `AgentEngine.run` | `openai4s/agent/engine.py` |
| `route_action` / `FinalizeAction` / `CodeCell` | `openai4s/agent/actions.py` |
| `with_finalize_response` | `openai4s/agent/finalize.py` |
| `RuntimeActionLedger` | `openai4s/agent/ledger.py` |
| `Agent.run` / `LazyKernel` assembly | `openai4s/agent/loop.py` |
| `_execute_read_only_waves` | `openai4s/agent/control.py` |
| `HostDispatcher` | `openai4s/host_dispatch.py` |
| `host.submit_output` / `host.bash` | `openai4s/sdk/host.py` |
| `TOOL_TYPES` | `openai4s/tools/registry.py` |
| `SessionRunner._loop` | `openai4s/server/gateway.py` |
| `send` / `admissionId` | `openai4s/server/webui/app.js` |
| `WebActionExecutor` / `WebEventSink` | `openai4s/server/agent_run.py` |
| `SessionBranchingService` | `openai4s/server/session_branching.py` |
| `SUPPORTED_WIRES` / `ark` | `openai4s/llm/capabilities.py` |
| `SkillLoader` | `openai4s/skills_loader/loader.py` |

### 23.3 Related docs in this repo

- series comparison and catalog: `总目录.md`, source analyses under `项目分析/`  
- companion tutorial: `从零构建OpenAI4S-开发全流程教程.md` / `从零构建OpenAI4S-开发全流程教程.html`  
- upstream: https://github.com/PKU-YuanGroup/OpenAI4S  

---

## Closing

OpenAI4S’s source charm is not “yet another Agent while loop,” but that it **splits the loop in two and still refuses to let them compete for a turn**, and at the kernel-protocol layer shows an almost allergic engineering taste against “silent confusion / fake provenance”: the JSON control plane owns auditable orchestration and permissions; the Python/R science plane owns leaving real compute in a persistent kernel, and mid-cell `host` RPC patches the piece tool_use is missing. Dual-track completion of `finalize_response` and `host.submit_output`, `TOOL_TYPES`’s explicit absence of shell, and Web details like `sub()`-before-`POST` / client `admissionId` — “small enough to look like a bugfix, large enough to look like a product contract” — together constitute its differentiation versus the Claude Code / Open Design / Manus family.

Baseline stays at `85e9fa0`. When checking this lecture, treat that commit as authoritative; as mainline continues to evolve, first check whether the four load-bearing walls still hold: `agent/actions.py`, `agent/engine.py`, `tools/registry.py`, and `server/webui/app.js::send`.

---

*Document-generation note: written in this series’ “PM + senior engineer” register; claims try to return to source. Companion HTML: `OpenAI4S-解析.html`.*