Compare commits

...

6 Commits

Author SHA1 Message Date
cfdaily a953fc0bc7 [moz] fix(spawner): PromptContext 缺少 event_type/event_data 导致通知显示「事件类型: 未知」
CI / lint (pull_request) Successful in 7s
CI / test (pull_request) Successful in 29s
CI / frontend (pull_request) Successful in 12s
CI / notify-on-failure (pull_request) Successful in 0s
根因:spawner 构建 PromptContext 时只传了 action_type/action_steps,
遗漏了 event_type 和 event_data。ToolchainContextSection.render() 从
context.event_type 取值,为空时回退到 '未知'。

修复:从 must_haves JSON 同时提取 event_type 和 context(→event_data)。
2026-06-17 07:19:04 +08:00
pangtong-fujunshi 7f17ee69d7 Merge PR #82: [moz] docs: §18 Mail Handler Verify/Prompt 强化设计
Deploy / ci (push) Failing after 7s
Deploy / deploy (push) Has been skipped
Deploy / notify-deploy-failure (push) Successful in 1s
Deploy / notify-deploy-success (push) Successful in 0s
2026-06-16 23:16:08 +00:00
cfdaily f1e513cba2 [moz] docs: §18 Mail Handler Verify/Prompt 强化设计
CI / lint (pull_request) Successful in 7s
CI / test (pull_request) Successful in 28s
CI / frontend (pull_request) Successful in 11s
CI / notify-on-failure (pull_request) Successful in 0s
2026-06-16 23:14:23 +00:00
pangtong-fujunshi 627982db09 Merge PR #81: [moz] feat: Runaway Guard per-task dispatch 上限
Deploy / ci (push) Failing after 7s
Deploy / deploy (push) Has been skipped
Deploy / notify-deploy-failure (push) Successful in 1s
Deploy / notify-deploy-success (push) Successful in 0s
2026-06-16 23:10:42 +00:00
cfdaily 9ec601d747 [moz] feat: Runaway Guard per-task dispatch 上限
CI / lint (pull_request) Successful in 8s
CI / test (pull_request) Successful in 29s
CI / frontend (pull_request) Successful in 12s
CI / notify-on-failure (pull_request) Successful in 1s
§15 Runaway Guard — per-task dispatch_count 上限,防止无限循环 dispatch

问题:mail/toolchain task 走 handler auto-working(跳过 claim),不受
claim_timeout 3 次重试兜底保护。如果反复 spawn 但永远到不了 done/failed,
会无限循环消耗资源(实际案例:2026-06-15 mention 重复投递事件)。

设计:
- tasks 表新增 dispatch_count 字段
- 每次 ticker 成功 dispatch 时递增
- dispatch_count >= 10 时自动标 failed(reason=runaway_guard)
- 覆盖所有非终态(pending/working/claimed)
- 参考 Hermes v0.13 §3 Per-Task 重试上限

改动文件:
- src/blackboard/db.py: _safe_add_column dispatch_count
- src/blackboard/models.py: Task dataclass 加 dispatch_count
- src/daemon/ticker.py: dispatch 递增 + _check_timeouts runaway guard
- docs/design/15-runaway-guard.md: 设计文档
- tests/integration/test_ticker_integration.py: E13 测试 3 个

测试:456 passed, 3 skipped
2026-06-16 23:10:27 +00:00
pangtong-fujunshi cc5c7f5ad1 Merge PR #80
Deploy / ci (push) Failing after 8s
Deploy / deploy (push) Has been skipped
Deploy / notify-deploy-failure (push) Successful in 0s
Deploy / notify-deploy-success (push) Successful in 0s
2026-06-16 14:49:17 +00:00
7 changed files with 426 additions and 2 deletions
+215 -2
View File
@@ -1,10 +1,11 @@
---
title: "TaskTypeRegistry + Handler 架构重构"
created: 2026-06-10
version: v3.0
version: v3.1
---
> 状态: ✅ 已完成(Step 1-5 全部合并,394 passed
> v3.1 新增 §18Mail Handler Verify/Prompt 强化(2026-06-16,进行中)
# §1 现状分析(v3.0 更新说明:§1-§13 保留原样,新增 §14-§18,更新 §3/§5/§7
@@ -985,7 +986,219 @@ handler.post_complete(task_id, agent_id, outcome, db_path)
---
## §14. Mail 失败通知机制
## §18. Mail Handler Verify/Prompt 强化
> 日期:2026-06-16 | 作者:庞统 | 状态:方向 1-5 全部已确认
## 18.1 问题背景
### 触发事件
2026-06-12 daemon 重启后,_mail DB 中积压的 E2E 测试遗留邮件(5/18~6/1 创建,type=requestperformative="text")被 dispatch 给 agent。agent 正常处理并输出文本(如"已阅,无需处理"),但 `verify_completion` 判定 no_reply → 标 failed → 触发 `notify_mail_failed` → 产生 38 封 `[投递失败]` 通知邮件,每 ~2.5 分钟一轮,持续 10 轮。
### 根因链
```
E2E 测试脚本 bugtype="text"
→ mail_routes.py 不校验 type 值,直接透传
→ performative="text" ≠ "inform" → 走 _check_reply
→ _check_reply 查 in_reply_to taskagent 没用 Mail API 回复
→ verify 失败 → on_failure 标 failed
→ notify_mail_failed 发 [投递失败] 通知
→ 通知本身也是 task,循环触发
```
### 三种 handler verify 对比
| 维度 | TaskHandler | MailHandler | ToolchainHandler |
|------|------------|-------------|------------------|
| verify 信号 | output / comment(≥50字) / terminal_status(三信号) | in_reply_to task(单信号) | action_report / output / comment(≥20字)(三层 fallback |
| inform 处理 | N/A | 直接通过(不检查执行证据) | N/A |
| verify 失败后 | **留 working**(覆盖 post_complete | **标 failed**base post_complete + mail on_failure | 标 failedbase post_complete + tc on_failure |
| agent 输出持久化 | 靠 agent 主动 POST output/comment | **无**agent 输出只在内存) | 靠 agent 主动 POST action_report |
**关键发现**
1. MailHandler 继承 BaseTaskHandler,未覆盖 `post_complete` → verify 失败时走 base 的 `on_failure` → 标 failed
2. TaskHandler 覆盖了 `post_complete` → verify 失败时留 working,让 ticker 重试
3. MailHandler 的 verify 只有 `in_reply_to` 一条路径,没有 fallback
4. inform 类型直接通过(`VerifyResult(True)`),不检查任何执行证据——inform 是"无需回复"不是"无需检查"
5. E2E 测试用 `TestClient(app)` 写生产 `_mail DB`,且测试脚本用了非标准 `type="text"`
## 18.2 修复方向
### 方向 1mail verify 对齐 toolchain 模式(✅ 已确认)
**问题**mail verify 只有 in_reply_to task 一条路径。task/toolchain 都有多层 fallbackoutputs / comments)。
**方案**mail 对齐 toolchain 模式——prompt 加 action report 要求,verify 优先查 action_report → fallback outputs → fallback comments。in_reply_to 回复邮件从唯一信号降为 request 类型的第 4 优先级信号。
#### prompt 强化(MailApiSection
参照 ToolchainApiSection,在 mail prompt 中追加 action report 要求:
```
### 完成后必须提交 action report
执行完邮件处理后,必须提交 action report
curl -s -X POST "http://localhost:8083/api/projects/_mail/tasks/{task_id}/comments" \
-H "Content-Type: application/json" \
-d '{"author": "{agent_id}", "comment_type": "action_report", "body": "处理结果摘要"}'
⚠️ 不提交 action report 的任务会被标记为 failed。
```
#### verify 改造(MailHandler.verify_completion
```python
def verify_completion(self, task_id, db_path) -> VerifyResult:
performative = self._parse_performative(task_id, db_path)
# 1. 优先检查 action_report comment(所有类型通用)
if self._has_action_report(task_id, db_path):
return VerifyResult(True, "has_action_report", "action_report found")
# 2. fallback: outputs
if self._has_outputs(task_id, db_path):
return VerifyResult(True, "has_output", f"output_count={count}")
# 3. fallback: 有实质内容的 comment(≥20字,非 system
if self._has_comment(task_id, db_path):
return VerifyResult(True, "has_comment", f"comment_count={count}")
# 4. request 特有:检查 in_reply_to 回复邮件
if performative == "request":
if self._check_reply(task_id, db_path):
return VerifyResult(True, "has_reply", "in_reply_to found")
return VerifyResult(False, "no_action",
"no action_report, no output, no comment, no reply")
```
注意:action_report 提交到 moziplus DBcomments 表),不是 Gitea。Gitea comment 是跨 agent 协作用的,不是 verify 检查的依据。
### 方向 2:prompt 约束强化(✅ 已确认)
**问题**:当前 mail prompt 只给了 curl 示例,没有硬约束要求 agent 必须输出处理结果。agent 判断"已阅"后直接跳过,不创建 in_reply_to task。
**方案**mail request/inform prompt 加 JSON 输出约束(参考 toolchain 的 Red Flags 模式)。
#### MailContextSection 强化
**request 类型**追加:
```
### 输出要求
- 你的回复必须包含对邮件的实际处理结果
- 如果是第一次收到:正常处理,输出处理结果
- 如果是重复邮件(你之前处理过相同 ID 的邮件):输出"此前已处理" + 之前的处理结果摘要
- ⚠️ "已阅""无需处理"不是有效处理结果
```
**inform 类型**追加:
```
### 输出要求
- 你的回复必须确认已处理(读取/执行/记录),不能只说"已阅"
- 如果是重复邮件:输出"此前已处理" + 处理结果摘要
- ⚠️ "已阅"不是有效输出
```
**MailConstraintsSection** 追加 Red Flags
```
| Agent 想法 | Red Flag 驳回 |
|------------|--------------|
| "已阅即可" | ❌ 错!必须输出处理结果或确认执行 |
| "重复邮件忽略" | ❌ 错!输出"此前已处理" + 结果摘要 |
| "无需回复" | ❌ 错!request 必须回复,inform 必须确认处理 |
```
### 方向 3:inform 也要检查执行证据(✅ 已确认)
**问题**:当前 inform verify 直接返回 `VerifyResult(True)`,不检查任何执行证据。inform 是"无需回复"不是"无需检查"。
**方案**inform verify 改为检查 agent 是否有实质输出(comment/output),和 request 走不同的验证路径但都需要验证。
**改动文件**`src/daemon/mail_handler.py` `verify_completion` 方法
### 方向 4verify 失败保持 working(✅ 已确认)
**问题**MailHandler 继承 BaseTaskHandlerverify 失败时走 base 的 `on_failure` → 标 failed。而 TaskHandler 覆盖了 `post_complete`verify 失败时留 working。
**原始设计意图**(§2 设计文档):"不通过 → 留 workingticker 重查(最多 3 次,然后标 failed"。
**方案**MailHandler 覆盖 `post_complete`verify 失败时不标 failed,保持 working。ticker 的 `_check_timeouts` 超时兜底:
- `check_completion` 通过(有回复)→ done
- `check_completion` 不通过 → 超时后标 failed
- Runaway Guard(§15 dispatch_count ≥ 10)兜底防止无限循环
**改动文件**`src/daemon/mail_handler.py`,新增 `post_complete` 覆盖
### 方向 5type 校验 + E2E 修复 + DB 清理(✅ 已确认)
#### 5.1 mail_routes.py type 校验
**问题**`mail_type = body.get("type")` 直接透传,传什么存什么。`"text"` 不是标准值。
**方案**:创建时校验 type 只允许 `inform` / `request`,非法值默认 `request`。
```python
mail_type = body.get("type")
if mail_type is None:
mail_type = "inform" if in_reply_to else "request"
elif mail_type not in ("inform", "request"):
# 非标准值,校正为默认值
mail_type = "inform" if in_reply_to else "request"
```
**改动文件**`src/api/mail_routes.py`
#### 5.2 _parse_performative 容错
**问题**`meta.get("performative", meta.get("type", "request"))` 当 performative="text" 时返回 "text",不等于 "inform" → 走 _check_reply。
**方案**:只认 `inform` 和 `request` 两个值,其他一律当 `request`。
```python
def _parse_performative(self, task_id, db_path) -> str:
raw = meta.get("performative", meta.get("type", "request"))
if raw == "inform":
return "inform"
return "request" # 非标准值一律当 request
```
**改动文件**`src/daemon/mail_handler.py` `_parse_performative` 方法
#### 5.3 E2E 测试修复
**问题**`tests/e2e/test_e2e_v27.py` 用 `type="text"` 创建测试邮件,且用 `TestClient(app)` 写生产 `_mail DB`。
**修复**
1. `type="text"` 全部改为 `type="inform"` 或 `type="request"`
2. E2E 测试跑完后清理测试邮件(`mail_ids` 列表中记录的 task
**改动文件**`tests/e2e/test_e2e_v27.py`
#### 5.4 生产 DB 清理
**问题**:生产 `_mail DB` 中残留大量 E2E 测试邮件(5/18~6/3 创建的"筛选测试""详情测试""已读测试""任务分配"等)。
**方案**:手动清理这些测试残留(一次性操作,不需要代码改动)。
## 18.3 影响范围
| 文件 | 改动类型 | 影响面 |
|------|---------|--------|
| `src/daemon/mail_handler.py` | verify + post_complete + prompt section | MailHandler 核心逻辑 |
| `src/api/mail_routes.py` | type 校验 | Mail API 创建入口 |
| `tests/e2e/test_e2e_v27.py` | type 值修正 + 清理 | E2E 测试 |
| 生产 `_mail DB` | 清理测试残留 | 一次性操作 |
## 18.4 验证计划
1. 单元测试:mail_handler verify/prompt 变更
2. 集成测试:mail dispatch → verify → done/working 全链路
3. 回归测试:`pytest -m "not e2e"` 全量
4. 手工验证:创建 inform/request 邮件,确认 verify 行为正确
---
# §14. Mail 失败通知机制
### 20.1 背景
+61
View File
@@ -0,0 +1,61 @@
# §15 Runaway Guard — Per-Task Dispatch 上限
> 设计文档 v1.0 | 2026-06-16
## 问题
mail/toolchain task 走 handler auto-working(跳过 claim 阶段),不受 claim_timeout 的 3 次重试兜底保护。如果一个 auto-working task 反复 spawn 但永远到不了 done/failed,会无限循环消耗资源。
### 实际案例
2026-06-15 mention 重复投递事件:`spawn_full_agent``use_main_session=True` 时返回 `None`ticker `_process_mentions` 误判为失败,每次 tick(30s)都重试。同一 mention 投递了 4 次,直到 retry_count 达到 mention_queue 的 5 次上限才停止。
直接根因已由 PR #80 修复,但如果类似 bug 再次出现,当前没有任何机制阻止 task 层面的无限循环。
## 设计
### 机制
tasks 表新增 `dispatch_count` 字段,每次 ticker 成功 dispatch 一个 task 时递增。当 `dispatch_count >= 10`(全局默认)时,自动标 failed。
### 默认值选择
全局默认 10 次。参考 Hermes v0.13 Best Practices §3 "Per-Task 重试上限"
- 简单任务重试 1 次
- 复杂任务重试 3 次
- crash recovery3 次)+ api_retry3 次)余量 = ~10 次
### 适用范围
所有 task 类型(task/mail/toolchain),所有非终态(pending/working/claimed)。
### 检查时机
`_check_timeouts` 方法开头,先于现有的 claimed/working 超时检查执行。
### 与现有机制的关系
| 机制 | 覆盖场景 | 触发动作 |
|------|---------|---------|
| claim_timeout retry_count >= 3 | 广播任务无人认领 | 升级庞统 |
| crash_limit 3/30min | working 状态 crash | 标 failed |
| api_retry_count | API 连续失败 | 标 failed |
| 续杯 max_retries 3 | 续杯耗尽 | 标 failed |
| working timeout | working 超时 | 标 failed 或 done |
| **runaway_guard 10 次** | **任何状态的无限循环** | **标 failed** |
runaway_guard 是最后一道防线,覆盖所有其他机制遗漏的循环场景。
## 改动文件
| 文件 | 改动 |
|------|------|
| `src/blackboard/db.py` | `_safe_add_column(conn, "tasks", "dispatch_count", "INTEGER DEFAULT 0")` |
| `src/blackboard/models.py` | Task dataclass 加 `dispatch_count: int = 0` |
| `src/daemon/ticker.py` | `_dispatch_pending` / `_dispatch_reviews` 递增 dispatch_count`_check_timeouts` 加 runaway guard 检查 |
## 参考
- Hermes v0.13 Kanban Best Practices §3 "Per-Task 重试上限"
- 实际案例:2026-06-15 mention 重复投递事件(PR #80 修复了直接根因,runaway guard 作为兜底)
+1
View File
@@ -117,6 +117,7 @@ def _migrate_v28(conn: sqlite3.Connection) -> None:
_safe_add_column(conn, "tasks", "round_count", "INTEGER DEFAULT 0")
_safe_add_column(conn, "tasks", "resumed_from", "TEXT")
_safe_add_column(conn, "tasks", "dispatch_count", "INTEGER DEFAULT 0")
# 3. checkpoints 表(M3
conn.execute("""CREATE TABLE IF NOT EXISTS checkpoints (
+2
View File
@@ -41,6 +41,8 @@ class Task:
resumed_from: Optional[str] = None # 暂停前状态,恢复时回到原状态
# v2.9 四相循环
round_count: int = 0 # 庞统 review 轮次计数
# §15 Runaway Guard
dispatch_count: int = 0 # 被 ticker dispatch 的总次数
# v2.8 归档
archived: bool = False
archived_at: Optional[str] = None
+5
View File
@@ -288,6 +288,8 @@ class AgentSpawner:
mail_type = ""
action_type = ""
action_steps = []
event_type = ""
event_data = {}
try:
meta = json.loads(must_haves) if must_haves else {}
from_agent = meta.get("from", "")
@@ -295,6 +297,8 @@ class AgentSpawner:
# toolchain 字段提取
action_type = meta.get("action_type", "")
action_steps = meta.get("steps", [])
event_type = meta.get("event_type", "")
event_data = meta.get("context", {})
except Exception:
pass
ctx = PromptContext(
@@ -304,6 +308,7 @@ class AgentSpawner:
spawn_type=spawn_type,
from_agent=from_agent, mail_type=mail_type,
action_type=action_type, action_steps=action_steps,
event_type=event_type, event_data=event_data,
)
return handler.build_prompt(ctx)
+51
View File
@@ -1084,6 +1084,19 @@ Parent Task ID: {parent_task.id}
broadcast_ids = await self._broadcast_claim(broadcast_tasks, db_path, project_id)
dispatched.extend(broadcast_ids)
# §15 Runaway Guard: 统一递增 dispatch_count
if dispatched:
conn = get_connection(db_path)
try:
for tid in dispatched:
conn.execute(
"UPDATE tasks SET dispatch_count = COALESCE(dispatch_count, 0) + 1 WHERE id=?",
(tid,),
)
conn.commit()
finally:
conn.close()
return dispatched
async def _broadcast_claim(self, tasks: list, db_path: Path,
@@ -1376,6 +1389,19 @@ Parent Task ID: {parent_task.id}
except Exception:
logger.exception("Review dispatch failed for %s", task.id)
# §15 Runaway Guard: 统一递增 dispatch_count (review)
if dispatched:
conn = get_connection(db_path)
try:
for tid in dispatched:
conn.execute(
"UPDATE tasks SET dispatch_count = COALESCE(dispatch_count, 0) + 1 WHERE id=?",
(tid,),
)
conn.commit()
finally:
conn.close()
return dispatched
# ------------------------------------------------------------------
@@ -1388,6 +1414,31 @@ Parent Task ID: {parent_task.id}
reclaimed: List[str] = []
now = datetime.utcnow() # UTC,与 SQLite datetime('now') 一致
# §15 Runaway Guard: per-task dispatch_count 上限检查
# 覆盖所有状态,防止无限循环 dispatch
MAX_DISPATCH_COUNT = 10
for status_to_check in ("pending", "working", "claimed"):
tasks_to_check = queries.tasks_by_status(status_to_check)
for task in tasks_to_check:
dispatch_count = getattr(task, 'dispatch_count', 0) or 0
if dispatch_count >= MAX_DISPATCH_COUNT:
conn = get_connection(db_path)
try:
ok = self._transition_status(
conn, task.id, "failed",
agent="daemon",
detail={"reason": "runaway_guard",
"dispatch_count": dispatch_count,
"message": f"dispatch {dispatch_count} 次仍未完成,自动标 failed"},
)
if ok:
reclaimed.append(task.id)
logger.error(
"Task %s: runaway guard triggered (dispatch_count=%d, status=%s), marking failed",
task.id, dispatch_count, status_to_check)
finally:
conn.close()
# claimed 超时 → 重置为 pending(如果 retry_count >= 3 则升级庞统)
claimed = queries.tasks_by_status("claimed")
for task in claimed:
@@ -543,3 +543,94 @@ class TestCheckTimeoutsUnified:
reclaimed = ticker._check_timeouts(db_path)
assert "t-review-dead" not in reclaimed
# ---------------------------------------------------------------------------
# E13: §15 Runaway Guard — per-task dispatch_count 上限
# ---------------------------------------------------------------------------
class TestRunawayGuard:
"""E13: dispatch_count >= 10 → 自动标 failed(覆盖所有非终态)"""
@pytest.fixture
def guard_project(self, tmp_path):
"""创建项目 + 任务"""
data_root = tmp_path / "projects"
registry = ProjectRegistry(data_root)
registry.create_project("guard-proj", "Guard Test", agents=["agent-a"])
db_path = data_root / "guard-proj" / "blackboard.db"
bb = Blackboard(db_path)
return registry, db_path, bb
def test_runaway_guard_triggers_working(self, guard_project):
"""E13.1: working 状态 dispatch_count >= 10 → 标 failed"""
registry, db_path, bb = guard_project
bb.create_task(Task(
id="t-runaway", title="Runaway Task", status="working",
assigned_by="daemon", current_agent="agent-a",
))
conn = bb._conn()
try:
conn.execute(
"UPDATE tasks SET dispatch_count = 10 WHERE id = ?", ("t-runaway",))
conn.commit()
finally:
conn.close()
ticker = Ticker(registry, tick_interval=30)
reclaimed = ticker._check_timeouts(db_path)
assert "t-runaway" in reclaimed
task = Queries(db_path).task_by_id("t-runaway")
assert task.status == "failed"
def test_runaway_guard_triggers_pending(self, guard_project):
"""E13.2: pending 状态 dispatch_count >= 10 → 标 failed"""
registry, db_path, bb = guard_project
bb.create_task(Task(
id="t-pending-runaway", title="Pending Runaway", status="pending",
assigned_by="daemon",
))
conn = bb._conn()
try:
conn.execute(
"UPDATE tasks SET dispatch_count = 10 WHERE id = ?",
("t-pending-runaway",))
conn.commit()
finally:
conn.close()
ticker = Ticker(registry, tick_interval=30)
reclaimed = ticker._check_timeouts(db_path)
assert "t-pending-runaway" in reclaimed
task = Queries(db_path).task_by_id("t-pending-runaway")
assert task.status == "failed"
def test_runaway_guard_not_triggered(self, guard_project):
"""E13.3: dispatch_count < 10 → 正常流程不受影响"""
registry, db_path, bb = guard_project
bb.create_task(Task(
id="t-normal", title="Normal Task", status="working",
assigned_by="daemon", current_agent="agent-a",
))
conn = bb._conn()
try:
conn.execute(
"UPDATE tasks SET dispatch_count = 5 WHERE id = ?", ("t-normal",))
conn.commit()
finally:
conn.close()
ticker = Ticker(registry, tick_interval=30)
reclaimed = ticker._check_timeouts(db_path)
assert "t-normal" not in reclaimed
task = Queries(db_path).task_by_id("t-normal")
assert task.status == "working"