auto-sync: 2026-04-27 21:35:37
This commit is contained in:
cfdaily
@@ -68799,3 +68799,21 @@ fatal: refusing to merge unrelated histories
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
/bin/sh: ./management/sanguo_auto_sync/auto-sync.sh: No such file or directory
|
||||
|
||||
@@ -0,0 +1,432 @@
|
||||
# 群晖NAS部署sanguo_vnpy回测服务方案
|
||||
|
||||
**编制人**：姜维 伯约
|
||||
**日期**：2026-04-27
|
||||
**状态**：调研完成，待实施
|
||||
|
||||
---
|
||||
|
||||
## 一、问题复现与根因分析
|
||||
|
||||
### 1.1 现象
|
||||
|
||||
从局域网任意机器访问：
|
||||
- `curl http://192.168.2.154:8888/` → ✅ 302（Jupyter正常）
|
||||
- `curl http://192.168.2.154:8088/api/backtest/health` → ❌ Connection refused
|
||||
|
||||
### 1.2 根因
|
||||
|
||||
**三重问题叠加**：
|
||||
|
||||
| # | 问题 | 影响 |
|
||||
|---|------|------|
|
||||
| **1** | entrypoint.sh 未启动 backtest-service | 容器内8088端口根本无进程监听 |
|
||||
| **2** | 容器内代码与本地不同步 | `start_backtest_service.py` 仍用旧import路径 `from scripts.backtest_service.main` |
|
||||
| **3** | main.py 使用相对导入 `from .config` | 无法作为独立脚本直接运行，需通过包方式启动 |
|
||||
|
||||
**证据**：
|
||||
|
||||
```bash
|
||||
# 容器内进程 — 无uvicorn/FastAPI
|
||||
$ docker exec sanguo_vnpy ss -tlnp
|
||||
LISTEN 0.0.0.0:22 (sshd)
|
||||
LISTEN 0.0.0.0:8888 (jupyter-lab)
|
||||
# 8088 无监听！
|
||||
|
||||
# 容器内启动backtest-service报错
|
||||
$ docker exec sanguo_vnpy python3 /app/scripts/start_backtest_service.py
|
||||
ModuleNotFoundError: No module named 'scripts'
|
||||
|
||||
# Docker端口映射正常
|
||||
$ iptables -t nat -L DOCKER -n | grep 8088
|
||||
DNAT tcp 0.0.0.0/0 0.0.0.0/0 tcp dpt:8088 to:172.17.0.2:8088
|
||||
```
|
||||
|
||||
### 1.3 排除项
|
||||
|
||||
- ❌ 端口映射问题：iptables DNAT规则正确，8088→172.17.0.2:8088
|
||||
- ❌ DSM防火墙问题：Jupyter使用同模式可访问
|
||||
- ❌ Docker网络问题：bridge模式正常工作
|
||||
- ❌ 进程崩溃问题：根本没启动过
|
||||
|
||||
---
|
||||
|
||||
## 二、现有资源盘点
|
||||
|
||||
### 2.1 NAS硬件
|
||||
|
||||
| 项目 | 规格 |
|
||||
|------|------|
|
||||
| 型号 | DS216+II |
|
||||
| CPU | Intel Celeron N3060 @ 1.60GHz (2核) |
|
||||
| 内存 | 8GB (可用约6.4GB) |
|
||||
| 磁盘 | 3.5TB 总量, 1.6TB 可用 |
|
||||
| Docker | 20.10.3 (需sudo) |
|
||||
|
||||
### 2.2 当前容器状态
|
||||
|
||||
```
|
||||
容器ID: 8fc55af3d27d
|
||||
镜像: sanguo_vnpy:with-scripts
|
||||
状态: Up 13 days
|
||||
端口: 2222→22, 8000→8000, 8080→8080, 8088→8088, 8888→8888, 2018→2018, 4102→4102
|
||||
```
|
||||
|
||||
### 2.3 代码状态
|
||||
|
||||
| 文件 | 容器内版本 | 本地版本 | 问题 |
|
||||
|------|-----------|---------|------|
|
||||
| `start_backtest_service.py` | `from scripts.backtest_service.main` (旧) | `from main import main` (新) | **容器内代码过旧** |
|
||||
| `backtest-service/main.py` | `from .config import settings` | 同 | **相对导入无法直接运行** |
|
||||
| `entrypoint.sh` | 不含backtest-service | 同 | **缺失启动命令** |
|
||||
|
||||
### 2.4 数据资源（NAS SMB共享）
|
||||
|
||||
```
|
||||
/Volumes/stock/A股数据/日线数据/ — 日线历史行情
|
||||
/Volumes/stock/A股数据/分钟线数据/ — 分钟线数据
|
||||
/Volumes/stock/A股数据/财务数据/ — 财务数据
|
||||
/Volumes/stock/sanguo_vnpy/data/ — 项目数据
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 三、部署方案
|
||||
|
||||
### 3.1 方案选择
|
||||
|
||||
| 方案 | 描述 | 优点 | 缺点 |
|
||||
|------|------|------|------|
|
||||
| **A: 原地修复** | docker cp更新文件+修改entrypoint | 最快，无需重建镜像 | 容器重建后丢失 |
|
||||
| **B: 重建镜像** | 修复代码后docker build | 持久化，可复现 | 构建耗时(约10-20min) |
|
||||
| **C: Volume挂载** | 代码通过volume挂载 | 代码更新无需重建 | 需调整容器启动 |
|
||||
|
||||
**推荐方案：A+B组合**
|
||||
- 先用方案A快速验证（5分钟内出结果）
|
||||
- 验证通过后用方案B持久化（下班后或闲时构建）
|
||||
|
||||
### 3.2 架构图
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 群晖 NAS 192.168.2.154 │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────┐ │
|
||||
│ │ Docker Container: sanguo_vnpy │ │
|
||||
│ │ │ │
|
||||
│ │ entrypoint.sh │ │
|
||||
│ │ ├── sshd -D (22) ──→ 2222 │ │
|
||||
│ │ ├── jupyter-lab (8888) ──→ 8888 ✅ 已有 │ │
|
||||
│ │ └── backtest-service (8088) ──→ 8088 🆕 新增 │ │
|
||||
│ │ │ │
|
||||
│ │ /app/scripts/backtest-service/ │ │
|
||||
│ │ /app/backtest_jobs/ (回测结果) │ │
|
||||
│ │ /app/data/ (volume→/volume1/stock/A股数据) │ │
|
||||
│ └─────────────────────────────────────────────────────────┘ │
|
||||
│ ↑ Docker bridge (172.17.0.0/16) │
|
||||
│ ↑ iptables DNAT: 8088→172.17.0.2:8088 │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
↑
|
||||
局域网 192.168.2.0/24
|
||||
↑
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ Mac mini │ │ Windows Node │ │ 其他将军 │
|
||||
│ 192.168.2.153 │ │ 192.168.2.33 │ │ │
|
||||
│ curl :8088 │ │ 回测任务提交 │ │ curl :8088 │
|
||||
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||
```
|
||||
|
||||
### 3.3 修复步骤（方案A：快速验证）
|
||||
|
||||
#### Step 1: 修复main.py的相对导入问题
|
||||
|
||||
将 `backtest-service/main.py` 中的相对导入改为绝对导入，使其可直接运行：
|
||||
|
||||
```python
|
||||
# 修改前（相对导入，不能直接 python main.py）
|
||||
from .config import settings
|
||||
from .api import router
|
||||
from .task_queue import task_queue
|
||||
|
||||
# 修改后（支持两种方式）
|
||||
import sys
|
||||
import os
|
||||
# 当作为包的子模块时，.config 正常；当直接运行时，需要添加路径
|
||||
if __name__ == "__main__" or __package__ is None:
|
||||
_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
if _dir not in sys.path:
|
||||
sys.path.insert(0, _dir)
|
||||
from config import settings
|
||||
from api import router
|
||||
from task_queue import task_queue
|
||||
else:
|
||||
from .config import settings
|
||||
from .api import router
|
||||
from .task_queue import task_queue
|
||||
```
|
||||
|
||||
#### Step 2: 更新容器内代码
|
||||
|
||||
```bash
|
||||
# 从本地复制修复后的代码到容器
|
||||
docker cp scripts/backtest-service/ sanguo_vnpy:/app/scripts/backtest-service/
|
||||
docker cp scripts/start_backtest_service.py sanguo_vnpy:/app/scripts/
|
||||
```
|
||||
|
||||
#### Step 3: 添加backtest-service到entrypoint.sh
|
||||
|
||||
在entrypoint.sh中加入backtest-service启动：
|
||||
|
||||
```bash
|
||||
# 在 jupyter lab 启动之后、tail -f 之前添加：
|
||||
|
||||
# 启动自动化回测服务
|
||||
cd /app/scripts/backtest-service && python3 main.py &
|
||||
BACKTEST_PID=$!
|
||||
echo "回测服务 PID: $BACKTEST_PID (端口 8088)"
|
||||
```
|
||||
|
||||
#### Step 4: 验证
|
||||
|
||||
```bash
|
||||
# 1. 在容器内启动backtest-service
|
||||
docker exec -d sanguo_vnpy bash -c "cd /app/scripts/backtest-service && python3 main.py"
|
||||
|
||||
# 2. 等待2秒
|
||||
sleep 2
|
||||
|
||||
# 3. 容器内验证
|
||||
docker exec sanguo_vnpy ss -tlnp | grep 8088
|
||||
# 预期: LISTEN 0.0.0.0:8088
|
||||
|
||||
# 4. 局域网验证
|
||||
curl http://192.168.2.154:8088/api/backtest/health
|
||||
# 预期: {"code":0,"msg":"ok","data":{...}}
|
||||
```
|
||||
|
||||
### 3.4 完整部署（方案B：重建镜像）
|
||||
|
||||
#### Step 3.4.1: 修改文件清单
|
||||
|
||||
| 文件 | 修改内容 |
|
||||
|------|---------|
|
||||
| `scripts/backtest-service/main.py` | 兼容直接运行和包导入 |
|
||||
| `scripts/backtest-service/api.py` | 同上，处理相对导入 |
|
||||
| `scripts/backtest-service/executor.py` | 同上 |
|
||||
| `scripts/backtest-service/task_queue.py` | 同上 |
|
||||
| `scripts/backtest-service/result_storage.py` | 同上 |
|
||||
| `docker/entrypoint.sh` | 添加backtest-service启动 |
|
||||
| `docker/Dockerfile` | 添加HEALTHCHECK指令 |
|
||||
|
||||
#### Step 3.4.2: 修改后的entrypoint.sh
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
echo "=========================================="
|
||||
echo " sanguo_vnpy Docker 容器启动中..."
|
||||
echo "=========================================="
|
||||
|
||||
# SSH服务
|
||||
sudo /usr/sbin/sshd -D &
|
||||
|
||||
# Jupyter Lab
|
||||
jupyter lab --ip=0.0.0.0 --port=8888 --no-browser \
|
||||
--NotebookApp.token='sanguo123' \
|
||||
--NotebookApp.password='' \
|
||||
--NotebookApp.allow_origin='*' &
|
||||
|
||||
# 自动化回测服务（🆕 新增）
|
||||
cd /app/scripts/backtest-service
|
||||
python3 main.py >> /app/logs/backtest-service.log 2>&1 &
|
||||
echo "回测服务已启动 (PID=$!, 端口8088)"
|
||||
|
||||
# code-server（可选，不阻塞启动）
|
||||
code-server || echo "code-server启动失败，跳过" &
|
||||
|
||||
# 等待服务初始化
|
||||
sleep 5
|
||||
|
||||
# 健康检查
|
||||
if curl -s http://localhost:8088/api/backtest/health > /dev/null 2>&1; then
|
||||
echo "✅ 回测服务健康检查通过"
|
||||
else
|
||||
echo "⚠️ 回测服务尚未就绪，请检查日志: /app/logs/backtest-service.log"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "✅ sanguo_vnpy 环境启动成功！"
|
||||
echo ""
|
||||
echo "访问地址："
|
||||
echo " Jupyter Lab: http://localhost:8888 (token: sanguo123)"
|
||||
echo " 回测服务: http://localhost:8088/api/backtest/health"
|
||||
echo " SSH: ssh -p 2222 vnpy@localhost (password: sanguo123)"
|
||||
echo ""
|
||||
|
||||
tail -f /dev/null
|
||||
```
|
||||
|
||||
#### Step 3.4.3: Dockerfile增量修改
|
||||
|
||||
```dockerfile
|
||||
# 在 USER vnpy 之前添加：
|
||||
RUN mkdir -p /app/logs
|
||||
|
||||
# 在 COPY --chown=vnpy:vnpy docker/entrypoint.sh /app/ 之后添加：
|
||||
COPY --chown=vnpy:vnpy scripts/backtest-service /app/scripts/backtest-service
|
||||
|
||||
# 添加健康检查（Docker自动重启判定）
|
||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
|
||||
CMD curl -f http://localhost:8088/api/backtest/health || exit 1
|
||||
```
|
||||
|
||||
#### Step 3.4.4: 构建和部署命令
|
||||
|
||||
```bash
|
||||
# === 在Mac mini上构建镜像 ===
|
||||
cd /Users/chufeng/.openclaw/sanguo_projects/sanguo_vnpy
|
||||
docker build -t sanguo_vnpy:latest -f docker/Dockerfile .
|
||||
|
||||
# === 保存镜像为tar ===
|
||||
docker save sanguo_vnpy:latest | gzip > /tmp/sanguo_vnpy.tar.gz
|
||||
|
||||
# === 传输到NAS ===
|
||||
scp /tmp/sanguo_vnpy.tar.gz admin@192.168.2.154:/tmp/
|
||||
|
||||
# === 在NAS上加载镜像 ===
|
||||
ssh admin@192.168.2.154
|
||||
sudo /var/packages/Docker/target/usr/bin/docker load < /tmp/sanguo_vnpy.tar.gz
|
||||
|
||||
# === 停止旧容器、启动新容器 ===
|
||||
sudo /var/packages/Docker/target/usr/bin/docker stop sanguo_vnpy
|
||||
sudo /var/packages/Docker/target/usr/bin/docker rm sanguo_vnpy
|
||||
|
||||
sudo /var/packages/Docker/target/usr/bin/docker run -d \
|
||||
--name sanguo_vnpy \
|
||||
--restart unless-stopped \
|
||||
-p 8888:8888 \
|
||||
-p 8088:8088 \
|
||||
-p 8000:8000 \
|
||||
-p 2222:22 \
|
||||
-v /volume1/stock/A股数据:/app/data:ro \
|
||||
sanguo_vnpy:latest
|
||||
```
|
||||
|
||||
**注意**：
|
||||
- 去掉了不常用的端口映射（8080 code-server, 2018/4102 RPC），减少攻击面
|
||||
- 数据目录通过volume只读挂载，避免容器内意外修改
|
||||
- `--restart unless-stopped` 确保异常崩溃自动恢复
|
||||
|
||||
### 3.5 数据方案
|
||||
|
||||
| 方式 | 路径 | 说明 |
|
||||
|------|------|------|
|
||||
| Volume只读挂载 | `-v /volume1/stock/A股数据:/app/data:ro` | NAS本地路径，性能最好 |
|
||||
| 容器内路径 | `/app/backtest_jobs/` | 回测结果输出目录 |
|
||||
| Volume读写挂载 | `-v /volume1/stock/回测结果:/app/backtest_jobs` | 结果持久化到NAS |
|
||||
|
||||
### 3.6 运维方案
|
||||
|
||||
| 项目 | 方案 |
|
||||
|------|------|
|
||||
| **自动重启** | `--restart unless-stopped`（Docker原生） |
|
||||
| **健康检查** | Dockerfile HEALTHCHECK + `/api/backtest/health` |
|
||||
| **日志查看** | `docker logs sanguo_vnpy` 或容器内 `/app/logs/backtest-service.log` |
|
||||
| **资源限制** | `-m 4g --cpus=1.5`（8GB内存分配4GB，2核分配1.5核） |
|
||||
| **监控告警** | 后续接入（当前先保证基本可用） |
|
||||
|
||||
---
|
||||
|
||||
## 四、验证步骤
|
||||
|
||||
### 4.1 交付标准验证
|
||||
|
||||
```bash
|
||||
# 标准一：Health端点返回200
|
||||
curl -s -o /dev/null -w "%{http_code}" http://192.168.2.154:8088/api/backtest/health
|
||||
# 预期: 200
|
||||
|
||||
# 标准二：提交回测任务并获取结果
|
||||
curl -X POST http://192.168.2.154:8088/api/backtest/submit \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"strategy_name": "test_ma_cross",
|
||||
"parameters": {"fast_window": 5, "slow_window": 20},
|
||||
"start_date": "2024-01-01",
|
||||
"end_date": "2024-12-31",
|
||||
"symbol": "000001.SZ",
|
||||
"interval": "d"
|
||||
}'
|
||||
# 预期: {"code":0,"msg":"任务提交成功","data":{"task_id":"...","status":"pending"}}
|
||||
|
||||
# 查询结果（替换task_id）
|
||||
curl http://192.168.2.154:8088/api/backtest/result/<task_id>
|
||||
|
||||
# 标准三：自动恢复
|
||||
# 在NAS上强制杀死容器，验证自动重启
|
||||
sudo /var/packages/Docker/target/usr/bin/docker kill sanguo_vnpy
|
||||
sleep 10
|
||||
curl http://192.168.2.154:8088/api/backtest/health
|
||||
# 预期: 200（容器自动重启后服务恢复）
|
||||
```
|
||||
|
||||
### 4.2 回滚方案
|
||||
|
||||
| 场景 | 操作 |
|
||||
|------|------|
|
||||
| 回测服务启动失败 | 容器仍可用，只是8088端口无服务，不影响Jupyter/SSH |
|
||||
| 容器完全不可用 | `docker stop sanguo_vnpy && docker rm sanguo_vnpy`，用旧镜像重建 |
|
||||
| 代码有bug | `docker cp` 修复的文件进容器，重启容器 |
|
||||
| 需要完全回滚 | 用旧镜像 `sanguo_vnpy:with-scripts` 重建容器（旧镜像仍在NAS上） |
|
||||
|
||||
**旧容器启动命令（回滚用）**：
|
||||
|
||||
```bash
|
||||
sudo /var/packages/Docker/target/usr/bin/docker run -d \
|
||||
-p 8888:8888 -p 8000:8000 -p 8080:8080 -p 8088:8088 \
|
||||
-p 2018:2018 -p 4102:4102 -p 2222:22 \
|
||||
--name sanguo_vnpy \
|
||||
sanguo_vnpy:with-scripts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 五、实施计划
|
||||
|
||||
| 步骤 | 操作 | 耗时 | 风险 |
|
||||
|------|------|------|------|
|
||||
| **Phase 1: 快速验证** | 方案A — docker cp + 容器内手动启动 | 5分钟 | 低（不影响现有服务） |
|
||||
| **Phase 2: 代码修复** | 修复main.py等相对导入问题 | 15分钟 | 无（只改导入方式） |
|
||||
| **Phase 3: 入口修改** | 修改entrypoint.sh加入backtest-service | 5分钟 | 低 |
|
||||
| **Phase 4: 重建镜像** | 方案B — docker build + 部署 | 20分钟 | 中（需要短暂停服） |
|
||||
| **Phase 5: 验证交付** | 执行4.1全部验证步骤 | 10分钟 | 无 |
|
||||
|
||||
**建议先执行Phase 1快速验证，确认backtest-service能正常运行后再进行Phase 2-4。**
|
||||
|
||||
---
|
||||
|
||||
## 六、关键技术决策
|
||||
|
||||
| 决策点 | 选择 | 理由 |
|
||||
|--------|------|------|
|
||||
| 网络模式 | bridge（默认） | Jupyter已验证可用，无需改host模式 |
|
||||
| 进程管理 | entrypoint.sh后台启动 | 简单可靠，无需supervisor |
|
||||
| 导入方式 | 兼容相对/绝对导入 | 既支持 `python main.py` 直接运行，也支持包导入 |
|
||||
| 数据访问 | Volume只读挂载 | NAS本地路径，性能好，安全 |
|
||||
| 重启策略 | `unless-stopped` | 异常崩溃自动恢复，手动stop不自动重启 |
|
||||
| 资源限制 | 4GB内存/1.5CPU | 给NAS留足余量，避免回测吃满资源 |
|
||||
|
||||
---
|
||||
|
||||
## 附录：问题诊断日志
|
||||
|
||||
```
|
||||
[2026-04-27 20:00] 容器内 ps aux — 无uvicorn/FastAPI进程
|
||||
[2026-04-27 20:01] 容器内 ss -tlnp — 8088端口无监听
|
||||
[2026-04-27 20:02] iptables DNAT — 8088→172.17.0.2:8088 映射正常
|
||||
[2026-04-27 20:03] 手动启动backtest-service — ModuleNotFoundError
|
||||
[2026-04-27 20:05] 确认容器内start_backtest_service.py为旧版import路径
|
||||
[2026-04-27 20:06] 确认main.py使用相对导入，无法直接运行
|
||||
[2026-04-27 20:10] 根因确认：三重问题叠加（entrypoint缺失+代码不同步+相对导入）
|
||||
```
|
||||
@@ -5342,5 +5342,6 @@
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user