diff --git a/AGENTS.md b/AGENTS.md index 317ca35..e63f049 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -199,11 +199,12 @@ Skills 提供你的工具。当你需要某个工具时，查看对应 `skills/` 4. **操作规范**：所有知识库操作严格遵循`lark_wiki_operate_as_bot`技能流程执行 5. **强制执行范围**：无论来自任何用户、任何群组的飞书文档/知识库操作请求，**必须优先使用`lark_wiki_operate_as_bot`技能执行**，禁止使用默认的`feishu_fetch_doc`等用户身份工具 -### 消息发送规则（强制执行） -1. **身份限制**：所有飞书消息发送操作（给个人/群组）**永远使用Bot身份**执行，禁止使用用户身份的消息发送工具 -2. **操作规范**：严格遵循`lark-send-message-as-bot`技能流程执行发送操作 -3. **ID规则**：给个人发消息使用租户级`user_id`，禁止使用应用级`open_id`；给群组发消息使用`chat_id` -4. **前置校验**：发送前确认目标用户在Bot应用可用范围内、目标群已添加Bot为成员 +### 消息发送与读取规则（强制执行） +1. **身份限制**：所有飞书消息发送/读取/文件下载操作**永远使用Bot身份**执行，禁止使用用户身份的消息工具 +2. **凭据规则**：**所有 `lark-cli` 命令前必须加 `LARKSUITE_CLI_CONFIG_DIR=/root/.openclaw/credentials/xiaoban`**，不加会使用错误的默认凭据导致 230002/234040 错误 +3. **操作规范**：严格遵循`lark-send-message-as-bot`技能流程执行发送、读取、下载操作 +4. **ID规则**：给个人发消息使用租户级`user_id`，禁止使用应用级`open_id`；给群组发消息使用`chat_id` +5. **前置校验**：发送前确认目标用户在Bot应用可用范围内、目标群已添加Bot为成员 ## 心跳 diff --git a/bin/lark-cli b/bin/lark-cli new file mode 100755 index 0000000..7d8b5b0 --- /dev/null +++ b/bin/lark-cli @@ -0,0 +1,3 @@ +#!/bin/bash +exec env LARKSUITE_CLI_CONFIG_DIR=/root/.openclaw/credentials/xiaoban \ + /root/.nvm/versions/node/v24.14.0/bin/lark-cli "$@" diff --git a/logs/backup.log b/logs/backup.log index eceb5e0..7ef40f2 100644 --- a/logs/backup.log +++ b/logs/backup.log @@ -612,3 +612,11 @@ To https://git.valavala.com/ai_member_only/ai_member_xiaoban 1187d1a..3f25e3f master -> master [2026-06-15 08:10:01] 工作区备份成功：自动备份 2026-06-15 08:10:01 [2026-06-16 08:10:01] 开始备份工作区... +[master 3703b73] 自动备份 2026-06-16 08:10:01 + 2 files changed, 1 insertion(+), 3 deletions(-) + delete mode 100644 tmp_daily_summary.md +remote: . Processing 1 references +remote: Processed 1 references in total +To https://git.valavala.com/ai_member_only/ai_member_xiaoban + 3f25e3f..3703b73 master -> master +[2026-06-16 08:11:15] 工作区备份成功：自动备份 2026-06-16 08:10:01 diff --git a/memory/.dreams/events.jsonl b/memory/.dreams/events.jsonl index e665875..fcabb94 100644 --- a/memory/.dreams/events.jsonl +++ b/memory/.dreams/events.jsonl @@ -82,3 +82,10 @@ {"type":"memory.recall.recorded","timestamp":"2026-06-12T07:16:32.880Z","query":"S1 U16 用户数据 查询","resultCount":5,"results":[{"path":"memory/2026-05-28.md","startLine":504,"endLine":536,"score":1},{"path":"memory/2026-05-28.md","startLine":337,"endLine":366,"score":1},{"path":"memory/2026-05-28.md","startLine":530,"endLine":557,"score":1},{"path":"memory/2026-05-24.md","startLine":46,"endLine":71,"score":1},{"path":"memory/2026-05-24.md","startLine":66,"endLine":92,"score":1}]} {"type":"memory.recall.recorded","timestamp":"2026-06-12T07:16:32.881Z","query":"用户学习数据导出 脚本","resultCount":4,"results":[{"path":"memory/2026-05-24.md","startLine":46,"endLine":71,"score":1},{"path":"memory/2026-05-24.md","startLine":66,"endLine":92,"score":1},{"path":"memory/2026-05-28.md","startLine":596,"endLine":624,"score":1},{"path":"memory/2026-05-28.md","startLine":572,"endLine":602,"score":1}]} {"type":"memory.recall.recorded","timestamp":"2026-06-15T04:54:12.059Z","query":"达人解码局 ou_361f1540608c192aa0c8e0d4f916698e","resultCount":4,"results":[{"path":"memory/2026-06-02.md","startLine":1,"endLine":32,"score":1},{"path":"memory/2026-06-02.md","startLine":30,"endLine":47,"score":1},{"path":"memory/2026-05-28.md","startLine":551,"endLine":580,"score":1},{"path":"memory/2026-05-28.md","startLine":530,"endLine":557,"score":1}]} +{"type":"memory.recall.recorded","timestamp":"2026-06-16T06:19:06.189Z","query":"渠道归属分类 Z列 销售转化场景","resultCount":4,"results":[{"path":"memory/2026-05-24.md","startLine":122,"endLine":132,"score":1},{"path":"memory/2026-05-28.md","startLine":551,"endLine":580,"score":1},{"path":"memory/2026-05-28.md","startLine":530,"endLine":557,"score":1},{"path":"memory/2026-05-28.md","startLine":572,"endLine":602,"score":1}]} +{"type":"memory.recall.recorded","timestamp":"2026-06-16T06:22:11.809Z","query":"销售转化渠道分类规则 渠道归属 端内 销转 达人 直购","resultCount":4,"results":[{"path":"memory/2026-05-28.md","startLine":572,"endLine":602,"score":1},{"path":"memory/2026-05-28.md","startLine":551,"endLine":580,"score":1},{"path":"memory/2026-05-24.md","startLine":122,"endLine":132,"score":1},{"path":"memory/2026-05-28.md","startLine":530,"endLine":557,"score":1}]} +{"type":"memory.recall.recorded","timestamp":"2026-06-16T06:22:32.406Z","query":"渠道归属 classify_channel 端内 销转 达人 直购 分类规则 王虹茗","resultCount":5,"results":[{"path":"memory/2026-05-24.md","startLine":122,"endLine":132,"score":1},{"path":"memory/2026-05-28.md","startLine":551,"endLine":580,"score":1},{"path":"memory/2026-05-28.md","startLine":530,"endLine":557,"score":1},{"path":"memory/2026-05-24.md","startLine":66,"endLine":92,"score":1},{"path":"memory/2026-06-02.md","startLine":1,"endLine":32,"score":1}]} +{"type":"memory.recall.recorded","timestamp":"2026-06-16T09:36:31.306Z","query":"sales_leads_full_refresh 订单汇总 merge 细水入海 三表","resultCount":6,"results":[{"path":"memory/2026-06-16.md","startLine":1,"endLine":19,"score":1},{"path":"memory/2026-05-28.md","startLine":31,"endLine":60,"score":1},{"path":"memory/2026-05-28.md","startLine":389,"endLine":418,"score":1},{"path":"memory/2026-05-28.md","startLine":199,"endLine":227,"score":1},{"path":"memory/2026-05-28.md","startLine":551,"endLine":580,"score":1},{"path":"memory/2026-05-28.md","startLine":530,"endLine":557,"score":1}]} +{"type":"memory.recall.recorded","timestamp":"2026-06-16T11:17:42.687Z","query":"王璐辰 user data export S1 U16","resultCount":4,"results":[{"path":"memory/2026-05-28.md","startLine":504,"endLine":536,"score":1},{"path":"memory/2026-05-28.md","startLine":173,"endLine":205,"score":1},{"path":"memory/2026-05-28.md","startLine":31,"endLine":60,"score":1},{"path":"memory/2026-05-28.md","startLine":389,"endLine":418,"score":1}]} +{"type":"memory.recall.recorded","timestamp":"2026-06-16T11:18:05.577Z","query":"S1 U16 课程 unit 用户数据","resultCount":6,"results":[{"path":"memory/2026-05-28.md","startLine":504,"endLine":536,"score":1},{"path":"memory/2026-05-28.md","startLine":337,"endLine":366,"score":1},{"path":"memory/2026-05-28.md","startLine":530,"endLine":557,"score":1},{"path":"memory/2026-05-24.md","startLine":122,"endLine":132,"score":1},{"path":"memory/2026-05-24.md","startLine":66,"endLine":92,"score":1},{"path":"memory/2026-05-24.md","startLine":46,"endLine":71,"score":1}]} +{"type":"memory.recall.recorded","timestamp":"2026-06-16T11:21:05.649Z","query":"童瑶 user data S1 U16 学习数据","resultCount":4,"results":[{"path":"memory/2026-05-28.md","startLine":504,"endLine":536,"score":1},{"path":"memory/2026-05-28.md","startLine":173,"endLine":205,"score":1},{"path":"memory/2026-05-28.md","startLine":31,"endLine":60,"score":1},{"path":"memory/2026-05-28.md","startLine":389,"endLine":418,"score":1}]} diff --git a/memory/.dreams/short-term-recall.json b/memory/.dreams/short-term-recall.json index ee8bc4b..07f851d 100644 --- a/memory/.dreams/short-term-recall.json +++ b/memory/.dreams/short-term-recall.json @@ -1,6 +1,6 @@ { "version": 1, - "updatedAt": "2026-06-15T04:54:12.059Z", + "updatedAt": "2026-06-16T11:21:05.649Z", "entries": { "memory:memory/2026-05-24.md:1:30": { "key": "memory:memory/2026-05-24.md:1:30", @@ -174,13 +174,13 @@ "endLine": 71, "source": "memory", "snippet": "### 增强: 报告开头加入角色基本信息 (2026-05-24) [刘庆逊提出] 在 studytime-analysis 输出中加入角色基本信息，包括： - 角色ID、账号ID、角色名字、性别、年龄、账号手机号后4位 **数据源（新增）**： - MySQL Online `vala_user` 库 - `vala_app_character` 表：id, account_id, nickname, gender(0=女/1=男), birthday(varchar \"YYYY-MM-DD\") - `vala_app_account` 表：id, tel(已脱敏如 186****1625) - 手机号已脱敏，直接取后4位；年龄从 birthday 计算 **修改文件**： - `skills/studytime-analysis/scripts/studytime_analysis.py`：新增 MySQL 连接函数 `get_mysql_connection()` 和 `fetch_role_info(role_id)`，更新 `format_report()` 输出基本角色信息 - 已验证 2895 正常运行输出 - 已同步 SkillHub + Git ### Unit 显示修复: 季度名称 → 全局单元编号 (2026-05-24) [刘庆逊提出] HTML 报告中 Unit 列显示错误——显示的是季度名称（如\"小镇时光\"\"钢铁之心\"）而非单元数字（0-48）。 **根因分析**： - `vala_game_chapter`（MySQL）无 `unit_", - "recallCount": 23, + "recallCount": 24, "dailyCount": 0, "groundedCount": 0, - "totalScore": 23, + "totalScore": 24, "maxScore": 1, "firstRecalledAt": "2026-05-25T05:47:41.388Z", - "lastRecalledAt": "2026-06-12T07:16:32.881Z", + "lastRecalledAt": "2026-06-16T11:18:05.577Z", "queryHashes": [ "9aff8ec9594a", "566b5958861e", @@ -204,7 +204,8 @@ "b2c23ece6608", "206acd60d69d", "363241a84e3c", - "6d1afbed352e" + "6d1afbed352e", + "a87866c0fa75" ], "recallDays": [ "2026-05-25", @@ -215,7 +216,8 @@ "2026-06-04", "2026-06-09", "2026-06-10", - "2026-06-12" + "2026-06-12", + "2026-06-16" ], "conceptTags": [ "studytime-analysis", @@ -399,13 +401,13 @@ "endLine": 92, "source": "memory", "snippet": "**根因分析**： - `vala_game_chapter`（MySQL）无 `unit_index` 字段 - `big_map_chapter`（PostgreSQL）有 `unit_index` 字段，但仅包含 A1 数据，且与 `vala_game_chapter` 无直接关联键 - 两者 ID 空间不重叠（big_map: ~1720-2070，game_chapter: ~55-399），UUID 也不匹配 **映射方案**： - 每个 season_package 内，`lesson_type=1` 的章节按 `id` 排序，每 5 个连续章节组成一个单元 - Season 0（序章/L1-U0）：所有章节属于 Unit 0 - Season 1-4：每个 season 有 12 个单元（60 个 lesson 章节） - 全局 unit_index = base_offset(season_of_quarter) + unit_within_season - base_offset: 0→0, 1→1, 2→13, 3→25, 4→37 **关键 Bug**：初版按 `season_of_quarter` 分组时 A1 和 A2 混在一起，因为相同季度值合并了。修复：改为按 `(level, season_of_quarter)` 分组。 **验证结果**： - A1: Unit 0-48（49 个单元），与 big_map_chapter 的 unit_index 范围一致 - A2: Unit 0-49（50 个单元，比 A1 多 1 个） *", - "recallCount": 10, + "recallCount": 12, "dailyCount": 0, "groundedCount": 0, - "totalScore": 10, + "totalScore": 12, "maxScore": 1, "firstRecalledAt": "2026-05-28T09:07:57.953Z", - "lastRecalledAt": "2026-06-12T07:16:32.881Z", + "lastRecalledAt": "2026-06-16T11:18:05.577Z", "queryHashes": [ "c6c7ff4ed75d", "c59410788b42", @@ -416,7 +418,9 @@ "6e3a2daa0a9f", "5d71b876843a", "363241a84e3c", - "6d1afbed352e" + "6d1afbed352e", + "02e468d377f4", + "a87866c0fa75" ], "recallDays": [ "2026-05-28", @@ -424,7 +428,8 @@ "2026-05-30", "2026-06-04", "2026-06-09", - "2026-06-12" + "2026-06-12", + "2026-06-16" ], "conceptTags": [ "vala-game-chapter", @@ -642,13 +647,13 @@ "endLine": 536, "source": "memory", "snippet": "| 2b | 客户主表 → 订单 | ✅ | 公式就绪 | | 3 | 日报 C1HVN2 | ✅ | 四段刷新 | | 3a/3c | 口径审计 | ✅ | 推群全过 | | 4a | 结算月汇总 | ✅ | 公式就绪 | **结论：当前表内所有数据与重拉完全一致，零差异，无需刷写。** ### 18:36 小溪行课触发实跑 - 已推 1920 条到小溪查询表（新 Sheet） - 已 @小溪 到小红书数据需求群，请处理 ~5 条待查询记录 - 表格：`RFIJsXT8FhGHhctY4RwczcOfnac` - 等待小溪回填后跑 `pull_xiaoxi_results.py` 回收结果 ### 18:50 PG 实时行课数据全量覆盖 [陈逸鸫确认] 不用等小溪回填那 5 条，直接用 PG `user_course_detail` 表实时数据覆盖： - 448 人有新进展（相比小溪历史快照） - pull → 3PRySY 口径对齐 12/12 ✅ - lesson_cache → C1HVN2 16 格 ✅ - 变化有限（PG=正式课，小溪=体验课，口径不同） - sync_base (4b) → 多维表格 8 张 ✅ 21.2s ### 18:54 漏斗看板发布待解决 - funnel HTML 已构建（scripts/build_funnel_dashboard.py ✅） - 妙搭发布 `apps +html-publish` 需要 `--as user`，bot 模式不支持 - 系统 lark-cli `/usr/local/lib/node_mod", - "recallCount": 11, + "recallCount": 14, "dailyCount": 0, "groundedCount": 0, - "totalScore": 11, + "totalScore": 14, "maxScore": 1, "firstRecalledAt": "2026-05-28T20:51:03.908Z", - "lastRecalledAt": "2026-06-12T07:16:32.880Z", + "lastRecalledAt": "2026-06-16T11:21:05.649Z", "queryHashes": [ "f22544a8757c", "2af907cea93d", @@ -660,7 +665,10 @@ "c5fe8ced03de", "a70555b22269", "78873c102522", - "363241a84e3c" + "363241a84e3c", + "b65a5ab98ae7", + "a87866c0fa75", + "5bcdd0ab32df" ], "recallDays": [ "2026-05-29", @@ -668,7 +676,8 @@ "2026-06-05", "2026-06-08", "2026-06-10", - "2026-06-12" + "2026-06-12", + "2026-06-16" ], "conceptTags": [ "3a/3c", @@ -688,13 +697,13 @@ "endLine": 366, "source": "memory", "snippet": "- 448 人有新进展（相比小溪历史快照） - pull → 3PRySY 口径对齐 12/12 ✅ - lesson_cache → C1HVN2 16 格 ✅ - 变化有限（PG=正式课，小溪=体验课，口径不同） - sync_base (4b) → 多维表格 8 张 ✅ 21.2s ### 18:54 漏斗看板发布待解决 - funnel HTML 已构建（scripts/build_funnel_dashboard.py ✅） - 妙搭发布 `apps +html-publish` 需要 `--as user`，bot 模式不支持 - 系统 lark-cli `/usr/local/lib/node_modules/@anthropic/lark-cli` 可能支持 - 待确认：服务器是否已 `lark-cli auth login --as user` ### 19:00 陈逸鸫派 Image2 生图任务 **任务：** L1-S1-U1《秘密基地》5 课投放用小地图底图 - 模型：gpt-image-2 · 3:4 · 2K · 不要文字 - 风格：太阳朋克 + L1 场景 - 5 张 PNG：U1-L1～U1-L5 - FUNCLOUD_API_KEY 在小研 workspace `.env` 中可用（`fc_eea138933b02b4797ce0779ffb637d8b8a6368db7b435dfdab7b4be1cd254d98`） - Brief 文档 `/docx/KsVadUTmooO7yYxHaGmc1R0Bn5b` + 投放手册 `/do", - "recallCount": 12, + "recallCount": 13, "dailyCount": 0, "groundedCount": 0, - "totalScore": 12, + "totalScore": 13, "maxScore": 1, "firstRecalledAt": "2026-05-28T20:51:03.908Z", - "lastRecalledAt": "2026-06-12T07:16:32.880Z", + "lastRecalledAt": "2026-06-16T11:18:05.577Z", "queryHashes": [ "f22544a8757c", "2af907cea93d", @@ -707,7 +716,8 @@ "c5fe8ced03de", "a70555b22269", "a90ba76a41cf", - "363241a84e3c" + "363241a84e3c", + "a87866c0fa75" ], "recallDays": [ "2026-05-29", @@ -716,7 +726,8 @@ "2026-06-03", "2026-06-08", "2026-06-09", - "2026-06-12" + "2026-06-12", + "2026-06-16" ], "conceptTags": [ "gpt", @@ -847,19 +858,14 @@ "endLine": 557, "source": "memory", "snippet": "- 妙搭发布 `apps +html-publish` 需要 `--as user`，bot 模式不支持 - 系统 lark-cli `/usr/local/lib/node_modules/@anthropic/lark-cli` 可能支持 - 待确认：服务器是否已 `lark-cli auth login --as user` ### 19:00 陈逸鸫派 Image2 生图任务 **任务：** L1-S1-U1《秘密基地》5 课投放用小地图底图 - 模型：gpt-image-2 · 3:4 · 2K · 不要文字 - 风格：太阳朋克 + L1 场景 - 5 张 PNG：U1-L1～U1-L5 - FUNCLOUD_API_KEY 在小研 workspace `.env` 中可用（`fc_eea138933b02b4797ce0779ffb637d8b8a6368db7b435dfdab7b4be1cd254d98`） - Brief 文档 `/docx/KsVadUTmooO7yYxHaGmc1R0Bn5b` + 投放手册 `/docx/QhYQdz7PvoN7Eaxmhu0c0Q5UnHe` — 均为个人文档，AGENTS.md 规则禁止读取 - 素材库入口：https://llm-dev.valavala.com/web_tools/material_prod --- ### 19:30 同事数据查询流程演练 [陈逸鸫测试] **场景：模拟王虹茗请求小龙 4/21-5/20 订单详情，验证三级查询流程** **小龙订单查询结果（数据源：3wcle8 销售订", - "recallCount": 35, + "recallCount": 40, "dailyCount": 0, "groundedCount": 0, - "totalScore": 35, + "totalScore": 40, "maxScore": 1, "firstRecalledAt": "2026-05-28T20:51:03.908Z", - "lastRecalledAt": "2026-06-15T04:54:12.059Z", + "lastRecalledAt": "2026-06-16T11:18:05.577Z", "queryHashes": [ - "f7ae50ae228d", - "3737f6af1445", - "cf12fd62a5e5", - "833509d09ccb", - "5b675d96f1da", "b8b71654e7aa", "f0e8e3da16e8", "771aaeed7600", @@ -886,7 +892,12 @@ "cfa4c0443735", "d248105a4ef8", "363241a84e3c", - "4c8704b4fc00" + "4c8704b4fc00", + "ff112f97114e", + "d7be4ba41be4", + "02e468d377f4", + "f6f7815a1493", + "a87866c0fa75" ], "recallDays": [ "2026-05-29", @@ -900,7 +911,8 @@ "2026-06-10", "2026-06-11", "2026-06-12", - "2026-06-15" + "2026-06-15", + "2026-06-16" ], "conceptTags": [ "gpt", @@ -1028,13 +1040,13 @@ "endLine": 132, "source": "memory", "snippet": "- HTML 模板使用 f-string 时，CSS 中的 `{` 必须双写 `{{`，且不能在 f-string 内做字符串拼接（会打断 f-string） - 年龄从 birthday（varchar \"YYYY-MM-DD\"）计算，非 DATE 类型 - 手机号在 MySQL 已脱敏（如 186****1625），直接取后4位 ### 已测试角色 - **32009 (zyl)**：26 条完课，651 条中互动(Perfect 82.6%、Good 2.9%)，26 节巩固全部满分。课程 A2，Apple App Store 购买，渠道 newmedia-daren-xhs-宣儿麻麻。设备小课屏E3。学习时间 2026-04-24 ~ 2026-05-24。 ### 同步 - 已推送到 SkillHub（`studycourse-analysis.xiaoban`） - 已 commit 8648b7b 到 Git 远程仓库 - 已通知 Cris（李若松）", - "recallCount": 9, + "recallCount": 13, "dailyCount": 0, "groundedCount": 0, - "totalScore": 9, + "totalScore": 13, "maxScore": 1, "firstRecalledAt": "2026-05-28T20:57:42.055Z", - "lastRecalledAt": "2026-06-09T03:15:16.782Z", + "lastRecalledAt": "2026-06-16T11:18:05.577Z", "queryHashes": [ "f139ebdae100", "f7ae50ae228d", @@ -1044,13 +1056,18 @@ "f0e8e3da16e8", "fbe762c60bc6", "1382c5611b17", - "a90ba76a41cf" + "a90ba76a41cf", + "ff112f97114e", + "d7be4ba41be4", + "02e468d377f4", + "a87866c0fa75" ], "recallDays": [ "2026-05-29", "2026-06-02", "2026-06-03", - "2026-06-09" + "2026-06-09", + "2026-06-16" ], "conceptTags": [ "f-string", @@ -1184,25 +1201,27 @@ "endLine": 227, "source": "memory", "snippet": "| 4a | 结算月汇总 | <1s | ✅ 真写成功（零警告） | | 3a/3c/3d/4c | 审计 | — | ⏭ dry-run 跳过 | **修复的 wrapper 兼容性问题：** - API 响应格式：`data.spreadsheet` → `data.sheets.sheets`（lark-cli 兼容） - 参数名：`--values` 与 `--data` 等价处理 - camelCase→snake_case：`sheetId` → `sheet_id`，`grid_properties` 包装 - `+info` 端点：v3 metadata → v2 metainfo（含 sheets 列表） - `update-dimension`：`PUT /dimension_range`，1-indexed ### 16:48 聚光凭证部署 **来源：陈逸鸫** — 从 Mac 发来 `.env.juguang.9180` 和 `.env.juguang.9181`，走方案 A（复制文件，不在服务器重 OAuth） - 9180（云智 adv=9746532）：refresh 验证通过 ✅ - 9181（谦禾 adv=9013261,9598861 / YTL adv=7242040,9891870,10157917,10562529）：refresh 验证通过 ✅ - OAuth 回调地址：`https://odourless-demetra-cany.ngrok-free.dev/callback.html`（ngrok 隧道） - `jugu", - "recallCount": 6, + "recallCount": 7, "dailyCount": 0, "groundedCount": 0, - "totalScore": 6, + "totalScore": 7, "maxScore": 1, "firstRecalledAt": "2026-05-29T02:53:52.924Z", - "lastRecalledAt": "2026-06-09T09:31:49.829Z", + "lastRecalledAt": "2026-06-16T09:36:31.306Z", "queryHashes": [ "d9f1601110da", "117aaedb584d", "c0b581bfb144", "d24884bfecf1", "50b1cd5a5d08", - "c3aab736911e" + "c3aab736911e", + "f6f7815a1493" ], "recallDays": [ "2026-05-29", "2026-06-04", - "2026-06-09" + "2026-06-09", + "2026-06-16" ], "conceptTags": [ "3a/3c/3d/4c", @@ -1222,13 +1241,13 @@ "endLine": 580, "source": "memory", "snippet": "- 18 单 | GMV ¥44,375 | 退款 ¥0 - 渠道：销转 11 / 达人 4 / 端内 3 - 客单价：¥599×2 / ¥1,999×9 / ¥3,598×7 - 订单日期分布在 11 天（04/23–05/15） **看板发布全流程梳理：** - 服务器 build HTML + Base sync → DM 陈逸鸫 → Mac 妙搭 `html-publish` + `access-scope-set` - 三个看板 App ID：漏斗 `app_4k886pmc9x6yt` · 指挥舱 `app_4k79smc6fa1kf` · 销售 `app_4k7qkz9wrga74` - 当前 cron 不自动发布看板，仅 build HTML **王虹茗身份确认：** wanghongming@makee.com，user_id 未获取 - 尝试 `lark-cli contact +search-user wanghongming@makee.com` → 失败：bot API 缺少 `search:user` scope - 替代方案：让她发消息给大麦（系统自动获取 user_id），或陈逸鸫截图资料页 **同事查询三级场景定稿：** 1. 常规只读 → 直接查，append 输出表 2. 权限外用户 → 先通知业务负责人，再决定是否返回 3. 写操作 → 回复「这会影响生产数据，已转 @陈逸鸫 确认」 ### 20:20 行课转化分析 [陈逸鸫需求] **需求：行课记录新增当日进线→当天行课 + 7天首课率 + 销售排名 + 日报展示** **", - "recallCount": 25, + "recallCount": 29, "dailyCount": 0, "groundedCount": 0, - "totalScore": 25, + "totalScore": 29, "maxScore": 1, "firstRecalledAt": "2026-05-29T06:11:40.432Z", - "lastRecalledAt": "2026-06-15T04:54:12.059Z", + "lastRecalledAt": "2026-06-16T09:36:31.306Z", "queryHashes": [ "82be33d1f911", "2aa08c6652fb", @@ -1254,7 +1273,11 @@ "206acd60d69d", "cfa4c0443735", "d248105a4ef8", - "4c8704b4fc00" + "4c8704b4fc00", + "ff112f97114e", + "d7be4ba41be4", + "02e468d377f4", + "f6f7815a1493" ], "recallDays": [ "2026-05-29", @@ -1266,7 +1289,8 @@ "2026-06-09", "2026-06-10", "2026-06-11", - "2026-06-15" + "2026-06-15", + "2026-06-16" ], "conceptTags": [ "04/23", @@ -1325,20 +1349,23 @@ "endLine": 602, "source": "memory", "snippet": "**需求：行课记录新增当日进线→当天行课 + 7天首课率 + 销售排名 + 日报展示** **数据摸底结果：** | 指标 | 5月数据 | 定义 | |------|---------|------| | 当日进线→当天行课 | 3.0% (4/135) | 进线当天有 chapter settlement | | 7天线索→首课 | 28.6% (10/35) | 过去7天进线中 小溪阶段≥5 | **按销售拆解：** | 销售 | 当日行课率 | 7天首课率 | |------|-----------|----------| | Bob | 7.4% (最高) | 28.6% | | Tom | 3.7% | 28.6% | | 吴迪 | 0% | 28.6% | | 小龙 | 0% | 14.3% (最低) | **改动方案（6处，待陈逸鸫确认后执行）：** 1. **2aNzzy 每日线索分配** — 加 2 列（首课日期 V、当日行课 W） 2. **3PRySY 行课情况** — 加 4 列（当日行课数、当日行课率、7天线索数、7天首课率） 3. **C1HVN2 日报** — 新增「五、行课转化」段 4. **Base 多维表格** — 3 张表加字段（tbl9HgnKU2qgj4gE / tblB27fjK7mDnxjY / tblxjCeGOPNoLHl6） 5. **funnel-daily 看板 HTML** — 新增「当日转化」+「7天首课」指标 6. **build_pipeline 脚本** — 新增 `scripts/compute_l", - "recallCount": 2, + "recallCount": 4, "dailyCount": 0, "groundedCount": 0, - "totalScore": 2, + "totalScore": 4, "maxScore": 1, "firstRecalledAt": "2026-05-29T06:12:52.521Z", - "lastRecalledAt": "2026-06-12T07:16:32.881Z", + "lastRecalledAt": "2026-06-16T06:22:11.809Z", "queryHashes": [ "2aa08c6652fb", - "6d1afbed352e" + "6d1afbed352e", + "ff112f97114e", + "d7be4ba41be4" ], "recallDays": [ "2026-05-29", - "2026-06-12" + "2026-06-12", + "2026-06-16" ], "conceptTags": [ "3.0", @@ -1805,13 +1832,13 @@ "endLine": 32, "source": "memory", "snippet": "# 2026-06-02 工作日志 ## 达播达人筛选 [苏雅] 苏雅在群里发多位达人让大麦分析是否适合带瓦拉英语。共分析 17 位达人。 ### 瓦拉英语客单价 - 1999 元 / 3598 元 [苏雅确认] ### 筛选标准 - 产出权重优先（场均GMV、品牌场均） - 品类匹配（教育品类占比） - 人群匹配（学龄段家长，6-12岁最佳） - 客单价匹配（瓦拉1999/3598） - 无MCN优先 --- ## 最终分级汇总（17人） ### 🥇 S级 — 优先建联推进（2人） | 达人 | 平台 | 粉丝 | 场均GMV | 品牌场均 | 教育品类 | 客单价 | 核心优势 | |------|------|------|------|------|------|------|------| | lilymum闹妈 | 小红书 | 34.3万 | 100-200万 | 2.5-5万 | 2.4% | 500+ | 带货断层领先，选品意向含教育，无MCN纯佣 | | 小朱妈妈Nancy | 小红书 | — | 10-25万 | 1-2.5万 | 66.87% | 500+ | 教育垂类+均价1515+选品极克制(11商家25商品)，南大硕士 | ### 🥈 A级 — 同步触达测试（4人） | 达人 | 平台 | 粉丝 | 场均GMV | 品牌场均 | 教育品类 | 客单价 | 核心优势 | |------|------|------|------|------|------|------|------| | CallMe王阿姨 | 小红书 | 24", - "recallCount": 12, + "recallCount": 13, "dailyCount": 0, "groundedCount": 0, - "totalScore": 12, + "totalScore": 13, "maxScore": 1, "firstRecalledAt": "2026-06-03T06:24:53.892Z", - "lastRecalledAt": "2026-06-15T04:54:12.059Z", + "lastRecalledAt": "2026-06-16T06:22:32.406Z", "queryHashes": [ "faadf692331b", "c2163a583b15", @@ -1824,7 +1851,8 @@ "ffe9b822f4a2", "235546f84d27", "d248105a4ef8", - "4c8704b4fc00" + "4c8704b4fc00", + "02e468d377f4" ], "recallDays": [ "2026-06-03", @@ -1832,7 +1860,8 @@ "2026-06-05", "2026-06-09", "2026-06-11", - "2026-06-15" + "2026-06-15", + "2026-06-16" ], "conceptTags": [ "6-12岁最佳", @@ -2055,19 +2084,23 @@ "endLine": 60, "source": "memory", "snippet": "| 2b | 客户主表→订单明细 | 5.1s | ✅ dry-run（真跑数据量大，预计几分钟） | | 3 | 日报 C1HVN2 | 38.3s | ✅ 真写成功 | | 4a | 结算月汇总 | <1s | ✅ 真写成功（零警告） | | 3a/3c/3d/4c | 审计 | — | ⏭ dry-run 跳过 | **修复的 wrapper 兼容性问题：** - API 响应格式：`data.spreadsheet` → `data.sheets.sheets`（lark-cli 兼容） - 参数名：`--values` 与 `--data` 等价处理 - camelCase→snake_case：`sheetId` → `sheet_id`，`grid_properties` 包装 - `+info` 端点：v3 metadata → v2 metainfo（含 sheets 列表） - `update-dimension`：`PUT /dimension_range`，1-indexed ### 16:48 聚光凭证部署 **来源：陈逸鸫** — 从 Mac 发来 `.env.juguang.9180` 和 `.env.juguang.9181`，走方案 A（复制文件，不在服务器重 OAuth） - 9180（云智 adv=9746532）：refresh 验证通过 ✅ - 9181（谦禾 adv=9013261,9598861 / YTL adv=7242040,9891870,10157917,10562529）：refresh 验证通过 ✅ - OAu", - "recallCount": 2, + "recallCount": 5, "dailyCount": 0, "groundedCount": 0, - "totalScore": 2, + "totalScore": 5, "maxScore": 1, "firstRecalledAt": "2026-06-09T09:31:36.077Z", - "lastRecalledAt": "2026-06-09T09:31:49.829Z", + "lastRecalledAt": "2026-06-16T11:21:05.649Z", "queryHashes": [ "50b1cd5a5d08", - "c3aab736911e" + "c3aab736911e", + "f6f7815a1493", + "b65a5ab98ae7", + "5bcdd0ab32df" ], "recallDays": [ - "2026-06-09" + "2026-06-09", + "2026-06-16" ], "conceptTags": [ "5.1s", @@ -2087,19 +2120,23 @@ "endLine": 418, "source": "memory", "snippet": "| 2b | 客户主表→订单明细 | 5.1s | ✅ dry-run（真跑数据量大，预计几分钟） | | 3 | 日报 C1HVN2 | 38.3s | ✅ 真写成功 | | 4a | 结算月汇总 | <1s | ✅ 真写成功（零警告） | | 3a/3c/3d/4c | 审计 | — | ⏭ dry-run 跳过 | **修复的 wrapper 兼容性问题：** - API 响应格式：`data.spreadsheet` → `data.sheets.sheets`（lark-cli 兼容） - 参数名：`--values` 与 `--data` 等价处理 - camelCase→snake_case：`sheetId` → `sheet_id`，`grid_properties` 包装 - `+info` 端点：v3 metadata → v2 metainfo（含 sheets 列表） - `update-dimension`：`PUT /dimension_range`，1-indexed ### 16:48 聚光凭证部署 **来源：陈逸鸫** — 从 Mac 发来 `.env.juguang.9180` 和 `.env.juguang.9181`，走方案 A（复制文件，不在服务器重 OAuth） - 9180（云智 adv=9746532）：refresh 验证通过 ✅ - 9181（谦禾 adv=9013261,9598861 / YTL adv=7242040,9891870,10157917,10562529）：refresh 验证通过 ✅ - OAu", - "recallCount": 2, + "recallCount": 5, "dailyCount": 0, "groundedCount": 0, - "totalScore": 2, + "totalScore": 5, "maxScore": 1, "firstRecalledAt": "2026-06-09T09:31:36.077Z", - "lastRecalledAt": "2026-06-09T09:31:49.829Z", + "lastRecalledAt": "2026-06-16T11:21:05.649Z", "queryHashes": [ "50b1cd5a5d08", - "c3aab736911e" + "c3aab736911e", + "f6f7815a1493", + "b65a5ab98ae7", + "5bcdd0ab32df" ], "recallDays": [ - "2026-06-09" + "2026-06-09", + "2026-06-16" ], "conceptTags": [ "5.1s", @@ -2142,6 +2179,69 @@ "分析", "增加" ] + }, + "memory:memory/2026-06-16.md:1:19": { + "key": "memory:memory/2026-06-16.md:1:19", + "path": "memory/2026-06-16.md", + "startLine": 1, + "endLine": 19, + "source": "memory", + "snippet": "# 2026-06-16 工作日志 ## 陈逸鸫 - 创建「细水入海」数据刷新 skill **来源：** 陈逸鸫（`ou_f981d4811369c954b3597908ca93a01c`） **需求：** 将小溪的 full_refresh 数据刷新流程固化为大麦的 skill，命名为「细水入海」（与 Cursor 协作 skill 同名）。 **完成内容：** - 创建 `skills/full-data-refresh/SKILL.md` - 梳理了四个脚本的完整逻辑： - `bot_sales_step2_refresh.py` — S2 刷新（销售三表 D/H/I/J + K-U + X/Y/Z） - `sales_leads_full_refresh.py` — 全量刷新（S2 + 订单汇总） - `refresh_order_summary.py` — 仅订单汇总镜像刷新 - `full_refresh_sales.py` — 旧版全量刷新（含过程数据） - 文档包含：表格配置、列结构、数据规则、三种执行模式、常见问题 **触发词：** 细水入海、全量刷新、跑全量、full refresh、刷新销售数据", + "recallCount": 1, + "dailyCount": 0, + "groundedCount": 0, + "totalScore": 1, + "maxScore": 1, + "firstRecalledAt": "2026-06-16T09:36:31.306Z", + "lastRecalledAt": "2026-06-16T09:36:31.306Z", + "queryHashes": [ + "f6f7815a1493" + ], + "recallDays": [ + "2026-06-16" + ], + "conceptTags": [ + "full-refresh", + "bot-sales-step2-refresh.py", + "d/h/i/j", + "k-u", + "x/y/z", + "sales-leads-full-refresh.py", + "refresh-order-summary.py", + "full-refresh-sales.py" + ] + }, + "memory:memory/2026-05-28.md:173:205": { + "key": "memory:memory/2026-05-28.md:173:205", + "path": "memory/2026-05-28.md", + "startLine": 173, + "endLine": 205, + "source": "memory", + "snippet": "- 核心职能：增长数据分析、营销效果评估、转化漏斗分析、商业化洞察 - Emoji：📚 → 🌾 - 已更新文件：IDENTITY.md、AGENTS.md（@规则同步变更）、MEMORY.md - SOUL.md 无需改动（行为方法论为通用框架） ### 16:20 pipeline 非聚光部分验证完成 **lark-cli wrapper v0.3 支持的 action：** | 类别 | action | 状态 | |------|--------|------| | sheets | +read, +write, +append, +info, +meta | ✅ | | sheets | +create-sheet, +delete-sheet | ✅ | | sheets | +batch-set-style (stub) | ✅ | | sheets | +merge-cells, +unmerge-cells | ✅ | | sheets | +update-dimension | ✅ | | bitable | +app, +tables, +records, +create, +update | ✅ | | auth | status | ✅ | | im | +messages-send (stub) | ✅ | **pipeline 试跑结果（--dry-run）：** | 步骤 | 说明 | 耗时 | 状态 | |------|------|------|------| | 1a/1a2 | 微伴 xlsx | — | ⏭ 跳过（无 xlsx）", + "recallCount": 2, + "dailyCount": 0, + "groundedCount": 0, + "totalScore": 2, + "maxScore": 1, + "firstRecalledAt": "2026-06-16T11:17:42.687Z", + "lastRecalledAt": "2026-06-16T11:21:05.649Z", + "queryHashes": [ + "b65a5ab98ae7", + "5bcdd0ab32df" + ], + "recallDays": [ + "2026-06-16" + ], + "conceptTags": [ + "identity.md", + "agents.md", + "memory.md", + "soul.md", + "lark-cli", + "v0.3", + "create-sheet", + "delete-sheet" + ] } } } diff --git a/memory/2026-06-16.md b/memory/2026-06-16.md new file mode 100644 index 0000000..aebc461 --- /dev/null +++ b/memory/2026-06-16.md @@ -0,0 +1,48 @@ +# 2026-06-16 工作日志 + +## 陈逸鸫 - 细水入海 full_refresh v2 定稿 + +**来源：** 陈逸鸫（`ou_f981d4811369c954b3597908ca93a01c`） + +**核心变更（6/16 定稿）：** + +### 1. 订单汇总列结构变更 +- 从 A-X(24列) → A-W(23列)：去掉原 W「有效成单」列，订单号从 X 左移到 W +- V=渠道归属，W=订单号 + +### 2. 线索只绑有效单 +- `pick_valid_order()`: GSV>0 · 非全额退 · K≥C，取最新一笔 +- Y=1 时 K/L/X/N/O/P/Z 全写有效主单真实值 +- 已退单不写旧 X/L +- 无有效单 → Y=0，K/L/X 留空 + +### 3. 汇总 gate 全量覆盖（非 DB 扩行） +- 唯一真源 = 三表 Y=1 gate 的 unique X +- Step 4 + Step 5 同一 run，共用 `pick_valid_order()` + `db_info` +- 汇总 W = 三表 X（gate 同源，不是 merge 再查 DB） +- clear → gate 全量覆盖，不保留旧 W +- 同 X 多进线 → 只保留行号最小的 1 行 + +### 4. 分工定稿 +- **Cursor**: 微伴/旧表同步 · 三键去重 · V/W 公式 · 验单 · 撞库消解 +- **大麦**: full_refresh · 手机/UID/行课回填 · 订单汇总 merge · 完成后群回「full_refresh 完成」 +- **小溪**: 不再参与 Bot 刷新 + +### 5. 验收标准 +- gate X = 汇总 W（当前 406=406） +- 绑单审计 E1–E9 全部 0 +- 孤儿 X = 0 + +### 6. 脚本修改清单 +- `bot_sales_step2_refresh.py`: DB 层改为逐单存储 + `pick_valid_order()` + Y≠1 不写 X +- `sales_leads_full_refresh.py`: 同上 + 汇总改为 gate 全量重建 +- `refresh_order_summary.py`: A-W(23列) + 渠道分类改用 L 列 +- 新增 `audit_lead_primary_order_bind.py`: 线索绑单审计脚本 + +### 7. 环境修复 +- `secrets.env` 需要软链接: `ln -sf /root/.openclaw/workspace/secrets.env /root/.openclaw/workspace-xiaoban/secrets.env` + +### 8. Skill 文档已更新 +- `skills/full-data-refresh/SKILL.md` → v2 定稿，含 6 条核心架构规则 +- 协作契约: `xhs-ark-dashboard/docs/bot-full-refresh-v2.md` +- 大麦侧主文档: `xhs-ark-dashboard/docs/damai-full-refresh-skill.md` diff --git a/output/u16_export/角色id_20624_导出时间_20260616.xlsx b/output/u16_export/角色id_20624_导出时间_20260616.xlsx new file mode 100644 index 0000000..50ab3e4 Binary files /dev/null and b/output/u16_export/角色id_20624_导出时间_20260616.xlsx differ diff --git a/output/u16_export/角色id_21533_导出时间_20260616.xlsx b/output/u16_export/角色id_21533_导出时间_20260616.xlsx new file mode 100644 index 0000000..772973d Binary files /dev/null and b/output/u16_export/角色id_21533_导出时间_20260616.xlsx differ diff --git a/output/u16_export/角色id_21809_导出时间_20260616.xlsx b/output/u16_export/角色id_21809_导出时间_20260616.xlsx new file mode 100644 index 0000000..2babe8f Binary files /dev/null and b/output/u16_export/角色id_21809_导出时间_20260616.xlsx differ diff --git a/output/u16_export/角色id_22003_导出时间_20260616.xlsx b/output/u16_export/角色id_22003_导出时间_20260616.xlsx new file mode 100644 index 0000000..42d150d Binary files /dev/null and b/output/u16_export/角色id_22003_导出时间_20260616.xlsx differ diff --git a/output/u16_export/角色id_2279_导出时间_20260616.xlsx b/output/u16_export/角色id_2279_导出时间_20260616.xlsx new file mode 100644 index 0000000..2700c10 Binary files /dev/null and b/output/u16_export/角色id_2279_导出时间_20260616.xlsx differ diff --git a/output/u16_export/角色id_2375_导出时间_20260616.xlsx b/output/u16_export/角色id_2375_导出时间_20260616.xlsx new file mode 100644 index 0000000..5a290b8 Binary files /dev/null and b/output/u16_export/角色id_2375_导出时间_20260616.xlsx differ diff --git a/output/u16_export/角色id_25976_导出时间_20260616.xlsx b/output/u16_export/角色id_25976_导出时间_20260616.xlsx new file mode 100644 index 0000000..c45502b Binary files /dev/null and b/output/u16_export/角色id_25976_导出时间_20260616.xlsx differ diff --git a/output/u16_export/角色id_385_导出时间_20260616.xlsx b/output/u16_export/角色id_385_导出时间_20260616.xlsx new file mode 100644 index 0000000..6c9bc06 Binary files /dev/null and b/output/u16_export/角色id_385_导出时间_20260616.xlsx differ diff --git a/output/u16_export/角色id_389_导出时间_20260616.xlsx b/output/u16_export/角色id_389_导出时间_20260616.xlsx new file mode 100644 index 0000000..b1fc675 Binary files /dev/null and b/output/u16_export/角色id_389_导出时间_20260616.xlsx differ diff --git a/output/u16_export/角色id_642_导出时间_20260616.xlsx b/output/u16_export/角色id_642_导出时间_20260616.xlsx new file mode 100644 index 0000000..e9bf228 Binary files /dev/null and b/output/u16_export/角色id_642_导出时间_20260616.xlsx differ diff --git a/scripts/audit_lead_primary_order_bind.py b/scripts/audit_lead_primary_order_bind.py new file mode 100644 index 0000000..9c9109e --- /dev/null +++ b/scripts/audit_lead_primary_order_bind.py @@ -0,0 +1,189 @@ +#!/usr/bin/env python3 +""" +线索绑单审计 — 交叉验证「线索只绑有效单」规则。 +与 Cursor 同口径，输出 fingerprint + E1–E9 计数。 +""" +import json, os, re, hashlib, requests +from datetime import datetime +from collections import defaultdict + +CRED_DIR = "/root/.openclaw/credentials/xiaoxi" +SPREADSHEET_TOKEN = "NoZqsFi47hIOHEt9j8WcfRtbnug" +SALES_SHEETS = [ + ("qJF4I", "小龙"), + ("f975f0", "吴迪"), + ("qJF4J", "成都"), +] + +def s(val, default=""): + if val is None: return default + return str(val).strip() + +def get_token(): + with open(os.path.join(CRED_DIR, "config.json")) as f: + cfg = json.load(f) + resp = requests.post( + "https://open.feishu.cn/open-apis/auth/v3/tenant_access_token/internal", + json={"app_id": cfg["apps"][0]["appId"], "app_secret": cfg["apps"][0]["appSecret"]}, + timeout=15 + ) + return resp.json()["tenant_access_token"] + +def get_sheet_data(token, sheet_id, range_str): + resp = requests.get( + f"https://open.feishu.cn/open-apis/sheets/v2/spreadsheets/{SPREADSHEET_TOKEN}/values/{sheet_id}?majorDimension=ROWS&range={sheet_id}!{range_str}", + headers={"Authorization": f"Bearer {token}"}, timeout=30 + ) + if resp.json().get("code") == 0: + return resp.json()["data"]["valueRange"]["values"] + return [] + +def parse_date(d): + """Parse 'M月D日 HH:MM:SS' or 'M月D日' → datetime""" + if not d: return None + m = re.match(r'(\d+)月(\d+)日', d) + if not m: return None + month, day = int(m.group(1)), int(m.group(2)) + # Extract time if present + hour, minute, second = 0, 0, 0 + time_match = re.search(r'(\d+):(\d+):(\d+)', d) + if time_match: + hour, minute, second = int(time_match.group(1)), int(time_match.group(2)), int(time_match.group(3)) + return datetime(2026, month, day, hour, minute, second) + +def main(): + token = get_token() + + all_rows = [] + for sheet_id, name in SALES_SHEETS: + data = get_sheet_data(token, sheet_id, f"A1:Z3000") + if not data: continue + rows = data[2:] if len(data) > 2 else [] + for i, row in enumerate(rows): + row_idx = i + 3 + all_rows.append({ + "sheet": name, + "row": row_idx, + "A": s(row[0]) if len(row) > 0 else "", # 销售 + "B": s(row[1]) if len(row) > 1 else "", # 昵称 + "C": s(row[2]) if len(row) > 2 else "", # 进线日期 + "D": s(row[3]) if len(row) > 3 else "", # 体验节数 + "E": s(row[4]) if len(row) > 4 else "", # 手机号 + "H": s(row[7]) if len(row) > 7 else "", # UID + "K": s(row[10]) if len(row) > 10 else "", # 下单日期 + "L": s(row[11]) if len(row) > 11 else "", # 成交渠道 + "N": s(row[13]) if len(row) > 13 else "", # GMV + "O": s(row[14]) if len(row) > 14 else "", # 退款 + "P": s(row[15]) if len(row) > 15 else "", # GSV + "X": s(row[23]) if len(row) > 23 else "", # 订单号 + "Y": s(row[24]) if len(row) > 24 else "", # 有效订单 + "Z": s(row[25]) if len(row) > 25 else "", # 渠道 + }) + + # Filter: rows with any data + active_rows = [r for r in all_rows if any(r[k] for k in ["A","B","C","E","H","K","X","Y"])] + print(f"扫描线索行: {len(active_rows)}") + + # ── E1: Y=1 but X empty ── + e1 = [r for r in active_rows if r["Y"] == "1" and not r["X"]] + print(f"E1 Y=1无X: {len(e1)}") + + # ── E2: Y=1 but K empty ── + e2 = [r for r in active_rows if r["Y"] == "1" and not r["K"]] + print(f"E2 Y=1无K: {len(e2)}") + + # ── E3: Y=1 but L empty ── + e3 = [r for r in active_rows if r["Y"] == "1" and not r["L"]] + print(f"E3 Y=1无L: {len(e3)}") + + # ── E4: Y=1 but N empty ── + e4 = [r for r in active_rows if r["Y"] == "1" and not r["N"]] + print(f"E4 Y=1无N: {len(e4)}") + + # ── E5: Y=1 but K= c_dt: + e7.append(r) + print(f"E7 Y=0有X且K≥C: {len(e7)}") + + # ── E8: Y=0 has X but K 1} + print(f"同手机多X手机数: {len(multi_x_phones)}") + + # ── 问题行合计 ── + problem_rows = set() + for lst in [e1, e2, e3, e4, e5, e6, e7, e8, e9]: + for r in lst: + problem_rows.add((r["sheet"], r["row"])) + print(f"问题行合计: {len(problem_rows)}") + + # ── Fingerprint ── + # Build canonical string from problem codes + counts + fp_parts = [] + for code, lst in [("E1", e1), ("E2", e2), ("E3", e3), ("E4", e4), ("E5", e5), + ("E6", e6), ("E7", e7), ("E8", e8), ("E9", e9)]: + fp_parts.append(f"{code}:{len(lst)}") + fp_parts.append(f"multiX:{len(multi_x_phones)}") + fp_parts.append(f"total:{len(problem_rows)}") + fp_str = "|".join(fp_parts) + fingerprint = hashlib.md5(fp_str.encode()).hexdigest()[:16] + print(f"\n指纹: {fingerprint}") + + # ── 详细输出 ── + if e5: + print(f"\nE5 明细 (Y=1但K= C(线索日期) + ④ 全额退清: 所有订单都退费 → N/O/P 全部清空 + ⑤ N/O/P 0留空, O整元 + ⑥ G列不动, 订单汇总不动 +覆盖列: D/H/I/J + K-U + X/Y/Z +""" +import json, re, time, sys, os, requests, psycopg2 +from datetime import datetime +from collections import defaultdict + +SCRIPTS_DIR = os.path.dirname(os.path.abspath(__file__)) +WORKSPACE = os.path.dirname(SCRIPTS_DIR) +CRED_DIR = "/root/.openclaw/credentials/xiaoxi" + +# XXTEA 加密 +sys.path.insert(0, SCRIPTS_DIR) +from phone_encrypt import encrypt_phone + +SPREADSHEET_TOKEN = "NoZqsFi47hIOHEt9j8WcfRtbnug" + +SALES_SHEETS = [ + ("qJF4I", "小龙", "A1:Z1200"), + ("f975f0", "吴迪", "A1:Z700"), + ("qJF4J", "成都", "A1:Z2500"), +] + +CS_MAP = {"吴迪": "吴迪", "小龙": "小龙", "Tom": "Tom", "Bob": "Bob"} + +GOODS_NAMES = { + 57: "瓦拉英语level1·单季", 60: "瓦拉英语level1", 63: "瓦拉英语level1·单季", + 31: "瓦拉英语年包", 32: "瓦拉英语单季度包", 33: "瓦拉英语level2", 54: "瓦拉英语季度包", + 61: "瓦拉英语level1+2", +} + +CHANNEL_MAP = { + "Apple App Store": "苹果", "科大讯飞学习机": "讯飞", "学而思学习机": "学而思", + "华为应用市场": "华为", "小米应用市场": "小米", "应用宝应用市场": "应用宝", + "希沃学习机": "希沃", "荣耀应用市场": "荣耀", "小度学习机": "小度", + "oppo应用市场": "OPPO", "vivo应用市场": "VIVO", "京东方学习机": "京东方", + "步步高学习机": "步步高", "作业帮学习机": "作业帮", "魅族应用市场": "魅族", + "官网": "官网", +} + +# Z列渠道归属分类规则 [王虹茗确认 2026-06-15] +def classify_channel(key_from): + """将 key_from 归类为: 端内 / 销转 / 达人 / 直购""" + if not key_from: + return "直购" + kf = key_from.strip() + if kf in ("app-active-h5-0-0", "app-sales-bj-qhm-0", "app-sales-bj-wd-0"): + return "端内" + if kf.startswith("sales-adp-"): + return "销转" + if kf.startswith("newmedia-daren-") or kf == "newmedia-dianpu-wwxx-0-0": + return "达人" + # 其余: dianpu(不含wwxx) + partner/stream/miniprogram/jingxuan/空/shuadan等 → 直购 + return "直购" + + +def parse_clue_date(date_str): + """解析进线日期: '6月14日 19:09:12' 或 '6月14日' → datetime.date""" + if not date_str: + return None + date_str = date_str.strip() + # 格式: "6月14日 19:09:12" 或 "6月14日" + m = re.match(r'(\d+)月(\d+)日', date_str) + if not m: + return None + month, day = int(m.group(1)), int(m.group(2)) + return datetime(2026, month, day).date() + +LOG_FILE = "/var/log/xiaoxi_step2_refresh.log" + +def log(msg): + ts = datetime.now().strftime("%Y-%m-%d %H:%M:%S") + line = f"[{ts}] {msg}" + print(line) + with open(LOG_FILE, "a") as f: + f.write(line + "\n") + +def get_secret(key): + with open(os.path.join(WORKSPACE, "secrets.env")) as f: + for line in f: + if line.startswith(f"{key}="): + return line.strip().split("=", 1)[1].strip("'\"") + +def get_fs_token(): + with open(os.path.join(CRED_DIR, "config.json")) as f: + cfg = json.load(f) + resp = requests.post( + "https://open.feishu.cn/open-apis/auth/v3/tenant_access_token/internal", + json={"app_id": cfg["apps"][0]["appId"], "app_secret": cfg["apps"][0]["appSecret"]}, + timeout=15 + ) + return resp.json()["tenant_access_token"] + +def read_sheet(token, sheet_id, range_str=None): + url = f"https://open.feishu.cn/open-apis/sheets/v2/spreadsheets/{SPREADSHEET_TOKEN}/values/{sheet_id}" + if range_str: + url += f"!{range_str}" + resp = requests.get(url, headers={"Authorization": f"Bearer {token}"}, timeout=30) + data = resp.json() + if data.get("code") != 0: + raise RuntimeError(f"读取失败 {sheet_id}: {data}") + return data["data"]["valueRange"]["values"] + +def put_values(token, sheet_id, range_str, values): + url = f"https://open.feishu.cn/open-apis/sheets/v2/spreadsheets/{SPREADSHEET_TOKEN}/values" + body = {"valueRange": {"range": f"{sheet_id}!{range_str}", "values": values}} + resp = requests.put(url, headers={ + "Authorization": f"Bearer {token}", + "Content-Type": "application/json" + }, json=body, timeout=30) + r = resp.json() + if r.get("code") != 0: + log(f" ❌ {range_str}: {r.get('code')} {r.get('msg')}") + return False + return True + +def batch_in(cur, sql_tpl, params, chunk=500): + results = [] + for i in range(0, len(params), chunk): + batch = params[i:i+chunk] + ph = ",".join(["%s"] * len(batch)) + cur.execute(sql_tpl % ph, batch) + results.extend(cur.fetchall()) + return results + +# ── Step 1: 解析销售三表 ── +def parse_sales_sheets(token): + """返回 {sheet_id: [(row_idx, sales_name, nickname, date_str, phone, existing_uid, g_val, existing_d, existing_i, existing_j), ...]}""" + all_data = {} + for sid, sname, rng in SALES_SHEETS: + rows = read_sheet(token, sid, rng) + entries = [] + for idx, row in enumerate(rows[2:], start=3): + if not row: + continue + sr = str(row[0]).strip() if len(row) > 0 and row[0] else "" + sales = None + for k, v in CS_MAP.items(): + if k in sr: + sales = v + break + if not sales: + continue + nickname = str(row[1]).strip() if len(row) > 1 and row[1] else "" + date_str = str(row[2]).strip() if len(row) > 2 and row[2] else "" + phone = "" + if len(row) > 4 and row[4]: + try: + phone = str(int(float(row[4]))) + except: + pass + uid = "" + if len(row) > 7 and row[7]: + try: + uid = str(int(float(row[7]))) + except: + pass + g_val = str(row[6]).strip() if len(row) > 6 and row[6] else "" + # 读取已有 D/I/J 值 (用于只补空判断) + d_val = str(row[3]).strip() if len(row) > 3 and row[3] else "" + i_val = str(row[8]).strip() if len(row) > 8 and row[8] else "" + j_val = str(row[9]).strip() if len(row) > 9 and row[9] else "" + entries.append((idx, sales, nickname, date_str, phone, uid, g_val, d_val, i_val, j_val)) + all_data[sid] = entries + log(f" {sname}: {len(entries)} rows, {sum(1 for e in entries if e[5] and e[5].isdigit() and int(e[5])>0)} with uid") + return all_data + +# ── Step 2: XXTEA 加密 → PG tel_encrypt 精确匹配 ── +def phone_to_uid_xxtea(all_entries): + """E列11位明文手机号 → XXTEA加密 → bi_vala_app_account.tel_encrypt精确匹配 → UID""" + # 收集所有 11 位手机号 + phone_rows = [] + for sid, entries in all_entries.items(): + for idx, sales, nick, date_str, phone, uid, g_val, d_val, i_val, j_val in entries: + if re.match(r'^\d{11}$', phone): + phone_rows.append((sid, idx, phone)) + + if not phone_rows: + return {} + + log(f" XXTEA 加密匹配: {len(phone_rows)} 个手机号") + + # 加密所有手机号 + phone_enc_map = {} # {encrypted: phone} + for _, _, phone in phone_rows: + try: + enc = encrypt_phone(phone) + phone_enc_map[enc] = phone + except Exception as e: + log(f" 加密失败 {phone}: {e}") + + log(f" 加密完成, 唯一密文: {len(phone_enc_map)}") + + # PG 精确查询 + conn = psycopg2.connect( + host="bj-postgres-16pob4sg.sql.tencentcdb.com", port=28591, + user="ai_member", password=get_secret("PG_ONLINE_PASSWORD"), + dbname="vala_bi", connect_timeout=30 + ) + cur = conn.cursor() + + enc_list = list(phone_enc_map.keys()) + phone_to_uid = {} + for i in range(0, len(enc_list), 500): + chunk = enc_list[i:i+500] + ph = ",".join(["%s"] * len(chunk)) + cur.execute( + f"SELECT id, tel_encrypt FROM bi_vala_app_account WHERE tel_encrypt IN ({ph}) AND status=1 AND deleted_at IS NULL", + chunk + ) + for uid, tel_enc in cur.fetchall(): + plain = phone_enc_map.get(tel_enc) + if plain: + phone_to_uid[plain] = str(uid) + time.sleep(0.05) + + cur.close() + conn.close() + log(f" 精确匹配到 {len(phone_to_uid)} 个 UID (via XXTEA)") + return phone_to_uid + +# ── Step 3: PostgreSQL 批量查询 ── +def query_all_pg(all_entries, phone_map): + """查询所有需要的数据""" + uid_set = set() + for sid, entries in all_entries.items(): + for idx, sales, nick, date_str, phone, uid, g_val, d_val, i_val, j_val in entries: + if re.match(r'^\d{11}$', phone) and phone in phone_map: + uid_set.add(int(phone_map[phone])) + if uid and uid.isdigit() and int(uid) > 0: + uid_set.add(int(uid)) + + uid_list = list(uid_set) + log(f" 有效 user_id: {len(uid_list)}") + + conn = psycopg2.connect( + host="bj-postgres-16pob4sg.sql.tencentcdb.com", port=28591, + user="ai_member", password=get_secret("PG_ONLINE_PASSWORD"), + dbname="vala_bi", connect_timeout=30 + ) + cur = conn.cursor() + + info = {uid: { + "reg_date": "", "download_channel": "", "trial_count": 0, + "has_order": False, "orders": [], + "activation": "", "lesson_progress": "", "lesson_time": "", "lesson_minutes": 0, + "max_lesson": 0, + } for uid in uid_set} + + # 3a. 注册信息 + log(" 查询注册信息...") + reg_info = batch_in(cur, + "SELECT id, created_at, download_channel FROM bi_vala_app_account WHERE id IN (%s) AND status=1 AND deleted_at IS NULL", + uid_list + ) + for aid, created_at, dc in reg_info: + if aid in info: + info[aid]["reg_date"] = created_at.strftime("%Y-%m-%d") if created_at else "" + raw_ch = dc or "" + info[aid]["download_channel"] = CHANNEL_MAP.get(raw_ch, raw_ch) + + # 3b. 体验节数 + log(" 查询体验节数...") + trial_info = batch_in(cur, + "SELECT account_id, COUNT(*) FROM bi_user_course_detail WHERE account_id IN (%s) AND expire_time IS NULL AND deleted_at IS NULL GROUP BY account_id", + uid_list + ) + for aid, cnt in trial_info: + if aid in info: + info[aid]["trial_count"] = cnt + + # 3c. 订单信息 + log(" 查询订单信息...") + orders = batch_in(cur, + "SELECT account_id, trade_no, pay_success_date, key_from, goods_id, pay_amount_int, order_status FROM bi_vala_order WHERE account_id IN (%s) AND pay_success_date IS NOT NULL AND order_status IN (3,4) ORDER BY pay_success_date DESC", + uid_list + ) + user_orders = defaultdict(list) + for o in orders: + user_orders[o[0]].append(o) + + trade_nos = [o[1] for o in orders if o[1]] + refund_map = {} + if trade_nos: + refunds = batch_in(cur, + "SELECT trade_no, refund_amount_int FROM bi_refund_order WHERE trade_no IN (%s) AND status=3", + trade_nos + ) + for tn, amt in refunds: + refund_map[tn] = amt + + for aid, olist in user_orders.items(): + if aid not in info: + continue + info[aid]["has_order"] = True + # 存储逐单数据（按 pay_success_date DESC），供后续选取有效单 + orders_data = [] + for o in olist: + trade_no = o[1] or "" + pay_dt = o[2] + key_from = o[3] or "" + goods_id = o[4] + pay_amount = o[5] / 100.0 + refund_amount = refund_map.get(trade_no, 0) / 100.0 + gsv = pay_amount - refund_amount + orders_data.append({ + "trade_no": trade_no, + "pay_dt": pay_dt, + "key_from": key_from, + "goods_id": goods_id, + "product": GOODS_NAMES.get(goods_id, f"商品{goods_id}"), + "pay_amount": pay_amount, + "refund_amount": refund_amount, + "gsv": gsv, + }) + info[aid]["orders"] = orders_data + + # 3d. 激活课程 + log(" 查询激活课程...") + try: + activations = batch_in(cur, + "SELECT t.account_id, t.season_package_level FROM bi_vala_seasonal_ticket t WHERE t.account_id IN (%s) AND t.status=1 AND t.deleted_at IS NULL AND t.season_package_level IN ('A1','A2')", + uid_list + ) + for aid, lvl in activations: + if aid in info: + info[aid]["activation"] = lvl + except Exception as e: + log(f" 激活查询异常: {e}") + + # 3e. 角色信息 + log(" 查询角色信息...") + char_info = batch_in(cur, + "SELECT account_id, id FROM bi_vala_app_character WHERE account_id IN (%s) AND deleted_at IS NULL", + uid_list + ) + account_chars = defaultdict(list) + char_to_account = {} + for aid, cid in char_info: + account_chars[aid].append(cid) + char_to_account[cid] = aid + char_ids = list(char_to_account.keys()) + log(f" 角色数: {len(char_ids)}") + + # 3f. 课程映射 + cur.execute("SELECT id, course_level, course_season, course_unit, course_lesson FROM bi_level_unit_lesson") + chapter_map = {} + for ch_id, cl, cs, cu, cl2 in cur.fetchall(): + chapter_map[ch_id] = (cl or "", cs or "", cu or "", cl2 or "") + + # 3g. 课时完成记录 + log(" 查询课时完成记录...") + char_plays = defaultdict(lambda: {"latest_time": None, "latest_chapter": None, "max_lesson_idx": 0, "total_ms": 0}) + for tbl_idx in range(8): + table = f"bi_user_chapter_play_record_{tbl_idx}" + try: + cur.execute( + f"SELECT user_id, chapter_id, created_at FROM {table} WHERE play_status=1 AND deleted_at IS NULL AND user_id = ANY(%s)", + (char_ids,) + ) + for uid, ch_id, created_at in cur.fetchall(): + ch_data = chapter_map.get(ch_id) + if not ch_data: + continue + rec = char_plays[uid] + if rec["latest_time"] is None or created_at > rec["latest_time"]: + rec["latest_time"] = created_at + rec["latest_chapter"] = (ch_id, ch_data) + cl, cs, cu, cl2 = ch_data + try: + u_num = int(cu[1:]) if cu and len(cu) >= 2 else 0 + l_num = int(cl2[1:]) if cl2 and len(cl2) >= 2 else 0 + lesson_idx = u_num * 5 + l_num + if lesson_idx > rec["max_lesson_idx"]: + rec["max_lesson_idx"] = lesson_idx + except: + pass + except Exception as e: + log(f" 警告 {table}: {e}") + + # 3h. 学习总耗时 + log(" 查询学习耗时...") + for tbl_idx in range(8): + table = f"bi_user_component_play_record_{tbl_idx}" + try: + cur.execute( + f"SELECT user_id, SUM(COALESCE(interval_time,0)) FROM {table} WHERE user_id = ANY(%s) AND deleted_at IS NULL GROUP BY user_id", + (char_ids,) + ) + for uid, total_ms in cur.fetchall(): + if uid in char_plays: + char_plays[uid]["total_ms"] += (total_ms or 0) + except Exception as e: + log(f" 警告 {table}: {e}") + + cur.close() + conn.close() + + # 汇总到 account 级别 + for aid in uid_set: + chars = account_chars.get(aid, []) + best_time = None + best_ch = None + max_lesson = 0 + total_ms = 0 + for cid in chars: + play = char_plays.get(cid) + if not play: + continue + if play["latest_chapter"]: + if best_time is None or play["latest_time"] > best_time: + best_time = play["latest_time"] + best_ch = play["latest_chapter"] + if play["max_lesson_idx"] > max_lesson: + max_lesson = play["max_lesson_idx"] + total_ms += play["total_ms"] + + info[aid]["max_lesson"] = max_lesson + info[aid]["lesson_minutes"] = round(total_ms / 60000, 1) + if info[aid]["lesson_minutes"] == int(info[aid]["lesson_minutes"]): + info[aid]["lesson_minutes"] = int(info[aid]["lesson_minutes"]) + + if best_ch: + ch_id, (cl, cs, cu, cl2) = best_ch + info[aid]["lesson_progress"] = f"{cl}-{cs}-{cu}-{cl2}" + info[aid]["lesson_time"] = best_time.strftime("%Y-%m-%d") if best_time else "" + + log(f" 数据库查询完成") + return info + +# ── Step 4: 写入销售三表 ── +def pick_valid_order(orders, clue_date): + """ + 从订单列表中选取有效主单。 + 规则: GSV>0 · 非全额退 · K≥C · 有时序 + 返回: (order_dict, is_valid) 或 (None, False) + """ + if not orders: + return None, False + clue_dt = parse_clue_date(clue_date) if clue_date else None + valid = [] + for o in orders: + gsv = o["gsv"] + pay_amount = o["pay_amount"] + refund_amount = o["refund_amount"] + is_full_refund = (pay_amount > 0 and pay_amount == refund_amount) + if gsv <= 0 or is_full_refund: + continue + pay_dt = o["pay_dt"] + if pay_dt and clue_dt and pay_dt.date() < clue_dt: + continue + valid.append(o) + if not valid: + return None, False + # 取最新一笔有效单 + valid.sort(key=lambda o: o["pay_dt"] or datetime.min, reverse=True) + return valid[0], True + + +def write_sales_sheets(token, all_entries, phone_map, db_info): + """全覆盖写入销售表的自动列: D/H/I/J + K-U + X/Y/Z""" + now_str = datetime.now().strftime("%Y-%m-%d %H:%M:%S") + + for sid, sname, _ in SALES_SHEETS: + entries = all_entries[sid] + log(f" 写入 {sname} ({sid})...") + + groups = [] + cur_grp = [] + for idx, sales, nick, date_str, phone, uid, g_val, d_val, i_val, j_val in entries: + item = {"row": idx, "phone": phone, "uid": uid, "g_val": g_val, + "date": date_str, "d_val": d_val, "i_val": i_val, "j_val": j_val} + if not cur_grp or idx == cur_grp[-1]["row"] + 1: + cur_grp.append(item) + else: + groups.append(cur_grp) + cur_grp = [item] + if cur_grp: + groups.append(cur_grp) + + for g in groups: + sr, er = g[0]["row"], g[-1]["row"] + + d_vals, h_vals, i_vals, j_vals = [], [], [], [] + k_vals, l_vals, m_vals, n_vals = [], [], [], [] + o_vals, p_vals, q_vals, r_vals = [], [], [], [] + s_vals, t_vals, u_vals = [], [], [] + x_vals, y_vals, z_vals = [], [], [] + + for item in g: + phone = item["phone"] + existing_uid = item["uid"] + existing_d = item.get("d_val", "") + existing_i = item.get("i_val", "") + existing_j = item.get("j_val", "") + clue_date = item.get("date", "") + + # 确定 UID: XXTEA 精确匹配优先 + aid = 0 + uid_str = "" + if re.match(r'^\d{11}$', phone) and phone in phone_map: + uid_str = phone_map[phone] + aid = int(uid_str) + elif existing_uid and existing_uid.isdigit() and int(existing_uid) > 0: + uid_str = existing_uid + aid = int(existing_uid) + + # H: UID — XXTEA匹配到就写，否则留空 + if re.match(r'^\d{11}$', phone) and phone in phone_map: + h_vals.append([phone_map[phone]]) + elif re.match(r'^\d{11}$', phone): + h_vals.append([""]) + elif existing_uid and existing_uid.isdigit(): + h_vals.append([existing_uid]) + else: + h_vals.append([""]) + + if aid > 0 and aid in db_info: + di = db_info[aid] + + # D: 体验节数 — 只补空 + if existing_d: + d_vals.append([existing_d]) + else: + tc = di["trial_count"] + d_vals.append([tc if tc > 0 else ""]) + + # I: 注册日 — 只补空 + if existing_i: + i_vals.append([existing_i]) + else: + i_vals.append([di["reg_date"]]) + + # J: 下载渠道 — 只补空 + if existing_j: + j_vals.append([existing_j]) + else: + j_vals.append([di["download_channel"]]) + + # 选取有效主单 + orders = di.get("orders", []) + valid_order, is_valid = pick_valid_order(orders, clue_date) + + if is_valid and valid_order: + # Y=1: K/L/X/N/O/P/Z 全写该有效单真实值 + pay_dt = valid_order["pay_dt"] + order_date = f"{pay_dt.month}月{pay_dt.day}日 {pay_dt.strftime('%H:%M:%S')}" if pay_dt else "" + k_vals.append([order_date]) + l_vals.append([valid_order["key_from"]]) + m_vals.append([valid_order["product"]]) + n_vals.append([int(valid_order["pay_amount"]) if valid_order["pay_amount"] > 0 else ""]) + o_vals.append([int(valid_order["refund_amount"]) if valid_order["refund_amount"] > 0 else ""]) + p_vals.append([int(valid_order["gsv"]) if valid_order["gsv"] > 0 else ""]) + x_vals.append([valid_order["trade_no"]]) + y_vals.append([1]) + z_vals.append([classify_channel(valid_order["key_from"])]) + elif di["has_order"]: + # 有订单但无有效单 → Y=0, K/L/X 留空, N/O/P 不写 + k_vals.append([""]) + l_vals.append([""]) + m_vals.append([""]) + n_vals.append([""]) + o_vals.append([""]) + p_vals.append([""]) + x_vals.append([""]) + y_vals.append([""]) + z_vals.append([""]) + else: + # 无订单 + k_vals.append([""]) + l_vals.append([""]) + m_vals.append([""]) + n_vals.append([""]) + o_vals.append([""]) + p_vals.append([""]) + x_vals.append([""]) + y_vals.append([""]) + z_vals.append([""]) + + # Q: 激活课程 + act = di["activation"] + if act: + q_vals.append([f"{act}体验课" if act in ("A1", "A2") else act]) + else: + q_vals.append([""]) + + # R: 行课进度, S: 最近行课时间 + lp = di["lesson_progress"] + r_vals.append([lp if lp else ""]) + s_vals.append([di["lesson_time"]]) + + # T: 学习时长 + lm = di["lesson_minutes"] + t_vals.append([lm if lm > 0 else ""]) + else: + for arr in [d_vals, i_vals, j_vals, k_vals, l_vals, m_vals, n_vals, + o_vals, p_vals, q_vals, r_vals, s_vals, t_vals, + x_vals, y_vals, z_vals]: + arr.append([""]) + + # U: 更新时间 + u_vals.append([now_str]) + + cols = [ + ("D", d_vals), ("H", h_vals), ("I", i_vals), ("J", j_vals), + ("K", k_vals), ("L", l_vals), ("M", m_vals), ("N", n_vals), + ("O", o_vals), ("P", p_vals), ("Q", q_vals), ("R", r_vals), + ("S", s_vals), ("T", t_vals), ("U", u_vals), + ("X", x_vals), ("Y", y_vals), ("Z", z_vals), + ] + for col_letter, vals in cols: + put_values(token, sid, f"{col_letter}{sr}:{col_letter}{er}", vals) + time.sleep(0.1) + + log(f" {sname}: {len(entries)} rows done") + +# ── Main ── +def main(): + log("=" * 50) + log("Bot 销转看板 Step2 刷新 (XXTEA精确匹配版) 启动") + + try: + token = get_fs_token() + + log("Step 1: 解析销售三表") + all_entries = parse_sales_sheets(token) + + log("Step 2: XXTEA 加密 → PG tel_encrypt 精确匹配") + phone_map = phone_to_uid_xxtea(all_entries) + + log("Step 3: PostgreSQL 批量查询") + db_info = query_all_pg(all_entries, phone_map) + + log("Step 4: 写入销售三表") + write_sales_sheets(token, all_entries, phone_map, db_info) + + log("✅ Step2 刷新完成 (XXTEA)") + return 0 + except Exception as e: + log(f"❌ ERROR: {e}") + import traceback + traceback.print_exc() + return 1 + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/feishu_sheet_utils.py b/scripts/feishu_sheet_utils.py new file mode 100644 index 0000000..6fd87c6 --- /dev/null +++ b/scripts/feishu_sheet_utils.py @@ -0,0 +1,142 @@ +#!/usr/bin/env python3 +""" +飞书表格安全写入工具 — 自动遵守 5000 格/次 API 上限 + +飞书 Open API 单次写入上限为 5000 格（行×列）。 +超过上限的请求会静默失败（API 不报错但数据不完整）， +导致旧数据残留、新数据被部分覆盖、末尾行丢失等问题。 + +本模块封装了安全的分批写入和清空逻辑，所有操作自动计算 +批大小确保 ≤ 4400 格/批（留 12% 安全余量）。 + +用法: + from feishu_sheet_utils import FeishuSheetWriter + + writer = FeishuSheetWriter(SPREADSHEET_TOKEN, token) + writer.clear(sheet_id, start_row=3, end_row=500, cols=26) + writer.write(sheet_id, start_row=3, rows=data, cols=26) +""" + +import time +import requests + +# 飞书 API 单次写入格数上限 +FEISHU_CELL_LIMIT = 5000 +# 安全余量系数（0.88，即实际使用 ≤ 4400 格/批） +SAFETY_FACTOR = 0.88 +# 单批最大格数 +SAFE_CELLS_PER_BATCH = int(FEISHU_CELL_LIMIT * SAFETY_FACTOR) # 4400 + + +def max_rows_per_batch(cols): + """根据列数计算单批最大行数（确保 ≤ 4400 格）。""" + return max(1, SAFE_CELLS_PER_BATCH // cols) + + +class FeishuSheetWriter: + """飞书表格安全写入器，自动分批遵守 5000 格上限。""" + + def __init__(self, spreadsheet_token, tenant_token): + self.spreadsheet_token = spreadsheet_token + self.token = tenant_token + self.base_url = "https://open.feishu.cn/open-apis/sheets/v2" + + def _put(self, sheet_id, range_str, values, retries=3): + """单次写入，含重试。""" + url = f"{self.base_url}/spreadsheets/{self.spreadsheet_token}/values" + body = {"valueRange": {"range": f"{sheet_id}!{range_str}", "values": values}} + for attempt in range(retries): + resp = requests.put(url, headers={ + "Authorization": f"Bearer {self.token}", + "Content-Type": "application/json" + }, json=body, timeout=30) + result = resp.json() + if result.get("code") == 0: + return True + print(f" Retry {attempt+1} for {range_str}: {result.get('msg','')}") + time.sleep(1) + print(f" FAILED {range_str}") + return False + + def _col_letter(self, idx): + """0-based column index → Excel column letter(s). 0→A, 25→Z, 26→AA.""" + result = "" + n = idx + while n >= 0: + result = chr(ord('A') + n % 26) + result + n = n // 26 - 1 + return result + + def _range_str(self, start_row, end_row, cols): + """生成范围字符串，如 A3:Z52。""" + end_col = self._col_letter(cols - 1) + return f"A{start_row}:{end_col}{end_row}" + + def clear(self, sheet_id, start_row, end_row, cols): + """ + 安全清空指定区域（写入空字符串）。 + 自动分批，每批 ≤ 4400 格。 + """ + if end_row < start_row: + return + batch_rows = max_rows_per_batch(cols) + total = end_row - start_row + 1 + print(f" Clearing {sheet_id} rows {start_row}-{end_row} " + f"({total} rows × {cols} cols, batch={batch_rows} rows)") + + for batch_start in range(start_row, end_row + 1, batch_rows): + batch_end = min(batch_start + batch_rows - 1, end_row) + n_rows = batch_end - batch_start + 1 + empty = [[""] * cols for _ in range(n_rows)] + rng = self._range_str(batch_start, batch_end, cols) + ok = self._put(sheet_id, rng, empty) + if not ok: + print(f" Clear batch {rng} failed, continuing...") + time.sleep(0.15) + + def write(self, sheet_id, start_row, rows, cols): + """ + 安全写入数据行。 + 自动分批，每批 ≤ 4400 格。 + rows: list of list，每行长度应为 cols。 + """ + if not rows: + return + batch_rows = max_rows_per_batch(cols) + total = len(rows) + print(f" Writing {sheet_id} {total} rows × {cols} cols " + f"(batch={batch_rows} rows, {batch_rows * cols} cells/batch)") + + for batch_start in range(0, total, batch_rows): + batch = rows[batch_start:batch_start + batch_rows] + sr = start_row + batch_start + er = sr + len(batch) - 1 + rng = self._range_str(sr, er, cols) + ok = self._put(sheet_id, rng, batch) + if not ok: + print(f" Write batch {rng} failed!") + time.sleep(0.3) + + def clear_excess(self, sheet_id, total_written, old_count, cols): + """清除超出新数据范围的旧行残留。""" + if old_count <= total_written: + return + clear_start = start_row_base = 3 # 假设数据从第3行开始 + actual_start = clear_start + total_written + actual_end = clear_start + old_count - 1 + if actual_start > actual_end: + return + print(f" Clearing excess rows {actual_start}-{actual_end}") + self.clear(sheet_id, actual_start, actual_end, cols) + + +def safe_clear_range(token, spreadsheet_token, sheet_id, start_row, end_row, cols): + """便捷函数：安全清空指定区域。""" + writer = FeishuSheetWriter(spreadsheet_token, token) + writer.clear(sheet_id, start_row, end_row, cols) + + +def safe_write_rows(token, spreadsheet_token, sheet_id, start_row, rows, cols): + """便捷函数：安全写入数据行。""" + writer = FeishuSheetWriter(spreadsheet_token, token) + writer.write(sheet_id, start_row, rows, cols) diff --git a/scripts/phone_encrypt.py b/scripts/phone_encrypt.py new file mode 100644 index 0000000..3189c32 --- /dev/null +++ b/scripts/phone_encrypt.py @@ -0,0 +1,66 @@ +#!/usr/bin/env python3 +""" +手机号加密工具 — 与 Go 端 Encrypt 函数完全一致 + +Go 原始逻辑: + func Encrypt(data string) string { + key := "K1pNOZ5O5+ZqTPSHA2kzPdoNOMOGcv6g" + encryptData := xxtea.Encrypt([]byte(data), []byte(key)) + n := base64.StdEncoding.EncodeToString(encryptData) + n = strings.ReplaceAll(n, "+", "-") + n = strings.ReplaceAll(n, "/", "_") + n = strings.ReplaceAll(n, "=", ".") + return n + } + +匹配方式: 加密明文手机号 → 与 bi_vala_app_account.tel_encrypt 比对 → 获取 account_id + +MD5 加密: tel_encrypt → 解密为明文 → MD5 → 用于跨系统关联 +""" +import xxtea +import base64 +import hashlib + +KEY = "K1pNOZ5O5+ZqTPSHA2kzPdoNOMOGcv6g" + + +def encrypt_phone(phone: str) -> str: + """加密明文手机号，返回与数据库 tel_encrypt 字段一致的密文""" + encrypted = xxtea.encrypt(phone.encode(), KEY.encode()) + result = base64.b64encode(encrypted).decode() + result = result.replace("+", "-").replace("/", "_").replace("=", ".") + return result + + +def encrypt_phones(phones: list[str]) -> dict[str, str]: + """批量加密手机号，返回 {密文: 明文手机号} 映射""" + return {encrypt_phone(p): p for p in phones} + + +def decrypt_phone(encrypted: str) -> str: + """解密 tel_encrypt 还原明文手机号（仅用于验证）""" + restored = encrypted.replace("-", "+").replace("_", "/").replace(".", "=") + decrypted = xxtea.decrypt(base64.b64decode(restored), KEY.encode()) + return decrypted.decode() + + +def phone_md5(phone: str) -> str: + """明文手机号 → MD5（32位小写十六进制）""" + return hashlib.md5(phone.encode()).hexdigest() + + +def tel_encrypt_to_md5(tel_encrypt: str) -> str: + """tel_encrypt 密文 → 解密 → MD5（一步到位）""" + phone = decrypt_phone(tel_encrypt) + return phone_md5(phone) + + +if __name__ == "__main__": + # 自测 + test_phones = ["13800138000", "15912345678", "18888888888"] + for p in test_phones: + enc = encrypt_phone(p) + dec = decrypt_phone(enc) + md5 = phone_md5(p) + status = "✓" if dec == p else "✗" + print(f"{p} → {enc} → {dec} → MD5:{md5} {status}") diff --git a/scripts/refresh_order_summary.py b/scripts/refresh_order_summary.py new file mode 100644 index 0000000..05a7430 --- /dev/null +++ b/scripts/refresh_order_summary.py @@ -0,0 +1,285 @@ +#!/usr/bin/env python3 +""" +订单汇总 A–X 全量镜像刷新 +触发：Step2（Cursor Step1 完成后 @小溪） +归属：小溪 (xiaoxi) + +进表条件：K=是 · O>0 · 非全额退(P空或P0 · 非全额退(P空或P 10 and raw[10] else "" + if k != "是": + continue + + # O > 0 + try: + o_val = float(raw[14]) if len(raw) > 14 and raw[14] not in (None, "") else 0 + except (ValueError, TypeError): + o_val = 0 + if o_val <= 0: + continue + + # 非全额退: P空或P 15 and raw[15] not in (None, "") else 0 + except (ValueError, TypeError): + p_val = 0 + if p_val > 0 and p_val >= o_val: + # 全额退 → 不进订单表 + continue + + # L ≥ C (C为空时跳过此检查，如直购用户无进线日期) + c_str = str(raw[2]).strip() if len(raw) > 2 and raw[2] else "" + l_str = str(raw[11]).strip() if len(raw) > 11 and raw[11] else "" + c_date = parse_date(c_str) + l_date = parse_date(l_str) + if c_date is not None and not date_le(c_date, l_date): + continue + + # 通过所有条件 + order_rows.append(r) + + print(f"Order rows after filter: {len(order_rows)}") + + # ── Step 2.5: 去重（同一人可能在三表中出现多次）── + # 按 (A销售归属, B微信昵称, O下单金额, P退款金额, L下单日期) 去重 + seen = set() + deduped = [] + for r in order_rows: + raw = r["raw"] + a = str(raw[0]).strip() if raw[0] else "" + b = str(raw[1]).strip() if len(raw) > 1 and raw[1] else "" + o = str(raw[14]).strip() if len(raw) > 14 and raw[14] else "" + p = str(raw[15]).strip() if len(raw) > 15 and raw[15] else "" + l = str(raw[11]).strip() if len(raw) > 11 and raw[11] else "" + key = (a, b, o, p, l) + if key not in seen: + seen.add(key) + deduped.append(r) + dup_count = len(order_rows) - len(deduped) + if dup_count > 0: + print(f" Removed {dup_count} duplicate rows") + order_rows = deduped + + # ── Step 3: 按 L 下单日降序 ── + order_rows.sort(key=lambda r: str(r["raw"][11]) if len(r["raw"]) > 11 and r["raw"][11] else "", reverse=True) + + # ── Step 4: 构建 A–W 行（23列）── + # 新契约: A-U镜像 + V=渠道归属 + W=订单号 + summary_rows = [] + for r in order_rows: + raw = r["raw"] + # A–U 原样镜像（21列） + new_row = list(raw[:21]) + + # V: 渠道归属（基于 L 成交渠道） + l_channel = str(raw[11]).strip() if len(raw) > 11 and raw[11] else "" + sales_name = str(raw[0]).strip() if len(raw) > 0 and raw[0] else "" + v = classify_w_channel(l_channel, sales_name) + new_row.append(v) + + # W: 订单号（原 X 列） + order_no = str(raw[23]).strip() if len(raw) > 23 and raw[23] else "" + new_row.append(order_no) + + summary_rows.append(new_row) + + print(f"Summary rows: {len(summary_rows)}") + + # ── Step 5: 写入订单汇总（使用安全写入工具，自动遵守 5000 格上限）── + print("Writing to 订单汇总...") + writer = FeishuSheetWriter(SPREADSHEET_TOKEN, token) + + # 先清空旧数据区（23 列，自动计算批大小 ≤ 4400 格/批） + writer.clear(SUMMARY_SHEET, start_row=3, end_row=2000, cols=23) + time.sleep(0.5) + + # 写入新数据（23 列 A-W，自动分批） + total = len(summary_rows) + writer.write(SUMMARY_SHEET, start_row=3, rows=summary_rows, cols=23) + + # ── Step 6: 清除多余旧行 ── + existing = read_sheet(token, SUMMARY_SHEET, "A3:A4000") + old_count = len([r for r in existing if r and any(c for c in r if c)]) + if old_count > total: + writer.clear(SUMMARY_SHEET, start_row=3 + total, end_row=3 + old_count - 1, cols=23) + + print(f"[{datetime.now():%Y-%m-%d %H:%M:%S}] ✅ 订单汇总刷新完成") + + +if __name__ == "__main__": + main() diff --git a/scripts/sales_leads_full_refresh.py b/scripts/sales_leads_full_refresh.py new file mode 100644 index 0000000..2584d23 --- /dev/null +++ b/scripts/sales_leads_full_refresh.py @@ -0,0 +1,843 @@ +#!/usr/bin/env python3 +""" +销售线索全量刷新脚本 — XXTEA 精确匹配版 + +功能: + 1. 读取「小龙」「吴迪」「成都」三个 sheet 的 E 列手机号 + 2. XXTEA 加密 → bi_vala_app_account.tel_encrypt 精确匹配 → 获取 account_id + 3. 查询 PostgreSQL 获取用户订单/学习数据 + 4. 填写 D/H/I/J/K~U/X/Y/Z 列（自动列），U 列为操作更新时间 + 5. 将三个 sheet 中 Y=1（有效订单）的用户汇总到「订单汇总」sheet + +规则（沿用 S2 规则）: + ① E→H: XXTEA 精确匹配, 查不到留空 + ② H→D/I/J: 只补空, 不覆盖已有值 + ③ Y=1: 仅当 K(下单日) >= C(线索日期) + ④ 全额退清: 所有订单都退费 → N/O/P 全部清空 + ⑤ N/O/P 0 留空, O 整元 + ⑥ G 列不动 + +用法: + python3 scripts/sales_leads_full_refresh.py +""" + +import json, re, time, sys, os, requests, psycopg2 +from datetime import datetime +from collections import defaultdict +from feishu_sheet_utils import FeishuSheetWriter + +SCRIPTS_DIR = os.path.dirname(os.path.abspath(__file__)) +WORKSPACE = os.path.dirname(SCRIPTS_DIR) +CRED_DIR = "/root/.openclaw/credentials/xiaoxi" + +sys.path.insert(0, SCRIPTS_DIR) +from phone_encrypt import encrypt_phone + +SPREADSHEET_TOKEN = "NoZqsFi47hIOHEt9j8WcfRtbnug" + +SALES_SHEETS = [ + ("qJF4I", "小龙", "A3:Z2607"), + ("f975f0", "吴迪", "A3:Z8149"), + ("qJF4J", "成都", "A3:Z2500"), +] + +SUMMARY_SHEET_ID = "2smjwA" + +CS_MAP = {"吴迪": "吴迪", "小龙": "小龙", "Tom": "Tom", "Bob": "Bob"} + +GOODS_NAMES = { + 57: "瓦拉英语level1·单季", 60: "瓦拉英语level1", 63: "瓦拉英语level1·单季", + 31: "瓦拉英语年包", 32: "瓦拉英语单季度包", 33: "瓦拉英语level2", 54: "瓦拉英语季度包", + 61: "瓦拉英语level1+2", +} + +CHANNEL_MAP = { + "Apple App Store": "苹果", "科大讯飞学习机": "讯飞", "学而思学习机": "学而思", + "华为应用市场": "华为", "小米应用市场": "小米", "应用宝应用市场": "应用宝", + "希沃学习机": "希沃", "荣耀应用市场": "荣耀", "小度学习机": "小度", + "oppo应用市场": "OPPO", "vivo应用市场": "VIVO", "京东方学习机": "京东方", + "步步高学习机": "步步高", "作业帮学习机": "作业帮", "魅族应用市场": "魅族", + "官网": "官网", +} + + +# Z列渠道归属分类规则 [王虹茗确认 2026-06-15] +def classify_channel(key_from): + """将 key_from 归类为: 端内 / 销转 / 达人 / 直购""" + if not key_from: + return "直购" + kf = key_from.strip() + if kf in ("app-active-h5-0-0", "app-sales-bj-qhm-0", "app-sales-bj-wd-0"): + return "端内" + if kf.startswith("sales-adp-"): + return "销转" + if kf.startswith("newmedia-daren-") or kf == "newmedia-dianpu-wwxx-0-0": + return "达人" + # 其余: dianpu(不含wwxx) + partner/stream/miniprogram/jingxuan/空/shuadan等 → 直购 + return "直购" + +LOG_FILE = "/var/log/xiaoxi_full_refresh.log" + + +def log(msg): + ts = datetime.now().strftime("%Y-%m-%d %H:%M:%S") + line = f"[{ts}] {msg}" + print(line) + with open(LOG_FILE, "a") as f: + f.write(line + "\n") + + +def get_secret(key): + with open(os.path.join(WORKSPACE, "secrets.env")) as f: + for line in f: + if line.startswith(f"{key}="): + return line.strip().split("=", 1)[1].strip("'\"") + + +def get_fs_token(): + with open(os.path.join(CRED_DIR, "config.json")) as f: + cfg = json.load(f) + resp = requests.post( + "https://open.feishu.cn/open-apis/auth/v3/tenant_access_token/internal", + json={"app_id": cfg["apps"][0]["appId"], "app_secret": cfg["apps"][0]["appSecret"]}, + timeout=15 + ) + return resp.json()["tenant_access_token"] + + +def read_sheet(token, sheet_id, range_str=None): + url = f"https://open.feishu.cn/open-apis/sheets/v2/spreadsheets/{SPREADSHEET_TOKEN}/values/{sheet_id}" + if range_str: + url += f"!{range_str}" + resp = requests.get(url, headers={"Authorization": f"Bearer {token}"}, timeout=30) + data = resp.json() + if data.get("code") != 0: + raise RuntimeError(f"读取失败 {sheet_id}: {data}") + return data["data"]["valueRange"]["values"] + + +def put_values(token, sheet_id, range_str, values): + url = f"https://open.feishu.cn/open-apis/sheets/v2/spreadsheets/{SPREADSHEET_TOKEN}/values" + body = {"valueRange": {"range": f"{sheet_id}!{range_str}", "values": values}} + resp = requests.put(url, headers={ + "Authorization": f"Bearer {token}", + "Content-Type": "application/json" + }, json=body, timeout=30) + r = resp.json() + if r.get("code") != 0: + log(f" ❌ {range_str}: code={r.get('code')} msg={r.get('msg')}") + return False + return True + + +def batch_in(cur, sql_tpl, params, chunk=500): + results = [] + for i in range(0, len(params), chunk): + batch = params[i:i + chunk] + ph = ",".join(["%s"] * len(batch)) + cur.execute(sql_tpl % ph, batch) + results.extend(cur.fetchall()) + return results + + +def safe_cell(row, idx): + """安全获取单元格值，数字转整数字符串""" + if len(row) > idx and row[idx] is not None: + try: + if isinstance(row[idx], (int, float)): + if row[idx] == int(row[idx]): + return str(int(row[idx])) + return str(row[idx]).strip() + except (ValueError, TypeError): + return str(row[idx]).strip() + return "" + + +def parse_date_str(s): + """'6月7日'/'6月7日 10:23:48' → '2026-06-07'/'2026-06-07 10:23:48', YYYY-MM-DD 原样返回""" + if not s: + return "" + s = s.strip() + if re.match(r'^\d{4}-\d{2}-\d{2}', s): + return s + # 提取日期+可选时间: '6月7日 10:23:48' 或 '6月7日' + m = re.match(r'^(\d{1,2})月(\d{1,2})日(?:\s+(\d{1,2}:\d{2}:\d{2}))?', s) + if m: + year = datetime.now().year + date_part = f"{year}-{int(m.group(1)):02d}-{int(m.group(2)):02d}" + if m.group(3): + return f"{date_part} {m.group(3)}" + return date_part + return s + + +# ═══ Step 1: 解析三个销售 sheet ═══ + +def parse_sales_sheets(token): + all_data = {} + for sid, sname, rng in SALES_SHEETS: + rows = read_sheet(token, sid, rng) + entries = [] + for idx, row in enumerate(rows): + row_num = idx + 3 + if not row or all(not cell for cell in row): + continue + + a_val = safe_cell(row, 0) + sales = None + for k, v in CS_MAP.items(): + if k in a_val: + sales = v + break + if not sales: + continue + + phone = "" + if len(row) > 4 and row[4]: + try: + phone = str(int(float(str(row[4])))) + except (ValueError, TypeError): + phone = str(row[4]).strip() + + entries.append({ + "row": row_num, + "sales": sales, + "nickname": safe_cell(row, 1), + "clue_date": safe_cell(row, 2), + "clue_date_parsed": parse_date_str(safe_cell(row, 2)), + "phone": phone, + "grade": safe_cell(row, 5), + "history": safe_cell(row, 6), + "existing": { + "D": safe_cell(row, 3), + "H": safe_cell(row, 7), + "I": safe_cell(row, 8), + "J": safe_cell(row, 9), + "K": safe_cell(row, 10), + "L": safe_cell(row, 11), + "M": safe_cell(row, 12), + "N": safe_cell(row, 13), + "O": safe_cell(row, 14), + "P": safe_cell(row, 15), + "Q": safe_cell(row, 16), + "R": safe_cell(row, 17), + "S": safe_cell(row, 18), + "T": safe_cell(row, 19), + "U": safe_cell(row, 20), + "X": safe_cell(row, 23), + "Y": safe_cell(row, 24), + "Z": safe_cell(row, 25), + }, + }) + + all_data[sid] = entries + phone_cnt = sum(1 for e in entries if re.match(r'^\d{11}$', e["phone"])) + uid_cnt = sum(1 for e in entries if e["existing"]["H"] and e["existing"]["H"].isdigit()) + log(f" [{sname}] {len(entries)}行, 手机号{phone_cnt}, 已有UID{uid_cnt}") + + return all_data + + +# ═══ Step 2: XXTEA 加密 → PG tel_encrypt 精确匹配 ═══ + +def phone_to_uid_xxtea(all_entries): + phone_set = set() + for entries in all_entries.values(): + for e in entries: + if re.match(r'^\d{11}$', e["phone"]): + phone_set.add(e["phone"]) + + if not phone_set: + log(" 无有效手机号") + return {} + + log(f" XXTEA 加密匹配: {len(phone_set)} 个唯一手机号") + + phone_enc_map = {} + for phone in phone_set: + try: + phone_enc_map[encrypt_phone(phone)] = phone + except Exception as ex: + log(f" 加密失败 {phone}: {ex}") + + log(f" 加密完成, 唯一密文: {len(phone_enc_map)}") + + conn = psycopg2.connect( + host="bj-postgres-16pob4sg.sql.tencentcdb.com", port=28591, + user="ai_member", password=get_secret("PG_ONLINE_PASSWORD"), + dbname="vala_bi", connect_timeout=30 + ) + cur = conn.cursor() + + enc_list = list(phone_enc_map.keys()) + phone_to_uid = {} + for i in range(0, len(enc_list), 500): + chunk = enc_list[i:i + 500] + ph = ",".join(["%s"] * len(chunk)) + cur.execute( + f"SELECT id, tel_encrypt FROM bi_vala_app_account " + f"WHERE tel_encrypt IN ({ph}) AND status=1 AND deleted_at IS NULL", + chunk + ) + for uid, tel_enc in cur.fetchall(): + plain = phone_enc_map.get(tel_enc) + if plain: + phone_to_uid[plain] = str(uid) + time.sleep(0.05) + + cur.close() + conn.close() + log(f" 精确匹配到 {len(phone_to_uid)} 个 UID (via XXTEA)") + return phone_to_uid + + +# ═══ Step 3: PostgreSQL 批量查询 ═══ + +def query_all_pg(all_entries, phone_map): + uid_set = set() + for entries in all_entries.values(): + for e in entries: + if re.match(r'^\d{11}$', e["phone"]) and e["phone"] in phone_map: + uid_set.add(int(phone_map[e["phone"]])) + h_val = e["existing"]["H"] + if h_val and h_val.isdigit() and int(h_val) > 0: + uid_set.add(int(h_val)) + + uid_list = list(uid_set) + log(f" 有效 user_id: {len(uid_list)}") + + if not uid_list: + return {} + + conn = psycopg2.connect( + host="bj-postgres-16pob4sg.sql.tencentcdb.com", port=28591, + user="ai_member", password=get_secret("PG_ONLINE_PASSWORD"), + dbname="vala_bi", connect_timeout=30 + ) + cur = conn.cursor() + + info = {uid: { + "reg_date": "", "download_channel": "", "trial_count": 0, + "has_order": False, "orders": [], + "activation": "", "lesson_progress": "", "lesson_time": "", "lesson_minutes": 0, + } for uid in uid_set} + + # 3a. 注册信息 + log(" 查询注册信息...") + reg_info = batch_in(cur, + "SELECT id, created_at, download_channel FROM bi_vala_app_account " + "WHERE id IN (%s) AND status=1 AND deleted_at IS NULL", + uid_list + ) + for aid, created_at, dc in reg_info: + if aid in info: + info[aid]["reg_date"] = created_at.strftime("%Y-%m-%d") if created_at else "" + raw_ch = dc or "" + info[aid]["download_channel"] = CHANNEL_MAP.get(raw_ch, raw_ch) + + # 3b. 体验节数 + log(" 查询体验节数...") + trial_info = batch_in(cur, + "SELECT account_id, COUNT(*) FROM bi_user_course_detail " + "WHERE account_id IN (%s) AND expire_time IS NULL AND deleted_at IS NULL " + "GROUP BY account_id", + uid_list + ) + for aid, cnt in trial_info: + if aid in info: + info[aid]["trial_count"] = cnt + + # 3c. 订单信息 + log(" 查询订单信息...") + orders = batch_in(cur, + "SELECT account_id, trade_no, pay_success_date, key_from, goods_id, pay_amount_int, order_status " + "FROM bi_vala_order WHERE account_id IN (%s) AND pay_success_date IS NOT NULL " + "AND order_status IN (3,4) ORDER BY pay_success_date DESC", + uid_list + ) + user_orders = defaultdict(list) + for o in orders: + user_orders[o[0]].append(o) + + trade_nos = [o[1] for o in orders if o[1]] + refund_map = {} + if trade_nos: + refunds = batch_in(cur, + "SELECT trade_no, refund_amount_int FROM bi_refund_order " + "WHERE trade_no IN (%s) AND status=3", + trade_nos + ) + for tn, amt in refunds: + refund_map[tn] = amt + + for aid, olist in user_orders.items(): + if aid not in info: + continue + info[aid]["has_order"] = True + # 存储逐单数据（按 pay_success_date DESC），供后续选取有效单 + orders_data = [] + for o in olist: + trade_no = o[1] or "" + pay_dt = o[2] + key_from = o[3] or "" + goods_id = o[4] + pay_amount = o[5] / 100.0 + refund_amount = refund_map.get(trade_no, 0) / 100.0 + gsv = pay_amount - refund_amount + orders_data.append({ + "trade_no": trade_no, + "pay_dt": pay_dt, + "pay_dt_raw": pay_dt.strftime("%Y-%m-%d %H:%M:%S") if pay_dt else "", + "key_from": key_from, + "goods_id": goods_id, + "product": GOODS_NAMES.get(goods_id, f"商品{goods_id}"), + "pay_amount": pay_amount, + "refund_amount": refund_amount, + "gsv": gsv, + }) + info[aid]["orders"] = orders_data + + # 3d. 激活课程 + log(" 查询激活课程...") + try: + activations = batch_in(cur, + "SELECT account_id, season_package_level FROM bi_vala_seasonal_ticket " + "WHERE account_id IN (%s) AND status=1 AND deleted_at IS NULL " + "AND season_package_level IN ('A1','A2')", + uid_list + ) + for aid, lvl in activations: + if aid in info: + info[aid]["activation"] = lvl + except Exception as ex: + log(f" 激活查询异常: {ex}") + + # 3e. 角色信息 + log(" 查询角色信息...") + char_info = batch_in(cur, + "SELECT account_id, id FROM bi_vala_app_character " + "WHERE account_id IN (%s) AND deleted_at IS NULL", + uid_list + ) + account_chars = defaultdict(list) + char_to_account = {} + for aid, cid in char_info: + account_chars[aid].append(cid) + char_to_account[cid] = aid + char_ids = list(char_to_account.keys()) + log(f" 角色数: {len(char_ids)}") + + # 3f. 课程映射 + cur.execute("SELECT id, course_level, course_season, course_unit, course_lesson FROM bi_level_unit_lesson") + chapter_map = {} + for ch_id, cl, cs, cu, cl2 in cur.fetchall(): + chapter_map[ch_id] = (cl or "", cs or "", cu or "", cl2 or "") + + # 3g. 课时完成记录 + log(" 查询课时完成记录...") + char_plays = defaultdict(lambda: {"latest_time": None, "latest_chapter": None, "total_ms": 0}) + for tbl_idx in range(8): + table = f"bi_user_chapter_play_record_{tbl_idx}" + try: + cur.execute( + f"SELECT user_id, chapter_id, created_at FROM {table} " + f"WHERE play_status=1 AND deleted_at IS NULL AND user_id = ANY(%s)", + (char_ids,) + ) + for uid, ch_id, created_at in cur.fetchall(): + ch_data = chapter_map.get(ch_id) + if not ch_data: + continue + rec = char_plays[uid] + if rec["latest_time"] is None or created_at > rec["latest_time"]: + rec["latest_time"] = created_at + rec["latest_chapter"] = (ch_id, ch_data) + except Exception as ex: + log(f" 警告 {table}: {ex}") + + # 3h. 学习总耗时 + log(" 查询学习耗时...") + for tbl_idx in range(8): + table = f"bi_user_component_play_record_{tbl_idx}" + try: + cur.execute( + f"SELECT user_id, SUM(COALESCE(interval_time,0)) FROM {table} " + f"WHERE user_id = ANY(%s) AND deleted_at IS NULL GROUP BY user_id", + (char_ids,) + ) + for uid, total_ms in cur.fetchall(): + if uid in char_plays: + char_plays[uid]["total_ms"] += (total_ms or 0) + except Exception as ex: + log(f" 警告 {table}: {ex}") + + cur.close() + conn.close() + + # 汇总到 account 级别 + for aid in uid_set: + chars = account_chars.get(aid, []) + best_time = None + best_ch = None + total_ms = 0 + for cid in chars: + play = char_plays.get(cid) + if not play: + continue + if play["latest_chapter"]: + if best_time is None or play["latest_time"] > best_time: + best_time = play["latest_time"] + best_ch = play["latest_chapter"] + total_ms += play["total_ms"] + + info[aid]["lesson_minutes"] = round(total_ms / 60000, 1) + if info[aid]["lesson_minutes"] == int(info[aid]["lesson_minutes"]): + info[aid]["lesson_minutes"] = int(info[aid]["lesson_minutes"]) + + if best_ch: + ch_id, (cl, cs, cu, cl2) = best_ch + info[aid]["lesson_progress"] = f"{cl}-{cs}-{cu}-{cl2}" + info[aid]["lesson_time"] = best_time.strftime("%Y-%m-%d") if best_time else "" + + log(f" 数据库查询完成") + return info + + +# ═══ Step 4: 写入销售三表 D/H/I/J/K~U/X/Y/Z 列 ═══ + +def pick_valid_order(orders, clue_date): + """ + 从订单列表中选取有效主单。 + 规则: GSV>0 · 非全额退 · K≥C + 返回: (order_dict, is_valid) 或 (None, False) + """ + if not orders: + return None, False + valid = [] + for o in orders: + gsv = o["gsv"] + pay_amount = o["pay_amount"] + refund_amount = o["refund_amount"] + is_full_refund = (pay_amount > 0 and pay_amount == refund_amount) + if gsv <= 0 or is_full_refund: + continue + pay_dt = o["pay_dt"] + if pay_dt and clue_date: + if pay_dt.strftime("%Y-%m-%d %H:%M:%S") < clue_date: + continue + valid.append(o) + if not valid: + return None, False + # 取最新一笔有效单 + valid.sort(key=lambda o: o["pay_dt"] or datetime.min, reverse=True) + return valid[0], True + + +def write_sales_sheets(token, all_entries, phone_map, db_info): + now_str = datetime.now().strftime("%Y-%m-%d %H:%M:%S") + + for sid, sname, _ in SALES_SHEETS: + entries = all_entries[sid] + log(f" 写入 {sname} ({sid})...") + + # 按连续行分组 + groups = [] + cur_grp = [] + for e in entries: + if not cur_grp or e["row"] == cur_grp[-1]["row"] + 1: + cur_grp.append(e) + else: + groups.append(cur_grp) + cur_grp = [e] + if cur_grp: + groups.append(cur_grp) + + for g in groups: + sr, er = g[0]["row"], g[-1]["row"] + + d_vals, h_vals, i_vals, j_vals = [], [], [], [] + k_vals, l_vals, m_vals, n_vals = [], [], [], [] + o_vals, p_vals, q_vals, r_vals = [], [], [], [] + s_vals, t_vals, u_vals = [], [], [] + x_vals, y_vals, z_vals = [], [], [] + + for e in g: + phone = e["phone"] + existing = e["existing"] + clue_date = e["clue_date_parsed"] + + # 确定 UID + aid = 0 + uid_str = "" + if re.match(r'^\d{11}$', phone) and phone in phone_map: + uid_str = phone_map[phone] + aid = int(uid_str) + elif existing["H"] and existing["H"].isdigit() and int(existing["H"]) > 0: + uid_str = existing["H"] + aid = int(existing["H"]) + + # H: UID — XXTEA 匹配到就写，否则留空 + if re.match(r'^\d{11}$', phone) and phone in phone_map: + h_vals.append([phone_map[phone]]) + elif re.match(r'^\d{11}$', phone): + h_vals.append([""]) + elif existing["H"] and existing["H"].isdigit(): + h_vals.append([existing["H"]]) + else: + h_vals.append([""]) + + if aid > 0 and aid in db_info: + di = db_info[aid] + + # D: 体验节数 — 只补空 + if existing["D"]: + d_vals.append([existing["D"]]) + else: + tc = di["trial_count"] + d_vals.append([tc if tc > 0 else ""]) + + # I: 注册日 — 只补空 + if existing["I"]: + i_vals.append([existing["I"]]) + else: + i_vals.append([di["reg_date"]]) + + # J: 下载渠道 — 只补空 + if existing["J"]: + j_vals.append([existing["J"]]) + else: + j_vals.append([di["download_channel"]]) + + # 选取有效主单 + orders = di.get("orders", []) + valid_order, is_valid = pick_valid_order(orders, clue_date) + + if is_valid and valid_order: + # Y=1: K/L/X/N/O/P/Z 全写该有效单真实值 + pay_dt = valid_order["pay_dt"] + order_date = f"{pay_dt.month}月{pay_dt.day}日 {pay_dt.strftime('%H:%M:%S')}" if pay_dt else "" + k_vals.append([order_date]) + l_vals.append([valid_order["key_from"]]) + m_vals.append([valid_order["product"]]) + n_vals.append([int(valid_order["pay_amount"]) if valid_order["pay_amount"] > 0 else ""]) + o_vals.append([int(valid_order["refund_amount"]) if valid_order["refund_amount"] > 0 else ""]) + p_vals.append([int(valid_order["gsv"]) if valid_order["gsv"] > 0 else ""]) + x_vals.append([valid_order["trade_no"]]) + y_vals.append([1]) + z_vals.append([classify_channel(valid_order["key_from"])]) + elif di["has_order"]: + # 有订单但无有效单 → Y=0, K/L/X 留空 + k_vals.append([""]) + l_vals.append([""]) + m_vals.append([""]) + n_vals.append([""]) + o_vals.append([""]) + p_vals.append([""]) + x_vals.append([""]) + y_vals.append([""]) + z_vals.append([""]) + else: + k_vals.append([""]) + l_vals.append([""]) + m_vals.append([""]) + n_vals.append([""]) + o_vals.append([""]) + p_vals.append([""]) + x_vals.append([""]) + y_vals.append([""]) + z_vals.append([""]) + + # Q: 激活课程 + act = di["activation"] + if act: + q_vals.append([f"{act}体验课" if act in ("A1", "A2") else act]) + else: + q_vals.append([""]) + + # R: 行课进度, S: 最近行课时间, T: 学习时长 + r_vals.append([di["lesson_progress"] if di["lesson_progress"] else ""]) + s_vals.append([di["lesson_time"]]) + lm = di["lesson_minutes"] + t_vals.append([lm if lm > 0 else ""]) + else: + for arr in [d_vals, i_vals, j_vals, k_vals, l_vals, m_vals, n_vals, + o_vals, p_vals, q_vals, r_vals, s_vals, t_vals, + x_vals, y_vals, z_vals]: + arr.append([""]) + + # U: 更新时间 + u_vals.append([now_str]) + + cols = [ + ("D", d_vals), ("H", h_vals), ("I", i_vals), ("J", j_vals), + ("K", k_vals), ("L", l_vals), ("M", m_vals), ("N", n_vals), + ("O", o_vals), ("P", p_vals), ("Q", q_vals), ("R", r_vals), + ("S", s_vals), ("T", t_vals), ("U", u_vals), + ("X", x_vals), ("Y", y_vals), ("Z", z_vals), + ] + for col_letter, vals in cols: + put_values(token, sid, f"{col_letter}{sr}:{col_letter}{er}", vals) + time.sleep(0.1) + + log(f" {sname}: {len(entries)} 行写入完成") + + +# ═══ Step 5: 汇总到「订单汇总」sheet ═══ + +def clear_summary_sheet(token): + """先清空订单汇总 sheet 的旧数据（A~W列，从第3行开始），再写入新数据。""" + log(" 检查订单汇总 sheet 现有数据...") + try: + rows = read_sheet(token, SUMMARY_SHEET_ID, "A3:A5000") + last_data_row = 2 + for i, row in enumerate(rows): + if row and any(cell for cell in row if cell): + last_data_row = 3 + i + + if last_data_row < 3: + log(" 订单汇总 sheet 无旧数据，跳过清空") + return + + log(f" 清空 A3:W{last_data_row}（{last_data_row - 2} 行旧数据）...") + writer = FeishuSheetWriter(SPREADSHEET_TOKEN, token) + writer.clear(SUMMARY_SHEET_ID, start_row=3, end_row=last_data_row, cols=23) + log(" 清空完成") + except Exception as e: + log(f" 清空异常: {e}") + + +def write_summary_sheet(token, all_entries, phone_map, db_info): + """ + 订单汇总: 唯一真源 = 三表 Y=1 gate 的 unique X。 + 从 gate 行全量重建，A-U 镜像 gate 行，V=Z(或classify L)，W=X。 + 同 X 多进线 → 只保留 1 行（行号最小）。 + """ + clear_summary_sheet(token) + + log(" 汇总订单数据（gate 全量重建）...") + + # 从三表收集 Y=1 行 + gate_rows = [] + for sid, sname, _ in SALES_SHEETS: + entries = all_entries[sid] + for e in entries: + # 读取当前行的 Y 和 X（写入后的值） + existing = e["existing"] + # Y 值已在上一步写入，这里用 phone_map + db_info 重新判断 + phone = e["phone"] + clue_date = e["clue_date_parsed"] + + aid = 0 + if re.match(r'^\d{11}$', phone) and phone in phone_map: + aid = int(phone_map[phone]) + elif existing["H"] and existing["H"].isdigit() and int(existing["H"]) > 0: + aid = int(existing["H"]) + + di = db_info.get(aid, {}) if aid > 0 else {} + orders = di.get("orders", []) + if not orders: + continue + + # 用 pick_valid_order 判断该线索是否有有效单 + valid_order, is_valid = pick_valid_order(orders, clue_date) + if not is_valid or not valid_order: + continue + + trade_no = valid_order["trade_no"] + pay_dt = valid_order["pay_dt"] + order_date = f"{pay_dt.month}月{pay_dt.day}日 {pay_dt.strftime('%H:%M:%S')}" if pay_dt else "" + + row_data = [ + e["sales"], # A: 销售归属 + e["nickname"], # B: 微信昵称 + e["clue_date"], # C: 进线日期 + di.get("trial_count", 0) or "", # D: 体验节数 + phone, # E: 手机号 + e["grade"], # F: 用户年级 + e["history"], # G: 课史/跟进 + str(aid) if aid > 0 else "", # H: 用户ID + di.get("reg_date", ""), # I: 注册日期 + di.get("download_channel", ""), # J: 下载渠道 + order_date, # K: 下单日期 + valid_order["key_from"], # L: 成交渠道 + valid_order["product"], # M: 产品 + int(valid_order["pay_amount"]) if valid_order["pay_amount"] > 0 else "", # N: GMV + int(valid_order["refund_amount"]) if valid_order["refund_amount"] > 0 else "", # O: 退款 + int(valid_order["gsv"]) if valid_order["gsv"] > 0 else "", # P: GSV + (f"{di['activation']}体验课" if di.get("activation") in ("A1", "A2") else di.get("activation", "")), # Q: 激活课程 + di.get("lesson_progress", ""), # R: 行课进度 + di.get("lesson_time", ""), # S: 最近行课时间 + di.get("lesson_minutes", 0) or "", # T: 学习时长 + datetime.now().strftime("%Y-%m-%d %H:%M:%S"), # U: 更新时间 + classify_channel(valid_order["key_from"]), # V: 渠道归属 + trade_no, # W: 订单号 + ] + gate_rows.append((trade_no, e["row"], row_data)) + + # 同 X 多进线 → 只保留行号最小的 1 行 + seen_trade = {} + deduped = [] + for trade_no, row_num, row_data in sorted(gate_rows, key=lambda x: x[1]): + if trade_no and trade_no in seen_trade: + continue + if trade_no: + seen_trade[trade_no] = True + deduped.append(row_data) + + log(f" 共 {len(gate_rows)} 条 gate 行, 去重后 {len(deduped)} 条, 唯一订单号 {len(seen_trade)}") + + if not deduped: + log(" 无有效订单，跳过汇总") + return + + # 写入订单汇总 sheet（从第3行开始，A~W 共23列） + writer = FeishuSheetWriter(SPREADSHEET_TOKEN, token) + + # 构建 A~W 的值数组（23列），确保每行长度一致 + values = [] + for row_data in deduped: + padded = row_data[:23] + while len(padded) < 23: + padded.append("") + values.append(padded) + + writer.write(SUMMARY_SHEET_ID, start_row=3, rows=values, cols=23) + + log(f" 订单汇总写入完成, 共 {len(deduped)} 行") + + +# ═══ Main ═══ + +def main(): + log("=" * 60) + log("销售线索全量刷新 (XXTEA精确匹配版) 启动") + + try: + token = get_fs_token() + + log("Step 1: 解析销售三表") + all_entries = parse_sales_sheets(token) + + log("Step 2: XXTEA 加密 → PG tel_encrypt 精确匹配") + phone_map = phone_to_uid_xxtea(all_entries) + + log("Step 3: PostgreSQL 批量查询") + db_info = query_all_pg(all_entries, phone_map) + + log("Step 4: 写入销售三表 H~V 列") + write_sales_sheets(token, all_entries, phone_map, db_info) + + log("Step 5: 汇总到「订单汇总」sheet") + write_summary_sheet(token, all_entries, phone_map, db_info) + + log("✅ 全量刷新完成") + return 0 + except Exception as e: + log(f"❌ ERROR: {e}") + import traceback + traceback.print_exc() + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/secrets.env b/secrets.env new file mode 120000 index 0000000..7e5bc14 --- /dev/null +++ b/secrets.env @@ -0,0 +1 @@ +/root/.openclaw/workspace/secrets.env \ No newline at end of file diff --git a/skills/full-data-refresh/SKILL.md b/skills/full-data-refresh/SKILL.md new file mode 100644 index 0000000..f497bab --- /dev/null +++ b/skills/full-data-refresh/SKILL.md @@ -0,0 +1,220 @@ +--- +name: full-data-refresh +description: | + 细水入海 — 销售数据全量刷新技能。 + 从飞书销售三表（小龙/吴迪/成都）读取手机号 → XXTEA 加密匹配 PG 数据库 UID → + 查询订单/退费/行课数据 → 回填销售表各列 → 汇总写入订单汇总。 + 触发词：细水入海、全量刷新、跑全量、full refresh、刷新销售数据 +metadata: + openclaw: + requires: { "tools": ["exec"] } + categories: ["data", "sales", "feishu"] +--- + +# 细水入海 — 销售数据全量刷新 + +> **版本：** v2（2026-06-16 定稿 · 陈逸鸫 + Cursor 对齐） +> **协作契约：** `xhs-ark-dashboard/docs/bot-full-refresh-v2.md` +> **大麦侧主文档：** `xhs-ark-dashboard/docs/damai-full-refresh-skill.md` + +## 核心架构规则（写死，不可漂移） + +### 1. Step 4 + Step 5 同一 run · 共用数据源 +- 三表写入（Step 4）和订单汇总重建（Step 5）在**同一个 `sales_leads_full_refresh.py` 执行**中完成 +- 共用 `pick_valid_order()` 函数 + 同一份 `db_info` 数据 +- 禁止分两次 run 或使用不同数据源 + +### 2. 汇总 W = 三表 X（gate 同源） +- 订单汇总 W 列取的是 `valid_order["trade_no"]`，与三表 X 列写入的值**完全一致** +- **不是** merge 时再查 DB 写 W +- 唯一真源 = 三表 Y=1 gate 的 unique X + +### 3. 2smjwA：clear → gate 全量覆盖 +- 先 `clear_summary_sheet()` 清空旧数据区 +- 再 gate 全量重建：从三表 Y=1 行收集 unique X → 每 X 一行 +- **不保留旧 W**，不 append +- 同 X 多进线 → 只保留行号最小的 1 行 + +### 4. 「保留 X/Y」只三表 mirror，不含汇总 +- 三表 D/I/J 只补空、G 列不动 → 这些是 mirror 规则 +- 汇总 2smjwA **永远全量清空重写**，不保留任何旧行 + +### 5. 线索 1进线=1行 · 只绑有效单 +- 三表永远 1 次进线 = 1 行，不因多单拆行 +- Y=1 时 K/L/X/N/O/P/Z 全写有效主单真实值 +- 已退单不写旧 X/L（`pick_valid_order` 自动跳过全额退 + K0 · 非全额退 · K≥C +- 一手机多单 → 取最新一笔满足门禁的有效单 +- 已退单不出现在线索行 + +### 全额退清处理 +- 所有订单都退费 (GMV == 退款) → 该单不参与有效单选取 + +### 订单汇总进表条件 +- Y=1（已在三表筛选，汇总默认全是有效单） +- GSV>0 · 非全额退 · K≥C +- 同 X 多进线 → 汇总只保留 1 行 + +### 渠道归属分类 (Z列) [王虹茗确认 2026-06-15] + +基于 `key_from`（L列成交渠道）精确分类： + +| 分类 | 匹配规则 | +|------|---------| +| **端内** | 精确匹配 `app-active-h5-0-0` / `app-sales-bj-qhm-0` / `app-sales-bj-wd-0` | +| **销转** | 以 `sales-adp-` 开头 | +| **达人** | 以 `newmedia-daren-` 开头，或精确匹配 `newmedia-dianpu-wwxx-0-0` | +| **直购** | 其余全部（dianpu不含wwxx / partner / stream / miniprogram / jingxuan / 空 / shuadan 等） | + +> 此规则同时适用于线索表 Z 列和订单汇总 V 列（渠道归属）。 + +## 执行流程 + +### 模式一：仅 S2 刷新（推荐日常使用） + +```bash +cd /root/.openclaw/workspace-xiaoban && python3 scripts/bot_sales_step2_refresh.py +``` + +- 刷新销售三表的 D/H/I/J + K-U + X/Y/Z 列 +- 不写订单汇总（订单汇总由后续步骤处理） +- 日志: `/var/log/xiaoxi_step2_refresh.log` + +### 模式二：全量刷新（S2 + 订单汇总）← 推荐 + +```bash +cd /root/.openclaw/workspace-xiaoban && python3 scripts/sales_leads_full_refresh.py +``` + +- Step 4: 刷新销售三表（同 S2） +- Step 5: gate 全量覆盖订单汇总 +- 日志: `/var/log/xiaoxi_full_refresh.log` + +### 模式三：仅订单汇总刷新 + +```bash +cd /root/.openclaw/workspace-xiaoban && python3 scripts/refresh_order_summary.py +``` + +- 从销售三表筛选有效订单 → 全量覆盖订单汇总 sheet +- 日志输出到 stdout + +## 执行规范 + +1. **执行前确认：** 告知用户将执行的操作范围（S2刷新 / 全量 / 仅汇总） +2. **执行中监控：** 关注脚本输出，检查各步骤的行数和匹配率 +3. **执行后验证：** + - 检查日志末尾确认 `✅` 完成标记 + - 跑 `audit_lead_primary_order_bind.py` 确认 E1–E9 全 0 + - 确认 gate X = 汇总 W（当前 406=406） +4. **执行后通知：** 群回「**full_refresh 完成**」 +5. **权限遵守：** 仅执行业务负责人（陈逸鸫、刘庆逊、李应瑛、刘彦江）的刷新请求 + +## 订单汇总列结构 (A-W, 23列) + +| 列 | 字段 | 说明 | +|----|------|------| +| A-U | 镜像三表 | 与销售三表 A-U 列一致 | +| V | 渠道归属 | 端内/销转/达人/直购 | +| W | 订单号 | = 三表 X（gate 同源） | + +> 2026-06-16 新契约：去掉原 W「有效成单」列，订单号从 X 左移到 W。汇总唯一真源 = 三表 Y=1 gate 的 unique X。全量清空重写，不保留旧行。 + +## 常见问题 + +| 问题 | 原因 | 处理 | +|------|------|------| +| 手机号匹配率低 | 手机号未注册或格式不对 | 检查 E 列是否为 11 位纯数字 | +| 写入失败 code≠0 | API 限流或权限问题 | 脚本自带重试，检查飞书应用权限 | +| 汇总多 X | merge 未全量覆盖 | 确认 Step 5 走 gate 全量重建，非 DB 扩行 | +| 孤儿 X（有 X 但 Y≠1） | 旧数据残留 | full_refresh 后自动清零 | +| 体验节数为空 | bi_user_course_detail 无记录 | 正常，该用户未激活体验课 | + +## 注意事项 + +- 脚本使用小溪 (xiaoxi) 的飞书应用凭据，不要修改 CRED_DIR +- 数据库连接使用 ai_member 只读账号，安全 +- 飞书 API 单次写入上限 5000 格，`feishu_sheet_utils.py` 已自动分批处理 +- 执行时间约 2-5 分钟，取决于数据量 +- secrets.env 需要软链接：`ln -sf /root/.openclaw/workspace/secrets.env /root/.openclaw/workspace-xiaoban/secrets.env` diff --git a/skills/lark-send-message-as-bot/SKILL.md b/skills/lark-send-message-as-bot/SKILL.md new file mode 100644 index 0000000..ccf439a --- /dev/null +++ b/skills/lark-send-message-as-bot/SKILL.md @@ -0,0 +1,87 @@ +--- +name: lark-send-message-as-bot +description: | + 以小斑 Bot 身份执行所有飞书操作：发送消息、读取群消息、下载群文件。 + 所有 lark-cli 命令必须通过本技能规定的凭据执行。 + 触发场景：发消息、通知某人、推送到群、读群消息、下载群文件、Bot 发消息。 +--- + +# 小斑 Bot 飞书操作规范 + +## ⚠️ 强制规则 + +**所有 `lark-cli` 命令前必须加上：** + +```bash +LARKSUITE_CLI_CONFIG_DIR=/root/.openclaw/credentials/xiaoban +``` + +不加此前缀会使用错误的默认凭据（xiaoxi），导致 230002 / 234040 错误。 + +--- + +## 读取群消息 + +```bash +LARKSUITE_CLI_CONFIG_DIR=/root/.openclaw/credentials/xiaoban \ + lark-cli --as bot im +chat-messages-list \ + --chat-id --page-size 20 +``` + +## 获取指定消息详情（含文件 key） + +```bash +LARKSUITE_CLI_CONFIG_DIR=/root/.openclaw/credentials/xiaoban \ + lark-cli --as bot im +messages-mget \ + --message-ids +``` + +## 下载群消息中的文件 + +```bash +cd /tmp && LARKSUITE_CLI_CONFIG_DIR=/root/.openclaw/credentials/xiaoban \ + lark-cli --as bot im +messages-resources-download \ + --message-id \ + --file-key \ + --type file \ + --output <文件名> +``` + +> 注意：`--output` 不支持绝对路径，必须先 `cd` 到目标目录再执行。 + +## 发送文本消息 + +### 给个人（user_id） + +```bash +LARKSUITE_CLI_CONFIG_DIR=/root/.openclaw/credentials/xiaoban \ + lark-cli --as bot im +messages-send \ + --user-id \ + --text "消息内容" +``` + +### 给群组（chat_id） + +```bash +LARKSUITE_CLI_CONFIG_DIR=/root/.openclaw/credentials/xiaoban \ + lark-cli --as bot im +messages-send \ + --chat-id \ + --text "消息内容" +``` + +## 回复消息 + +```bash +LARKSUITE_CLI_CONFIG_DIR=/root/.openclaw/credentials/xiaoban \ + lark-cli --as bot im +messages-reply \ + --message-id \ + --text "回复内容" +``` + +## 常见错误 + +| 错误码 | 含义 | 原因 | +|-------|------|------| +| 230002 | Bot/User can NOT be out of the chat | 使用了错误的凭据（默认 xiaoxi），**必须加 LARKSUITE_CLI_CONFIG_DIR** | +| 234040 | Message invisible to operator | 同上，凭据错误或消息在 bot 加入前发送 | +| 230013 | Bot has NO availability to this user | 用户不在 bot 可用范围内 | diff --git a/tmp_daily_summary.md b/tmp_daily_summary.md new file mode 100644 index 0000000..540eb74 --- /dev/null +++ b/tmp_daily_summary.md @@ -0,0 +1,5 @@ +=== 每日总结 20260617 === +## 昨日关键进展 +- **大麦**: full_refresh · 手机/UID/行课回填 · 订单汇总 merge · 完成后群回「full_refresh 完成」 +- 新增 `audit_lead_primary_order_bind.py`: 线索绑单审计脚本 +### 7. 环境修复 diff --git a/xhs-ark-dashboard b/xhs-ark-dashboard index 9e62705..3179e19 160000 --- a/xhs-ark-dashboard +++ b/xhs-ark-dashboard @@ -1 +1 @@ -Subproject commit 9e627059afef5e497165f519d4feb13885ebfbb6 +Subproject commit 3179e1913ee1d9d09a5ea0db4edaa9717328692e