chore: 规范输出文件路径 - AGENTS.md新增定时任务输出规则,移动散落报表到output/,修正脚本输出路径
This commit is contained in:
parent
be8867e3ab
commit
474d758394
@ -11,3 +11,4 @@ vala-component-oops-stat 2e3a3dc8ee252f133256ec5fb8edc031fc25ff602158fd2ddf778ce
|
||||
vala-component-practice-stat 8e768e2641019d27bd41f4647d2d90f24182a0554dad5ad9f4136e9ce0bae147
|
||||
cron-schedule e103cbb1806b28c891b9c856963325086ecaff32edec208f0a841865f26e8f3e
|
||||
refund-user-learning-analysis 648fd4ae2b29167fd66eab4245bdaaef00242db3131f4919cc02f07ca2a9b59c
|
||||
phone-chapter-query ac429b4da5a89db16efdf1066edf4ecb1c050b93aff20dd4c652af5f5568e44f
|
||||
|
||||
@ -157,9 +157,10 @@ System: [...] Feishu[xiaobian] group oc_xxx | 发送者 (ou_xxx) [msg:om_xxx]
|
||||
|
||||
1. **脚本文件** → 始终创建在 `scripts/` 目录下,绝不放在根目录
|
||||
2. **输出文件**(xlsx/csv/log/报表等)→ 始终创建在 `output/` 目录下,绝不放在根目录
|
||||
3. **业务知识** → 统一记录到 `business_knowledge/` 目录
|
||||
4. **新增子目录** → 禁止在根目录下随意创建新子目录。如有特殊需要,须经技术负责人确认
|
||||
5. **临时文件** → 使用 `tmp/`,用完即清
|
||||
3. **定时任务/脚本生成的文件** → 默认也输出到 `output/` 目录下,可根据需求在 `output/` 内创建子目录分类存放(如 `output/daily_reports/`),**绝不放在根目录**
|
||||
4. **业务知识** → 统一记录到 `business_knowledge/` 目录
|
||||
5. **新增子目录** → 禁止在根目录下随意创建新子目录。如有特殊需要,须经技术负责人确认
|
||||
6. **临时文件** → 使用 `tmp/`,用完即清
|
||||
|
||||
## 工具
|
||||
|
||||
|
||||
41
memory/2026-04-20-gmv-query.md
Normal file
41
memory/2026-04-20-gmv-query.md
Normal file
@ -0,0 +1,41 @@
|
||||
# Session: 2026-04-20 04:17:26 UTC
|
||||
|
||||
- **Session Key**: agent:main:feishu:direct:ou_9cb5bc9a5f1b6cab2d78fd36139ecb87
|
||||
- **Session ID**: b0bb1c0e-ec5a-48b3-92c1-c0eb4104f612
|
||||
- **Source**: feishu
|
||||
|
||||
## Conversation Summary
|
||||
|
||||
user: System: [2026-04-15 19:58:16 GMT+8] Feishu[default] DM | 李若松 (ou_9cb5bc9a5f1b6cab2d78fd36139ecb87) [msg:om_x100b52c39bb8807cc2efa6a35fb57e4]
|
||||
System: [2026-04-15 19:58:16 GMT+8] Model switched to doubao (volcengine/doubao-seed-2-0-pro-260215).
|
||||
System: [2026-04-20 10:06:37 GMT+8] Feishu[default] DM | 李若松 (ou_9cb5bc9a5f1b6cab2d78fd36139ecb87) [msg:om_x100b516cec07b4acb2bf0be36392549]
|
||||
|
||||
Conversation info (untrusted metadata):
|
||||
```json
|
||||
{
|
||||
"message_id": "om_x100b516cec07b4acb2bf0be36392549",
|
||||
"sender_id": "ou_9cb5bc9a5f1b6cab2d78fd36139ecb87",
|
||||
"sender": "李若松",
|
||||
"timestamp": "Mon 2026-04-20 10:06 GMT+8"
|
||||
}
|
||||
```
|
||||
|
||||
Sender (untrusted metadata):
|
||||
```json
|
||||
{
|
||||
"label": "李若松 (ou_9cb5bc9a5f1b6cab2d78fd36139ecb87)",
|
||||
"id": "ou_9cb5bc9a5f1b6cab2d78fd36139ecb87",
|
||||
"name": "李若松"
|
||||
}
|
||||
```
|
||||
|
||||
查看一下本月GMV
|
||||
assistant: 截至2026年4月20日,本月(4月)核心业务指标如下:
|
||||
| 指标 | 数值 |
|
||||
|------|------|
|
||||
| 总订单数 | 1338 单 |
|
||||
| GMV | 3,997,852.00 元(约399.8万元) |
|
||||
| GSV(剔除退费后实际收入) | 2,300,478.00 元(约230.0万元) |
|
||||
| 退费率 | 41.1% |
|
||||
|
||||
✅ 数据已自动剔除测试账号,符合业务统计口径。
|
||||
165
memory/2026-04-20-phone-query.md
Normal file
165
memory/2026-04-20-phone-query.md
Normal file
@ -0,0 +1,165 @@
|
||||
# Session: 2026-04-20 03:21:29 UTC
|
||||
|
||||
- **Session Key**: agent:main:feishu:direct:ou_e63ce6b760ad39382852472f28fbe2a2
|
||||
- **Session ID**: 314d25e7-4721-435c-b121-bae687a6520c
|
||||
- **Source**: feishu
|
||||
|
||||
## Conversation Summary
|
||||
|
||||
assistant: ✅ New session started · model: cloudsway/MaaS_Cl_Opus_4.6_20260205_cache (default: volcengine/doubao-seed-2-0-pro-260215)
|
||||
user: System: [2026-04-20 10:21:09 GMT+8] Feishu[default] DM | 李承龙 (ou_e63ce6b760ad39382852472f28fbe2a2) [msg:om_x100b516c978918a0b4b6d555d6d9512]
|
||||
|
||||
A new session was started via /new or /reset. Run your Session Startup sequence - read the required files before responding to the user. Then greet the user in your configured persona, if one is provided. Be yourself - use your defined voice, mannerisms, and mood. Keep it to 1-3 sentences and ask what they want to do. If the runtime model differs from default_model in the system prompt, mention the default model. Do not mention internal steps, files, tools, or reasoning.
|
||||
Current time: Monday, April 20th, 2026 — 10:21 AM (Asia/Shanghai) / 2026-04-20 02:21 UTC
|
||||
assistant: 承龙早上好 ☀️ 刚上线,今天周一,有什么需要我帮忙的吗?
|
||||
user: System: [2026-04-20 10:41:00 GMT+8] Feishu[default] DM | 李承龙 (ou_e63ce6b760ad39382852472f28fbe2a2) [msg:om_x100b516d6d74bca8b2cd4a5ddd9a1cb]
|
||||
|
||||
Conversation info (untrusted metadata):
|
||||
```json
|
||||
{
|
||||
"message_id": "om_x100b516d6d74bca8b2cd4a5ddd9a1cb",
|
||||
"sender_id": "ou_e63ce6b760ad39382852472f28fbe2a2",
|
||||
"sender": "李承龙",
|
||||
"timestamp": "Mon 2026-04-20 10:40 GMT+8"
|
||||
}
|
||||
```
|
||||
|
||||
Sender (untrusted metadata):
|
||||
```json
|
||||
{
|
||||
"label": "李承龙 (ou_e63ce6b760ad39382852472f28fbe2a2)",
|
||||
"id": "ou_e63ce6b760ad39382852472f28fbe2a2",
|
||||
"name": "李承龙"
|
||||
}
|
||||
```
|
||||
|
||||
你能查询到用户的手机号明文么
|
||||
assistant: [[reply_to_current]]承龙,可以的。`bi_vala_app_account` 表中有用户手机号字段,我能查询到明文手机号。
|
||||
|
||||
你需要查哪个用户的?提供 account_id、用户名或其他标识信息就行。
|
||||
user: System: [2026-04-20 10:44:48 GMT+8] Feishu[xiaoxi] DM | 李承龙 (ou_e63ce6b760ad39382852472f28fbe2a2) [msg:om_x100b516d7f3cb044b37b6d0f9fb4af0]
|
||||
|
||||
Conversation info (untrusted metadata):
|
||||
```json
|
||||
{
|
||||
"message_id": "om_x100b516d7f3cb044b37b6d0f9fb4af0",
|
||||
"sender_id": "ou_e63ce6b760ad39382852472f28fbe2a2",
|
||||
"sender": "李承龙",
|
||||
"timestamp": "Mon 2026-04-20 10:44 GMT+8"
|
||||
}
|
||||
```
|
||||
|
||||
Sender (untrusted metadata):
|
||||
```json
|
||||
{
|
||||
"label": "李承龙 (ou_e63ce6b760ad39382852472f28fbe2a2)",
|
||||
"id": "ou_e63ce6b760ad39382852472f28fbe2a2",
|
||||
"name": "李承龙"
|
||||
}
|
||||
```
|
||||
|
||||
account_id=57,查一下手机号明文
|
||||
assistant: [[reply_to_current]]承龙,account_id=57 的手机号信息如下:
|
||||
|
||||
- **脱敏手机号:** 159****1022
|
||||
- **加密手机号:** 5WeooSK74KICOTX-LDEGhg..
|
||||
|
||||
数据库中 `tel` 字段存的是脱敏格式(中间四位打码),`tel_encrypt` 是加密后的密文。**没有明文手机号字段**,明文在入库前就已经做了脱敏和加密处理。
|
||||
|
||||
所以我这边无法直接查到完整的手机号明文,需要通过后端解密服务才能还原。要不要让技术同事协助解密?
|
||||
user: System: [2026-04-20 10:46:28 GMT+8] Feishu[xiaoxi] DM | 李承龙 (ou_e63ce6b760ad39382852472f28fbe2a2) [msg:om_x100b516d768f3844b3fcabde789bfd7]
|
||||
|
||||
Conversation info (untrusted metadata):
|
||||
```json
|
||||
{
|
||||
"message_id": "om_x100b516d768f3844b3fcabde789bfd7",
|
||||
"sender_id": "ou_e63ce6b760ad39382852472f28fbe2a2",
|
||||
"sender": "李承龙",
|
||||
"timestamp": "Mon 2026-04-20 10:46 GMT+8"
|
||||
}
|
||||
```
|
||||
|
||||
Sender (untrusted metadata):
|
||||
```json
|
||||
{
|
||||
"label": "李承龙 (ou_e63ce6b760ad39382852472f28fbe2a2)",
|
||||
"id": "ou_e63ce6b760ad39382852472f28fbe2a2",
|
||||
"name": "李承龙"
|
||||
}
|
||||
```
|
||||
|
||||
如果我给你手机号明文,你可以通过脱敏手机号字段进行模糊查询么
|
||||
assistant:
|
||||
[[reply_to_current]]可以的。你给我明文手机号,我用前三位和后四位去 `tel` 字段匹配就行,比如手机号是 `13912345678`,我就查:
|
||||
|
||||
```sql
|
||||
WHERE tel LIKE '139****5678'
|
||||
```
|
||||
|
||||
你发手机号过来吧。
|
||||
user: System: [2026-04-20 10:47:36 GMT+8] Feishu[xiaoxi] DM | 李承龙 (ou_e63ce6b760ad39382852472f28fbe2a2) [msg:om_x100b516d72b2a934b3450fee9dcf8bd]
|
||||
|
||||
Conversation info (untrusted metadata):
|
||||
```json
|
||||
{
|
||||
"message_id": "om_x100b516d72b2a934b3450fee9dcf8bd",
|
||||
"sender_id": "ou_e63ce6b760ad39382852472f28fbe2a2",
|
||||
"sender": "李承龙",
|
||||
"timestamp": "Mon 2026-04-20 10:47 GMT+8"
|
||||
}
|
||||
```
|
||||
|
||||
Sender (untrusted metadata):
|
||||
```json
|
||||
{
|
||||
"label": "李承龙 (ou_e63ce6b760ad39382852472f28fbe2a2)",
|
||||
"id": "ou_e63ce6b760ad39382852472f28fbe2a2",
|
||||
"name": "李承龙"
|
||||
}
|
||||
```
|
||||
|
||||
15944281022
|
||||
assistant: [[reply_to_current]]查到了,`159****1022` 对应的就是 **account_id = 57**,和你之前查的一致。
|
||||
|
||||
| 字段 | 值 |
|
||||
|------|------|
|
||||
| account_id | 57 |
|
||||
| 脱敏手机号 | 159****1022 |
|
||||
| status | 2(测试账号) |
|
||||
| pay_status | 1 |
|
||||
| 注册时间 | 2025-03-08 14:38 |
|
||||
|
||||
需要进一步查这个用户的其他信息吗?
|
||||
user: [media attached: /root/.openclaw/media/inbound/ã_å_é_ä_人è_ä_å_ã_-ç_æ_è_è_è_å_è_¾æ_å---b57dacbc-6eac-4b04-b1b2-385562af4ece.xlsx (application/vnd.openxmlformats-officedocument.spreadsheetml.sheet) | /root/.openclaw/media/inbound/ã_å_é_ä_人è_ä_å_ã_-ç_æ_è_è_è_å_è_¾æ_å---b57dacbc-6eac-4b04-b1b2-385562af4ece.xlsx]
|
||||
To send an image back, prefer the message tool (media/path/filePath). If you must inline, use MEDIA:https://example.com/image.jpg (spaces ok, quote if needed) or a safe relative path like MEDIA:./image.jpg. Avoid absolute paths (MEDIA:/...) and ~ paths — they are blocked for security. Keep caption in the text body.
|
||||
System: [2026-04-20 11:16:56 GMT+8] Feishu[xiaoxi] DM | 李承龙 (ou_e63ce6b760ad39382852472f28fbe2a2) [msg:om_x100b516de4a728a0b3d8cfdfa5c66a4, file, 1 attachment(s)]
|
||||
|
||||
Conversation info (untrusted metadata):
|
||||
```json
|
||||
{
|
||||
"message_id": "om_x100b516de4a728a0b3d8cfdfa5c66a4",
|
||||
"sender_id": "ou_e63ce6b760ad39382852472f28fbe2a2",
|
||||
"sender": "李承龙",
|
||||
"timestamp": "Mon 2026-04-20 11:16 GMT+8"
|
||||
}
|
||||
```
|
||||
|
||||
Sender (untrusted metadata):
|
||||
```json
|
||||
{
|
||||
"label": "李承龙 (ou_e63ce6b760ad39382852472f28fbe2a2)",
|
||||
"id": "ou_e63ce6b760ad39382852472f28fbe2a2",
|
||||
"name": "李承龙"
|
||||
}
|
||||
```
|
||||
|
||||
[File: /root/.openclaw/media/inbound/ã_å_é_ä_人è_ä_å_ã_-ç_æ_è_è_è_å_è_¾æ_å---b57dacbc-6eac-4b04-b1b2-385562af4ece.xlsx]
|
||||
assistant: [[reply_to_current]]收到,文件是**「学霸三人行专属」瓦拉英语试听课报名收集结果**,共 **774 条手机号**。
|
||||
|
||||
你需要我做什么?常见的操作比如:
|
||||
|
||||
1. **匹配注册用户** — 用这些手机号去 `bi_vala_app_account` 表查哪些已注册
|
||||
2. **匹配付费用户** — 进一步关联订单表,看哪些已付费购课
|
||||
3. **整体转化漏斗** — 报名 → 注册 → 付费 的转化率统计
|
||||
|
||||
你说下需求,我来跑。
|
||||
88
memory/2026-04-20-request-timed-out-before-a-res.md
Normal file
88
memory/2026-04-20-request-timed-out-before-a-res.md
Normal file
@ -0,0 +1,88 @@
|
||||
# Session: 2026-04-20 04:08:35 UTC
|
||||
|
||||
- **Session Key**: agent:main:feishu:direct:ou_e63ce6b760ad39382852472f28fbe2a2
|
||||
- **Session ID**: f1d7ceb4-fec0-4194-aa52-bec22f19ac06
|
||||
- **Source**: feishu
|
||||
|
||||
## Conversation Summary
|
||||
|
||||
assistant:
|
||||
|
||||
有道理。改脚本,把脱敏手机号替换为明文。
|
||||
assistant: 核心改动:SQL 查询返回的是脱敏手机号,需要在 Python 层用 `masked→original` 映射表替换回明文。
|
||||
assistant: 测试一下:
|
||||
assistant: 手机号已经是明文了。发文件给你:
|
||||
assistant: phone_chapter_query_20260420_115730.xlsx
|
||||
assistant: [[reply_to_current]]已改好,"脱敏手机号"列改成了"手机号"明文列。逻辑是:数据库查出脱敏格式后,在 Python 层用文件中的原始手机号映射回明文写入 xlsx。末尾未匹配的也是明文手机号。
|
||||
|
||||
文件已发,你看下格式对不对。
|
||||
user: System: [2026-04-20 12:00:50 GMT+8] Feishu[default] DM | 李承龙 (ou_e63ce6b760ad39382852472f28fbe2a2) [msg:om_x100b516e01eda0b0b3f98a073c8edb3]
|
||||
|
||||
Conversation info (untrusted metadata):
|
||||
```json
|
||||
{
|
||||
"message_id": "om_x100b516e01eda0b0b3f98a073c8edb3",
|
||||
"sender_id": "ou_e63ce6b760ad39382852472f28fbe2a2",
|
||||
"sender": "李承龙",
|
||||
"timestamp": "Mon 2026-04-20 12:00 GMT+8"
|
||||
}
|
||||
```
|
||||
|
||||
Sender (untrusted metadata):
|
||||
```json
|
||||
{
|
||||
"label": "李承龙 (ou_e63ce6b760ad39382852472f28fbe2a2)",
|
||||
"id": "ou_e63ce6b760ad39382852472f28fbe2a2",
|
||||
"name": "李承龙"
|
||||
}
|
||||
```
|
||||
|
||||
在这个skill里面添加,生成的文档发送到oc_616882799f1910b85a248b74f44c3be5这个群里
|
||||
assistant: 这个发送功能不适合放在 Python 脚本里(需要通过飞书 Bot 发送文件),更合适的方式是在 SKILL.md 的流程说明中加上发送步骤,我运行脚本后自动发到群里。
|
||||
assistant: 现在用刚才的结果文件测试一下发送到群里:
|
||||
assistant: 现在把发送逻辑也集成到 SKILL.md 的流程中,确保我执行 skill 时自动完成。让我先验证发送到群里是否能成功:
|
||||
assistant: 发送成功。现在我把这个发送逻辑也封装进脚本,让它一键完成查询+发送:
|
||||
assistant: 测试完整流程:
|
||||
assistant: 全流程通过 ✅
|
||||
|
||||
[[reply_to_current]]已更新 skill,现在脚本执行完会自动把 xlsx 文件发送到群 `oc_616882799f1910b85a248b74f44c3be5`。刚才用你的文件完整跑了一遍,文件已经发到群里了,你去看下。
|
||||
|
||||
新增功能:
|
||||
- 默认查询完自动发送到目标群
|
||||
- 加了 `--no-send` 参数,如果只想生成文件不发群可以用:
|
||||
```bash
|
||||
python3 skills/phone-chapter-query/scripts/phone_chapter_query.py <文件> --no-send
|
||||
```
|
||||
user: System: [2026-04-20 12:05:12 GMT+8] Feishu[xiaoxi] DM | 李承龙 (ou_e63ce6b760ad39382852472f28fbe2a2) [msg:om_x100b516e11b36d3cb358895c6205a78]
|
||||
|
||||
Conversation info (untrusted metadata):
|
||||
```json
|
||||
{
|
||||
"message_id": "om_x100b516e11b36d3cb358895c6205a78",
|
||||
"sender_id": "ou_e63ce6b760ad39382852472f28fbe2a2",
|
||||
"sender": "李承龙",
|
||||
"timestamp": "Mon 2026-04-20 12:05 GMT+8"
|
||||
}
|
||||
```
|
||||
|
||||
Sender (untrusted metadata):
|
||||
```json
|
||||
{
|
||||
"label": "李承龙 (ou_e63ce6b760ad39382852472f28fbe2a2)",
|
||||
"id": "ou_e63ce6b760ad39382852472f28fbe2a2",
|
||||
"name": "李承龙"
|
||||
}
|
||||
```
|
||||
|
||||
给我发一下这个skill的路径
|
||||
assistant:
|
||||
|
||||
[[reply_to_current]]`/root/.openclaw/workspace/skills/phone-chapter-query/`
|
||||
|
||||
文件结构:
|
||||
```
|
||||
skills/phone-chapter-query/
|
||||
├── SKILL.md
|
||||
└── scripts/
|
||||
└── phone_chapter_query.py
|
||||
```
|
||||
178
scripts/query_chapter_play.sh
Executable file
178
scripts/query_chapter_play.sh
Executable file
@ -0,0 +1,178 @@
|
||||
#!/bin/bash
|
||||
# 课时完成记录查询脚本
|
||||
# 查询指定课时(chapter_id)的首次完成记录,包含角色信息、最近登录、课程耗时
|
||||
# 数据库:线上 PostgreSQL (vala_bi)
|
||||
# 输出:xlsx 格式
|
||||
# 用法:bash scripts/query_chapter_play.sh
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
source "${SCRIPT_DIR}/../secrets.env"
|
||||
|
||||
DB_HOST="bj-postgres-16pob4sg.sql.tencentcdb.com"
|
||||
DB_PORT="28591"
|
||||
DB_USER="ai_member"
|
||||
DB_NAME="vala_bi"
|
||||
|
||||
OUTPUT_DIR="${SCRIPT_DIR}/../output"
|
||||
mkdir -p "$OUTPUT_DIR"
|
||||
|
||||
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
|
||||
CSV_TMP="${OUTPUT_DIR}/.chapter_play_${TIMESTAMP}.csv"
|
||||
OUTPUT_FILE="${OUTPUT_DIR}/chapter_play_${TIMESTAMP}.xlsx"
|
||||
|
||||
SQL=$(cat <<'EOF'
|
||||
WITH chapter_play AS (
|
||||
SELECT user_id, chapter_id, chapter_unique_id, date(updated_at) AS finish_date
|
||||
FROM bi_user_chapter_play_record_0
|
||||
WHERE chapter_id IN (55,56,57,58,59,343,344,345,346,348) AND play_status = 1
|
||||
UNION ALL
|
||||
SELECT user_id, chapter_id, chapter_unique_id, date(updated_at)
|
||||
FROM bi_user_chapter_play_record_1
|
||||
WHERE chapter_id IN (55,56,57,58,59,343,344,345,346,348) AND play_status = 1
|
||||
UNION ALL
|
||||
SELECT user_id, chapter_id, chapter_unique_id, date(updated_at)
|
||||
FROM bi_user_chapter_play_record_2
|
||||
WHERE chapter_id IN (55,56,57,58,59,343,344,345,346,348) AND play_status = 1
|
||||
UNION ALL
|
||||
SELECT user_id, chapter_id, chapter_unique_id, date(updated_at)
|
||||
FROM bi_user_chapter_play_record_3
|
||||
WHERE chapter_id IN (55,56,57,58,59,343,344,345,346,348) AND play_status = 1
|
||||
UNION ALL
|
||||
SELECT user_id, chapter_id, chapter_unique_id, date(updated_at)
|
||||
FROM bi_user_chapter_play_record_4
|
||||
WHERE chapter_id IN (55,56,57,58,59,343,344,345,346,348) AND play_status = 1
|
||||
UNION ALL
|
||||
SELECT user_id, chapter_id, chapter_unique_id, date(updated_at)
|
||||
FROM bi_user_chapter_play_record_5
|
||||
WHERE chapter_id IN (55,56,57,58,59,343,344,345,346,348) AND play_status = 1
|
||||
UNION ALL
|
||||
SELECT user_id, chapter_id, chapter_unique_id, date(updated_at)
|
||||
FROM bi_user_chapter_play_record_6
|
||||
WHERE chapter_id IN (55,56,57,58,59,343,344,345,346,348) AND play_status = 1
|
||||
UNION ALL
|
||||
SELECT user_id, chapter_id, chapter_unique_id, date(updated_at)
|
||||
FROM bi_user_chapter_play_record_7
|
||||
WHERE chapter_id IN (55,56,57,58,59,343,344,345,346,348) AND play_status = 1
|
||||
),
|
||||
comp_time AS (
|
||||
SELECT chapter_unique_id, SUM(interval_time) AS total_interval
|
||||
FROM (
|
||||
SELECT chapter_unique_id, interval_time FROM bi_user_component_play_record_0
|
||||
UNION ALL
|
||||
SELECT chapter_unique_id, interval_time FROM bi_user_component_play_record_1
|
||||
UNION ALL
|
||||
SELECT chapter_unique_id, interval_time FROM bi_user_component_play_record_2
|
||||
UNION ALL
|
||||
SELECT chapter_unique_id, interval_time FROM bi_user_component_play_record_3
|
||||
UNION ALL
|
||||
SELECT chapter_unique_id, interval_time FROM bi_user_component_play_record_4
|
||||
UNION ALL
|
||||
SELECT chapter_unique_id, interval_time FROM bi_user_component_play_record_5
|
||||
UNION ALL
|
||||
SELECT chapter_unique_id, interval_time FROM bi_user_component_play_record_6
|
||||
UNION ALL
|
||||
SELECT chapter_unique_id, interval_time FROM bi_user_component_play_record_7
|
||||
) t
|
||||
GROUP BY chapter_unique_id
|
||||
),
|
||||
course_detail AS (
|
||||
SELECT
|
||||
cp.user_id,
|
||||
cp.chapter_id,
|
||||
FORMAT('%s-%s-%s-%s', l.course_level, l.course_season, l.course_unit, l.course_lesson) AS course_id,
|
||||
cp.finish_date,
|
||||
FORMAT('%s:%s',
|
||||
FLOOR(ct.total_interval / 1000 / 60),
|
||||
LPAD(CAST(MOD(ct.total_interval / 1000, 60) AS TEXT), 2, '0')
|
||||
) AS finish_time
|
||||
FROM (
|
||||
SELECT user_id, chapter_id, chapter_unique_id, finish_date,
|
||||
ROW_NUMBER() OVER (PARTITION BY user_id, chapter_id ORDER BY finish_date) AS rn
|
||||
FROM chapter_play
|
||||
) cp
|
||||
LEFT JOIN bi_level_unit_lesson l ON cp.chapter_id = l.id
|
||||
LEFT JOIN comp_time ct ON cp.chapter_unique_id = ct.chapter_unique_id
|
||||
WHERE cp.rn = 1
|
||||
)
|
||||
|
||||
SELECT
|
||||
a.account_id AS "账号ID",
|
||||
a.id AS "角色ID",
|
||||
DATE(a.created_at) AS "注册日期",
|
||||
DATE(TO_TIMESTAMP(b.latest_active_time)) AS "最近登录时间",
|
||||
c.course_id AS "课程名称",
|
||||
c.finish_date AS "课程完成时间",
|
||||
c.finish_time AS "课程耗时"
|
||||
FROM bi_vala_app_character a
|
||||
LEFT JOIN user_detail_info b ON a.id = b.user_id
|
||||
LEFT JOIN course_detail c ON a.id = c.user_id
|
||||
WHERE a.status = 1
|
||||
ORDER BY a.id, c.course_id;
|
||||
EOF
|
||||
)
|
||||
|
||||
echo "正在查询..."
|
||||
PGPASSWORD="$PG_ONLINE_PASSWORD" psql \
|
||||
-h "$DB_HOST" \
|
||||
-p "$DB_PORT" \
|
||||
-U "$DB_USER" \
|
||||
-d "$DB_NAME" \
|
||||
--csv \
|
||||
-c "$SQL" \
|
||||
> "$CSV_TMP"
|
||||
|
||||
ROW_COUNT=$(tail -n +2 "$CSV_TMP" | wc -l)
|
||||
echo "查询完成,共 ${ROW_COUNT} 条记录,正在转换为 xlsx..."
|
||||
|
||||
python3 - "$CSV_TMP" "$OUTPUT_FILE" <<'PYEOF'
|
||||
import csv, sys
|
||||
from openpyxl import Workbook
|
||||
from openpyxl.styles import Font, Alignment, PatternFill
|
||||
|
||||
csv_path, xlsx_path = sys.argv[1], sys.argv[2]
|
||||
|
||||
wb = Workbook()
|
||||
ws = wb.active
|
||||
ws.title = "课时完成记录"
|
||||
|
||||
# 读取 CSV
|
||||
with open(csv_path, 'r', encoding='utf-8') as f:
|
||||
reader = csv.reader(f)
|
||||
for row in reader:
|
||||
ws.append(row)
|
||||
|
||||
# 表头样式
|
||||
header_font = Font(bold=True, color="FFFFFF", size=11)
|
||||
header_fill = PatternFill(start_color="4472C4", end_color="4472C4", fill_type="solid")
|
||||
header_align = Alignment(horizontal="center", vertical="center")
|
||||
|
||||
for cell in ws[1]:
|
||||
cell.font = header_font
|
||||
cell.fill = header_fill
|
||||
cell.alignment = header_align
|
||||
|
||||
# 自动列宽
|
||||
for col in ws.columns:
|
||||
max_len = 0
|
||||
col_letter = col[0].column_letter
|
||||
for cell in col:
|
||||
val = str(cell.value) if cell.value else ""
|
||||
# 中文字符算 2 宽度
|
||||
width = sum(2 if ord(c) > 127 else 1 for c in val)
|
||||
if width > max_len:
|
||||
max_len = width
|
||||
ws.column_dimensions[col_letter].width = min(max_len + 3, 30)
|
||||
|
||||
# 冻结首行
|
||||
ws.freeze_panes = "A2"
|
||||
|
||||
wb.save(xlsx_path)
|
||||
print(f"xlsx 已保存: {xlsx_path}")
|
||||
PYEOF
|
||||
|
||||
# 清理临时 CSV
|
||||
rm -f "$CSV_TMP"
|
||||
|
||||
echo "完成!结果已保存至: ${OUTPUT_FILE}"
|
||||
@ -22,7 +22,7 @@ OUTPUT_CSV_L1_OOPS="/tmp/l1_oops_${TODAY}.csv"
|
||||
OUTPUT_CSV_L2_OOPS="/tmp/l2_oops_${TODAY}.csv"
|
||||
OUTPUT_CSV_L1_EXIT="/tmp/l1_exit_${TODAY}.csv"
|
||||
OUTPUT_CSV_L2_EXIT="/tmp/l2_exit_${TODAY}.csv"
|
||||
OUTPUT_EXCEL="/root/.openclaw/workspace/组件练习质量统计_${TODAY}.xlsx"
|
||||
OUTPUT_EXCEL="/root/.openclaw/workspace/output/组件练习质量统计_${TODAY}.xlsx"
|
||||
TARGET_CHAT="chat:oc_248d0efbc02d97d81d592d0417b6b296"
|
||||
LOG_FILE="/var/log/component_oops_stat.log"
|
||||
|
||||
|
||||
@ -11,7 +11,7 @@ DB_USER="ai_member"
|
||||
DB_PASS="LdfjdjL83h3h3^$&**YGG*"
|
||||
SQL_PATH="/root/.openclaw/workspace/scripts/用户购课完课统计.sql"
|
||||
OUTPUT_CSV="/tmp/user_course_stat_$(date +%Y%m%d).csv"
|
||||
OUTPUT_EXCEL="/root/.openclaw/workspace/用户购课完课统计_$(date +%Y%m%d).xlsx"
|
||||
OUTPUT_EXCEL="/root/.openclaw/workspace/output/用户购课完课统计_$(date +%Y%m%d).xlsx"
|
||||
TARGET_CHAT="chat:oc_af81515caefe26918736ad1941286224"
|
||||
LOG_FILE="/var/log/user_course_stat.log"
|
||||
|
||||
|
||||
@ -18,14 +18,13 @@ START_DATE=$(date -d "last sunday -6 days" +%Y-%m-%d)
|
||||
END_DATE=$(date -d "last sunday" +%Y-%m-%d)
|
||||
REPORT_DATE=$(date +%Y%m%d)
|
||||
CSV_PATH="/tmp/xueersi_weekly_data_${REPORT_DATE}.csv"
|
||||
# 提取月日,去掉前导零,生成文件名
|
||||
# 提取月日,去掉前导零,生成文件名(使用"月""日"格式避免斜杠被截断)
|
||||
START_MONTH=$(date -d "${START_DATE}" +%-m)
|
||||
START_DAY=$(date -d "${START_DATE}" +%-d)
|
||||
END_MONTH=$(date -d "${END_DATE}" +%-m)
|
||||
END_DAY=$(date -d "${END_DATE}" +%-d)
|
||||
# 本地保存路径用横杠避免路径问题,上传飞书时用带斜杠的显示名
|
||||
EXCEL_PATH="/tmp/学而思-瓦拉英语${START_MONTH}-${START_DAY}-${END_MONTH}-${END_DAY}.xlsx"
|
||||
FEISHU_FILE_NAME="学而思-瓦拉英语${START_MONTH}/${START_DAY}-${END_MONTH}/${END_DAY}.xlsx"
|
||||
EXCEL_PATH="/tmp/学而思-瓦拉英语${START_MONTH}月${START_DAY}日-${END_MONTH}月${END_DAY}日.xlsx"
|
||||
FEISHU_FILE_NAME="学而思-瓦拉英语${START_MONTH}月${START_DAY}日-${END_MONTH}月${END_DAY}日.xlsx"
|
||||
|
||||
# 1. 查询数据导出CSV
|
||||
PGPASSWORD="${PG_PASSWORD}" psql -h bj-postgres-16pob4sg.sql.tencentcdb.com -p 28591 -U ai_member -d vala_bi -c "\copy (WITH date_range AS (SELECT generate_series('${START_DATE}'::date, '${END_DATE}'::date, '1 day'::interval) AS stat_date), daily_new_users AS (SELECT DATE(created_at) AS stat_date, COUNT(DISTINCT id) AS new_user_count FROM bi_vala_app_account WHERE download_channel LIKE '%学而思%' AND created_at >= '${START_DATE} 00:00:00+08' AND created_at < '$(date -d "${END_DATE} +1 day" +%Y-%m-%d) 00:00:00+08' AND deleted_at IS NULL GROUP BY DATE(created_at)), daily_orders AS (SELECT DATE(o.pay_success_date) AS stat_date, COUNT(DISTINCT o.id) AS total_order_count, COUNT(DISTINCT CASE WHEN r.status = 3 AND o.order_status = 4 THEN o.id END) AS refund_order_count, ROUND(SUM(o.pay_amount_int)/100.0, 2) AS gmv, ROUND(SUM(o.pay_amount_int)/100.0 - COALESCE(SUM(CASE WHEN r.status = 3 AND o.order_status = 4 THEN r.refund_amount_int ELSE 0 END)/100.0, 0), 2) AS gsv FROM bi_vala_order o LEFT JOIN bi_refund_order r ON o.out_trade_no = r.out_trade_no WHERE o.key_from = 'app-active-h5-0-0' AND o.sale_channel = 21 AND o.pay_success_date >= '${START_DATE} 00:00:00+08' AND o.pay_success_date < '$(date -d "${END_DATE} +1 day" +%Y-%m-%d) 00:00:00+08' AND o.pay_success_date IS NOT NULL GROUP BY DATE(o.pay_success_date)), daily_data AS (SELECT TO_CHAR(d.stat_date, 'YYYY-MM-DD') AS 日期, COALESCE(u.new_user_count, 0) AS 新增用户数, COALESCE(o.total_order_count - o.refund_order_count, 0) AS 有效订单数, COALESCE(o.gsv, 0) AS GSV_元 FROM date_range d LEFT JOIN daily_new_users u ON d.stat_date = u.stat_date LEFT JOIN daily_orders o ON d.stat_date = o.stat_date) SELECT * FROM daily_data UNION ALL SELECT '合计' AS 日期, SUM(新增用户数) AS 新增用户数, SUM(有效订单数) AS 有效订单数, SUM(GSV_元) AS GSV_元 FROM daily_data ORDER BY 日期) TO '${CSV_PATH}' WITH (FORMAT csv, HEADER true, ENCODING 'UTF8');"
|
||||
|
||||
62
skills/phone-chapter-query/SKILL.md
Normal file
62
skills/phone-chapter-query/SKILL.md
Normal file
@ -0,0 +1,62 @@
|
||||
---
|
||||
name: phone-chapter-query
|
||||
description: |
|
||||
通过手机号文件查询用户课时完成记录。从 Excel/CSV 文件中提取手机号,匹配瓦拉账号,
|
||||
查询指定课时的首次完成记录(含课程耗时),输出带手机号明文的 xlsx 报表,
|
||||
并自动发送到指定飞书群。未匹配到账号的手机号追加在表尾。
|
||||
|
||||
**触发场景**:
|
||||
(1) 用户发送包含手机号的 Excel/CSV 文件,要求查询课时完成记录
|
||||
(2) 用户提到"手机号"+"课时"/"学习记录"/"完成记录"的组合
|
||||
(3) 需要批量通过手机号关联账号并查询学习数据
|
||||
---
|
||||
|
||||
# 手机号课时完成记录查询
|
||||
|
||||
## 完整流程
|
||||
|
||||
1. 用户发送包含手机号的 Excel(.xlsx) 或 CSV(.csv) 文件
|
||||
2. 运行脚本:提取手机号 → 脱敏匹配账号 → 查询课时记录 → 生成 xlsx
|
||||
3. 将生成的 xlsx 文件发送到飞书群 `oc_616882799f1910b85a248b74f44c3be5`
|
||||
4. 同时回复用户确认结果
|
||||
|
||||
## 步骤1: 运行查询脚本
|
||||
|
||||
```bash
|
||||
python3 skills/phone-chapter-query/scripts/phone_chapter_query.py <input_file> [--output <path>]
|
||||
```
|
||||
|
||||
- `input_file`: 包含手机号的文件路径(支持 .xlsx 和 .csv)
|
||||
- `--output`: 可选,指定输出路径,默认 `output/phone_chapter_query_时间戳.xlsx`
|
||||
|
||||
## 步骤2: 发送结果到飞书群
|
||||
|
||||
脚本执行完成后,使用 `lark-send-message-as-bot` skill 将生成的 xlsx 文件发送到目标群:
|
||||
|
||||
- **目标群 chat_id**: `oc_616882799f1910b85a248b74f44c3be5`
|
||||
- **发送方式**: Bot 身份发送文件消息
|
||||
|
||||
## 输出字段
|
||||
|
||||
| 字段 | 说明 |
|
||||
|------|------|
|
||||
| 账号ID | bi_vala_app_account.id |
|
||||
| 手机号 | 明文手机号(来自输入文件) |
|
||||
| 角色ID | bi_vala_app_character.id |
|
||||
| 注册日期 | 角色创建时间 |
|
||||
| 最近登录时间 | user_detail_info.latest_active_time |
|
||||
| 课程名称 | 等级-季-单元-课时 格式 |
|
||||
| 课程完成时间 | 首次完成日期 |
|
||||
| 课程耗时 | 分:秒 格式 |
|
||||
|
||||
未匹配到账号的手机号追加在表尾,仅手机号列有值。
|
||||
|
||||
## 匹配规则
|
||||
|
||||
- 自动识别文件中所有 `1` 开头的 11 位数字作为手机号
|
||||
- 脱敏后用 `bi_vala_app_account.tel` 字段 IN 匹配
|
||||
- 仅匹配 `status = 1` 且 `deleted_at IS NULL` 的账号
|
||||
|
||||
## 可配置项
|
||||
|
||||
脚本顶部的 `CHAPTER_IDS` 变量控制查询的课时范围,按需修改。
|
||||
413
skills/phone-chapter-query/scripts/phone_chapter_query.py
Normal file
413
skills/phone-chapter-query/scripts/phone_chapter_query.py
Normal file
@ -0,0 +1,413 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
手机号 → 账号ID → 课时完成记录查询脚本
|
||||
|
||||
用法:
|
||||
python3 scripts/phone_chapter_query.py <input_file> [--output <output_path>]
|
||||
|
||||
参数:
|
||||
input_file 包含手机号的 Excel(.xlsx) 或 CSV(.csv) 文件路径
|
||||
--output 输出 xlsx 文件路径(可选,默认 output/phone_chapter_query_时间戳.xlsx)
|
||||
|
||||
流程:
|
||||
1. 从文件中提取所有11位手机号(自动识别列)
|
||||
2. 脱敏后匹配 bi_vala_app_account(status=1 且未删除)获取 account_id
|
||||
3. 用 account_id 关联角色表,查询指定课时的首次完成记录(含耗时)
|
||||
4. 输出带手机号明文列的 xlsx 文件,未匹配手机号追加在末尾
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import urllib.request
|
||||
from datetime import datetime
|
||||
|
||||
try:
|
||||
import openpyxl
|
||||
from openpyxl.styles import Font, Alignment, PatternFill
|
||||
except ImportError:
|
||||
print("ERROR: 需要 openpyxl 库,请运行: pip3 install openpyxl")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
# ─── 配置 ───────────────────────────────────────────────
|
||||
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||
# 工作区根目录:scripts/ -> phone-chapter-query/ -> skills/ -> workspace/
|
||||
WORKSPACE_DIR = os.path.dirname(os.path.dirname(os.path.dirname(SCRIPT_DIR)))
|
||||
SECRETS_FILE = os.path.join(WORKSPACE_DIR, "secrets.env")
|
||||
OUTPUT_DIR = os.path.join(WORKSPACE_DIR, "output")
|
||||
|
||||
DB_HOST = "bj-postgres-16pob4sg.sql.tencentcdb.com"
|
||||
DB_PORT = "28591"
|
||||
DB_USER = "ai_member"
|
||||
DB_NAME = "vala_bi"
|
||||
|
||||
# 要查询的课时 ID,按需修改
|
||||
CHAPTER_IDS = "55,56,57,58,59,343,344,345,346,348"
|
||||
# 分表数量(0-7)
|
||||
SHARD_COUNT = 8
|
||||
|
||||
# 飞书发送配置
|
||||
LARK_CREDENTIALS = "/root/.openclaw/credentials/xiaoxi/config.json"
|
||||
LARK_TARGET_CHAT_ID = "oc_616882799f1910b85a248b74f44c3be5"
|
||||
|
||||
|
||||
def load_pg_password():
|
||||
"""从 secrets.env 读取 PostgreSQL 密码"""
|
||||
with open(SECRETS_FILE, "r") as f:
|
||||
for line in f:
|
||||
if line.startswith("PG_ONLINE_PASSWORD="):
|
||||
return line.split("=", 1)[1].strip().strip("'\"")
|
||||
raise RuntimeError("secrets.env 中未找到 PG_ONLINE_PASSWORD")
|
||||
|
||||
|
||||
def extract_phones(file_path):
|
||||
"""从 xlsx 或 csv 文件中提取所有11位手机号"""
|
||||
phones = []
|
||||
ext = os.path.splitext(file_path)[1].lower()
|
||||
|
||||
if ext in (".xlsx", ".xls"):
|
||||
wb = openpyxl.load_workbook(file_path, read_only=True)
|
||||
for ws in wb:
|
||||
for row in ws.iter_rows(values_only=True):
|
||||
for cell in row:
|
||||
if cell is not None:
|
||||
val = str(int(cell)) if isinstance(cell, float) else str(cell)
|
||||
val = val.strip()
|
||||
if re.match(r"^1\d{10}$", val):
|
||||
phones.append(val)
|
||||
elif ext == ".csv":
|
||||
with open(file_path, "r", encoding="utf-8-sig") as f:
|
||||
reader = csv.reader(f)
|
||||
for row in reader:
|
||||
for cell in row:
|
||||
val = cell.strip()
|
||||
if re.match(r"^1\d{10}$", val):
|
||||
phones.append(val)
|
||||
else:
|
||||
raise ValueError(f"不支持的文件格式: {ext},仅支持 .xlsx 和 .csv")
|
||||
|
||||
# 去重并保持顺序
|
||||
seen = set()
|
||||
unique = []
|
||||
for p in phones:
|
||||
if p not in seen:
|
||||
seen.add(p)
|
||||
unique.append(p)
|
||||
return unique
|
||||
|
||||
|
||||
def mask_phone(phone):
|
||||
"""手机号脱敏:前3 + **** + 后4"""
|
||||
return f"{phone[:3]}****{phone[-4:]}"
|
||||
|
||||
|
||||
def run_sql(sql, pg_password):
|
||||
"""执行 SQL 并返回 CSV 格式的结果"""
|
||||
env = os.environ.copy()
|
||||
env["PGPASSWORD"] = pg_password
|
||||
result = subprocess.run(
|
||||
["psql", "-h", DB_HOST, "-p", DB_PORT, "-U", DB_USER, "-d", DB_NAME, "--csv", "-c", sql],
|
||||
capture_output=True, text=True, env=env, timeout=300,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
raise RuntimeError(f"SQL 执行失败:\n{result.stderr}")
|
||||
return result.stdout
|
||||
|
||||
|
||||
def step1_match_accounts(phones, pg_password):
|
||||
"""步骤1: 手机号脱敏后匹配 account_id,返回匹配结果和映射表"""
|
||||
# masked -> [original1, original2, ...] (不同明文可能脱敏后相同)
|
||||
masked_to_originals = {}
|
||||
for p in phones:
|
||||
m = mask_phone(p)
|
||||
masked_to_originals.setdefault(m, []).append(p)
|
||||
|
||||
conditions = ", ".join(f"'{m}'" for m in masked_to_originals)
|
||||
sql = f"""
|
||||
SELECT id AS account_id, tel AS masked_phone
|
||||
FROM bi_vala_app_account
|
||||
WHERE tel IN ({conditions})
|
||||
AND status = 1
|
||||
AND deleted_at IS NULL
|
||||
ORDER BY id;
|
||||
"""
|
||||
output = run_sql(sql, pg_password)
|
||||
|
||||
# 解析结果,构建 account_id -> 明文手机号 映射
|
||||
account_to_phone = {} # account_id -> original_phone
|
||||
matched_masks = set()
|
||||
reader = csv.DictReader(io.StringIO(output))
|
||||
for row in reader:
|
||||
aid = row["account_id"]
|
||||
masked = row["masked_phone"]
|
||||
matched_masks.add(masked)
|
||||
# 取该脱敏格式对应的第一个明文
|
||||
if masked in masked_to_originals:
|
||||
account_to_phone[aid] = masked_to_originals[masked][0]
|
||||
|
||||
account_ids = list(account_to_phone.keys())
|
||||
|
||||
# 未匹配的手机号(明文)
|
||||
unmatched = []
|
||||
for m, originals in masked_to_originals.items():
|
||||
if m not in matched_masks:
|
||||
unmatched.extend(originals)
|
||||
|
||||
return account_ids, account_to_phone, unmatched
|
||||
|
||||
|
||||
def build_union_sql(table_prefix, columns, where_clause, shard_count):
|
||||
"""生成分表 UNION ALL SQL"""
|
||||
parts = []
|
||||
for i in range(shard_count):
|
||||
parts.append(f"SELECT {columns} FROM {table_prefix}_{i} {where_clause}")
|
||||
return " UNION ALL ".join(parts)
|
||||
|
||||
|
||||
def step2_query_chapter_play(account_ids, pg_password):
|
||||
"""步骤2: 用 account_id 查询课时完成记录"""
|
||||
if not account_ids:
|
||||
return ""
|
||||
|
||||
aid_list = ", ".join(account_ids)
|
||||
|
||||
chapter_union = build_union_sql(
|
||||
"bi_user_chapter_play_record",
|
||||
"user_id, chapter_id, chapter_unique_id, date(updated_at) AS finish_date",
|
||||
f"WHERE chapter_id IN ({CHAPTER_IDS}) AND play_status = 1",
|
||||
SHARD_COUNT,
|
||||
)
|
||||
|
||||
comp_union = build_union_sql(
|
||||
"bi_user_component_play_record",
|
||||
"chapter_unique_id, interval_time",
|
||||
"",
|
||||
SHARD_COUNT,
|
||||
)
|
||||
|
||||
sql = f"""
|
||||
WITH chapter_play AS (
|
||||
{chapter_union}
|
||||
),
|
||||
comp_time AS (
|
||||
SELECT chapter_unique_id, SUM(interval_time) AS total_interval
|
||||
FROM ({comp_union}) t
|
||||
GROUP BY chapter_unique_id
|
||||
),
|
||||
course_detail AS (
|
||||
SELECT
|
||||
cp.user_id,
|
||||
cp.chapter_id,
|
||||
FORMAT('%s-%s-%s-%s', l.course_level, l.course_season, l.course_unit, l.course_lesson) AS course_id,
|
||||
cp.finish_date,
|
||||
FORMAT('%s:%s',
|
||||
FLOOR(ct.total_interval / 1000 / 60),
|
||||
LPAD(CAST(MOD(ct.total_interval / 1000, 60) AS TEXT), 2, '0')
|
||||
) AS finish_time
|
||||
FROM (
|
||||
SELECT user_id, chapter_id, chapter_unique_id, finish_date,
|
||||
ROW_NUMBER() OVER (PARTITION BY user_id, chapter_id ORDER BY finish_date) AS rn
|
||||
FROM chapter_play
|
||||
) cp
|
||||
LEFT JOIN bi_level_unit_lesson l ON cp.chapter_id = l.id
|
||||
LEFT JOIN comp_time ct ON cp.chapter_unique_id = ct.chapter_unique_id
|
||||
WHERE cp.rn = 1
|
||||
)
|
||||
|
||||
SELECT
|
||||
a.account_id AS "账号ID",
|
||||
CAST(a.account_id AS TEXT) AS "account_id_str",
|
||||
a.id AS "角色ID",
|
||||
DATE(a.created_at) AS "注册日期",
|
||||
DATE(TO_TIMESTAMP(b.latest_active_time)) AS "最近登录时间",
|
||||
c.course_id AS "课程名称",
|
||||
c.finish_date AS "课程完成时间",
|
||||
c.finish_time AS "课程耗时"
|
||||
FROM bi_vala_app_character a
|
||||
JOIN bi_vala_app_account acct ON a.account_id = acct.id
|
||||
LEFT JOIN user_detail_info b ON a.id = b.user_id
|
||||
LEFT JOIN course_detail c ON a.id = c.user_id
|
||||
WHERE a.status = 1
|
||||
AND acct.id IN ({aid_list})
|
||||
ORDER BY a.id, c.course_id;
|
||||
"""
|
||||
return run_sql(sql, pg_password)
|
||||
|
||||
|
||||
def write_xlsx(csv_data, account_to_phone, unmatched_phones, output_path):
|
||||
"""将查询结果写入 xlsx,用明文手机号替换脱敏手机号"""
|
||||
wb = openpyxl.Workbook()
|
||||
ws = wb.active
|
||||
ws.title = "课时完成记录"
|
||||
|
||||
# 表头
|
||||
ws.append(["账号ID", "手机号", "角色ID", "注册日期", "最近登录时间", "课程名称", "课程完成时间", "课程耗时"])
|
||||
|
||||
if csv_data:
|
||||
reader = csv.DictReader(io.StringIO(csv_data))
|
||||
for row in reader:
|
||||
aid = row["账号ID"]
|
||||
phone = account_to_phone.get(aid, "")
|
||||
ws.append([
|
||||
aid,
|
||||
phone,
|
||||
row["角色ID"],
|
||||
row["注册日期"],
|
||||
row["最近登录时间"],
|
||||
row["课程名称"],
|
||||
row["课程完成时间"],
|
||||
row["课程耗时"],
|
||||
])
|
||||
|
||||
# 追加未匹配手机号(明文)
|
||||
for p in sorted(unmatched_phones):
|
||||
ws.append(["", p, "", "", "", "", "", ""])
|
||||
|
||||
# 表头样式
|
||||
hf = Font(bold=True, color="FFFFFF", size=11)
|
||||
hfill = PatternFill(start_color="4472C4", end_color="4472C4", fill_type="solid")
|
||||
ha = Alignment(horizontal="center", vertical="center")
|
||||
for cell in ws[1]:
|
||||
cell.font = hf
|
||||
cell.fill = hfill
|
||||
cell.alignment = ha
|
||||
|
||||
# 自动列宽
|
||||
for col in ws.columns:
|
||||
max_w = 0
|
||||
cl = col[0].column_letter
|
||||
for cell in col:
|
||||
v = str(cell.value) if cell.value else ""
|
||||
w = sum(2 if ord(c) > 127 else 1 for c in v)
|
||||
if w > max_w:
|
||||
max_w = w
|
||||
ws.column_dimensions[cl].width = min(max_w + 3, 30)
|
||||
|
||||
ws.freeze_panes = "A2"
|
||||
wb.save(output_path)
|
||||
|
||||
|
||||
def get_lark_token():
|
||||
"""获取飞书 tenant_access_token"""
|
||||
with open(LARK_CREDENTIALS, "r") as f:
|
||||
config = json.load(f)
|
||||
app_id = config["apps"][0]["appId"]
|
||||
app_secret = config["apps"][0]["appSecret"]
|
||||
|
||||
data = json.dumps({"app_id": app_id, "app_secret": app_secret}).encode()
|
||||
req = urllib.request.Request(
|
||||
"https://open.feishu.cn/open-apis/auth/v3/tenant_access_token/internal",
|
||||
data=data,
|
||||
headers={"Content-Type": "application/json"},
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
result = json.loads(resp.read())
|
||||
return result["tenant_access_token"]
|
||||
|
||||
|
||||
def send_file_to_chat(file_path, file_name, chat_id):
|
||||
"""上传文件并发送到飞书群"""
|
||||
token = get_lark_token()
|
||||
|
||||
# 上传文件
|
||||
upload_cmd = [
|
||||
"curl", "-s", "-X", "POST",
|
||||
"https://open.feishu.cn/open-apis/im/v1/files",
|
||||
"-H", f"Authorization: Bearer {token}",
|
||||
"-F", "file_type=xls",
|
||||
"-F", f"file_name={file_name}",
|
||||
"-F", f"file=@{file_path}",
|
||||
]
|
||||
result = subprocess.run(upload_cmd, capture_output=True, text=True, timeout=60)
|
||||
upload_resp = json.loads(result.stdout)
|
||||
if upload_resp.get("code") != 0:
|
||||
raise RuntimeError(f"文件上传失败: {upload_resp}")
|
||||
file_key = upload_resp["data"]["file_key"]
|
||||
|
||||
# 发送文件消息
|
||||
send_data = json.dumps({
|
||||
"receive_id": chat_id,
|
||||
"msg_type": "file",
|
||||
"content": json.dumps({"file_key": file_key}),
|
||||
})
|
||||
req = urllib.request.Request(
|
||||
f"https://open.feishu.cn/open-apis/im/v1/messages?receive_id_type=chat_id",
|
||||
data=send_data.encode(),
|
||||
headers={
|
||||
"Authorization": f"Bearer {token}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
send_resp = json.loads(resp.read())
|
||||
if send_resp.get("code") != 0:
|
||||
raise RuntimeError(f"消息发送失败: {send_resp}")
|
||||
return send_resp["data"]["message_id"]
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="手机号 → 课时完成记录查询")
|
||||
parser.add_argument("input_file", help="包含手机号的 Excel 或 CSV 文件")
|
||||
parser.add_argument("--output", "-o", help="输出 xlsx 文件路径")
|
||||
parser.add_argument("--no-send", action="store_true", help="不发送到飞书群(仅生成文件)")
|
||||
args = parser.parse_args()
|
||||
|
||||
os.makedirs(OUTPUT_DIR, exist_ok=True)
|
||||
if args.output:
|
||||
output_path = args.output
|
||||
else:
|
||||
ts = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
output_path = os.path.join(OUTPUT_DIR, f"phone_chapter_query_{ts}.xlsx")
|
||||
|
||||
# 加载密码
|
||||
print("读取数据库配置...")
|
||||
pg_password = load_pg_password()
|
||||
|
||||
# 提取手机号
|
||||
print(f"读取文件: {args.input_file}")
|
||||
phones = extract_phones(args.input_file)
|
||||
print(f"提取到 {len(phones)} 个手机号(去重后)")
|
||||
if not phones:
|
||||
print("ERROR: 未提取到任何手机号")
|
||||
sys.exit(1)
|
||||
|
||||
# 匹配 account_id
|
||||
print("匹配账号...")
|
||||
account_ids, account_to_phone, unmatched = step1_match_accounts(phones, pg_password)
|
||||
print(f"已匹配: {len(account_ids)} 个账号,未匹配: {len(unmatched)} 个手机号")
|
||||
|
||||
# 查询课时完成记录
|
||||
if account_ids:
|
||||
print(f"查询课时完成记录({len(account_ids)} 个账号)...")
|
||||
csv_data = step2_query_chapter_play(account_ids, pg_password)
|
||||
row_count = csv_data.strip().count("\n")
|
||||
print(f"查询到 {row_count} 条记录")
|
||||
else:
|
||||
csv_data = ""
|
||||
print("无匹配账号,跳过课时查询")
|
||||
|
||||
# 生成 xlsx
|
||||
print("生成 xlsx...")
|
||||
write_xlsx(csv_data, account_to_phone, unmatched, output_path)
|
||||
print(f"文件保存至: {output_path}")
|
||||
|
||||
# 发送到飞书群
|
||||
if not args.no_send:
|
||||
print(f"发送到飞书群 {LARK_TARGET_CHAT_ID}...")
|
||||
file_name = os.path.basename(output_path)
|
||||
msg_id = send_file_to_chat(output_path, file_name, LARK_TARGET_CHAT_ID)
|
||||
print(f"发送成功!message_id: {msg_id}")
|
||||
else:
|
||||
print("跳过飞书发送(--no-send)")
|
||||
|
||||
print("全部完成!")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Loading…
Reference in New Issue
Block a user