🤖 每日自动备份 - 2026-06-13 08:00:01
This commit is contained in:
小溪
parent
57ead02783
commit
f39592fcd3
72
memory/2026-06-12.md
Normal file
72
memory/2026-06-12.md
Normal file
@ -0,0 +1,72 @@
|
||||
# 2026-06-12 工作日志
|
||||
|
||||
## 陈逸鸫 - 3月拿单 UID/手机清单核对
|
||||
|
||||
收到陈逸鸫从Cursor发来的3月ROI被拿掉订单清单，分A/B/C三组共20笔，需要DB核对。
|
||||
|
||||
### 小龙表读取
|
||||
- 小龙表(qJF4I)共250行，但目标用户均不在该表中（通过昵称、UID、手机号搜索均未命中）
|
||||
|
||||
### Group A (有UID，10笔) - DB核对结果
|
||||
| # | 昵称 | UID | 手机(解密) | C日期 | DB L日期 | L≥C | GMV | GSV | 状态 |
|
||||
|---|------|-----|-----------|-------|---------|-----|-----|-----|------|
|
||||
| 1 | 卢彩苗 | 23814 | 13702058482 | 3/1 | 4/17 | ✓ | 1999 | 1999 | order_status=3, sales-adp-bj-jxl-0 |
|
||||
| 2 | 万万紫 | 17783 | 13702771124 | 3/8 | 无订单 | - | 0 | 0 | DB无任何订单 |
|
||||
| 3 | 哼哼&哈哈 | 16144 | 13327951305 | 3/1 | 无订单 | - | 0 | 0 | DB无任何订单 |
|
||||
| 4 | 毅毅 | 16078 | 19091764172 | 3/1 | 无订单 | - | 0 | 0 | DB无任何订单 |
|
||||
| 5 | 雪珂💗 | 7319 | 13672785535 | 3/6 | 3/6 | ✓ | 1999 | 1999 | 万物店铺, 另有2025-11-14已退单 |
|
||||
| 6 | 小乖大人 | 16158 | 13944890221 | 3/1 | 无订单 | - | 0 | 0 | DB无任何订单 |
|
||||
| 7 | 潘潘 | 16150 | 18610935696 | 3/1 | 无订单 | - | 0 | 0 | DB无任何订单 |
|
||||
| 8 | 张滢ya | 17894 | 13799768340 | 3/7 | 无订单 | - | 0 | 0 | DB无任何订单 |
|
||||
| 9 | sallywu | 17816 | 15998103065 | 3/7 | 无订单 | - | 0 | 0 | DB无任何订单 |
|
||||
| 10 | 🦁萨摩 | 21858 | 13685553716 | 3/8 | 4/8 | ✓ | 1999 | 1999 | 达人-学霸三人行 |
|
||||
|
||||
### Group B (有手机，4笔) - phone_encrypt查UID
|
||||
| # | 昵称 | 手机 | 加密结果 | DB匹配 |
|
||||
|---|------|------|---------|--------|
|
||||
| 11 | 潘提提 | 13427741613 | IiShdIaiY1oy7B_Xn4EH3g.. | 无匹配 |
|
||||
| 12 | 狸小路 | 18622850293 | YPAQ-740vKwxroqZGkeGyQ.. | 无匹配 |
|
||||
| 13 | 希小希 | 18086665321 | c8zfpqBrN1nikMkwAj64aQ.. | 无匹配 |
|
||||
| 14 | 曼 | 13520255515 | NBVtGuxEge7f7hdkyK3y7Q.. | 无匹配 |
|
||||
|
||||
### Group C (无手机/UID，6笔) - DB反查
|
||||
| # | 昵称 | C日期 | 匹配UID | 手机 | DB L | L≥C | 备注 |
|
||||
|---|------|-------|---------|------|------|-----|------|
|
||||
| 15 | Rachel | 3/5 | 10994 | 13510564547 | 3/7 | ✓ | sales-adp-bj-jxl-0, GMV=1999 |
|
||||
| 16 | soul | 3/2 | 17387 | 15640464255 | 3/12 | ✓ | sales-adp-bj-jxl-0, GMV=1999 |
|
||||
| 17 | 红 | 3/7 | 17025 | 13533955004 | 3/14 | ✓ | sales-adp-bj-jxl-0, GMV=1999 |
|
||||
| 18 | 一笑轩渠 | 3/8 | 17425 | 15017528458 | 3/11 | ✓ | sales-adp-bj-jxl-0, GMV=1999 |
|
||||
| 19 | 蜗牛 | 3/2 | ❓ | ❓ | ❓ | - | 晚柠5/15订单数百笔，无手机/UID无法定位 |
|
||||
| 20 | c_瑶 | 3/6 | ❓ | ❓ | ❓ | - | "直购"渠道DB不存在，3/14无3998/1999匹配 |
|
||||
|
||||
### 关键发现
|
||||
1. Group A 中 7/10 用户 DB 中无任何订单（pre汇总有GMV但DB不存在）
|
||||
2. Group B 4个手机号全部未注册（H=未注册 确认正确）
|
||||
3. Group C #15-#18 4笔 jxl-0 均 L≥C，pre怀疑#18 L<C不成立
|
||||
4. #19 蜗牛和 #20 c_瑶 无法定位
|
||||
|
||||
## 陈逸鸫 - full_refresh (S2+S3) 联调
|
||||
|
||||
### 执行记录
|
||||
- 时间：2026-06-12 18:00 左右
|
||||
- S2：三表 XXTEA 匹配 2001 个 UID，D/H/I/J + K–V 已刷新
|
||||
- S3：筛选 K=是 · O>0 · P<O · L≥C → 381 行写入订单汇总 2smjwA，W 渠道归属 + X=1 已写入
|
||||
|
||||
### 发现的问题
|
||||
1. **订单汇总前 8 行无 M/N（成交渠道/产品）**：S2 脚本读小龙 sheet 时 API 只返回了 1198 行（截断），后面约 200 行没被处理。已手动补 M/N。
|
||||
2. **小龙 sheet 第 20 行"赫尔辛基的阳光"**：E 列手机号和 H 列 UID 均为空，但 K/O/R/S/T/U 有值。原因是之前 S2 刷新时 E 列有手机号匹配到了 UID，后来手机号被人清掉但订单数据残留。下次 S2 会清空该行自动列。
|
||||
3. **Bot 应用无多维表格创建权限**：缺少 `bitable:app` 或 `base:app:create` 权限，需技术负责人在飞书开放平台开通。
|
||||
|
||||
### 小红书直购数据（2026-03-12 ~ 2026-06-12）
|
||||
- DB 中 25 个直购用户（26 单），订单汇总仅覆盖 11 行
|
||||
- 差距原因：直购用户在销售表无手机号 → S2 匹配不到 UID → 进不了订单汇总
|
||||
- 整体小红书（含达人+直购）：880 用户、969 单、328 万 GMV
|
||||
|
||||
### 直购表 + S3 联调完成
|
||||
- 时间：2026-06-12 18:40 左右
|
||||
- 新建直购 sheet `1sosYE`（Bot 工作簿内），独立存放小红书直购订单
|
||||
- 新建 `scripts/refresh_direct_sheet.py`：从 DB 直查 `dianpu-xhs` / `stream-xhs` 订单，不依赖手机号，全量 73 条
|
||||
- 修改 `scripts/refresh_order_summary.py`：S3 读取三表 + 直购表，合并写入订单汇总
|
||||
- 直购行 C 列为空时跳过 L≥C 检查（直购无进线日期概念）
|
||||
- full_refresh 流程：先 `refresh_direct_sheet.py` → 再 `refresh_order_summary.py`
|
||||
- 本次验证：381（三表）+ 73（直购）= 454 行写入订单汇总
|
||||
218
scripts/refresh_direct_sheet.py
Normal file
218
scripts/refresh_direct_sheet.py
Normal file
@ -0,0 +1,218 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
直购表全量刷新（小红书店铺 dianpu-xhs + stream-xhs）
|
||||
触发：full_refresh 时先跑此脚本，再跑 refresh_order_summary.py
|
||||
归属：小溪 (xiaoxi)
|
||||
|
||||
直购用户不依赖销售表手机号匹配，直接从 DB 查 key_from 含 dianpu-xhs/stream-xhs 的订单。
|
||||
"""
|
||||
import json, time, os, requests, psycopg2
|
||||
from datetime import datetime
|
||||
|
||||
# ── 配置 ──
|
||||
APP_ID = "cli_a929ae22e0b8dcc8"
|
||||
APP_SECRET = "OtFjMy7p3qE3VvLbMdcWidwgHOnGD4FJ"
|
||||
SPREADSHEET_TOKEN = "NoZqsFi47hIOHEt9j8WcfRtbnug"
|
||||
DIRECT_SHEET = "1sosYE"
|
||||
|
||||
HEADER = [
|
||||
"销售归属", "微信昵称", "进线日期", "体验节数", "手机号", "用户年级",
|
||||
"课史/跟进", "用户ID", "注册日期", "下载渠道", "是否下单", "下单日期",
|
||||
"成交渠道", "产品", "下单金额", "退款金额", "实际收入", "激活课程",
|
||||
"当前行课进度", "最近行课时间", "累计学习时长", "更新时间", "渠道归属", "有效成单"
|
||||
]
|
||||
|
||||
|
||||
def _get_pg_password():
|
||||
secrets_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "secrets.env")
|
||||
with open(secrets_path) as f:
|
||||
for line in f:
|
||||
if line.startswith("PG_ONLINE_PASSWORD="):
|
||||
return line.strip().split("=", 1)[1].strip('"').strip("'")
|
||||
raise RuntimeError("PG_ONLINE_PASSWORD not found in secrets.env")
|
||||
|
||||
|
||||
PG_CONFIG = {
|
||||
"host": "bj-postgres-16pob4sg.sql.tencentcdb.com", "port": 28591,
|
||||
"user": "ai_member", "password": _get_pg_password(), "database": "vala_bi",
|
||||
}
|
||||
|
||||
|
||||
def get_token():
|
||||
r = requests.post("https://open.feishu.cn/open-apis/auth/v3/tenant_access_token/internal",
|
||||
json={"app_id": APP_ID, "app_secret": APP_SECRET}, timeout=15)
|
||||
return r.json()["tenant_access_token"]
|
||||
|
||||
|
||||
def put_values(token, sheet_id, range_str, values, retries=3):
|
||||
url = f"https://open.feishu.cn/open-apis/sheets/v2/spreadsheets/{SPREADSHEET_TOKEN}/values"
|
||||
body = {"valueRange": {"range": f"{sheet_id}!{range_str}", "values": values}}
|
||||
for attempt in range(retries):
|
||||
r = requests.put(url, headers={
|
||||
"Authorization": f"Bearer {token}",
|
||||
"Content-Type": "application/json"
|
||||
}, json=body, timeout=30)
|
||||
result = r.json()
|
||||
if result.get("code") == 0:
|
||||
return True
|
||||
time.sleep(1)
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
print(f"[{datetime.now():%Y-%m-%d %H:%M:%S}] 直购表刷新 启动")
|
||||
|
||||
# ── Step 1: DB 查询 ──
|
||||
conn = psycopg2.connect(**PG_CONFIG, connect_timeout=30)
|
||||
cur = conn.cursor()
|
||||
|
||||
cur.execute("""
|
||||
SELECT
|
||||
o.account_id,
|
||||
a.name,
|
||||
a.tel,
|
||||
a.created_at,
|
||||
a.download_channel,
|
||||
o.pay_success_date,
|
||||
o.key_from,
|
||||
o.goods_name,
|
||||
o.pay_amount_int,
|
||||
o.order_status,
|
||||
COALESCE(r.refund_amount_int, 0) as refund_amount
|
||||
FROM bi_vala_order o
|
||||
JOIN bi_vala_app_account a ON o.account_id = a.id AND a.status = 1
|
||||
LEFT JOIN bi_refund_order r ON o.trade_no = r.trade_no AND r.status = 3
|
||||
WHERE o.pay_success_date IS NOT NULL
|
||||
AND o.order_status IN (3,4)
|
||||
AND (o.key_from LIKE '%dianpu-xhs%' OR o.key_from LIKE '%stream-xhs%')
|
||||
ORDER BY o.pay_success_date DESC
|
||||
""")
|
||||
orders = cur.fetchall()
|
||||
|
||||
# Dedup by account_id
|
||||
seen = set()
|
||||
unique_orders = []
|
||||
for o in orders:
|
||||
if o[0] not in seen:
|
||||
seen.add(o[0])
|
||||
unique_orders.append(o)
|
||||
|
||||
uids = [o[0] for o in unique_orders]
|
||||
|
||||
# 体验节数
|
||||
cur.execute("""
|
||||
SELECT account_id, COUNT(*) FROM bi_user_course_detail
|
||||
WHERE account_id = ANY(%s) AND expire_time IS NULL AND deleted_at IS NULL
|
||||
GROUP BY account_id
|
||||
""", (uids,))
|
||||
exp_map = {r[0]: r[1] for r in cur.fetchall()}
|
||||
|
||||
# 激活课程
|
||||
cur.execute("""
|
||||
SELECT DISTINCT ON (account_id) account_id, course_level
|
||||
FROM bi_user_course_detail
|
||||
WHERE account_id = ANY(%s) AND expire_time IS NOT NULL AND deleted_at IS NULL
|
||||
ORDER BY account_id, expire_time DESC
|
||||
""", (uids,))
|
||||
course_map = {r[0]: r[1] for r in cur.fetchall()}
|
||||
|
||||
# 学习进度
|
||||
cur.execute("""
|
||||
SELECT DISTINCT ON (c.user_id) a.id, blul.course_level, blul.course_season,
|
||||
blul.course_unit, blul.course_lesson, c.created_at
|
||||
FROM bi_user_chapter_play_record_0 c
|
||||
JOIN bi_vala_app_character ch ON c.user_id = ch.id
|
||||
JOIN bi_vala_app_account a ON ch.account_id = a.id
|
||||
JOIN bi_level_unit_lesson blul ON c.chapter_id = blul.id
|
||||
WHERE a.id = ANY(%s) AND c.play_status = 1
|
||||
ORDER BY c.user_id, c.created_at DESC
|
||||
""", (uids,))
|
||||
study_map = {}
|
||||
study_time_map = {}
|
||||
for r in cur.fetchall():
|
||||
aid = r[0]
|
||||
if aid not in study_map:
|
||||
study_map[aid] = f"{r[1]}-{r[2]}-{r[3]}-{r[4]}"
|
||||
study_time_map[aid] = str(r[5])[:19]
|
||||
|
||||
# 学习时长
|
||||
cur.execute("""
|
||||
SELECT a.id, SUM(comp.interval_time)
|
||||
FROM bi_user_chapter_play_record_0 c
|
||||
JOIN bi_vala_app_character ch ON c.user_id = ch.id
|
||||
JOIN bi_vala_app_account a ON ch.account_id = a.id
|
||||
JOIN bi_user_component_play_record_0 comp ON c.chapter_unique_id = comp.chapter_unique_id
|
||||
WHERE a.id = ANY(%s) AND c.play_status = 1
|
||||
GROUP BY a.id
|
||||
""", (uids,))
|
||||
time_map = {r[0]: r[1] for r in cur.fetchall()}
|
||||
|
||||
cur.close()
|
||||
conn.close()
|
||||
|
||||
# ── Step 2: 构建行 ──
|
||||
now_str = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||
new_rows = []
|
||||
|
||||
for o in unique_orders:
|
||||
uid = o[0]
|
||||
name = o[1] or ""
|
||||
tel = o[2] or ""
|
||||
reg = str(o[3])[:10] if o[3] else ""
|
||||
dl = o[4] or ""
|
||||
l_date_raw = o[5]
|
||||
l_date = str(l_date_raw)[:10] if l_date_raw else ""
|
||||
key_from = o[6]
|
||||
goods = o[7].strip() if o[7] else ""
|
||||
amount = o[8] / 100
|
||||
refund = o[10] / 100 if o[10] else 0
|
||||
|
||||
# 全额退跳过
|
||||
if refund > 0 and refund >= amount:
|
||||
continue
|
||||
|
||||
exp = exp_map.get(uid, 0)
|
||||
course = course_map.get(uid, "")
|
||||
study = study_map.get(uid, "")
|
||||
last_study = study_time_map.get(uid, "")
|
||||
total_min = round(time_map.get(uid, 0) / 60000, 1) if time_map.get(uid, 0) else ""
|
||||
gsv = amount - refund
|
||||
|
||||
row = [
|
||||
"直购", name, "", exp, tel, "", "", uid, reg, dl,
|
||||
"是", l_date, key_from, goods, amount,
|
||||
refund if refund > 0 else "", gsv, course, study,
|
||||
last_study, total_min, now_str, "直购", 1,
|
||||
]
|
||||
new_rows.append(row)
|
||||
|
||||
# 按下单日期降序
|
||||
new_rows.sort(key=lambda r: r[11], reverse=True)
|
||||
|
||||
print(f"有效直购订单: {len(new_rows)}")
|
||||
|
||||
# ── Step 3: 写入飞书 ──
|
||||
token = get_token()
|
||||
|
||||
# 写表头
|
||||
put_values(token, DIRECT_SHEET, "A1:X1", [HEADER])
|
||||
|
||||
# 写数据
|
||||
for batch_start in range(0, len(new_rows), 20):
|
||||
batch = new_rows[batch_start:batch_start + 20]
|
||||
sr = 2 + batch_start
|
||||
er = sr + len(batch) - 1
|
||||
put_values(token, DIRECT_SHEET, f"A{sr}:X{er}", batch)
|
||||
time.sleep(0.3)
|
||||
|
||||
# 清除多余旧行
|
||||
total = len(new_rows)
|
||||
clear_start = 2 + total
|
||||
empty = [[""] * 24 for _ in range(50)]
|
||||
put_values(token, DIRECT_SHEET, f"A{clear_start}:X{clear_start + 49}", empty)
|
||||
|
||||
print(f"[{datetime.now():%Y-%m-%d %H:%M:%S}] ✅ 直购表刷新完成 ({total} 行)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@ -18,6 +18,7 @@ APP_ID = "cli_a929ae22e0b8dcc8"
|
||||
APP_SECRET = "OtFjMy7p3qE3VvLbMdcWidwgHOnGD4FJ"
|
||||
SPREADSHEET_TOKEN = "NoZqsFi47hIOHEt9j8WcfRtbnug"
|
||||
SALES_SHEETS = {"f975f0": "吴迪", "qJF4I": "小龙", "qJF4J": "成都"}
|
||||
DIRECT_SHEET = "1sosYE" # 直购表（小红书店铺+stream-xhs，不依赖手机号匹配）
|
||||
SUMMARY_SHEET = "2smjwA"
|
||||
|
||||
def _get_pg_password():
|
||||
@ -186,6 +187,24 @@ def main():
|
||||
|
||||
print(f"Total rows: {len(all_rows)}")
|
||||
|
||||
# ── Step 1.5: 读取直购表 ──
|
||||
print(f"Reading 直购...")
|
||||
direct_vals = read_sheet(token, DIRECT_SHEET, "A2:V10000")
|
||||
direct_rows = []
|
||||
for i, row in enumerate(direct_vals):
|
||||
while len(row) < 22:
|
||||
row.append("")
|
||||
b = str(row[1]).strip() if row[1] else ""
|
||||
h = str(row[7]).strip() if row[7] else ""
|
||||
if b or h:
|
||||
direct_rows.append({
|
||||
"sid": DIRECT_SHEET, "name": "直购", "row": i + 2,
|
||||
"raw": row[:22],
|
||||
})
|
||||
print(f" {len(direct_rows)} non-empty rows")
|
||||
all_rows.extend(direct_rows)
|
||||
print(f"Total rows (with 直购): {len(all_rows)}")
|
||||
|
||||
# ── Step 2: 筛选进订单汇总的行 ──
|
||||
# 条件：K=是 · O>0 · 非全额退(P空或P<O) · L≥C
|
||||
order_rows = []
|
||||
@ -212,12 +231,12 @@ def main():
|
||||
# 全额退 → 不进订单表
|
||||
continue
|
||||
|
||||
# L ≥ C
|
||||
# L ≥ C (C为空时跳过此检查，如直购用户无进线日期)
|
||||
c_str = str(raw[2]).strip() if len(raw) > 2 and raw[2] else ""
|
||||
l_str = str(raw[11]).strip() if len(raw) > 11 and raw[11] else ""
|
||||
c_date = parse_date(c_str)
|
||||
l_date = parse_date(l_str)
|
||||
if not date_le(c_date, l_date):
|
||||
if c_date is not None and not date_le(c_date, l_date):
|
||||
continue
|
||||
|
||||
# 通过所有条件
|
||||
@ -225,6 +244,26 @@ def main():
|
||||
|
||||
print(f"Order rows after filter: {len(order_rows)}")
|
||||
|
||||
# ── Step 2.5: 去重（同一人可能在三表+直购中出现多次）──
|
||||
# 按 (A销售归属, B微信昵称, O下单金额, P退款金额, L下单日期) 去重
|
||||
seen = set()
|
||||
deduped = []
|
||||
for r in order_rows:
|
||||
raw = r["raw"]
|
||||
a = str(raw[0]).strip() if raw[0] else ""
|
||||
b = str(raw[1]).strip() if len(raw) > 1 and raw[1] else ""
|
||||
o = str(raw[14]).strip() if len(raw) > 14 and raw[14] else ""
|
||||
p = str(raw[15]).strip() if len(raw) > 15 and raw[15] else ""
|
||||
l = str(raw[11]).strip() if len(raw) > 11 and raw[11] else ""
|
||||
key = (a, b, o, p, l)
|
||||
if key not in seen:
|
||||
seen.add(key)
|
||||
deduped.append(r)
|
||||
dup_count = len(order_rows) - len(deduped)
|
||||
if dup_count > 0:
|
||||
print(f" Removed {dup_count} duplicate rows")
|
||||
order_rows = deduped
|
||||
|
||||
# ── Step 3: 按 L 下单日降序 ──
|
||||
order_rows.sort(key=lambda r: str(r["raw"][11]) if len(r["raw"]) > 11 and r["raw"][11] else "", reverse=True)
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user