diff --git a/.vala_skill_hashes b/.vala_skill_hashes index 8170083..daa3db8 100644 --- a/.vala_skill_hashes +++ b/.vala_skill_hashes @@ -13,4 +13,4 @@ refund-user-learning-analysis 648fd4ae2b29167fd66eab4245bdaaef00242db3131f4919cc vala-component-practice-stat 8e768e2641019d27bd41f4647d2d90f24182a0554dad5ad9f4136e9ce0bae147 phone-chapter-query a28b6bac101d422a5b4f2d0124ada48a14fb9a737da680d5de5501dba4c6b421 vala-order-amortization-stat c2ba3c2a82cf0c0a43ba9bbb7b2e16b62120f4fe00026212dc04ae4fd45d32ed -welfare-user-list ba2bb1f5bf5a55bfdf852689d90a8d93509dfed2ed74a84226fa5527176fbe28 +welfare-user-list f8d3a56c5d4b8358be260249d9d292d636f5c296f54cf94a3cd7aba967ae3ce1 diff --git a/MEMORY.md b/MEMORY.md index 28fcb4a..f22a1da 100644 --- a/MEMORY.md +++ b/MEMORY.md @@ -126,19 +126,20 @@ - **核心业务指标口径定义：** - **测试账号剔除规则（所有订单统计前置校验）：** 计算订单数、GMV、GSV、退费金额、退费率等所有订单相关指标时，必须关联`bi_vala_app_account`表（关联逻辑：`bi_vala_order.account_id = bi_vala_app_account.id`），仅保留`bi_vala_app_account.status = 1`的非测试账号订单，自动剔除所有`status≠1`的测试账号订单。 - GMV：全部营销金额，包含退费金额，不剔除退费 - - GSV：实际收入，为GMV剔除退费金额后的金额 + - GSV：实际收入，为GMV剔除退费金额后的金额（退费金额按上述退费订单校验规则累加计算） - 退费率： - 单日退费率：当日成交的订单中，发生退费的订单数占当日总成交订单数的比例（退费订单不限定退费时间，只要对应订单是当日成交的即计入） - 时间段/整体退费率：同口径，统计时间段内成交的订单中发生退费的订单数占该时间段总成交订单数的比例 - - **退费订单校验规则：** 统计退费订单时必须同时满足两个条件： - 1. `bi_refund_order` 表中 `status = 3`（退费成功） - 2. `bi_vala_order` 表中 `order_status = 4`（订单状态为已退款） - 两个条件缺一不可，避免统计错误。 + - **退费订单校验规则（[李承龙确认] 2026-06-18 更新）：** 一笔订单可能存在多次退费，需分情况处理： + - **情况一：`bi_vala_order.order_status = 4`（已退款）** → 直接视为退费订单 + - **情况二：`bi_vala_order.order_status = 3`（已完成）** → 必须在 `bi_refund_order` 表中存在 `status = 3` 的退费记录才视为退费订单 + - **退费金额计算（重要）：** 无论哪种情况，退费金额都必须对同一笔订单在 `bi_refund_order` 表中的**所有** `status = 3` 退费记录做 `SUM(refund_amount)` 累加，而非只取单笔。因为一笔订单可能对应多笔退费记录。 + - **关联方式：** `bi_vala_order.trade_no = bi_refund_order.trade_no` 或 `bi_vala_order.out_trade_no = bi_refund_order.out_trade_no` - **转化率 / 7日转化率 / 14日转化率（端内注册转付费，[李承龙确认] 2026-05-11）：** - **转化率 = 端内付费用户数 / 注册用户数 × 100%** - **分母：** 按注册日期（`bi_vala_app_account.created_at`）分组，`status=1` 且 `deleted_at IS NULL` 的非测试、未删除账号 - **分子（含退费）：** 分母用户中，在端内（`key_from IN ('app-active-h5-0-0', 'app-sales-bj-qhm-0', 'app-sales-bj-wd-0')`）有支付成功订单的去重用户数 - - **分子（剔除退费）：** 同上，但仅剔除端内订单**全部被退费**的用户——即只要用户还有任何一笔未退费的端内订单就保留（退费判定：`bi_refund_order.status=3` 且 `bi_vala_order.order_status=4`） + - **分子（剔除退费）：** 同上，但仅剔除端内订单**全部被退费**的用户——即只要用户还有任何一笔未退费的端内订单就保留（退费判定：按上述退费订单校验规则，`order_status=4` 直接判定退费，`order_status=3` 需匹配退费表中 `status=3` 的记录） - **订单状态限定：** 端内订单筛选 `order_status IN (3, 4)`，即已完成或已退款 - **时间基准：** 按用户注册日期分组，不限制订单发生时间（7日/14日除外） - **订单时间字段：** `pay_success_date`（支付成功时间） @@ -173,7 +174,7 @@ 2. 指标说明： - 订单数：符合条件的订单总数量 - GMV：符合条件的订单`pay_amount_int`求和/100（单位：元） - - GSV：GMV 减去符合条件的订单中已完成退费的金额总和（单位：元） + - GSV：GMV 减去符合条件的订单中已完成退费的金额总和（单位：元，退费金额按退费订单校验规则对多笔退费记录累加） - 退费率：符合条件的订单中已完成退费的订单数 / 订单总数量 * 100%，保留1位小数 - **渠道映射规则（key_from字段匹配）：** - 端内购买：`app-active-h5-0-0`、`app-sales-bj-qhm-0`、`app-sales-bj-wd-0`（三个值匹配任意一个即属于端内购买） diff --git a/SKILL_REGISTRY.md b/SKILL_REGISTRY.md index 37f47cd..c1158f6 100644 --- a/SKILL_REGISTRY.md +++ b/SKILL_REGISTRY.md @@ -22,6 +22,7 @@ - **创建时间：** 2026-05-06 - **变更记录：** - 2026-05-28 | 计税时机改为「下单即计税」、退后订单历史累计摊销仅统计账期内退费订单 | 李承龙 + - 2026-06-18 | 退费规则更新：同时按 out_trade_no 和 trade_no 关联退费表、退费金额 SUM 累加、order_status=4 直接视为退费、order_status=3 需退费表有记录才视为退费 | 李承龙 ### welfare-user-list - **创建来源：** 李承龙（`ou_e63ce6b760ad39382852472f28fbe2a2`） @@ -219,6 +220,9 @@ | `rewrite_daily_report_formulas.py` | 来源不可追溯 | 重写日报公式 | 2026-06-02 | | `style_sheets.py` | 来源不可追溯 | 样式 Sheet | 2026-06-01 | | `batch_update_sheet.py` | 来源不可追溯 | 批量更新 Sheet | 2026-05-23 | +| `finance_orders_refresh.py` | 李承龙（`ou_e63ce6b760ad39382852472f28fbe2a2`） | 财务口径订单刷新，从订单汇总 2smjwA 出发，逐行匹配全量订单并写入财务 tab 2hSLSg | 2026-06-05 | +| **变更记录：** | | | | +| 2026-06-18 | 退费规则更新：同时按 trade_no 和 out_trade_no 匹配退费表、退费金额 SUM 累加、order_status=4 直接视为退费 | 李承龙 | | | ### 定时/备份/运维 | 脚本 | 创建来源 | 需求描述 | 创建时间 | diff --git a/memory/.dreams/short-term-recall.json b/memory/.dreams/short-term-recall.json index 5af5d2a..8a6e674 100644 --- a/memory/.dreams/short-term-recall.json +++ b/memory/.dreams/short-term-recall.json @@ -1,6 +1,6 @@ { "version": 1, - "updatedAt": "2026-06-16T04:37:44.473Z", + "updatedAt": "2026-06-18T09:12:10.290Z", "entries": { "memory:memory/2026-05-06.md:1:20": { "key": "memory:memory/2026-05-06.md:1:20", @@ -639,18 +639,20 @@ "endLine": 38, "source": "memory", "snippet": "| 6 | 小乖大人 | 16158 | 13944890221 | 3/1 | 无订单 | - | 0 | 0 | DB无任何订单 | | 7 | 潘潘 | 16150 | 18610935696 | 3/1 | 无订单 | - | 0 | 0 | DB无任何订单 | | 8 | 张滢ya | 17894 | 13799768340 | 3/7 | 无订单 | - | 0 | 0 | DB无任何订单 | | 9 | sallywu | 17816 | 15998103065 | 3/7 | 无订单 | - | 0 | 0 | DB无任何订单 | | 10 | 🦁萨摩 | 21858 | 13685553716 | 3/8 | 4/8 | ✓ | 1999 | 1999 | 达人-学霸三人行 | ### Group B (有手机，4笔) - phone_encrypt查UID | # | 昵称 | 手机 | 加密结果 | DB匹配 | |---|------|------|---------|--------| | 11 | 潘提提 | 13427741613 | IiShdIaiY1oy7B_Xn4EH3g.. | 无匹配 | | 12 | 狸小路 | 18622850293 | YPAQ-740vKwxroqZGkeGyQ.. | 无匹配 | | 13 | 希小希 | 18086665321 | c8zfpqBrN1nikMkwAj64aQ.. | 无匹配 | | 14 | 曼 | 13520255515 | NBVtGuxEge7f7hdkyK3y7Q.. | 无匹配", - "recallCount": 1, + "recallCount": 2, "dailyCount": 0, "groundedCount": 0, - "totalScore": 1, + "totalScore": 2, "maxScore": 1, "firstRecalledAt": "2026-06-14T06:58:06.164Z", - "lastRecalledAt": "2026-06-14T06:58:06.164Z", + "lastRecalledAt": "2026-06-18T09:12:10.290Z", "queryHashes": [ - "6769ba9ebb36" + "6769ba9ebb36", + "3f85aec3e063" ], "recallDays": [ - "2026-06-14" + "2026-06-14", + "2026-06-18" ], "conceptTags": [ "3/1", @@ -670,22 +672,24 @@ "endLine": 56, "source": "memory", "snippet": "| 15 | Rachel | 3/5 | 10994 | 13510564547 | 3/7 | ✓ | sales-adp-bj-jxl-0, GMV=1999 | | 16 | soul | 3/2 | 17387 | 15640464255 | 3/12 | ✓ | sales-adp-bj-jxl-0, GMV=1999 | | 17 | 红 | 3/7 | 17025 | 13533955004 | 3/14 | ✓ | sales-adp-bj-jxl-0, GMV=1999 | | 18 | 一笑轩渠 | 3/8 | 17425 | 15017528458 | 3/11 | ✓ | sales-adp-bj-jxl-0, GMV=1999 | | 19 | 蜗牛 | 3/2 | ❓ | ❓ | ❓ | - | 晚柠5/15订单数百笔，无手机/UID无法定位 | | 20 | c_瑶 | 3/6 | ❓ | ❓ | ❓ | - | \"直购\"渠道DB不存在，3/14无3998/1999匹配 | ### 关键发现 1. Group A 中 7/10 用户 DB 中无任何订单（pre汇总有GMV但DB不存在） 2. Group B 4个手机号全部未注册（H=未注册 确认正确） 3. Group C #15-#18 4笔 jxl-0 均 L≥C，pre怀疑#18 L 2 else None + date_str = excel_serial_to_md(date_serial) if date_serial else "" + # E: 微信昵称 (col 4) + nickname = str(row[4]).strip() if len(row) > 4 and row[4] else "" + # F: 手机号 (col 5) + phone = "" + if len(row) > 5 and row[5]: + try: + phone = str(int(float(row[5]))) + except: + phone = str(row[5]).strip() + # G: 孩子年龄/年级 (col 6) + grade = str(row[6]).strip() if len(row) > 6 and row[6] else "" + # H: 英语基础和在学课程 (col 7) → 课史/跟进 + followup = str(row[7]).strip() if len(row) > 7 and row[7] else "" + # N: 用户ID (col 13) - LP 自带的 + lp_uid = "" + if len(row) > 13 and row[13]: + try: + lp_uid = str(int(float(row[13]))) + except: + pass + # T: U0行课进度 (col 18) → 体验节数 + u0_progress = str(row[18]).strip() if len(row) > 18 and row[18] else "" + exp_lessons = u0_progress if u0_progress else "" + + if not phone or len(phone) != 11: + continue # skip rows without valid phone + + all_leads.append({ + "sales": sales_name, + "nickname": nickname, + "date_str": date_str, + "date_serial": date_serial, + "exp_lessons": exp_lessons, + "phone": phone, + "grade": grade, + "followup": followup, + "lp_uid": lp_uid, + }) + log(f" {sales_name}: {len([l for l in all_leads if l['sales']==sales_name])} valid leads") + log(f" 共 {len(all_leads)} 条有效线索") + return all_leads + +# ── Step 2: DB 批量查询 ── +def query_db(conn, leads): + """查询注册信息、订单、退款、行课""" + cur = conn.cursor() + + # 加密所有手机号 + phone_enc_map = {} + for lead in leads: + enc = encrypt_phone(lead["phone"]) + phone_enc_map[enc] = lead["phone"] + lead["tel_encrypt"] = enc + + enc_list = list(phone_enc_map.keys()) + + # 2a. 注册信息 + log(" 查询注册信息...") + reg_info = batch_in(cur, + "SELECT id, tel_encrypt, created_at, download_channel FROM bi_vala_app_account WHERE tel_encrypt IN (%s) AND status=1 AND deleted_at IS NULL", + enc_list + ) + tel_to_account = {} + for aid, tel_enc, created_at, dc in reg_info: + tel_to_account[tel_enc] = { + "account_id": aid, + "reg_date": created_at.strftime("%Y-%m-%d") if created_at else "", + "download_channel": dc or "", + } + + # 填充到 leads + account_ids = set() + for lead in leads: + acc = tel_to_account.get(lead["tel_encrypt"], {}) + lead["account_id"] = acc.get("account_id") + lead["reg_date"] = acc.get("reg_date", "") + lead["download_channel"] = acc.get("download_channel", "") + if lead["account_id"]: + account_ids.add(lead["account_id"]) + + aid_list = list(account_ids) + log(f" 匹配到 account_id: {len(aid_list)}") + + # 2b. 订单信息 + log(" 查询订单信息...") + lead["orders"] = [] + lead["valid_order"] = None + if aid_list: + orders = batch_in(cur, + "SELECT account_id, trade_no, pay_success_date, key_from, goods_id, pay_amount_int, order_status FROM bi_vala_order WHERE account_id IN (%s) AND pay_success_date IS NOT NULL AND order_status IN (3,4) ORDER BY pay_success_date DESC", + aid_list + ) + # 按 account_id 分组 + aid_orders = defaultdict(list) + for o in orders: + aid_orders[o[0]].append(o) + + # 退款 + trade_nos = [o[1] for o in orders if o[1]] + refund_map = defaultdict(int) + if trade_nos: + refunds = batch_in(cur, + "SELECT trade_no, refund_amount_int FROM bi_refund_order WHERE trade_no IN (%s) AND status=3", + trade_nos + ) + for tn, amt in refunds: + refund_map[tn] += amt # SUM 多笔退费 + + # 为每个 account 找有效订单 + for lead in leads: + aid = lead.get("account_id") + if not aid: + continue + olist = aid_orders.get(aid, []) + lead["all_orders"] = olist + + # 找有效订单: GSV>0, 非全额退, 下单日期≥进线日期 + lead_date = excel_serial_to_date(lead.get("date_serial")) + for o in olist: + trade_no = o[1] + pay_date = o[2] + key_from = o[3] + goods_id = o[4] + gmv = o[5] + order_status = o[6] + + total_refund = refund_map.get(trade_no, 0) + gsv = gmv - total_refund + + if gsv <= 0: + continue # 全额退或 GSV≤0 + if gmv == total_refund: + continue # 全额退 + + pay_date_str = pay_date.strftime("%Y-%m-%d") if pay_date else "" + if lead_date and pay_date_str < lead_date: + continue # 下单早于进线 + + lead["valid_order"] = { + "trade_no": trade_no, + "pay_date": pay_date_str, + "pay_date_md": f"{pay_date.month}月{pay_date.day}日" if pay_date else "", + "key_from": key_from or "", + "goods_id": goods_id, + "gmv": gmv / 100.0, + "refund": total_refund / 100.0, + "gsv": gsv / 100.0, + "product": GOODS_NAMES.get(goods_id, f"商品{goods_id}"), + "channel_class": classify_sales_channel(key_from), + } + break # 取第一个符合条件的（最新） + + # 2c. 激活课程 + log(" 查询激活课程...") + if aid_list: + try: + activations = batch_in(cur, + "SELECT account_id, season_package_level FROM bi_vala_seasonal_ticket WHERE account_id IN (%s) AND status=1 AND deleted_at IS NULL AND season_package_level IN ('A1','A2')", + aid_list + ) + aid_activation = {} + for aid, lvl in activations: + aid_activation[aid] = lvl + for lead in leads: + aid = lead.get("account_id") + if aid: + lead["activation"] = aid_activation.get(aid, "") + else: + lead["activation"] = "" + except Exception as e: + log(f" 激活查询异常: {e}") + for lead in leads: + lead["activation"] = "" + + # 2d. 角色 + 行课 + log(" 查询角色信息...") + lead["lesson_progress"] = "" + lead["lesson_time"] = "" + lead["lesson_minutes"] = 0 + if aid_list: + char_info = batch_in(cur, + "SELECT account_id, id FROM bi_vala_app_character WHERE account_id IN (%s) AND deleted_at IS NULL", + aid_list + ) + account_chars = defaultdict(list) + char_to_account = {} + for aid, cid in char_info: + account_chars[aid].append(cid) + char_to_account[cid] = aid + char_ids = list(char_to_account.keys()) + log(f" 角色数: {len(char_ids)}") + + # 课程映射 + cur.execute("SELECT id, course_level, course_season, course_unit, course_lesson FROM bi_level_unit_lesson") + chapter_map = {} + for ch_id, cl, cs, cu, cl2 in cur.fetchall(): + chapter_map[ch_id] = (cl or "", cs or "", cu or "", cl2 or "") + + # 课时完成记录 + log(" 查询课时完成记录...") + char_plays = defaultdict(lambda: {"latest_time": None, "latest_chapter": None}) + for tbl_idx in range(8): + table = f"bi_user_chapter_play_record_{tbl_idx}" + try: + cur.execute( + f"SELECT user_id, chapter_id, created_at FROM {table} WHERE play_status=1 AND deleted_at IS NULL AND user_id = ANY(%s)", + (char_ids,) + ) + for uid, ch_id, created_at in cur.fetchall(): + ch_data = chapter_map.get(ch_id) + if not ch_data: + continue + rec = char_plays[uid] + if rec["latest_time"] is None or created_at > rec["latest_time"]: + rec["latest_time"] = created_at + rec["latest_chapter"] = ch_data + except Exception as e: + log(f" 警告 {table}: {e}") + + # 学习总耗时 + log(" 查询学习耗时...") + for tbl_idx in range(8): + table = f"bi_user_component_play_record_{tbl_idx}" + try: + cur.execute( + f"SELECT user_id, SUM(COALESCE(interval_time,0)) FROM {table} WHERE user_id = ANY(%s) AND deleted_at IS NULL GROUP BY user_id", + (char_ids,) + ) + for uid, total_ms in cur.fetchall(): + if uid in char_plays: + char_plays[uid]["total_ms"] = char_plays[uid].get("total_ms", 0) + (total_ms or 0) + except Exception as e: + log(f" 警告 {table}: {e}") + + # 汇总到 account 级别 + for lead in leads: + aid = lead.get("account_id") + if not aid: + continue + chars = account_chars.get(aid, []) + best_time = None + best_ch = None + total_ms = 0 + for cid in chars: + play = char_plays.get(cid) + if not play: + continue + if play.get("latest_chapter"): + if best_time is None or play["latest_time"] > best_time: + best_time = play["latest_time"] + best_ch = play["latest_chapter"] + total_ms += play.get("total_ms", 0) + + if best_ch: + cl, cs, cu, cl2 = best_ch + lead["lesson_progress"] = f"{cl}-{cs}-{cu}-{cl2}" + if best_time: + lead["lesson_time"] = best_time.strftime("%Y-%m-%d") + lead["lesson_minutes"] = round(total_ms / 60000, 1) if total_ms > 0 else 0 + + cur.close() + log(" DB 查询完成") + return leads + +# ── Step 3: 写入目标表 ── +def write_target_sheet(token, leads): + """Clear A3:Z500, 写入所有线索行""" + log(" 写入 4koH9C...") + + # 先清空 A1 的迁移提示 + put_values(token, TARGET_SHEET, "A1:A1", [[""]]) + + # 恢复标准表头 r1 + header = [["销售归属", "微信昵称", "进线日期", "体验节数", "手机号", "用户年级", + "课史/跟进", "用户ID", "注册日期", "下载渠道", "下单日期", "成交渠道", + "产品", "下单金额(GMV)", "退款金额", "实际收入(GSV)", "激活课程", + "当前行课进度", "最近行课时间", "累计学习时长(min)", "更新时间", + "微伴补充", "进线早于下单", "订单号", "有效订单", "渠道归属"]] + put_values(token, TARGET_SHEET, "A1:Z1", header) + + # Clear r2 + put_values(token, TARGET_SHEET, "A2:Z2", [[""] * 26]) + + # 构建数据行 + update_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S") + rows = [] + for lead in leads: + vo = lead.get("valid_order") + row = [ + lead["sales"], # A: 销售归属 + lead["nickname"], # B: 微信昵称 + lead["date_str"], # C: 进线日期 + lead["exp_lessons"], # D: 体验节数 + lead["phone"], # E: 手机号 + lead["grade"], # F: 用户年级 + lead["followup"], # G: 课史/跟进 + lead.get("account_id", "") or "", # H: 用户ID + lead.get("reg_date", ""), # I: 注册日期 + lead.get("download_channel", ""), # J: 下载渠道 + vo["pay_date_md"] if vo else "", # K: 下单日期 + vo["key_from"] if vo else "", # L: 成交渠道 + vo["product"] if vo else "", # M: 产品 + vo["gmv"] if vo else "", # N: GMV + vo["refund"] if vo else "", # O: 退款金额 + vo["gsv"] if vo else "", # P: GSV + lead.get("activation", ""), # Q: 激活课程 + lead.get("lesson_progress", ""), # R: 当前行课进度 + lead.get("lesson_time", ""), # S: 最近行课时间 + lead.get("lesson_minutes", 0) or "", # T: 累计学习时长 + update_time, # U: 更新时间 + "", # V: 微伴补充 (不填) + "", # W: 进线早于下单 (Cursor 填) + vo["trade_no"] if vo else "", # X: 订单号 + 1 if vo else 0, # Y: 有效订单 + vo["channel_class"] if vo else "", # Z: 渠道归属 + ] + rows.append(row) + + total = len(rows) + log(f" 共 {total} 行，Y=1: {sum(1 for r in rows if r[24]==1)}") + + # 分批写入 (每批最多 20 行 × 26 列 = 520 格，远低于 4400) + for batch_start in range(0, total, 20): + batch = rows[batch_start:batch_start+20] + sr = 3 + batch_start + er = sr + len(batch) - 1 + put_values(token, TARGET_SHEET, f"A{sr}:Z{er}", batch) + time.sleep(0.3) + + # 清除多余旧行 + if total < 498: + clear_start = 3 + total + clear_end = 500 + empty_rows = [[""] * 26 for _ in range(clear_end - clear_start + 1)] + put_values(token, TARGET_SHEET, f"A{clear_start}:Z{clear_end}", empty_rows) + log(f" 清除多余行 A{clear_start}:Z{clear_end}") + + log(f" 写入完成") + +# ── Main ── +def main(): + log("=" * 50) + log("北京试点线索刷新 启动") + + try: + token = get_fs_token() + conn = psycopg2.connect( + host=PG_HOST, port=PG_PORT, user=PG_USER, + password=get_pg_password(), dbname=PG_DB, connect_timeout=30 + ) + + # Step 1: 解析 LP 表 + log("Step 1: 解析 LP 表") + leads = parse_lp_sheets(token) + + # Step 2: DB 查询 + log("Step 2: DB 查询") diff --git a/scripts/finance_orders_refresh.py b/scripts/finance_orders_refresh.py index e24a1b6..bb3ce2f 100644 --- a/scripts/finance_orders_refresh.py +++ b/scripts/finance_orders_refresh.py @@ -207,23 +207,34 @@ def query_all_orders(uid_set): log(" 查询全量订单...") orders = batch_in(cur, - "SELECT account_id, trade_no, pay_success_date, key_from, goods_id, " + "SELECT account_id, trade_no, out_trade_no, pay_success_date, key_from, goods_id, " "pay_amount_int, order_status " "FROM bi_vala_order WHERE account_id IN (%s) AND pay_success_date IS NOT NULL " "ORDER BY pay_success_date DESC", uid_list ) + # 收集所有 trade_no 和 out_trade_no 用于退费匹配 trade_nos = [o[1] for o in orders if o[1]] + out_trade_nos = [o[2] for o in orders if o[2]] + all_order_nos = list(set(trade_nos + out_trade_nos)) refund_map = {} - if trade_nos: - refunds = batch_in(cur, - "SELECT trade_no, refund_amount_int FROM bi_refund_order " - "WHERE trade_no IN (%s) AND status=3", - trade_nos + if all_order_nos: + # 同时按 trade_no 和 out_trade_no 匹配退费记录，SUM 累加退费金额 [李承龙确认 2026-06-18] + refunds_by_trade = batch_in(cur, + "SELECT trade_no, SUM(refund_amount_int) FROM bi_refund_order " + "WHERE trade_no IN (%s) AND status=3 GROUP BY trade_no", + all_order_nos ) - for tn, amt in refunds: - refund_map[tn] = amt + for tn, amt in refunds_by_trade: + refund_map[tn] = refund_map.get(tn, 0) + (amt or 0) + refunds_by_out = batch_in(cur, + "SELECT out_trade_no, SUM(refund_amount_int) FROM bi_refund_order " + "WHERE out_trade_no IN (%s) AND status=3 GROUP BY out_trade_no", + all_order_nos + ) + for otn, amt in refunds_by_out: + refund_map[otn] = refund_map.get(otn, 0) + (amt or 0) cur.close() conn.close() @@ -232,11 +243,16 @@ def query_all_orders(uid_set): for o in orders: aid = o[0] tn = o[1] - gmv = o[5] / 100.0 - refund = refund_map.get(tn, 0) / 100.0 + otn = o[2] + gmv = o[6] / 100.0 + order_status = o[7] + # 退费金额：同时按 trade_no 和 out_trade_no 匹配，SUM 累加 [李承龙确认 2026-06-18] + refund = (refund_map.get(tn, 0) + refund_map.get(otn, 0)) / 100.0 + # 退费判定：order_status=4 直接视为退费；order_status=3 需退费表有记录 + is_refunded = (order_status == 4) or (order_status == 3 and refund > 0) gsv = gmv - refund - dt = o[2] + dt = o[3] order_date = f"{dt.month}月{dt.day}日 {dt.strftime('%H:%M:%S')}" if dt else "" order_date_raw = dt.strftime("%Y-%m-%d %H:%M:%S") if dt else "" @@ -244,12 +260,13 @@ def query_all_orders(uid_set): "trade_no": tn or "", "order_date": order_date, "order_date_raw": order_date_raw, - "key_from": o[3] or "", - "product": GOODS_NAMES.get(o[4], f"商品{o[4]}"), + "key_from": o[4] or "", + "product": GOODS_NAMES.get(o[5], f"商品{o[5]}"), "gmv": int(gmv), "refund": int(refund), "gsv": int(gsv), - "order_status": o[6], + "order_status": order_status, + "is_refunded": is_refunded, }) log(f" 全量订单: {sum(len(v) for v in uid_orders.values())} 条") @@ -312,7 +329,8 @@ def write_finance_sheet(token, entries, phone_to_uid, uid_orders): gmv_val = o["gmv"] refund_val = o["refund"] gsv_val = o["gsv"] - is_full_refund = (gmv_val > 0 and gmv_val == refund_val) + is_refunded = o.get("is_refunded", False) + is_full_refund = (gmv_val > 0 and gmv_val == refund_val) or (o["order_status"] == 4 and refund_val == 0) order_valid = (gsv_val > 0 and not is_full_refund) # 进线早于下单检查 diff --git a/scripts/full_refresh_sales.py b/scripts/full_refresh_sales.py index 5b4625d..50eb35b 100644 --- a/scripts/full_refresh_sales.py +++ b/scripts/full_refresh_sales.py @@ -203,16 +203,16 @@ def query_all_db(conn, all_entries): for o in orders: user_orders[o[0]].append(o) - # 退款 + # 退款（SUM 聚合，修复多笔退费覆盖 bug） trade_nos = [o[1] for o in orders if o[1]] - refund_map = {} + refund_map = defaultdict(int) if trade_nos: refunds = batch_in(cur, "SELECT trade_no, refund_amount_int FROM bi_refund_order WHERE trade_no IN (%s) AND status=3", trade_nos ) for tn, amt in refunds: - refund_map[tn] = amt + refund_map[tn] += amt for aid, olist in user_orders.items(): if aid not in info: continue @@ -229,6 +229,7 @@ def query_all_db(conn, all_entries): info[aid]["refund"] = total_refund info[aid]["gsv"] = total_gmv - total_refund info[aid]["is_paid"] = True + info[aid]["channel_class"] = classify_sales_channel(latest[3]) # 2c. 激活课程 log(" 查询激活课程...") @@ -368,58 +369,64 @@ def write_sales_sheets(token, all_entries, db_info): d_vals = [] # 体验节数 i_vals = [] # 注册日期 j_vals = [] # 下载渠道 - k_vals = [] # 是否下单 - l_vals = [] # 下单日期 - m_vals = [] # 成交渠道 - n_vals = [] # 产品 - o_vals = [] # GMV - p_vals = [] # 退款 - q_vals = [] # GSV - r_vals = [] # 激活课程 - s_vals = [] # 行课进度 - t_vals = [] # 最近行课 - u_vals = [] # 学习时长 - v_vals = [] # 更新时间 + k_vals = [] # 下单日期 + l_vals = [] # 成交渠道 + m_vals = [] # 产品 + n_vals = [] # GMV + o_vals = [] # 退款 + p_vals = [] # GSV + q_vals = [] # 激活课程 + r_vals = [] # 行课进度 + s_vals = [] # 最近行课 + t_vals = [] # 学习时长 + u_vals = [] # 更新时间 + x_vals = [] # 有效订单 0/1 + y_vals = [] # 渠道归属 for item in g: uid = item["uid"] aid = int(uid) if uid and uid.isdigit() and int(uid) > 0 else 0 if aid > 0 and aid in db_info: di = db_info[aid] - # 体验节数：用 max_lesson 换算 trial_count = di["max_lesson"] d_vals.append([trial_count if trial_count > 0 else ""]) i_vals.append([di["reg_date"]]) j_vals.append([di["download_channel"]]) - k_vals.append([di["has_order"]]) - l_vals.append([di["order_date"]]) - m_vals.append([di["order_channel"]]) - n_vals.append([di["product"] if di["has_order"] == "是" else ""]) - o_vals.append([int(di["gmv"]) if di["gmv"] > 0 else ""]) - p_vals.append([int(di["refund"]) if di["refund"] > 0 else ""]) - q_vals.append([int(di["gsv"]) if di["gsv"] > 0 else ""]) - # 激活课程 + k_vals.append([di["order_date"]]) # 下单日期 + l_vals.append([di["order_channel"]]) + m_vals.append([di["product"] if di["has_order"] == "是" else ""]) + n_vals.append([int(di["gmv"]) if di["gmv"] > 0 else ""]) + o_vals.append([int(di["refund"]) if di["refund"] > 0 else ""]) + p_vals.append([int(di["gsv"]) if di["gsv"] > 0 else ""]) act = di["activation"] if act: - r_vals.append([f"{act}体验课" if act in ("A1","A2") else act]) + q_vals.append([f"{act}体验课" if act in ("A1","A2") else act]) else: - r_vals.append([""]) - s_vals.append([di["lesson_progress"]]) - t_vals.append([di["lesson_time"]]) - u_vals.append([di["lesson_minutes"]]) + q_vals.append([""]) + r_vals.append([di["lesson_progress"]]) + s_vals.append([di["lesson_time"]]) + t_vals.append([di["lesson_minutes"]]) + # X: 有效 0/1 (GSV>0, 非全额退) + gmv = di["gmv"] + refund = di["refund"] + gsv = di["gsv"] + is_full_refund = (gmv > 0 and gmv == refund) + x_vals.append([1 if (gsv > 0 and not is_full_refund) else 0]) + y_vals.append([di.get("channel_class", "")]) else: - # 无有效 user_id，留空 for arr in [d_vals, i_vals, j_vals, k_vals, l_vals, m_vals, n_vals, - o_vals, p_vals, q_vals, r_vals, s_vals, t_vals, u_vals]: + o_vals, p_vals, q_vals, r_vals, s_vals, t_vals]: arr.append([""]) - v_vals.append([now_str]) + x_vals.append([0]) + y_vals.append([""]) + u_vals.append([now_str]) - # 写入各列 + # 写入各列（A-Y 共25列，V/W 为微伴/公式不动，X/Y 为有效/渠道） cols = [ ("D", d_vals), ("I", i_vals), ("J", j_vals), ("K", k_vals), ("L", l_vals), ("M", m_vals), ("N", n_vals), ("O", o_vals), ("P", p_vals), ("Q", q_vals), ("R", r_vals), ("S", s_vals), - ("T", t_vals), ("U", u_vals), ("V", v_vals), + ("T", t_vals), ("U", u_vals), ("X", x_vals), ("Y", y_vals), ] for col_letter, vals in cols: put_values(token, sid, f"{col_letter}{sr}:{col_letter}{er}", vals) @@ -538,31 +545,47 @@ def write_process_data(token, all_entries, db_info): # ── Step 5: 订单汇总 ── def write_order_summary(token, all_entries, db_info): - """从销售三表筛选 Y=1 的行，全量替换订单汇总 A-X（r3+），按 K 下单日降序""" + """从销售三表筛选 X=1 的行，全量替换订单汇总 A-W（r3+），按 K 下单日降序""" log(" 写入订单汇总（全量替换）...") - # 从销售三表读取已更新的数据，筛选 Y=1 - order_rows = [] + # 从销售三表读取已更新的数据，筛选 X=1，同 UID 多表去重（保留行号最小） + uid_best = {} # uid -> (row_num, row_data) for sid, sname, rng in SALES_SHEETS: rows = read_sheet(token, sid, rng) sheet_count = 0 for idx, row in enumerate(rows[2:], start=3): if not row: continue - # Y 列 (index 24) = 1 - y_val = "" - if len(row) > 24 and row[24] not in (None, ""): - y_val = str(row[24]).strip() - if y_val not in ("1", 1): continue - # 复制 A-X (indices 0-23) + # X 列 (index 23) = 1（飞书返回可能是 float 1.0） + x_val = None + if len(row) > 23 and row[23] not in (None, ""): + try: + x_val = int(float(row[23])) + except: pass + if x_val != 1: continue + # 取 UID (H 列 index 7) + uid = "" + if len(row) > 7 and row[7] not in (None, ""): + try: + uid = str(int(float(row[7]))) + except: pass + if not uid: continue + # 复制 A-U (indices 0-20) + V=渠道(=三表Y) + W=留空 row_data = [] - for ci in range(24): + for ci in range(21): # A-U if ci < len(row): row_data.append(row[ci]) else: row_data.append("") - order_rows.append(row_data) + # V = 渠道 (= 三表 Y col index 24) + row_data.append(str(row[24]).strip() if len(row) > 24 and row[24] not in (None, "") else "") + # W = 留空 + row_data.append("") + # 同 UID 去重：保留行号最小的 + if uid not in uid_best or idx < uid_best[uid][0]: + uid_best[uid] = (idx, row_data) sheet_count += 1 log(f" {sname}: {sheet_count} 条") + order_rows = [v[1] for v in uid_best.values()] # 按 K 列 (index 10, 下单日期) 降序 order_rows.sort(key=lambda r: str(r[10]) if len(r) > 10 and r[10] else "", reverse=True) @@ -570,12 +593,12 @@ def write_order_summary(token, all_entries, db_info): total = len(order_rows) log(f" 共 {total} 条订单，写入订单汇总 r3+") - # 全量写入 A-X 从 row 3 开始 + # 全量写入 A-W 从 row 3 开始（23列） for batch_start in range(0, total, 20): batch = order_rows[batch_start:batch_start+20] sr = 3 + batch_start er = sr + len(batch) - 1 - put_values(token, ORDER_SHEET, f"A{sr}:X{er}", batch) + put_values(token, ORDER_SHEET, f"A{sr}:W{er}", batch) time.sleep(0.5) # 清除多余旧行 @@ -585,9 +608,9 @@ def write_order_summary(token, all_entries, db_info): if old_count > total: clear_start = 3 + total clear_end = 3 + old_count - 1 - empty_rows = [[""] * 24 for _ in range(clear_end - clear_start + 1)] - put_values(token, ORDER_SHEET, f"A{clear_start}:X{clear_end}", empty_rows) - log(f" 清除多余行 A{clear_start}:X{clear_end}") + empty_rows = [[""] * 23 for _ in range(clear_end - clear_start + 1)] + put_values(token, ORDER_SHEET, f"A{clear_start}:W{clear_end}", empty_rows) + log(f" 清除多余行 A{clear_start}:W{clear_end}") except Exception as e: log(f" 清除多余行跳过: {e}") @@ -622,11 +645,10 @@ def main(): log("Step 4: 过程数据") write_process_data(token, all_entries, db_info) - # Step 5: 订单汇总 - log("Step 5: 订单汇总") - write_order_summary(token, all_entries, db_info) + # Step 5: 订单汇总 — 由 Cursor 负责，小溪不写 + log("Step 5: 跳过（Cursor 负责汇总）") - log("✅ 全覆盖刷新完成") + log("✅ 全覆盖刷新完成（Step4 only）") return 0 except Exception as e: log(f"❌ ERROR: {e}") diff --git a/skills/vala-order-amortization-stat/SKILL.md b/skills/vala-order-amortization-stat/SKILL.md index e5dac35..1bbd74f 100644 --- a/skills/vala-order-amortization-stat/SKILL.md +++ b/skills/vala-order-amortization-stat/SKILL.md @@ -19,6 +19,7 @@ 3. 表关联规则： - bi_vala_order.out_trade_no ↔ bi_refund_order.out_trade_no 关联 - bi_vala_order.trade_no ↔ bi_refund_order.trade_no 关联 + - 两种关联方式 UNION ALL 后聚合去重，退费金额 SUM 累加 [李承龙确认 2026-06-18] ## 执行方式 @@ -69,6 +70,8 @@ vala-order-amortization-stat/ - 关联 bi_vala_app_account 表（不限制 status，不剔除测试账号） 2. 退费范围： - 退费记录范围：截至账期结束日前的所有退费成功记录，用于标记订单退费状态 + - 退费金额 SUM 累加：同时按 `out_trade_no` 和 `trade_no` 关联 `bi_refund_order`，对同一订单的多笔 `status=3` 退费记录做 SUM(refund_amount) 累加 [李承龙确认 2026-06-18] + - 退费判定：`order_status=4` 直接视为退费；`order_status=3` 需退费表有 `status=3` 记录才视为退费 - 账期内退费判定：bi_refund_order.updated_at 在账期起止范围内，status = 3 3. 账期前退费订单处理规则： - 全额退费（is_full_refund = 1，累计退费金额 ≥ 原订单金额）：本账期完全不统计该订单 diff --git a/skills/vala-order-amortization-stat/sql/_common.sql b/skills/vala-order-amortization-stat/sql/_common.sql index ceb26be..10c09ea 100644 --- a/skills/vala-order-amortization-stat/sql/_common.sql +++ b/skills/vala-order-amortization-stat/sql/_common.sql @@ -4,16 +4,36 @@ WITH -- 步骤0：获取所有历史退费记录（不受当前账期限制） +-- 同时按 out_trade_no 和 trade_no 关联，UNION ALL 后聚合去重 +-- 退费金额 SUM 累加，支持一笔订单多笔退费记录 [李承龙确认 2026-06-18] all_refund_records AS ( SELECT - out_trade_no AS order_no, - SUM(refund_amount_int)::numeric / 100 AS total_refund_amount, + order_no, + SUM(total_refund_amount) AS total_refund_amount, MAX(refund_type) AS refund_type, - MAX(DATE(updated_at)) AS latest_refund_date - FROM bi_refund_order - WHERE status = 3 - AND DATE(updated_at) <= '{period_end}'::date - GROUP BY out_trade_no + MAX(latest_refund_date) AS latest_refund_date + FROM ( + SELECT + out_trade_no AS order_no, + SUM(refund_amount_int)::numeric / 100 AS total_refund_amount, + MAX(refund_type) AS refund_type, + MAX(DATE(updated_at)) AS latest_refund_date + FROM bi_refund_order + WHERE status = 3 + AND DATE(updated_at) <= '{period_end}'::date + GROUP BY out_trade_no + UNION ALL + SELECT + trade_no AS order_no, + SUM(refund_amount_int)::numeric / 100 AS total_refund_amount, + MAX(refund_type) AS refund_type, + MAX(DATE(updated_at)) AS latest_refund_date + FROM bi_refund_order + WHERE status = 3 + AND DATE(updated_at) <= '{period_end}'::date + GROUP BY trade_no + ) combined + GROUP BY order_no ), -- 步骤1：获取所有符合条件的订单基础信息 order_base AS ( @@ -32,16 +52,24 @@ order_base AS ( a.id AS account_id, o.key_from, o.sale_channel, - CASE WHEN ar.order_no IS NOT NULL THEN 1 ELSE 0 END AS has_refund, - COALESCE(ar.total_refund_amount, 0) AS total_refund_amount, - CASE WHEN ar.order_no IS NOT NULL - AND COALESCE(ar.total_refund_amount, 0) >= o.pay_amount_int::numeric / 100 + -- has_refund: order_status=4 直接视为退费；order_status=3 需退费表有记录 [李承龙确认 2026-06-18] + CASE WHEN ar.order_no IS NOT NULL OR o.order_status = 4 THEN 1 ELSE 0 END AS has_refund, + -- total_refund_amount: 退费表 SUM 累加；order_status=4 且退费表无记录时用订单金额兜底 + COALESCE(ar.total_refund_amount, + CASE WHEN o.order_status = 4 THEN o.pay_amount_int::numeric / 100 ELSE 0 END + ) AS total_refund_amount, + -- is_full_refund: 退费金额>=订单金额 或 order_status=4且退费表无记录 + CASE WHEN (ar.order_no IS NOT NULL AND COALESCE(ar.total_refund_amount, 0) >= o.pay_amount_int::numeric / 100) + OR (o.order_status = 4 AND ar.order_no IS NULL) THEN 1 ELSE 0 END AS is_full_refund, ar.refund_type, - ar.latest_refund_date + -- latest_refund_date: order_status=4 且退费表无记录时用下单日期兜底 + COALESCE(ar.latest_refund_date, + CASE WHEN o.order_status = 4 THEN DATE(o.pay_success_date) ELSE NULL END + ) AS latest_refund_date FROM bi_vala_order o JOIN bi_vala_app_account a ON o.account_id = a.id - LEFT JOIN all_refund_records ar ON o.out_trade_no = ar.order_no + LEFT JOIN all_refund_records ar ON o.out_trade_no = ar.order_no OR o.trade_no = ar.order_no WHERE o.pay_success_date >= '2025-06-01' AND o.pay_success_date <= '{period_end}'::date + INTERVAL '1 day' - INTERVAL '1 second' diff --git a/skills/vala-order-amortization-stat/sql/prepaid.sql b/skills/vala-order-amortization-stat/sql/prepaid.sql index 7f11c05..68168b3 100644 --- a/skills/vala-order-amortization-stat/sql/prepaid.sql +++ b/skills/vala-order-amortization-stat/sql/prepaid.sql @@ -13,17 +13,35 @@ month_range AS ( (DATE_TRUNC('month', '{period_start}'::date) - INTERVAL '1 day')::date AS last_month_end, (DATE_TRUNC('month', '{period_start}'::date) - INTERVAL '1 month')::date AS last_month_start ), --- 所有退费记录 +-- 所有退费记录（同时按 out_trade_no 和 trade_no 关联，SUM 累加，支持多笔退费）[李承龙确认 2026-06-18] all_refund_records AS ( SELECT - out_trade_no AS order_no, - SUM(refund_amount_int)::numeric / 100 AS total_refund_amount, - MAX(CASE WHEN refund_type = 2 THEN 1 ELSE 0 END) AS is_full_refund, - MAX(DATE(updated_at)) AS latest_refund_date - FROM bi_refund_order - WHERE status = 3 - AND DATE(updated_at) <= '{period_end}'::date - GROUP BY out_trade_no + order_no, + SUM(total_refund_amount) AS total_refund_amount, + MAX(is_full_refund) AS is_full_refund, + MAX(latest_refund_date) AS latest_refund_date + FROM ( + SELECT + out_trade_no AS order_no, + SUM(refund_amount_int)::numeric / 100 AS total_refund_amount, + MAX(CASE WHEN refund_type = 2 THEN 1 ELSE 0 END) AS is_full_refund, + MAX(DATE(updated_at)) AS latest_refund_date + FROM bi_refund_order + WHERE status = 3 + AND DATE(updated_at) <= '{period_end}'::date + GROUP BY out_trade_no + UNION ALL + SELECT + trade_no AS order_no, + SUM(refund_amount_int)::numeric / 100 AS total_refund_amount, + MAX(CASE WHEN refund_type = 2 THEN 1 ELSE 0 END) AS is_full_refund, + MAX(DATE(updated_at)) AS latest_refund_date + FROM bi_refund_order + WHERE status = 3 + AND DATE(updated_at) <= '{period_end}'::date + GROUP BY trade_no + ) combined + GROUP BY order_no ), -- 所有订单基础信息（含退费标记） order_base AS ( @@ -31,9 +49,16 @@ order_base AS ( o.out_trade_no AS order_no, DATE(o.pay_success_date) AS order_date, o.pay_amount_int::numeric / 100 AS pay_amount, - CASE WHEN ar.order_no IS NOT NULL THEN 1 ELSE 0 END AS has_refund, - COALESCE(ar.total_refund_amount, 0) AS total_refund_amount, - CASE WHEN ar.order_no IS NOT NULL AND COALESCE(ar.total_refund_amount, 0) >= o.pay_amount_int::numeric / 100 THEN 1 ELSE 0 END AS is_full_refund, + -- has_refund: order_status=4 直接视为退费；order_status=3 需退费表有记录 [李承龙确认 2026-06-18] + CASE WHEN ar.order_no IS NOT NULL OR o.order_status = 4 THEN 1 ELSE 0 END AS has_refund, + -- total_refund_amount: 退费表 SUM 累加；order_status=4 且退费表无记录时用订单金额兜底 + COALESCE(ar.total_refund_amount, + CASE WHEN o.order_status = 4 THEN o.pay_amount_int::numeric / 100 ELSE 0 END + ) AS total_refund_amount, + -- is_full_refund: 退费金额>=订单金额 或 order_status=4且退费表无记录 + CASE WHEN (ar.order_no IS NOT NULL AND COALESCE(ar.total_refund_amount, 0) >= o.pay_amount_int::numeric / 100) + OR (o.order_status = 4 AND ar.order_no IS NULL) + THEN 1 ELSE 0 END AS is_full_refund, ar.latest_refund_date, -- 转正日期 = 下单日 + 7天 DATE(o.pay_success_date) + INTERVAL '7 days' AS amortization_start_date, @@ -45,7 +70,7 @@ order_base AS ( o.order_status FROM bi_vala_order o JOIN bi_vala_app_account a ON o.account_id = a.id - LEFT JOIN all_refund_records ar ON o.out_trade_no = ar.order_no + LEFT JOIN all_refund_records ar ON o.out_trade_no = ar.order_no OR o.trade_no = ar.order_no CROSS JOIN month_range mr WHERE o.pay_success_date >= mr.last_month_start