Compare commits

...

3 Commits

Author SHA1 Message Date
5dab499930 默认只点击图片 2026-04-23 19:27:24 +08:00
c835182a44 初步搞定 很酷 2026-04-23 19:23:59 +08:00
90881c4191 解决了图片点击问题 2026-04-23 18:39:49 +08:00
9 changed files with 445 additions and 78 deletions

View File

@ -28,15 +28,20 @@ wechat_clicker/
### 关键设计决策 ### 关键设计决策
- 微信 v4.1.9 点击聊天会打开**独立窗口**(非页内导航),状态检测基于窗口计数
- 聊天列表项为 AXStaticText无 AXPress使用 **CGEvent 鼠标坐标点击** - 聊天列表项为 AXStaticText无 AXPress使用 **CGEvent 鼠标坐标点击**
- AXValue 位置/尺寸需用 AXValueGetValue 解包 CGPoint/CGSize - AXValue 位置/尺寸需用 AXValueGetValue 解包 CGPoint/CGSize
- 消息类型通过 title 内容判断:`"图片"` → 图片,`"文件\n..."` → 文件 - 消息类型通过 title 内容判断:`"图片"` → 图片,`"文件\n..."` → 文件
- **图片/文件气泡偏移点击**AXStaticText 覆盖整行575px宽实际气泡仅在左侧 ~80-170px 区域,使用 `click_at_element_offset(x=120, y=height/2)` 命中
- 图片处理:点击缩略图 → 点击"..." → 点击"使用预览打开" → 关闭 Preview.app - 图片处理:点击缩略图 → 点击"..." → 点击"使用预览打开" → 关闭 Preview.app
- 文件处理:直接点击触发下载 - 文件处理:直接点击触发下载
- 进入聊天后先**滚到底部**(最新消息),再向上滚动 5 轮加载历史消息 - **滚动方向**macOS CGEvent ScrollWheel **负值=向下滚**(看新消息),**正值=向上滚**(看旧消息)
- 进入聊天后先滚到底部(负值),再向上滚动 5 轮(正值)加载历史消息
- 滚动使用 **kCGEventMouseMoved + ScrollWheel**(不触发点击),避免误点 UI 元素 - 滚动使用 **kCGEventMouseMoved + ScrollWheel**(不触发点击),避免误点 UI 元素
- "..."按钮搜索限制在预览区域(独立窗口或主窗口 x>200排除侧边栏 - "..."按钮搜索限制在预览区域(独立窗口或主窗口 x>200排除侧边栏
- 媒体去重基于消息列表**子元素索引**child index同一元素跨滚动轮次只处理一次
- 可见性检查基于元素**中心点**是否在消息列表可见区域内30px margin
- 状态检测基于窗口计数 + **AXMinimized 属性**:区分窗口存在但最小化 vs 真正可见
- **窗口恢复策略**NSRunningApplication.unhide隐藏→ AX API 设置 AXMinimized=False最小化→ activate前台AppleScript 不可靠(微信不支持 miniaturized 属性)
## 使用方法 ## 使用方法
@ -53,17 +58,18 @@ python main.py # 持续运行
python main.py --once # 单次扫描 python main.py --once # 单次扫描
python main.py --dry-run # 只扫描不点击 python main.py --dry-run # 只扫描不点击
python main.py --dump-ui # 输出 UI 元素树 python main.py --dump-ui # 输出 UI 元素树
python main.py --dump-ui --chat "聊天名" # 进入指定聊天 dump 消息元素树(诊断用)
python main.py --debug # 详细日志 python main.py --debug # 详细日志
``` ```
## 配置重点 ## 配置重点
- `config.yaml` 中可设置扫描间隔、延迟范围、白/黑名单、工作时间、媒体类型开关 - `config.yaml` 中可设置扫描间隔、延迟范围、白/黑名单、工作时间、媒体类型开关
- 默认不处理视频(`media.click_videos: false` - 默认只点击图片(`media.click_files: false`, `media.click_videos: false`
- 默认黑名单包含微信系统账号 - 默认黑名单包含微信系统账号
## 注意事项 ## 注意事项
- 微信窗口需要保持可见(不能最小化) - 微信窗口最小化或隐藏时工具会自动恢复(通过 AX API 设置 AXMinimized=False + NSRunningApplication activate
- 运行时会占用微信前台操作 - 运行时会占用微信前台操作
- 建议在专用电脑上运行 - 建议在专用电脑上运行

View File

@ -40,7 +40,7 @@ filter:
# 媒体处理 # 媒体处理
media: media:
click_images: true # 是否点击图片 click_images: true # 是否点击图片
click_files: true # 是否点击文件 click_files: false # 是否点击文件(默认关闭,只点击图片)
click_videos: false # 是否点击视频(视频可能很大,默认关闭) click_videos: false # 是否点击视频(视频可能很大,默认关闭)
max_media_per_chat: 20 # 每个聊天最多点击几个媒体 max_media_per_chat: 20 # 每个聊天最多点击几个媒体

View File

@ -36,6 +36,10 @@ def main():
"--dump-ui", action="store_true", "--dump-ui", action="store_true",
help="输出微信 UI 元素树后退出 (调试用)" help="输出微信 UI 元素树后退出 (调试用)"
) )
parser.add_argument(
"--chat",
help="配合 --dump-ui 使用:进入指定聊天并 dump 消息区域元素树"
)
args = parser.parse_args() args = parser.parse_args()
# 加载配置 # 加载配置
@ -63,7 +67,7 @@ def main():
# 执行 # 执行
if args.dump_ui: if args.dump_ui:
automator.dump_ui_tree() automator.dump_ui_tree(chat_name=args.chat)
elif args.once: elif args.once:
automator.run_once() automator.run_once()
else: else:

View File

@ -33,16 +33,43 @@
- [x] **修复: 滚动误点击**`_scroll_at()` 原先用 mouseDown/mouseUp 聚焦导致误点侧边栏按钮,改用 kCGEventMouseMoved 仅移动鼠标不触发点击 - [x] **修复: 滚动误点击**`_scroll_at()` 原先用 mouseDown/mouseUp 聚焦导致误点侧边栏按钮,改用 kCGEventMouseMoved 仅移动鼠标不触发点击
- [x] **修复: 未跳转最新消息** — 进入聊天后先 `scroll_to_bottom()` 滚到消息列表底部(最新消息),再向上滚动加载历史 - [x] **修复: 未跳转最新消息** — 进入聊天后先 `scroll_to_bottom()` 滚到消息列表底部(最新消息),再向上滚动加载历史
- [x] **修复: "..."按钮误匹配**`_find_more_button_in_preview()` 限制搜索范围,主窗口中只搜索 x>200 区域(排除左侧侧边栏),增加位置日志 - [x] **修复: "..."按钮误匹配**`_find_more_button_in_preview()` 限制搜索范围,主窗口中只搜索 x>200 区域(排除左侧侧边栏),增加位置日志
- [x] **修复: 重复点击同一图片** — 用消息列表子元素索引(child index)做去重processed_indices 跟踪已处理元素,跨滚动轮次不重复
- [x] **修复: 裁剪图片点击无效** — 只处理完全在消息列表可见区域内的媒体(上下各 30px margin跳过部分被裁剪的元素
- [x] **增强: 全链路调试日志** — click_at_element 记录坐标/角色/标题get_media_messages 输出每个媒体详情find_menu_item 记录搜索过程,状态检测记录所有窗口标题 - [x] **增强: 全链路调试日志** — click_at_element 记录坐标/角色/标题get_media_messages 输出每个媒体详情find_menu_item 记录搜索过程,状态检测记录所有窗口标题
### v0.4.0 诊断工具与窗口恢复 (2026/04/23)
- [x] **新增: --dump-ui --chat 诊断工具**`python main.py --dump-ui --chat "聊天名"` 进入指定聊天,滚到底部,以 max_depth=10 dump 每个消息子元素的完整 AX 树含索引、position、actions、identifier用于诊断不同图片元素结构差异
- [x] **新增: dump_element 增强** — 输出 AXPosition、AXActions、AXIdentifier 字段
- [x] **新增: 微信窗口最小化自动恢复** — 新增 `WECHAT_MINIMIZED` 状态detect_state 区分"未运行"和"已最小化",通过 AppleScript 自动取消最小化并恢复前台
- [x] **增强: verify_setup 自动恢复** — 启动时找不到主窗口先尝试恢复最小化,再判断失败
### v0.5.0 图片点击与滚动修复 (2026/04/23)
- [x] **修复: 图片点击不生效(根本原因)** — AXStaticText 覆盖整行575px宽实际图片气泡仅在左侧 80~170px 区域。改用 `click_at_element_offset(x=120, y=height/2)` 命中实际缩略图区域(通过逐像素探测确认可点击边界)
- [x] **修复: 滚动方向反转** — macOS CGEvent ScrollWheel 正值=向上滚(看旧消息),负值=向下滚(看新消息)。`scroll_to_bottom` 从 +20 改为 -20历史加载滚动从 -10 改为 +10
- [x] **修复: 可见性判断过严** — 原先要求元素完全在可见区域内30px margin289px 高图片经常被判定裁剪。改为只检查元素**中心点**是否在可见区域内
- [x] **新增: click_at_element_offset** — ax_bridge 新增带偏移量的点击方法,文件/视频点击也使用偏移量
- [x] **已验证: 完整图片处理流程** — 蜗牛聊天 4 张不同尺寸图片289px、155px、180px全部成功缩略图→预览窗口→...按钮→使用预览打开→关闭 Preview→恢复
- [x] **已验证: 带滚动的媒体处理** — scroll_to_bottom 正确到达最新消息向上滚动正确加载历史去重正确processed_indices=[45,47,49,50]
### v0.6.0 最小化窗口恢复修复 (2026/04/23)
- [x] **修复: 最小化时恢复失败(根本原因)** — 原 AppleScript `tell application "WeChat"` 方案存在两个致命问题:(1) 微信的 localizedName 是"微信"不是"WeChat"AppleScript 找错应用或静默失败导致打开了别的窗口;(2) 微信不支持标准 AppleScript `miniaturized` 属性(返回 error -10006
- [x] **修复: 最小化检测遗漏** — 微信窗口最小化后 AXWindows 仍返回该窗口AXMinimized=True`detect_state``ensure_wechat_visible` 未检查此属性,误判窗口已可见
- [x] **改用 AX API 恢复窗口** — 通过 `AXUIElementSetAttributeValue(window, "AXMinimized", False)` 取消最小化,比 AppleScript 更可靠
- [x] **多策略窗口恢复** — unhideCmd+H隐藏→ AXMinimized=False最小化→ activate前台覆盖所有不可见场景
- [x] **恢复重试与验证**`ensure_wechat_visible` 增加 3 次重试循环,每次验证主窗口 AXMinimized=False 后才返回成功
- [x] **已验证** — 两次完整测试通过:最小化微信→运行 `--once --debug`→自动恢复→正常扫描处理
### v0.7.0 默认只点击图片 (2026/04/23)
- [x] **配置变更: click_files 默认关闭**`media.click_files` 默认值从 `true` 改为 `false`,默认只点击图片不点击文件。需要点击文件时在 config.yaml 中设置 `click_files: true`
### 待验证 ### 待验证
- [ ] 验证 CGEvent 鼠标点击能否正确打开聊天会话 - [x] ~~验证最小化窗口自动恢复功能(需手动最小化微信窗口测试)~~
- [ ] 验证 scroll_to_bottom 是否能到达最新消息 - [ ] 有未读图片消息时的完整 --once 自动化流程
- [ ] 验证图片预览中"..."按钮和"使用预览打开"菜单项的查找
- [ ] 验证滚动不再触发误点击
- [ ] 验证消息列表滚动能否加载历史消息
- [ ] 验证 Preview.app 的检测与关闭
- [ ] 长时间运行稳定性测试 - [ ] 长时间运行稳定性测试
### 未来可能的改进 ### 未来可能的改进

View File

@ -13,12 +13,12 @@ from .ax_bridge import AXBridge
from .config import Config from .config import Config
from .human_like import HumanBehavior from .human_like import HumanBehavior
from .state_machine import StateMachine, UIState from .state_machine import StateMachine, UIState
from .wechat_ui import WeChatUI from .wechat_ui import WeChatUI, MessageItem
logger = logging.getLogger("wechat_clicker.automator") logger = logging.getLogger("wechat_clicker.automator")
SCROLL_ROUNDS = 5 SCROLL_ROUNDS = 5
SCROLL_LINES_PER_ROUND = -10 SCROLL_LINES_PER_ROUND = 10
class WeChatAutomator: class WeChatAutomator:
@ -53,6 +53,11 @@ class WeChatAutomator:
return False return False
main_win = self.ui.get_main_window() main_win = self.ui.get_main_window()
if main_win is None:
logger.info("未找到微信主窗口,尝试恢复最小化窗口...")
if self.ui.ensure_wechat_visible():
main_win = self.ui.get_main_window()
if main_win is None: if main_win is None:
logger.error("未找到微信主窗口,请确保微信已登录并可见") logger.error("未找到微信主窗口,请确保微信已登录并可见")
return False return False
@ -200,78 +205,148 @@ class WeChatAutomator:
# ---------------------------------------------------------------- # ----------------------------------------------------------------
def _process_media_with_scroll(self, main_win, msg_list, chat_name: str) -> int: def _process_media_with_scroll(self, main_win, msg_list, chat_name: str) -> int:
"""先滚到底部看最新消息,再向上滚动加载历史并处理媒体。""" """先滚到底部看最新消息,再向上滚动加载历史并处理媒体。
用消息列表子元素索引做去重避免同一张图片被重复点击"""
total_clicked = 0 total_clicked = 0
processed_indices = set()
# 先滚到消息列表底部(看到最新消息) # 先滚到消息列表底部(看到最新消息)
logger.info(f" {chat_name}: 滚动到最新消息...") logger.info(f" {chat_name}: 滚动到最新消息...")
self.ax.scroll_to_bottom(msg_list, rounds=10, lines_per_round=20) self.ax.scroll_to_bottom(msg_list, rounds=10, lines_per_round=-20)
self.human.random_delay(1.0, 2.0) self.human.random_delay(1.0, 2.0)
# 处理当前可见的媒体(最新消息) # 处理当前可见的媒体(最新消息)
clicked = self._process_visible_media(main_win, chat_name) clicked = self._process_visible_media(
main_win, msg_list, chat_name, processed_indices
)
total_clicked += clicked total_clicked += clicked
# 向上滚动加载更多历史消息 # 向上滚动加载更多历史消息
for round_idx in range(SCROLL_ROUNDS): for round_idx in range(SCROLL_ROUNDS):
# 安全网:滚动前确保没有残留的预览窗口
state = self.state.detect_state()
if state != UIState.MAIN_CHAT_LIST:
logger.debug(
f" 滚动前状态不干净: {state.value}, 清理额外窗口"
)
self._close_extra_windows()
self.human.delay("between_messages") self.human.delay("between_messages")
logger.debug(f" {chat_name}: 向上滚动第 {round_idx + 1}/{SCROLL_ROUNDS}") logger.debug(f" {chat_name}: 向上滚动第 {round_idx + 1}/{SCROLL_ROUNDS}")
self.ax.scroll_at_element(msg_list, lines=SCROLL_LINES_PER_ROUND) self.ax.scroll_at_element(msg_list, lines=SCROLL_LINES_PER_ROUND)
self.human.random_delay(1.0, 2.5) self.human.random_delay(1.0, 2.5)
clicked = self._process_visible_media(main_win, chat_name) clicked = self._process_visible_media(
main_win, msg_list, chat_name, processed_indices
)
total_clicked += clicked total_clicked += clicked
logger.debug(
f" {chat_name}: 滚动结束, 已处理索引={sorted(processed_indices)}"
)
return total_clicked return total_clicked
def _process_visible_media(self, main_win, chat_name: str) -> int: def _process_visible_media(
"""处理当前可见的所有媒体消息。""" self, main_win, msg_list, chat_name: str, processed_indices: set
media_messages = self.ui.get_media_messages(main_win) ) -> int:
if not media_messages: """处理当前可见且未处理过的媒体消息。
logger.debug(f" {chat_name}: 当前视图无媒体消息") 通过子元素索引去重通过可见区域过滤跳过裁剪元素"""
return 0 # 获取消息列表可见区域
list_pos = self.ax.get_position(msg_list)
list_size = self.ax.get_size(msg_list)
visible_top = list_pos[1]
visible_bottom = list_pos[1] + list_size[1]
margin = 30
# 遍历消息列表的所有子元素,带索引
children = self.ax.get_children(msg_list)
targets = [] targets = []
for msg in media_messages:
if msg.msg_type == "image" and self.config.click_images: for idx, child in enumerate(children):
targets.append(msg) if idx in processed_indices:
elif msg.msg_type == "file" and self.config.click_files: continue
targets.append(msg)
elif msg.msg_type == "video" and self.config.click_videos: role = self.ax.get_role(child)
targets.append(msg) if role != "AXStaticText":
continue
title = self.ax.get_title(child)
if not title:
continue
size = self.ax.get_size(child)
if size[0] == 0 or size[1] == 0:
continue
msg_type = WeChatUI._classify_message(title, size)
if msg_type not in ("image", "file", "video"):
continue
# 检查配置是否处理该类型
if msg_type == "image" and not self.config.click_images:
continue
if msg_type == "file" and not self.config.click_files:
continue
if msg_type == "video" and not self.config.click_videos:
continue
# 可见性检查:元素中心必须在消息列表可见区域内(带 margin
pos = self.ax.get_position(child)
elem_center_y = pos[1] + size[1] // 2
if elem_center_y < visible_top + margin:
logger.debug(
f" 跳过(中心在顶部外): idx={idx} type={msg_type} "
f"center_y={elem_center_y} < list_top+margin={visible_top + margin}"
)
continue
if elem_center_y > visible_bottom - margin:
logger.debug(
f" 跳过(中心在底部外): idx={idx} type={msg_type} "
f"center_y={elem_center_y} > list_bottom-margin={visible_bottom - margin}"
)
continue
targets.append((idx, child, msg_type, title, size, pos))
if not targets: if not targets:
logger.debug( logger.debug(f" {chat_name}: 当前视图无可处理的新媒体")
f" {chat_name}: 发现 {len(media_messages)} 个媒体, "
f"但无符合配置的处理目标"
)
return 0 return 0
logger.info(f" {chat_name}: 当前可见 {len(targets)} 个媒体消息")
clicked = 0
for idx, msg in enumerate(targets):
self.human.delay("before_click_media")
pos = self.ax.get_position(msg.element)
short_title = msg.title.replace("\n", " ")[:50]
logger.info( logger.info(
f" [{msg.msg_type}] {short_title} " f" {chat_name}: 当前可见 {len(targets)} 个新媒体 "
f"(pos={pos}, size={msg.size}, #{idx+1}/{len(targets)})" f"(已处理 {len(processed_indices)} 个)"
) )
if msg.msg_type == "image": clicked = 0
success = self._click_image(msg) for idx, child, msg_type, title, size, pos in targets:
elif msg.msg_type == "file": self.human.delay("before_click_media")
success = self._click_file(msg)
short_title = title.replace("\n", " ")[:50]
logger.info(
f" [{msg_type}] {short_title} "
f"(idx={idx}, pos={pos}, size={size})"
)
msg_item = MessageItem(
element=child, msg_type=msg_type, title=title, size=size
)
if msg_type == "image":
success = self._click_image(msg_item)
elif msg_type == "file":
success = self._click_file(msg_item)
else: else:
success = self._click_generic(msg) success = self._click_generic(msg_item)
# 无论成功失败都标记为已处理,避免重复尝试
processed_indices.add(idx)
if success: if success:
clicked += 1 clicked += 1
logger.debug(f" 处理成功: {msg.msg_type}") logger.debug(f" 处理成功: idx={idx} {msg_type}")
else: else:
logger.warning(f" 处理失败: {msg.msg_type}") logger.warning(f" 处理失败: idx={idx} {msg_type}")
self.human.delay("between_messages") self.human.delay("between_messages")
@ -289,8 +364,12 @@ class WeChatAutomator:
def _click_image(self, msg) -> bool: def _click_image(self, msg) -> bool:
"""处理一张图片:点开大图 -> 点... -> 点使用预览打开 -> 关闭 Preview.app。""" """处理一张图片:点开大图 -> 点... -> 点使用预览打开 -> 关闭 Preview.app。"""
# Step 1: 点击图片缩略图打开大图预览 # Step 1: 点击图片缩略图打开大图预览
# 图片气泡在消息行 AXStaticText 的左侧区域(非中心),
# 用偏移 (120, height//2) 命中实际缩略图
logger.debug(" Step1: 点击图片缩略图") logger.debug(" Step1: 点击图片缩略图")
if not self.ax.click_at_element(msg.element): x_off = 120
y_off = msg.size[1] // 2
if not self.ax.click_at_element_offset(msg.element, x_off, y_off):
logger.warning(" 图片缩略图点击失败") logger.warning(" 图片缩略图点击失败")
return False return False
@ -346,17 +425,12 @@ class WeChatAutomator:
self.human.random_delay(2.0, 4.0) self.human.random_delay(2.0, 4.0)
# Step 7: 关闭 Preview.app # Step 7: 关闭所有额外窗口Preview.app + 微信预览窗口)
if self.state.is_preview_app_running(): logger.debug(" Step7: 清理预览窗口")
logger.debug(" Step7: 关闭 Preview.app") self._close_extra_windows()
self.state.close_preview_app()
else:
logger.debug(" Step7: Preview.app 未运行,跳过关闭")
# Step 8: 确保微信回到前台,关闭大图预览 # Step 8: Escape 关闭可能残留的内嵌预览
logger.debug(" Step8: 恢复微信前台Escape 关闭预览") logger.debug(" Step8: Escape 关闭内嵌预览")
self.ui.ensure_wechat_frontmost()
time.sleep(0.3)
self.ax.send_escape_key() self.ax.send_escape_key()
time.sleep(0.5) time.sleep(0.5)
@ -399,19 +473,21 @@ class WeChatAutomator:
# ---------------------------------------------------------------- # ----------------------------------------------------------------
def _click_file(self, msg) -> bool: def _click_file(self, msg) -> bool:
"""点击文件触发下载。""" """点击文件触发下载,然后关闭可能弹出的预览窗口。"""
if not self.ax.click_at_element(msg.element): x_off = 120
y_off = msg.size[1] // 2
if not self.ax.click_at_element_offset(msg.element, x_off, y_off):
logger.warning(" 文件点击失败") logger.warning(" 文件点击失败")
return False return False
self.human.delay("after_click_media") self.human.delay("after_click_media")
# 文件点击后可能会打开文件预览,关闭它
self.human.random_delay(1.0, 2.0) self.human.random_delay(1.0, 2.0)
state = self.state.detect_state() state = self.state.detect_state()
if state == UIState.MEDIA_PREVIEW: logger.debug(f" 文件点击后状态={state.value}")
self.ax.send_escape_key()
time.sleep(0.5) if state != UIState.MAIN_CHAT_LIST:
self._close_extra_windows()
return True return True
@ -420,7 +496,9 @@ class WeChatAutomator:
# ---------------------------------------------------------------- # ----------------------------------------------------------------
def _click_generic(self, msg) -> bool: def _click_generic(self, msg) -> bool:
if not self.ax.click_at_element(msg.element): x_off = 120
y_off = msg.size[1] // 2
if not self.ax.click_at_element_offset(msg.element, x_off, y_off):
return False return False
self.human.delay("after_click_media") self.human.delay("after_click_media")
self.human.micro_jitter() self.human.micro_jitter()
@ -428,17 +506,45 @@ class WeChatAutomator:
time.sleep(0.5) time.sleep(0.5)
return True return True
# ----------------------------------------------------------------
# 窗口清理:关闭预览等额外窗口
# ----------------------------------------------------------------
def _close_extra_windows(self):
"""关闭所有非主窗口(文件预览、图片预览等)和 Preview.app恢复干净状态。"""
if self.state.is_preview_app_running():
logger.debug(" 关闭 Preview.app")
self.state.close_preview_app()
conv_wins = self.ui.get_conversation_windows()
for win in conv_wins:
title = self.ax.get_title(win)
logger.debug(f" 关闭额外窗口: {title}")
close_btn = self.ui.get_close_button(win)
if close_btn:
self.ax.press(close_btn)
else:
self.ax.send_cmd_w()
time.sleep(0.5)
self.ui.ensure_wechat_frontmost()
time.sleep(0.3)
# ---------------------------------------------------------------- # ----------------------------------------------------------------
# 调试工具 # 调试工具
# ---------------------------------------------------------------- # ----------------------------------------------------------------
def dump_ui_tree(self): def dump_ui_tree(self, chat_name: str = None):
if not self.verify_setup(): if not self.verify_setup():
return return
self.ui.ensure_wechat_frontmost() self.ui.ensure_wechat_frontmost()
time.sleep(0.5) time.sleep(0.5)
if chat_name:
self._dump_chat_messages(chat_name)
return
main_win = self.ui.get_main_window() main_win = self.ui.get_main_window()
if main_win: if main_win:
print("=== 主窗口 (微信) ===") print("=== 主窗口 (微信) ===")
@ -461,6 +567,84 @@ class WeChatAutomator:
f"| pos={pos} size={size}" f"| pos={pos} size={size}"
) )
def _dump_chat_messages(self, chat_name: str):
"""进入指定聊天dump 消息列表中每个子元素的完整子树。"""
# 确保在聊天列表
if not self.state.recover_to_chat_list():
print(f"ERROR: 无法恢复到聊天列表")
return
# 在聊天列表中查找匹配的聊天项(子串匹配)
items = self.ui.get_chat_items()
target = None
for item in items:
if chat_name in item.name:
target = item
break
if target is None:
print(f"ERROR: 聊天列表中未找到包含 \"{chat_name}\" 的聊天")
print("当前可见聊天:")
for item in items:
print(f" - {item.name}")
return
print(f"找到聊天: {target.name} (未读: {target.unread_count})")
print(f"点击进入...")
# 点击聊天项(先点其他聊天再点回来,避免重复选中不加载)
other = next((i for i in items if i.name != target.name), None)
if other:
self.ax.click_at_element(other.element)
time.sleep(0.5)
if not self.ax.click_at_element(target.element):
print(f"ERROR: 点击聊天项失败")
return
time.sleep(2.0)
# 在主窗口中找消息列表(重试几次,等待 UI 加载)
main_win = self.ui.get_main_window()
if main_win is None:
print("ERROR: 主窗口丢失")
return
msg_list = None
for attempt in range(3):
msg_list = self.ui.get_message_list(main_win)
if msg_list is not None:
break
print(f" 等待消息列表加载... (尝试 {attempt + 1}/3)")
time.sleep(1.5)
if msg_list is None:
print("ERROR: 未找到消息列表dump 主窗口结构以诊断:")
print(self.ax.dump_element(main_win, max_depth=8))
return
# 滚到底部看最新消息
print("滚动到最新消息...")
self.ax.scroll_to_bottom(msg_list, rounds=10, lines_per_round=-20)
time.sleep(1.5)
# 遍历消息列表所有子元素
children = self.ax.get_children(msg_list)
print(f"\n=== 消息列表 ({target.name}) ===")
print(f"子元素总数: {len(children)}")
print()
for idx, child in enumerate(children):
role = self.ax.get_role(child)
title = self.ax.get_title(child)
size = self.ax.get_size(child)
pos = self.ax.get_position(child)
short_title = (title or "").replace("\n", "\\n")[:60]
print(f"--- [{idx}] role={role} title=\"{short_title}\" size={size} pos={pos} ---")
print(self.ax.dump_element(child, max_depth=10))
print()
# ---------------------------------------------------------------- # ----------------------------------------------------------------
# 统计 # 统计
# ---------------------------------------------------------------- # ----------------------------------------------------------------

View File

@ -15,6 +15,7 @@ from ApplicationServices import (
AXUIElementCopyAttributeNames, AXUIElementCopyAttributeNames,
AXUIElementCopyActionNames, AXUIElementCopyActionNames,
AXUIElementPerformAction, AXUIElementPerformAction,
AXUIElementSetAttributeValue,
) )
from Cocoa import ( from Cocoa import (
NSRunningApplication, NSRunningApplication,
@ -102,6 +103,64 @@ class AXBridge:
app = apps[0] app = apps[0]
return app.activateWithOptions_(NSApplicationActivateIgnoringOtherApps) return app.activateWithOptions_(NSApplicationActivateIgnoringOtherApps)
def is_app_running(self, bundle_id: str) -> bool:
"""检查应用是否正在运行(不依赖窗口状态)。"""
apps = NSRunningApplication.runningApplicationsWithBundleIdentifier_(bundle_id)
return apps is not None and len(apps) > 0
def unminimize_app(self, bundle_id: str) -> bool:
"""取消应用窗口的最小化/隐藏状态,多策略依次尝试。"""
apps = NSRunningApplication.runningApplicationsWithBundleIdentifier_(bundle_id)
if not apps or len(apps) == 0:
logger.error(f"未找到运行中的应用: {bundle_id}")
return False
app = apps[0]
app_name = app.localizedName() or "WeChat"
# 策略1: NSRunningApplication unhide处理 Cmd+H 隐藏)
if app.isHidden():
logger.info(f"{app_name} 处于隐藏状态,调用 unhide")
app.unhide()
time.sleep(0.3)
# 策略2: 通过 AX API 取消所有窗口的最小化(最可靠的方式)
app_ref = self.get_app_ref(bundle_id)
if app_ref:
windows = self.get_windows(app_ref)
for win in windows:
if self.is_window_minimized(win):
title = self.get_title(win) or "?"
logger.info(f"通过 AX API 取消窗口最小化: {title}")
self.set_window_minimized(win, False)
time.sleep(0.5)
# 策略3: activate 带到前台
app.activateWithOptions_(NSApplicationActivateIgnoringOtherApps)
time.sleep(0.3)
logger.info(f"窗口恢复流程完成: {app_name}")
return True
def is_window_minimized(self, window) -> bool:
"""检查窗口是否处于最小化状态。"""
try:
err, value = AXUIElementCopyAttributeValue(window, "AXMinimized", None)
if err == kAXErrorSuccess:
return bool(value)
except Exception:
pass
return False
def set_window_minimized(self, window, minimized: bool) -> bool:
"""通过 AX API 设置窗口最小化状态。"""
try:
err = AXUIElementSetAttributeValue(window, "AXMinimized", minimized)
return err == kAXErrorSuccess
except Exception as e:
logger.error(f"设置 AXMinimized 失败: {e}")
return False
# ---------------------------------------------------------------- # ----------------------------------------------------------------
# 属性读取 # 属性读取
# ---------------------------------------------------------------- # ----------------------------------------------------------------
@ -268,6 +327,23 @@ class AXBridge:
) )
return self._mouse_click(cx, cy) return self._mouse_click(cx, cy)
def click_at_element_offset(self, element, x_offset: int, y_offset: int) -> bool:
"""通过鼠标事件点击元素指定偏移位置。"""
pos = self.get_position(element)
size = self.get_size(element)
if pos == (0, 0) and size == (0, 0):
logger.warning("元素位置/尺寸不可用,无法点击")
return False
cx = pos[0] + x_offset
cy = pos[1] + y_offset
role = self.get_role(element)
title = (self.get_title(element) or "").replace("\n", "\\n")[:40]
logger.debug(
f"click_at_element_offset: ({cx}, {cy}) offset=({x_offset},{y_offset}) "
f"role={role} title=\"{title}\" pos={pos} size={size}"
)
return self._mouse_click(cx, cy)
def _mouse_click(self, x: int, y: int) -> bool: def _mouse_click(self, x: int, y: int) -> bool:
"""在屏幕坐标 (x, y) 处执行鼠标左键点击。""" """在屏幕坐标 (x, y) 处执行鼠标左键点击。"""
try: try:
@ -331,8 +407,8 @@ class AXBridge:
cy = pos[1] + size[1] // 2 cy = pos[1] + size[1] // 2
self._scroll_at(cx, cy, lines) self._scroll_at(cx, cy, lines)
def scroll_to_bottom(self, element, rounds: int = 10, lines_per_round: int = 20): def scroll_to_bottom(self, element, rounds: int = 10, lines_per_round: int = -20):
"""向下滚动多次,尽量到达元素(如消息列表)的底部。""" """向下滚动多次,尽量到达元素(如消息列表)的底部。负值=向下滚动。"""
pos = self.get_position(element) pos = self.get_position(element)
size = self.get_size(element) size = self.get_size(element)
if pos == (0, 0) and size == (0, 0): if pos == (0, 0) and size == (0, 0):
@ -422,11 +498,13 @@ class AXBridge:
name_attr = self.get_attribute(element, "AXValue") or "" name_attr = self.get_attribute(element, "AXValue") or ""
desc = self.get_description(element) desc = self.get_description(element)
size = self.get_size(element) size = self.get_size(element)
pos = self.get_position(element)
actions = self.get_action_names(element)
identifier = self.get_attribute(element, "AXIdentifier") or ""
prefix = " " * indent prefix = " " * indent
info_parts = [f"role={role}"] info_parts = [f"role={role}"]
if title: if title:
# 截断过长的 title用单行表示
short_title = title.replace("\n", "\\n")[:80] short_title = title.replace("\n", "\\n")[:80]
info_parts.append(f'title="{short_title}"') info_parts.append(f'title="{short_title}"')
if name_attr: if name_attr:
@ -436,6 +514,12 @@ class AXBridge:
info_parts.append(f'desc="{desc}"') info_parts.append(f'desc="{desc}"')
if size != (0, 0): if size != (0, 0):
info_parts.append(f"size={size[0]}x{size[1]}") info_parts.append(f"size={size[0]}x{size[1]}")
if pos != (0, 0):
info_parts.append(f"pos=({pos[0]},{pos[1]})")
if actions:
info_parts.append(f"actions={actions}")
if identifier:
info_parts.append(f'id="{identifier}"')
lines.append(f"{prefix}[{', '.join(info_parts)}]") lines.append(f"{prefix}[{', '.join(info_parts)}]")

View File

@ -39,7 +39,7 @@ DEFAULTS = {
}, },
"media": { "media": {
"click_images": True, "click_images": True,
"click_files": True, "click_files": False,
"click_videos": False, "click_videos": False,
"max_media_per_chat": 20, "max_media_per_chat": 20,
}, },

View File

@ -26,6 +26,7 @@ class UIState(Enum):
CONVERSATION_OPEN = "conversation_open" CONVERSATION_OPEN = "conversation_open"
MEDIA_PREVIEW = "media_preview" MEDIA_PREVIEW = "media_preview"
WECHAT_NOT_RUNNING = "wechat_not_running" WECHAT_NOT_RUNNING = "wechat_not_running"
WECHAT_MINIMIZED = "wechat_minimized"
class StateMachine: class StateMachine:
@ -54,20 +55,35 @@ class StateMachine:
windows = self.ui.get_all_windows() windows = self.ui.get_all_windows()
if not windows: if not windows:
if self.ax.is_app_running(self.ui.bundle_id):
self._current_state = UIState.WECHAT_MINIMIZED
self._conversation_name = None
logger.debug("微信在运行但无可见窗口(可能已最小化或隐藏)")
else:
self._current_state = UIState.WECHAT_NOT_RUNNING self._current_state = UIState.WECHAT_NOT_RUNNING
self._conversation_name = None self._conversation_name = None
logger.debug("未检测到微信窗口") logger.debug("微信未运行")
return self._current_state return self._current_state
# 分析窗口 # 检查主窗口是否最小化AXWindows 会包含最小化的窗口)
main_window = None main_window = None
all_minimized = True
other_windows = [] other_windows = []
for win in windows: for win in windows:
title = self.ax.get_title(win) title = self.ax.get_title(win)
is_mini = self.ax.is_window_minimized(win)
if title == "微信": if title == "微信":
main_window = win main_window = win
elif title: elif title:
other_windows.append((win, title)) other_windows.append((win, title))
if not is_mini:
all_minimized = False
if all_minimized:
self._current_state = UIState.WECHAT_MINIMIZED
self._conversation_name = None
logger.debug("微信所有窗口均已最小化")
return self._current_state
if main_window is None: if main_window is None:
self._current_state = UIState.UNKNOWN self._current_state = UIState.UNKNOWN
@ -117,6 +133,15 @@ class StateMachine:
logger.error("微信未运行,无法恢复") logger.error("微信未运行,无法恢复")
return False return False
if state == UIState.WECHAT_MINIMIZED:
logger.info("微信窗口已最小化,正在恢复...")
if self.ui.ensure_wechat_visible():
time.sleep(0.5)
continue
else:
logger.error("微信窗口恢复失败")
return False
if state == UIState.MEDIA_PREVIEW: if state == UIState.MEDIA_PREVIEW:
# 先关闭预览Escape # 先关闭预览Escape
logger.info("关闭媒体预览...") logger.info("关闭媒体预览...")

View File

@ -10,6 +10,7 @@
import logging import logging
import re import re
import time
from .ax_bridge import AXBridge from .ax_bridge import AXBridge
@ -75,6 +76,42 @@ class WeChatUI:
"""确保微信在最前台。""" """确保微信在最前台。"""
return self.ax.bring_to_front(self.bundle_id) return self.ax.bring_to_front(self.bundle_id)
def ensure_wechat_visible(self, max_retries: int = 3) -> bool:
"""确保微信窗口可见(取消最小化/隐藏并带到前台)。
多次重试每次重试后验证主窗口是否真正可见非最小化
"""
if not self.ax.is_app_running(self.bundle_id):
logger.error("微信未运行")
return False
# 先检查是否已经可见且非最小化
main_win = self.get_main_window()
if main_win is not None and not self.ax.is_window_minimized(main_win):
self.ensure_wechat_frontmost()
return True
for attempt in range(1, max_retries + 1):
logger.info(f"微信窗口不可见或已最小化,恢复尝试 {attempt}/{max_retries}...")
self.ax.unminimize_app(self.bundle_id)
time.sleep(1.0)
# 清除缓存的 app_ref强制重新获取
self.invalidate_app_ref()
main_win = self.get_main_window()
if main_win is not None and not self.ax.is_window_minimized(main_win):
logger.info("微信主窗口已恢复可见")
self.ensure_wechat_frontmost()
return True
logger.warning(f"恢复尝试 {attempt} 后窗口仍不可见,等待后重试...")
time.sleep(1.0)
logger.error(f"经过 {max_retries} 次尝试,微信窗口恢复失败")
return False
# ---------------------------------------------------------------- # ----------------------------------------------------------------
# 窗口查找 # 窗口查找
# ---------------------------------------------------------------- # ----------------------------------------------------------------