解决了图片点击问题
This commit is contained in:
parent
47f120f7fc
commit
90881c4191
11
CLAUDE.md
11
CLAUDE.md
@ -28,15 +28,19 @@ wechat_clicker/
|
||||
|
||||
### 关键设计决策
|
||||
|
||||
- 微信 v4.1.9 点击聊天会打开**独立窗口**(非页内导航),状态检测基于窗口计数
|
||||
- 聊天列表项为 AXStaticText(无 AXPress),使用 **CGEvent 鼠标坐标点击**
|
||||
- AXValue 位置/尺寸需用 AXValueGetValue 解包 CGPoint/CGSize
|
||||
- 消息类型通过 title 内容判断:`"图片"` → 图片,`"文件\n..."` → 文件
|
||||
- **图片/文件气泡偏移点击**:AXStaticText 覆盖整行(575px宽),实际气泡仅在左侧 ~80-170px 区域,使用 `click_at_element_offset(x=120, y=height/2)` 命中
|
||||
- 图片处理:点击缩略图 → 点击"..." → 点击"使用预览打开" → 关闭 Preview.app
|
||||
- 文件处理:直接点击触发下载
|
||||
- 进入聊天后先**滚到底部**(最新消息),再向上滚动 5 轮加载历史消息
|
||||
- **滚动方向**:macOS CGEvent ScrollWheel **负值=向下滚**(看新消息),**正值=向上滚**(看旧消息)
|
||||
- 进入聊天后先滚到底部(负值),再向上滚动 5 轮(正值)加载历史消息
|
||||
- 滚动使用 **kCGEventMouseMoved + ScrollWheel**(不触发点击),避免误点 UI 元素
|
||||
- "..."按钮搜索限制在预览区域(独立窗口或主窗口 x>200),排除侧边栏
|
||||
- 媒体去重基于消息列表**子元素索引**(child index),同一元素跨滚动轮次只处理一次
|
||||
- 可见性检查基于元素**中心点**是否在消息列表可见区域内(30px margin)
|
||||
- 状态检测基于窗口计数:1 窗口=聊天列表,2 窗口=有预览/会话打开
|
||||
|
||||
## 使用方法
|
||||
|
||||
@ -53,6 +57,7 @@ python main.py # 持续运行
|
||||
python main.py --once # 单次扫描
|
||||
python main.py --dry-run # 只扫描不点击
|
||||
python main.py --dump-ui # 输出 UI 元素树
|
||||
python main.py --dump-ui --chat "聊天名" # 进入指定聊天 dump 消息元素树(诊断用)
|
||||
python main.py --debug # 详细日志
|
||||
```
|
||||
|
||||
@ -64,6 +69,6 @@ python main.py --debug # 详细日志
|
||||
|
||||
## 注意事项
|
||||
|
||||
- 微信窗口需要保持可见(不能最小化)
|
||||
- 微信窗口最小化时工具会自动尝试恢复(通过 AppleScript 取消最小化)
|
||||
- 运行时会占用微信前台操作
|
||||
- 建议在专用电脑上运行
|
||||
|
||||
6
main.py
6
main.py
@ -36,6 +36,10 @@ def main():
|
||||
"--dump-ui", action="store_true",
|
||||
help="输出微信 UI 元素树后退出 (调试用)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--chat",
|
||||
help="配合 --dump-ui 使用:进入指定聊天并 dump 消息区域元素树"
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
# 加载配置
|
||||
@ -63,7 +67,7 @@ def main():
|
||||
|
||||
# 执行
|
||||
if args.dump_ui:
|
||||
automator.dump_ui_tree()
|
||||
automator.dump_ui_tree(chat_name=args.chat)
|
||||
elif args.once:
|
||||
automator.run_once()
|
||||
else:
|
||||
|
||||
26
project.md
26
project.md
@ -33,16 +33,30 @@
|
||||
- [x] **修复: 滚动误点击** — `_scroll_at()` 原先用 mouseDown/mouseUp 聚焦导致误点侧边栏按钮,改用 kCGEventMouseMoved 仅移动鼠标不触发点击
|
||||
- [x] **修复: 未跳转最新消息** — 进入聊天后先 `scroll_to_bottom()` 滚到消息列表底部(最新消息),再向上滚动加载历史
|
||||
- [x] **修复: "..."按钮误匹配** — `_find_more_button_in_preview()` 限制搜索范围,主窗口中只搜索 x>200 区域(排除左侧侧边栏),增加位置日志
|
||||
- [x] **修复: 重复点击同一图片** — 用消息列表子元素索引(child index)做去重,processed_indices 跟踪已处理元素,跨滚动轮次不重复
|
||||
- [x] **修复: 裁剪图片点击无效** — 只处理完全在消息列表可见区域内的媒体(上下各 30px margin),跳过部分被裁剪的元素
|
||||
- [x] **增强: 全链路调试日志** — click_at_element 记录坐标/角色/标题,get_media_messages 输出每个媒体详情,find_menu_item 记录搜索过程,状态检测记录所有窗口标题
|
||||
|
||||
### v0.4.0 诊断工具与窗口恢复 (2026/04/23)
|
||||
|
||||
- [x] **新增: --dump-ui --chat 诊断工具** — `python main.py --dump-ui --chat "聊天名"` 进入指定聊天,滚到底部,以 max_depth=10 dump 每个消息子元素的完整 AX 树(含索引、position、actions、identifier),用于诊断不同图片元素结构差异
|
||||
- [x] **新增: dump_element 增强** — 输出 AXPosition、AXActions、AXIdentifier 字段
|
||||
- [x] **新增: 微信窗口最小化自动恢复** — 新增 `WECHAT_MINIMIZED` 状态,detect_state 区分"未运行"和"已最小化",通过 AppleScript 自动取消最小化并恢复前台
|
||||
- [x] **增强: verify_setup 自动恢复** — 启动时找不到主窗口先尝试恢复最小化,再判断失败
|
||||
|
||||
### v0.5.0 图片点击与滚动修复 (2026/04/23)
|
||||
|
||||
- [x] **修复: 图片点击不生效(根本原因)** — AXStaticText 覆盖整行(575px宽),实际图片气泡仅在左侧 80~170px 区域。改用 `click_at_element_offset(x=120, y=height/2)` 命中实际缩略图区域(通过逐像素探测确认可点击边界)
|
||||
- [x] **修复: 滚动方向反转** — macOS CGEvent ScrollWheel 正值=向上滚(看旧消息),负值=向下滚(看新消息)。`scroll_to_bottom` 从 +20 改为 -20,历史加载滚动从 -10 改为 +10
|
||||
- [x] **修复: 可见性判断过严** — 原先要求元素完全在可见区域内(30px margin),289px 高图片经常被判定裁剪。改为只检查元素**中心点**是否在可见区域内
|
||||
- [x] **新增: click_at_element_offset** — ax_bridge 新增带偏移量的点击方法,文件/视频点击也使用偏移量
|
||||
- [x] **已验证: 完整图片处理流程** — 蜗牛聊天 4 张不同尺寸图片(289px、155px、180px)全部成功:缩略图→预览窗口→...按钮→使用预览打开→关闭 Preview→恢复
|
||||
- [x] **已验证: 带滚动的媒体处理** — scroll_to_bottom 正确到达最新消息,向上滚动正确加载历史,去重正确(processed_indices=[45,47,49,50])
|
||||
|
||||
### 待验证
|
||||
|
||||
- [ ] 验证 CGEvent 鼠标点击能否正确打开聊天会话
|
||||
- [ ] 验证 scroll_to_bottom 是否能到达最新消息
|
||||
- [ ] 验证图片预览中"..."按钮和"使用预览打开"菜单项的查找
|
||||
- [ ] 验证滚动不再触发误点击
|
||||
- [ ] 验证消息列表滚动能否加载历史消息
|
||||
- [ ] 验证 Preview.app 的检测与关闭
|
||||
- [ ] 验证最小化窗口自动恢复功能(需手动最小化微信窗口测试)
|
||||
- [ ] 有未读图片消息时的完整 --once 自动化流程
|
||||
- [ ] 长时间运行稳定性测试
|
||||
|
||||
### 未来可能的改进
|
||||
|
||||
@ -13,12 +13,12 @@ from .ax_bridge import AXBridge
|
||||
from .config import Config
|
||||
from .human_like import HumanBehavior
|
||||
from .state_machine import StateMachine, UIState
|
||||
from .wechat_ui import WeChatUI
|
||||
from .wechat_ui import WeChatUI, MessageItem
|
||||
|
||||
logger = logging.getLogger("wechat_clicker.automator")
|
||||
|
||||
SCROLL_ROUNDS = 5
|
||||
SCROLL_LINES_PER_ROUND = -10
|
||||
SCROLL_LINES_PER_ROUND = 10
|
||||
|
||||
|
||||
class WeChatAutomator:
|
||||
@ -54,8 +54,13 @@ class WeChatAutomator:
|
||||
|
||||
main_win = self.ui.get_main_window()
|
||||
if main_win is None:
|
||||
logger.error("未找到微信主窗口,请确保微信已登录并可见")
|
||||
return False
|
||||
logger.info("未找到微信主窗口,尝试恢复最小化窗口...")
|
||||
if self.ui.ensure_wechat_visible():
|
||||
main_win = self.ui.get_main_window()
|
||||
|
||||
if main_win is None:
|
||||
logger.error("未找到微信主窗口,请确保微信已登录并可见")
|
||||
return False
|
||||
|
||||
logger.info("环境检查通过")
|
||||
return True
|
||||
@ -200,78 +205,148 @@ class WeChatAutomator:
|
||||
# ----------------------------------------------------------------
|
||||
|
||||
def _process_media_with_scroll(self, main_win, msg_list, chat_name: str) -> int:
|
||||
"""先滚到底部看最新消息,再向上滚动加载历史并处理媒体。"""
|
||||
"""先滚到底部看最新消息,再向上滚动加载历史并处理媒体。
|
||||
用消息列表子元素索引做去重,避免同一张图片被重复点击。"""
|
||||
total_clicked = 0
|
||||
processed_indices = set()
|
||||
|
||||
# 先滚到消息列表底部(看到最新消息)
|
||||
logger.info(f" {chat_name}: 滚动到最新消息...")
|
||||
self.ax.scroll_to_bottom(msg_list, rounds=10, lines_per_round=20)
|
||||
self.ax.scroll_to_bottom(msg_list, rounds=10, lines_per_round=-20)
|
||||
self.human.random_delay(1.0, 2.0)
|
||||
|
||||
# 处理当前可见的媒体(最新消息)
|
||||
clicked = self._process_visible_media(main_win, chat_name)
|
||||
clicked = self._process_visible_media(
|
||||
main_win, msg_list, chat_name, processed_indices
|
||||
)
|
||||
total_clicked += clicked
|
||||
|
||||
# 向上滚动加载更多历史消息
|
||||
for round_idx in range(SCROLL_ROUNDS):
|
||||
# 安全网:滚动前确保没有残留的预览窗口
|
||||
state = self.state.detect_state()
|
||||
if state != UIState.MAIN_CHAT_LIST:
|
||||
logger.debug(
|
||||
f" 滚动前状态不干净: {state.value}, 清理额外窗口"
|
||||
)
|
||||
self._close_extra_windows()
|
||||
|
||||
self.human.delay("between_messages")
|
||||
logger.debug(f" {chat_name}: 向上滚动第 {round_idx + 1}/{SCROLL_ROUNDS} 轮")
|
||||
self.ax.scroll_at_element(msg_list, lines=SCROLL_LINES_PER_ROUND)
|
||||
self.human.random_delay(1.0, 2.5)
|
||||
|
||||
clicked = self._process_visible_media(main_win, chat_name)
|
||||
clicked = self._process_visible_media(
|
||||
main_win, msg_list, chat_name, processed_indices
|
||||
)
|
||||
total_clicked += clicked
|
||||
|
||||
logger.debug(
|
||||
f" {chat_name}: 滚动结束, 已处理索引={sorted(processed_indices)}"
|
||||
)
|
||||
return total_clicked
|
||||
|
||||
def _process_visible_media(self, main_win, chat_name: str) -> int:
|
||||
"""处理当前可见的所有媒体消息。"""
|
||||
media_messages = self.ui.get_media_messages(main_win)
|
||||
if not media_messages:
|
||||
logger.debug(f" {chat_name}: 当前视图无媒体消息")
|
||||
return 0
|
||||
def _process_visible_media(
|
||||
self, main_win, msg_list, chat_name: str, processed_indices: set
|
||||
) -> int:
|
||||
"""处理当前可见且未处理过的媒体消息。
|
||||
通过子元素索引去重,通过可见区域过滤跳过裁剪元素。"""
|
||||
# 获取消息列表可见区域
|
||||
list_pos = self.ax.get_position(msg_list)
|
||||
list_size = self.ax.get_size(msg_list)
|
||||
visible_top = list_pos[1]
|
||||
visible_bottom = list_pos[1] + list_size[1]
|
||||
margin = 30
|
||||
|
||||
# 遍历消息列表的所有子元素,带索引
|
||||
children = self.ax.get_children(msg_list)
|
||||
targets = []
|
||||
for msg in media_messages:
|
||||
if msg.msg_type == "image" and self.config.click_images:
|
||||
targets.append(msg)
|
||||
elif msg.msg_type == "file" and self.config.click_files:
|
||||
targets.append(msg)
|
||||
elif msg.msg_type == "video" and self.config.click_videos:
|
||||
targets.append(msg)
|
||||
|
||||
for idx, child in enumerate(children):
|
||||
if idx in processed_indices:
|
||||
continue
|
||||
|
||||
role = self.ax.get_role(child)
|
||||
if role != "AXStaticText":
|
||||
continue
|
||||
|
||||
title = self.ax.get_title(child)
|
||||
if not title:
|
||||
continue
|
||||
|
||||
size = self.ax.get_size(child)
|
||||
if size[0] == 0 or size[1] == 0:
|
||||
continue
|
||||
|
||||
msg_type = WeChatUI._classify_message(title, size)
|
||||
if msg_type not in ("image", "file", "video"):
|
||||
continue
|
||||
|
||||
# 检查配置是否处理该类型
|
||||
if msg_type == "image" and not self.config.click_images:
|
||||
continue
|
||||
if msg_type == "file" and not self.config.click_files:
|
||||
continue
|
||||
if msg_type == "video" and not self.config.click_videos:
|
||||
continue
|
||||
|
||||
# 可见性检查:元素中心必须在消息列表可见区域内(带 margin)
|
||||
pos = self.ax.get_position(child)
|
||||
elem_center_y = pos[1] + size[1] // 2
|
||||
|
||||
if elem_center_y < visible_top + margin:
|
||||
logger.debug(
|
||||
f" 跳过(中心在顶部外): idx={idx} type={msg_type} "
|
||||
f"center_y={elem_center_y} < list_top+margin={visible_top + margin}"
|
||||
)
|
||||
continue
|
||||
if elem_center_y > visible_bottom - margin:
|
||||
logger.debug(
|
||||
f" 跳过(中心在底部外): idx={idx} type={msg_type} "
|
||||
f"center_y={elem_center_y} > list_bottom-margin={visible_bottom - margin}"
|
||||
)
|
||||
continue
|
||||
|
||||
targets.append((idx, child, msg_type, title, size, pos))
|
||||
|
||||
if not targets:
|
||||
logger.debug(
|
||||
f" {chat_name}: 发现 {len(media_messages)} 个媒体, "
|
||||
f"但无符合配置的处理目标"
|
||||
)
|
||||
logger.debug(f" {chat_name}: 当前视图无可处理的新媒体")
|
||||
return 0
|
||||
|
||||
logger.info(f" {chat_name}: 当前可见 {len(targets)} 个媒体消息")
|
||||
logger.info(
|
||||
f" {chat_name}: 当前可见 {len(targets)} 个新媒体 "
|
||||
f"(已处理 {len(processed_indices)} 个)"
|
||||
)
|
||||
|
||||
clicked = 0
|
||||
for idx, msg in enumerate(targets):
|
||||
for idx, child, msg_type, title, size, pos in targets:
|
||||
self.human.delay("before_click_media")
|
||||
|
||||
pos = self.ax.get_position(msg.element)
|
||||
short_title = msg.title.replace("\n", " ")[:50]
|
||||
short_title = title.replace("\n", " ")[:50]
|
||||
logger.info(
|
||||
f" [{msg.msg_type}] {short_title} "
|
||||
f"(pos={pos}, size={msg.size}, #{idx+1}/{len(targets)})"
|
||||
f" [{msg_type}] {short_title} "
|
||||
f"(idx={idx}, pos={pos}, size={size})"
|
||||
)
|
||||
|
||||
if msg.msg_type == "image":
|
||||
success = self._click_image(msg)
|
||||
elif msg.msg_type == "file":
|
||||
success = self._click_file(msg)
|
||||
msg_item = MessageItem(
|
||||
element=child, msg_type=msg_type, title=title, size=size
|
||||
)
|
||||
|
||||
if msg_type == "image":
|
||||
success = self._click_image(msg_item)
|
||||
elif msg_type == "file":
|
||||
success = self._click_file(msg_item)
|
||||
else:
|
||||
success = self._click_generic(msg)
|
||||
success = self._click_generic(msg_item)
|
||||
|
||||
# 无论成功失败都标记为已处理,避免重复尝试
|
||||
processed_indices.add(idx)
|
||||
|
||||
if success:
|
||||
clicked += 1
|
||||
logger.debug(f" 处理成功: {msg.msg_type}")
|
||||
logger.debug(f" 处理成功: idx={idx} {msg_type}")
|
||||
else:
|
||||
logger.warning(f" 处理失败: {msg.msg_type}")
|
||||
logger.warning(f" 处理失败: idx={idx} {msg_type}")
|
||||
|
||||
self.human.delay("between_messages")
|
||||
|
||||
@ -289,8 +364,12 @@ class WeChatAutomator:
|
||||
def _click_image(self, msg) -> bool:
|
||||
"""处理一张图片:点开大图 -> 点... -> 点使用预览打开 -> 关闭 Preview.app。"""
|
||||
# Step 1: 点击图片缩略图打开大图预览
|
||||
# 图片气泡在消息行 AXStaticText 的左侧区域(非中心),
|
||||
# 用偏移 (120, height//2) 命中实际缩略图
|
||||
logger.debug(" Step1: 点击图片缩略图")
|
||||
if not self.ax.click_at_element(msg.element):
|
||||
x_off = 120
|
||||
y_off = msg.size[1] // 2
|
||||
if not self.ax.click_at_element_offset(msg.element, x_off, y_off):
|
||||
logger.warning(" 图片缩略图点击失败")
|
||||
return False
|
||||
|
||||
@ -346,17 +425,12 @@ class WeChatAutomator:
|
||||
|
||||
self.human.random_delay(2.0, 4.0)
|
||||
|
||||
# Step 7: 关闭 Preview.app
|
||||
if self.state.is_preview_app_running():
|
||||
logger.debug(" Step7: 关闭 Preview.app")
|
||||
self.state.close_preview_app()
|
||||
else:
|
||||
logger.debug(" Step7: Preview.app 未运行,跳过关闭")
|
||||
# Step 7: 关闭所有额外窗口(Preview.app + 微信预览窗口)
|
||||
logger.debug(" Step7: 清理预览窗口")
|
||||
self._close_extra_windows()
|
||||
|
||||
# Step 8: 确保微信回到前台,关闭大图预览
|
||||
logger.debug(" Step8: 恢复微信前台,Escape 关闭预览")
|
||||
self.ui.ensure_wechat_frontmost()
|
||||
time.sleep(0.3)
|
||||
# Step 8: Escape 关闭可能残留的内嵌预览
|
||||
logger.debug(" Step8: Escape 关闭内嵌预览")
|
||||
self.ax.send_escape_key()
|
||||
time.sleep(0.5)
|
||||
|
||||
@ -399,19 +473,21 @@ class WeChatAutomator:
|
||||
# ----------------------------------------------------------------
|
||||
|
||||
def _click_file(self, msg) -> bool:
|
||||
"""点击文件触发下载。"""
|
||||
if not self.ax.click_at_element(msg.element):
|
||||
"""点击文件触发下载,然后关闭可能弹出的预览窗口。"""
|
||||
x_off = 120
|
||||
y_off = msg.size[1] // 2
|
||||
if not self.ax.click_at_element_offset(msg.element, x_off, y_off):
|
||||
logger.warning(" 文件点击失败")
|
||||
return False
|
||||
|
||||
self.human.delay("after_click_media")
|
||||
|
||||
# 文件点击后可能会打开文件预览,关闭它
|
||||
self.human.random_delay(1.0, 2.0)
|
||||
|
||||
state = self.state.detect_state()
|
||||
if state == UIState.MEDIA_PREVIEW:
|
||||
self.ax.send_escape_key()
|
||||
time.sleep(0.5)
|
||||
logger.debug(f" 文件点击后状态={state.value}")
|
||||
|
||||
if state != UIState.MAIN_CHAT_LIST:
|
||||
self._close_extra_windows()
|
||||
|
||||
return True
|
||||
|
||||
@ -420,7 +496,9 @@ class WeChatAutomator:
|
||||
# ----------------------------------------------------------------
|
||||
|
||||
def _click_generic(self, msg) -> bool:
|
||||
if not self.ax.click_at_element(msg.element):
|
||||
x_off = 120
|
||||
y_off = msg.size[1] // 2
|
||||
if not self.ax.click_at_element_offset(msg.element, x_off, y_off):
|
||||
return False
|
||||
self.human.delay("after_click_media")
|
||||
self.human.micro_jitter()
|
||||
@ -428,17 +506,45 @@ class WeChatAutomator:
|
||||
time.sleep(0.5)
|
||||
return True
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# 窗口清理:关闭预览等额外窗口
|
||||
# ----------------------------------------------------------------
|
||||
|
||||
def _close_extra_windows(self):
|
||||
"""关闭所有非主窗口(文件预览、图片预览等)和 Preview.app,恢复干净状态。"""
|
||||
if self.state.is_preview_app_running():
|
||||
logger.debug(" 关闭 Preview.app")
|
||||
self.state.close_preview_app()
|
||||
|
||||
conv_wins = self.ui.get_conversation_windows()
|
||||
for win in conv_wins:
|
||||
title = self.ax.get_title(win)
|
||||
logger.debug(f" 关闭额外窗口: {title}")
|
||||
close_btn = self.ui.get_close_button(win)
|
||||
if close_btn:
|
||||
self.ax.press(close_btn)
|
||||
else:
|
||||
self.ax.send_cmd_w()
|
||||
time.sleep(0.5)
|
||||
|
||||
self.ui.ensure_wechat_frontmost()
|
||||
time.sleep(0.3)
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# 调试工具
|
||||
# ----------------------------------------------------------------
|
||||
|
||||
def dump_ui_tree(self):
|
||||
def dump_ui_tree(self, chat_name: str = None):
|
||||
if not self.verify_setup():
|
||||
return
|
||||
|
||||
self.ui.ensure_wechat_frontmost()
|
||||
time.sleep(0.5)
|
||||
|
||||
if chat_name:
|
||||
self._dump_chat_messages(chat_name)
|
||||
return
|
||||
|
||||
main_win = self.ui.get_main_window()
|
||||
if main_win:
|
||||
print("=== 主窗口 (微信) ===")
|
||||
@ -461,6 +567,84 @@ class WeChatAutomator:
|
||||
f"| pos={pos} size={size}"
|
||||
)
|
||||
|
||||
def _dump_chat_messages(self, chat_name: str):
|
||||
"""进入指定聊天,dump 消息列表中每个子元素的完整子树。"""
|
||||
# 确保在聊天列表
|
||||
if not self.state.recover_to_chat_list():
|
||||
print(f"ERROR: 无法恢复到聊天列表")
|
||||
return
|
||||
|
||||
# 在聊天列表中查找匹配的聊天项(子串匹配)
|
||||
items = self.ui.get_chat_items()
|
||||
target = None
|
||||
for item in items:
|
||||
if chat_name in item.name:
|
||||
target = item
|
||||
break
|
||||
|
||||
if target is None:
|
||||
print(f"ERROR: 聊天列表中未找到包含 \"{chat_name}\" 的聊天")
|
||||
print("当前可见聊天:")
|
||||
for item in items:
|
||||
print(f" - {item.name}")
|
||||
return
|
||||
|
||||
print(f"找到聊天: {target.name} (未读: {target.unread_count})")
|
||||
print(f"点击进入...")
|
||||
|
||||
# 点击聊天项(先点其他聊天再点回来,避免重复选中不加载)
|
||||
other = next((i for i in items if i.name != target.name), None)
|
||||
if other:
|
||||
self.ax.click_at_element(other.element)
|
||||
time.sleep(0.5)
|
||||
|
||||
if not self.ax.click_at_element(target.element):
|
||||
print(f"ERROR: 点击聊天项失败")
|
||||
return
|
||||
|
||||
time.sleep(2.0)
|
||||
|
||||
# 在主窗口中找消息列表(重试几次,等待 UI 加载)
|
||||
main_win = self.ui.get_main_window()
|
||||
if main_win is None:
|
||||
print("ERROR: 主窗口丢失")
|
||||
return
|
||||
|
||||
msg_list = None
|
||||
for attempt in range(3):
|
||||
msg_list = self.ui.get_message_list(main_win)
|
||||
if msg_list is not None:
|
||||
break
|
||||
print(f" 等待消息列表加载... (尝试 {attempt + 1}/3)")
|
||||
time.sleep(1.5)
|
||||
|
||||
if msg_list is None:
|
||||
print("ERROR: 未找到消息列表,dump 主窗口结构以诊断:")
|
||||
print(self.ax.dump_element(main_win, max_depth=8))
|
||||
return
|
||||
|
||||
# 滚到底部看最新消息
|
||||
print("滚动到最新消息...")
|
||||
self.ax.scroll_to_bottom(msg_list, rounds=10, lines_per_round=-20)
|
||||
time.sleep(1.5)
|
||||
|
||||
# 遍历消息列表所有子元素
|
||||
children = self.ax.get_children(msg_list)
|
||||
print(f"\n=== 消息列表 ({target.name}) ===")
|
||||
print(f"子元素总数: {len(children)}")
|
||||
print()
|
||||
|
||||
for idx, child in enumerate(children):
|
||||
role = self.ax.get_role(child)
|
||||
title = self.ax.get_title(child)
|
||||
size = self.ax.get_size(child)
|
||||
pos = self.ax.get_position(child)
|
||||
|
||||
short_title = (title or "").replace("\n", "\\n")[:60]
|
||||
print(f"--- [{idx}] role={role} title=\"{short_title}\" size={size} pos={pos} ---")
|
||||
print(self.ax.dump_element(child, max_depth=10))
|
||||
print()
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# 统计
|
||||
# ----------------------------------------------------------------
|
||||
|
||||
@ -5,6 +5,7 @@
|
||||
|
||||
import logging
|
||||
import re
|
||||
import subprocess
|
||||
import time
|
||||
|
||||
from ApplicationServices import (
|
||||
@ -102,6 +103,28 @@ class AXBridge:
|
||||
app = apps[0]
|
||||
return app.activateWithOptions_(NSApplicationActivateIgnoringOtherApps)
|
||||
|
||||
def is_app_running(self, bundle_id: str) -> bool:
|
||||
"""检查应用是否正在运行(不依赖窗口状态)。"""
|
||||
apps = NSRunningApplication.runningApplicationsWithBundleIdentifier_(bundle_id)
|
||||
return apps is not None and len(apps) > 0
|
||||
|
||||
def unminimize_app(self, app_name: str) -> bool:
|
||||
"""通过 AppleScript 取消应用窗口的最小化状态。"""
|
||||
script = (
|
||||
f'tell application "{app_name}" to '
|
||||
f'set miniaturized of every window to false'
|
||||
)
|
||||
try:
|
||||
subprocess.run(
|
||||
["osascript", "-e", script],
|
||||
capture_output=True, timeout=5
|
||||
)
|
||||
logger.info(f"已取消 {app_name} 窗口最小化")
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"取消最小化失败: {e}")
|
||||
return False
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# 属性读取
|
||||
# ----------------------------------------------------------------
|
||||
@ -268,6 +291,23 @@ class AXBridge:
|
||||
)
|
||||
return self._mouse_click(cx, cy)
|
||||
|
||||
def click_at_element_offset(self, element, x_offset: int, y_offset: int) -> bool:
|
||||
"""通过鼠标事件点击元素指定偏移位置。"""
|
||||
pos = self.get_position(element)
|
||||
size = self.get_size(element)
|
||||
if pos == (0, 0) and size == (0, 0):
|
||||
logger.warning("元素位置/尺寸不可用,无法点击")
|
||||
return False
|
||||
cx = pos[0] + x_offset
|
||||
cy = pos[1] + y_offset
|
||||
role = self.get_role(element)
|
||||
title = (self.get_title(element) or "").replace("\n", "\\n")[:40]
|
||||
logger.debug(
|
||||
f"click_at_element_offset: ({cx}, {cy}) offset=({x_offset},{y_offset}) "
|
||||
f"role={role} title=\"{title}\" pos={pos} size={size}"
|
||||
)
|
||||
return self._mouse_click(cx, cy)
|
||||
|
||||
def _mouse_click(self, x: int, y: int) -> bool:
|
||||
"""在屏幕坐标 (x, y) 处执行鼠标左键点击。"""
|
||||
try:
|
||||
@ -331,8 +371,8 @@ class AXBridge:
|
||||
cy = pos[1] + size[1] // 2
|
||||
self._scroll_at(cx, cy, lines)
|
||||
|
||||
def scroll_to_bottom(self, element, rounds: int = 10, lines_per_round: int = 20):
|
||||
"""向下滚动多次,尽量到达元素(如消息列表)的底部。"""
|
||||
def scroll_to_bottom(self, element, rounds: int = 10, lines_per_round: int = -20):
|
||||
"""向下滚动多次,尽量到达元素(如消息列表)的底部。负值=向下滚动。"""
|
||||
pos = self.get_position(element)
|
||||
size = self.get_size(element)
|
||||
if pos == (0, 0) and size == (0, 0):
|
||||
@ -422,11 +462,13 @@ class AXBridge:
|
||||
name_attr = self.get_attribute(element, "AXValue") or ""
|
||||
desc = self.get_description(element)
|
||||
size = self.get_size(element)
|
||||
pos = self.get_position(element)
|
||||
actions = self.get_action_names(element)
|
||||
identifier = self.get_attribute(element, "AXIdentifier") or ""
|
||||
|
||||
prefix = " " * indent
|
||||
info_parts = [f"role={role}"]
|
||||
if title:
|
||||
# 截断过长的 title,用单行表示
|
||||
short_title = title.replace("\n", "\\n")[:80]
|
||||
info_parts.append(f'title="{short_title}"')
|
||||
if name_attr:
|
||||
@ -436,6 +478,12 @@ class AXBridge:
|
||||
info_parts.append(f'desc="{desc}"')
|
||||
if size != (0, 0):
|
||||
info_parts.append(f"size={size[0]}x{size[1]}")
|
||||
if pos != (0, 0):
|
||||
info_parts.append(f"pos=({pos[0]},{pos[1]})")
|
||||
if actions:
|
||||
info_parts.append(f"actions={actions}")
|
||||
if identifier:
|
||||
info_parts.append(f'id="{identifier}"')
|
||||
|
||||
lines.append(f"{prefix}[{', '.join(info_parts)}]")
|
||||
|
||||
|
||||
@ -26,6 +26,7 @@ class UIState(Enum):
|
||||
CONVERSATION_OPEN = "conversation_open"
|
||||
MEDIA_PREVIEW = "media_preview"
|
||||
WECHAT_NOT_RUNNING = "wechat_not_running"
|
||||
WECHAT_MINIMIZED = "wechat_minimized"
|
||||
|
||||
|
||||
class StateMachine:
|
||||
@ -54,9 +55,14 @@ class StateMachine:
|
||||
windows = self.ui.get_all_windows()
|
||||
|
||||
if not windows:
|
||||
self._current_state = UIState.WECHAT_NOT_RUNNING
|
||||
self._conversation_name = None
|
||||
logger.debug("未检测到微信窗口")
|
||||
if self.ax.is_app_running(self.ui.bundle_id):
|
||||
self._current_state = UIState.WECHAT_MINIMIZED
|
||||
self._conversation_name = None
|
||||
logger.debug("微信在运行但无可见窗口(可能已最小化)")
|
||||
else:
|
||||
self._current_state = UIState.WECHAT_NOT_RUNNING
|
||||
self._conversation_name = None
|
||||
logger.debug("微信未运行")
|
||||
return self._current_state
|
||||
|
||||
# 分析窗口
|
||||
@ -117,6 +123,15 @@ class StateMachine:
|
||||
logger.error("微信未运行,无法恢复")
|
||||
return False
|
||||
|
||||
if state == UIState.WECHAT_MINIMIZED:
|
||||
logger.info("微信窗口已最小化,正在恢复...")
|
||||
if self.ui.ensure_wechat_visible():
|
||||
time.sleep(0.5)
|
||||
continue
|
||||
else:
|
||||
logger.error("微信窗口恢复失败")
|
||||
return False
|
||||
|
||||
if state == UIState.MEDIA_PREVIEW:
|
||||
# 先关闭预览(Escape)
|
||||
logger.info("关闭媒体预览...")
|
||||
|
||||
@ -10,6 +10,7 @@
|
||||
|
||||
import logging
|
||||
import re
|
||||
import time
|
||||
|
||||
from .ax_bridge import AXBridge
|
||||
|
||||
@ -75,6 +76,30 @@ class WeChatUI:
|
||||
"""确保微信在最前台。"""
|
||||
return self.ax.bring_to_front(self.bundle_id)
|
||||
|
||||
def ensure_wechat_visible(self) -> bool:
|
||||
"""确保微信窗口可见(取消最小化并带到前台)。"""
|
||||
windows = self.get_all_windows()
|
||||
if windows:
|
||||
return self.ensure_wechat_frontmost()
|
||||
|
||||
if not self.ax.is_app_running(self.bundle_id):
|
||||
logger.error("微信未运行")
|
||||
return False
|
||||
|
||||
logger.info("微信窗口不可见(可能已最小化),尝试恢复...")
|
||||
self.ax.unminimize_app("WeChat")
|
||||
time.sleep(0.5)
|
||||
self.ax.bring_to_front(self.bundle_id)
|
||||
time.sleep(0.5)
|
||||
|
||||
windows = self.get_all_windows()
|
||||
if windows:
|
||||
logger.info("微信窗口已恢复可见")
|
||||
return True
|
||||
|
||||
logger.error("微信窗口恢复失败")
|
||||
return False
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# 窗口查找
|
||||
# ----------------------------------------------------------------
|
||||
|
||||
Loading…
Reference in New Issue
Block a user