python~windows自动化工具 uiautomation库

微软提供了关于自动化操作PC端桌面应用程序的工具，官方描述为：

　　Microsoft UI Automation is an accessibility framework that enables Windows applications to provide and consume programmatic information about user interfaces (UIs). It provides programmatic access to most UI elements on the desktop. It enables assistive technology products, such as screen readers, to provide information about the UI to end users and to manipulate the UI by means other than standard input. UI Automation also allows automated test scripts to interact with the UI.

　　Python库为我们封装了shoppingmode 微软UI Automation的API，即uiautomation库。

资料

微软官网关于UI Automation的介绍：

1、IUIAutomationElement interface:

https://docs.microsoft.com/en-us/windows/win32/api/uiautomationclient/nn-uiautomationclient-iuiautomationelement

2、Control Types and Their Supported Control Patterns，查看全部的Control-Pattern支持表格。

https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-controlpatternmapping

3、uiautomation库的具体用法，GitHub上开发者发布的教程：

https://github.com/yinkaisheng/Python-UIAutomation-for-Windows/blob/master/readme_cn.md

4、作者自己也有博客

https://www.cnblogs.com/Yinkaisheng/p/3444132.html

安装automation

1、安装uiautomation库

# pip install uiautomation

python中默认的安装目录为: C:\Users\XXX\AppData\Local\Programs\Python\Python310\Scripts

2、使用

进入windwos PowerShell，到automation.py的目录，通过命令: python automation.py +参数：

因为加入和环境变量，也可以直接的cmd中，输入: automation.py -t3

1）、查看帮助

PS C:\Users\gXXX\AppData\Local\Programs\Python\Python310\Scripts> python automation.py -h
UIAutomation 2.0.16 (Python 3.10.5, 64 bit)
usage
-h      show command help
-t      delay time, default 3 seconds, begin to enumerate after Value seconds, this must be an integer
        you can delay a few seconds and make a window active so automation can enumerate the active window
-d      enumerate tree depth, this must be an integer, if it is null, enumerate the whole tree
-r      enumerate from root:Desktop window, if it is null, enumerate from foreground window
-f      enumerate from focused control, if it is null, enumerate from foreground window
-c      enumerate the control under cursor, if depth is < 0, enumerate from its ancestor up to depth
-a      show ancestors of the control under cursor
-n      show control full name, if it is null, show first 30 characters of control's name in console,
        always show full name in log file @AutomationLog.txt
-p      show process id of controls

运行automation.py遍历控件时，支持下列参数

-t int value, 延迟时间time秒，延迟指定秒数再遍历控件，
-r, 从树的根部(root: Desktop)遍历，如果不指定，从当前窗口遍历
-d, int Value, 遍历控件树的的深度depth，如果不指定，遍历整个树，和-c一起使用时，可以为负值
-f, 遍历焦点focus控件，如果不指定，从当前窗口遍历
-c, 遍历光标下的控件，如果不指定，从当前窗口遍历，如果同时指定-d, -d可以为负值，比如-d-2表示从光标下控件向上找到两层父控件，遍历这个父控件

-a, 获取光标下控件及其所有祖先(ancestor)控件

-n, 显示控件的完整name, 如果不指定，只显示前30个字符

举例：

automation.py –t3， 3秒后遍历当前窗口所有控件
automation.py –d2 –t3， 3秒后遍历当前窗口前三层控件
automation.py –r –d1 –t0 -n， 0秒后遍历root的第一层子控件，并显示控件完整名称

automation.py –c –t3， 3秒后遍历鼠标光标下面的控件信息

automation.py –c –t3 -d-2， 3秒后遍历鼠标光标下面的控件向上两层的父控件

2）、抓取微信控件

-t3，表示延迟3s抓取；

此处执行命令后，将鼠标移至微信，抓取信息如下：

C:\Users\gongl1>automation.py -d4 -t3
UIAutomation 2.0.16 (Python 3.10.5, 64 bit)
please wait for 3 seconds

2022-07-11 11:58:54.240 automation.py[79] main -> Starts, Current Cursor Position: (285, 379)
ControlType: PaneControl ClassName: #32769 AutomationId: Rect: (0,0,1920,1080)[1920x1080] Name: 'Desktop 1' Handle: 0x10010(65552) Depth: 0 SupportedPattern: LegacyIAccessiblePattern
ControlType: WindowControl ClassName: WeChatMainWndForPC AutomationId: Rect: (30,95,1168,895)[1138x800] Name: 'WeChat' Handle: 0x20742(132930) Depth: 1 SupportedPattern: LegacyIAccessiblePattern ScrollItemPattern TransformPattern WindowPattern
ControlType: ButtonControl ClassName: AutomationId: Rect: (41,140,86,185)[45x45] Name: '磊' Handle: 0x0(0) Depth: 5 SupportedPattern: InvokePattern LegacyIAccessiblePattern ScrollItemPattern
ControlType: ButtonControl ClassName: AutomationId: Rect: (32,204,95,259)[63x55] Name: 'Chats' Handle: 0x0(0) Depth: 5 SupportedPattern: InvokePattern LegacyIAccessiblePattern ScrollItemPattern
ControlType: ButtonControl ClassName: AutomationId: Rect: (32,255,95,310)[63x55] Name: 'Contacts' Handle: 0x0(0) Depth: 5 SupportedPattern: InvokePattern LegacyIAccessiblePattern ScrollItemPattern
ControlType: ButtonControl ClassName: AutomationId: Rect: (32,306,95,361)[63x55] Name: 'Favorites' Handle: 0x0(0) Depth: 5 SupportedPattern: InvokePattern LegacyIAccessiblePattern ScrollItemPattern
ControlType: ButtonControl ClassName: AutomationId: Rect: (32,357,95,412)[63x55] Name: 'WeChat Files' Handle: 0x0(0) Depth: 5 SupportedPattern: InvokePattern LegacyIAccessiblePattern ScrollItemPattern
ControlType: ButtonControl ClassName: AutomationId: Rect: (32,408,95,463)[63x55] Name: 'Moments' Handle: 0x0(0) Depth: 5 SupportedPattern: InvokePattern LegacyIAccessiblePattern ScrollItemPattern
ControlType: ButtonControl ClassName: AutomationId: Rect: (32,561,95,616)[63x55] Name: 'Search' Handle: 0x0(0) Depth: 5 SupportedPattern: InvokePattern LegacyIAccessiblePattern ScrollItemPattern
ControlType: ButtonControl ClassName: AutomationId: Rect: (48,741,79,772)[31x31] Name: 'Mini Programs Panel' Handle: 0x0(0) Depth: 5 SupportedPattern: InvokePattern LegacyIAccessiblePattern ScrollItemPattern
2022-07-11 11:58:55.270 automation.py[114] main -> Ends

C:\Users\gongl1>

这些参数就对应：

对于未读消息，可以查看到：

ScrollItemPattern
ControlType: ListControl ClassName: AutomationId: Rect: (82,212,394,934)[312x722] Name: '会话' Handle: 0x0(0) Depth: 9 ScrollPattern.HorizontalScrollPercent: 0.0 ScrollPattern.VerticalScrollPercent: 0.0060790273556231 SupportedPattern: LegacyIAccessiblePattern ScrollItemPattern ScrollPattern SelectionPattern
ControlType: ListItemControl ClassName: AutomationId: Rect: (82,212,394,292)[312x80] Name: '群test' Handle: 0x0(0) Depth: 10 SelectionItemPattern.IsSelected: False ValuePattern.Value: '张三：未读信息' SupportedPattern: LegacyIAccessiblePattern ScrollItemPattern

# 绑定微信的主程序窗口

wx=iautomation.WindowControl(Name='WeChat')

# 切到到程序窗口

wx.SwitchToThisWindow()

# 绑定会话窗口

hw=wx.ListControl(Name='Chats')