【未解决】uiautomator2中如何获取到欢乐大作战游戏界面中的元素

折腾：

【未解决】给安卓手机小米9中欢乐大作战的游戏实现自动挂机

期间，已经可以从weditor中看到界面了。

但是发现dum出来的元素，没有包含具体按钮等位置。

只有部分的大的区域的节点。

所以去想办法去写代码看看如何能获取到游戏界面中的其他子元素

然后先启动代码：

def autoTestAndroidGame():
  initLogging()
 
  d = u2.connect(AndroidDeviceId_Xiaomi9)
  d.debug = True
  logging.info("d.info=%s", d.info)

可以输出了基本信息：

➜  autoTestAndroidGameHappyBigBattle cd /Users/crifan/dev/dev_root/python/autoTestAndroid/autoTestAndroidGameHappyBigBattle ; env PYTHONIOENCODING=UTF-8 PYTHONUNBUFFERED=1 /Users/crifan/.local/share/virtualenvs/autoTestAndroidGameHappyBigBattle-F2NzwZ7Z/bin/python /Users/crifan/.vscode/extensions/ms-python.python-2019.10.44104/pythonFiles/ptvsd_launcher.py --default --client --host localhost --port 53023 /Users/crifan/dev/dev_root/python/autoTestAndroid/autoTestAndroidGameHappyBigBattle/autoTestAndroidGame.py 
21:01:32.421 $ curl -X POST -d 'b'{"jsonrpc": "2.0", "id": "9b58bea5d6c1335115fdde220e74bc45", "method": "deviceInfo", "params": {}}'' '
http://127.0.0.1:51426/jsonrpc/0
'
21:01:32.574 Response (153 ms) >>>
{"jsonrpc":"2.0","id":"9b58bea5d6c1335115fdde220e74bc45","result":{"currentPackageName":"com.tencent.tmgp.hldzzsyol","displayHeight":1080,"displayRotation":3,"displaySizeDpX":851,"displaySizeDpY":393,"displayWidth":2135,"productName":"cepheus","sdkInt":29,"naturalOrientation":false}}
<<< END
20191122 09:01:32 autoTestAndroidGame.py:26   INFO    d.info={'currentPackageName': 'com.tencent.tmgp.hldzzsyol', 'displayHeight': 1080, 'displayRotation': 3, 'displaySizeDpX': 851, 'displaySizeDpY': 393, 'displayWidth': 2135, 'productName': 'cepheus', 'sdkInt': 29, 'naturalOrientation': False}

然后接着要去启动游戏app

所以要知道主页面：

【已解决】分析安卓游戏app欢乐大作战的主页面mainActivity

然后：

【已解决】uiautomator2用app_start启动app报错：uiautomator2.exceptions.BaseError package not found

启动了，可以看到此app启动后，会经历几个界面后，才能进入主页面。

注：在之前已经登录了账户的前提下。

（1）启动logo

（2）宣传图

初始化组件

再去

连接服务器

以及：

正在登录账号

（3）加载进度条

最后才是主页

所以现在要分别去检测出不同页面，以及页面中有的元素

随便试了试代码：

1	`d(text="初始化组件").wait(timeout=10.0)`

等了10秒后，结果没找到而报错：

22:15:47.696 $ curl -X POST -d 'b'{"jsonrpc": "2.0", "id": "a9dd75ae8b645b6e6493113d9ea09bfe", "method": "waitForExists", "params": [{"mask": 1, "childOrSibling": [], "childOrSiblingSelector": [], "text": "\\u521d\\u59cb\\u5316\\u7ec4\\u4ef6"}, 20000]}'' 'http://127.0.0.1:51426/jsonrpc/0'
22:16:09.165 Response (21469 ms) >>>
{"jsonrpc":"2.0","id":"a9dd75ae8b645b6e6493113d9ea09bfe","result":false}
<<< END
Traceback (most recent call last):
  File "/Users/crifan/.vscode/extensions/ms-python.python-2019.10.44104/pythonFiles/ptvsd_launcher.py", line 43, in <module>
    main(ptvsdArgs)
  File "/Users/crifan/.vscode/extensions/ms-python.python-2019.10.44104/pythonFiles/lib/python/old_ptvsd/ptvsd/__main__.py", line 432, in main
    run()
  File "/Users/crifan/.vscode/extensions/ms-python.python-2019.10.44104/pythonFiles/lib/python/old_ptvsd/ptvsd/__main__.py", line 316, in run_file
    runpy.run_path(target, run_name='__main__')
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/crifan/dev/dev_root/python/autoTestAndroid/autoTestAndroidGameHappyBigBattle/autoTestAndroidGame.py", line 55, in <module>
    autoTestAndroidGame()
  File "/Users/crifan/dev/dev_root/python/autoTestAndroid/autoTestAndroidGameHappyBigBattle/autoTestAndroidGame.py", line 50, in autoTestAndroidGame
    d(text="初始化组件").click()
  File "/Users/crifan/.local/share/virtualenvs/autoTestAndroidGameHappyBigBattle-F2NzwZ7Z/lib/python3.7/site-packages/uiautomator2/session.py", line 38, in _inner
    return fn(self, *args, **kwargs)
  File "/Users/crifan/.local/share/virtualenvs/autoTestAndroidGameHappyBigBattle-F2NzwZ7Z/lib/python3.7/site-packages/uiautomator2/session.py", line 803, in click
    self.must_wait(timeout=timeout)
  File "/Users/crifan/.local/share/virtualenvs/autoTestAndroidGameHappyBigBattle-F2NzwZ7Z/lib/python3.7/site-packages/uiautomator2/session.py", line 998, in must_wait
    raise UiObjectNotFoundError({'code': -32002, 'method': 'wait'})
uiautomator2.exceptions.UiObjectNotFoundError: -32002 Client error: <> data: , method: None

因为此时游戏已进入主界面

算了，接着去想办法检测主界面

weditor检测了半天，还是只是找到有限的可用元素

比如找个：描述是Game view

想办法获取到当前页面的所有节点

https://github.com/openatx/uiautomator2/blob/master/uiautomator2/ext/xpath/README.md

  allElementList = d.xpath("//*").all()
  if allElementList:
    allElementListLen = len(allElementList)
    logging.info("allElementListLen=%s", allElementListLen)
    for eachElement in allElementList:
      logging.info("eachElement=%s", eachElement)

调试了下，就只有6个元素

其中的内容，和weditor看到的是基本上一样的：

感觉最底层的那个节点：

android.view.View

估计是内部用canvas去绘图的？

所以抓不到子节点？

Common issues · openatx/uiautomator2 Wiki

uiautomator2 游戏抓不到元素

uiautomator2 game app no xml node

uiautomator2 游戏抓取节点

另外试了试，点击其中的战斗按钮，结果会报错：

Traceback (most recent call last):
  File "/Users/crifan/.local/share/virtualenvs/autoTestAndroidGameHappyBigBattle-F2NzwZ7Z/lib/python3.7/site-packages/weditor/web/handlers/page.py", line 231, in run
    ret = eval(code, self._global)
  File "<string>", line 1, in <module>
  File "/Users/crifan/.local/share/virtualenvs/autoTestAndroidGameHappyBigBattle-F2NzwZ7Z/lib/python3.7/site-packages/uiautomator2/session.py", line 365, in click
    self._click(x, y)
  File "/Users/crifan/.local/share/virtualenvs/autoTestAndroidGameHappyBigBattle-F2NzwZ7Z/lib/python3.7/site-packages/uiautomator2/utils.py", line 54, in inner
    ret = fn(self, *args, **kwargs)
  File "/Users/crifan/.local/share/virtualenvs/autoTestAndroidGameHappyBigBattle-F2NzwZ7Z/lib/python3.7/site-packages/uiautomator2/session.py", line 369, in _click
    self.jsonrpc.click(x, y)
  File "/Users/crifan/.local/share/virtualenvs/autoTestAndroidGameHappyBigBattle-F2NzwZ7Z/lib/python3.7/site-packages/uiautomator2/__init__.py", line 440, in __call__
    params, http_timeout)
  File "</Users/crifan/.local/share/virtualenvs/autoTestAndroidGameHappyBigBattle-F2NzwZ7Z/lib/python3.7/site-packages/decorator.py:decorator-gen-12>", line 2, in jsonrpc_retry_call
  File "/Users/crifan/.local/share/virtualenvs/autoTestAndroidGameHappyBigBattle-F2NzwZ7Z/lib/python3.7/site-packages/retry/api.py", line 74, in retry_decorator
    logger)
  File "/Users/crifan/.local/share/virtualenvs/autoTestAndroidGameHappyBigBattle-F2NzwZ7Z/lib/python3.7/site-packages/retry/api.py", line 33, in __retry_internal
    return f()
  File "/Users/crifan/.local/share/virtualenvs/autoTestAndroidGameHappyBigBattle-F2NzwZ7Z/lib/python3.7/site-packages/uiautomator2/__init__.py", line 457, in jsonrpc_retry_call
    return self.jsonrpc_call(*args, **kwargs)
  File "/Users/crifan/.local/share/virtualenvs/autoTestAndroidGameHappyBigBattle-F2NzwZ7Z/lib/python3.7/site-packages/uiautomator2/__init__.py", line 542, in jsonrpc_call
    raise err
uiautomator2.exceptions.JsonRpcError: 0 Unknown error: <Injecting to another application requires INJECT_EVENTS permission> data: {'exceptionTypeName': 'java.lang.SecurityException', 'message': 'Injecting to another application requires INJECT_EVENTS permission'}, method: click
[Finished in 0.216s]