引文：

因?yàn)槲覘売迷瓉鞟TX框架中的uiautomator的東西，所以現(xiàn)在要把 UiSelector().text("XXX")這部分的功能給重新實(shí)現(xiàn)下。

所以這篇文章介紹的是抓取到頁面中的文字還有文字的位置的方法及其分析。

現(xiàn)有的方法

1，UiSelector.text /自動化測試框架

大多數(shù)測試框架使用的方法。好像需要在手機(jī)上安裝一個(gè)測試的app，沒動手實(shí)踐

2，Layout Inspector

android studio->Tools->android->Layout Inspector

顯示很友好，沒找到開源部分的代碼

3，uiautomator dump

adb shell uiautomator dump

dump出來的東西會自動保存在/sdcard/window_dump.xml中，內(nèi)容大概是這樣

...

resource-id="com.android.contacts:id/contacts_unavailable_view"

class="android.widget.FrameLayout"

package="com.android.contacts"

content-desc="" checkable="false" checked="false"

clickable="false" enabled="true" focusable="false"

focused="false" scrollable="false" long-clickable="false"

password="false" selected="false"

bounds="[0,408][1080,1776]">

resource-id="com.android.contacts:id/contacts_unavailable_container" class="android.widget.FrameLayout"

package="com.android.contacts"

content-desc="" checkable="false" checked="false"

clickable="false" enabled="true" focusable="false"

focused="false" scrollable="false" long-clickable="false"

password="false" selected="false"

bounds="[0,1776]">

resource-id="com.android.contacts:id/floating_action_button"

class="android.widget.ImageButton"

package="com.android.contacts"

content-desc="添加新聯(lián)系人" checkable="false"

checked="false" clickable="true"

enabled="true" focusable="true" focused="false"

scrollable="false" long-clickable="false"

password="false" selected="false"

bounds="[864,1560][1032,1728]"/>

...

...

數(shù)據(jù)類型都是類似的，都是node節(jié)點(diǎn)，bounds就是各個(gè)view的邊界了，舉個(gè)例子

添加新聯(lián)系人

bounds= 864,1560 1032,1728

所以重心就是 (948，1644)

adb shell input tap 948 1644

就點(diǎn)擊到文字了。

ok，下面也是談實(shí)現(xiàn)細(xì)節(jié)。dump調(diào)用鏈?zhǔn)沁@樣的

DumpCommand:run

-> automationWrapper.getUiAutomation().getRootInActiveWindow()

-> AccessibilityInteractionClient:findAccessibilityNodeInfoByAccessibilityId

-> binder

-> ViewRootImpl.findAccessibilityNodeInfoByAccessibilityId

-> AccessibilityNodeInfoDumper.dumpWindowToFile

序列化的節(jié)點(diǎn)node的數(shù)據(jù)存在這里

AccessibilityNodeInfo.java

可以看到，它是一顆樹的節(jié)點(diǎn)

public class AccessibilityNodeInfo implements Parcelable {

...

private static final int MAX_POOL_SIZE = 50;

private static final SynchronizedPool sPool =

new SynchronizedPool<>(MAX_POOL_SIZE);

...

}

節(jié)點(diǎn)是這樣建的

view.java

public AccessibilityNodeInfo createAccessibilityNodeInfoInternal() {

AccessibilityNodeProvider provider = getAccessibilityNodeProvider();

if (provider != null) {

return provider.createAccessibilityNodeInfo(AccessibilityNodeProvider.HOST_VIEW_ID);

} else {

AccessibilityNodeInfo info = AccessibilityNodeInfo.obtain(this);

onInitializeAccessibilityNodeInfo(info);

return info;

}

}

public void onInitializeAccessibilityNodeInfoInternal(AccessibilityNodeInfo info) {

...

getDrawingRect(bounds);

info.setBoundsInParent(bounds);

}

在TextView

Override

public void onInitializeAccessibilityNodeInfoInternal(AccessibilityNodeInfo info) {

super.onInitializeAccessibilityNodeInfoInternal(info);

final boolean isPassword = hasPasswordTransformationMethod();

info.setPassword(isPassword);

if (!isPassword || shouldSpeakPasswordsForAccessibility()) {

info.setText(getTextForAccessibility());

}

...

}

所以AccessibilityNodeInfo更像是在View中存了一個(gè)副本，這個(gè)副本可以用于輔助操作。

// TODO 更細(xì)節(jié)的分析

4,hierarchyviewer1

這個(gè)是在源碼中找到的工具，發(fā)現(xiàn)它抓取的信息出乎意料的很全，比3中的信息還要多許多，是這樣用的

.out/host/linux-x86/bin/hierarchyviewer1

是一個(gè)gui工具，打開之后，選中某個(gè)元素，可以看到Property中有absolute_x、absolute_y、getHeight、getWidth、mText。更加這些信息我們可以判斷找的這個(gè)元素是不是在屏幕內(nèi)，在的話，具體是哪個(gè)位置。

/sdk/hierachyviewer/ com.android.hierarchyviewer.scene.ViewHierarchyLoader

package com.android.hierarchyviewer.scene;

public class ViewHierarchyLoader {

public static ViewHierarchyScene loadScene(IDevice device,Window window) {

...

System.out.println("==> Starting client");

socket = new Socket();

socket.connect(new InetSocketAddress("127.0.0.1",

DeviceBridge.getDeviceLocalPort(device)));

out = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));

in = new BufferedReader(new InputStreamReader(socket.getInputStream(),"utf-8"));

System.out.println("==> DUMP");

out.write("DUMP " + window.encode());

out.newLine();

out.flush();

Stack stack = new Stack();

boolean setRoot = true;

ViewNode lastNode = null;

int lastWhitespaceCount = Integer.MAX_VALUE;

while ((line = in.readLine()) != null) {

// debug by yeshen：

// System.out.println(line);

if ("DONE.".equalsIgnoreCase(line)) {

break;

}

...

}

...

}

}

看樣子是自己建了一個(gè)socket取連本地的socket server，在代碼里面打log，發(fā)現(xiàn)line的數(shù)據(jù)是類似這樣的：

android.widget.Button@eb7e4a6

text:mCurTextColor=9,-13224394text:mGravity=2,17

text:mText=6,創(chuàng)建新聯(lián)系人

getEllipsize()=4,null

text:getScaledTextSize()=4,14.0

text:getSelectionEnd()=2,-1

text:getSelectionStart()=2,-1

text:getTextSize()=4,42.0

text:getTypefaceStyle()=6,NORMAL

layout:mBottom=3,144

theme:com.android.contacts:style/PeopleTheme()=6,forced

theme:android:style/Theme.DeviceDefault()=6,forced

fg_=4,null

mID=24,id/create_contact_button

drawing:mLayerType=4,NONE

layout:mLeft=1,0

measurement:mMeasuredHeight=3,144

measurement:mMeasuredWidth=3,324

measurement:mMinHeight=3,144

measurement:mMinWidth=3,264

padding:mPaddingBottom=2,30

padding:mPaddingLeft=2,36

padding:mPaddingRight=2,36

padding:mPaddingTop=2,30

mPrivateFlags_DRAWN=4,0x20

mPrivateFlags=9,0x1028830

layout:mRight=3,324

scrolling:mScrollX=1,0

scrolling:mScrollY=1,0

mSystemUiVisibility_SYSTEM_UI_FLAG_VISIBLE=3,0x0

mSystemUiVisibility=3,0x0

layout:mTop=1,0

padding:mUserPaddingBottom=2,30

padding:mUserPaddingEnd=11,-2147483648

padding:mUserPaddingLeft=2,36

padding:mUserPaddingRight=2,36

padding:mUserPaddingStart=11,-2147483648

mViewFlags=10,0x18004001

drawing:getAlpha()=3,1.0

layout:getBaseline()=2,88

accessibility:getContentDescription()=4,null

drawing:getElevation()=3,6.0

getFilterTouchesWhenObscured()=5,false

getFitsSystemWindows()=5,false

layout:getHeight()=3,144

accessibility:getImportantForAccessibility()=3,yes

accessibility:getLabelFor()=2,-1

layout:getLayoutDirection()=22,RESOLVED_DIRECTION_LTR

layout:layout_gravity=4,NONE

layout:layout_weight=3,0.0

layout:layout_bottomMargin=2,45

layout:layout_endMargin=11,-2147483648

layout:layout_leftMargin=1,0

layout:layout_mMarginFlags_LEFT_MARGIN_UNDEFINED_MASK=3,0x4

layout:layout_mMarginFlags_RIGHT_MARGIN_UNDEFINED_MASK=3,0x8

layout:layout_mMarginFlags=4,0x0C

layout:layout_rightMargin=1,0

layout:layout_startMargin=11,-2147483648

layout:layout_topMargin=1,0

layout:layout_height=12,WRAP_CONTENT

layout:layout_width=12,MATCH_PARENT

layout:getLocationOnScreen_x()=3,378

layout:getLocationOnScreen_y()=3,757

measurement:getMeasuredHeightAndState()=3,144

measurement:getMeasuredWidthAndState()=3,324

drawing:getPivotX()=5,162.0

drawing:getPivotY()=4,72.0

layout:getRawLayoutDirection()=7,INHERIT

text:getRawTextAlignment()=7,GRAVITY

text:getRawTextDirection()=7,INHERIT

drawing:getRotation()=3,0.0

drawing:getRotationX()=3,0.0

drawing:getRotationY()=3,0.0

drawing:getScaleX()=3,1.0

drawing:getScaleY()=3,1.0

getScrollBarStyle()=14,INSIDE_OVERLAY

drawing:getSolidColor()=1,0

getTag()=4,null

text:getTextAlignment()=7,GRAVITY

text:getTextDirection()=12,FIRST_STRONG

drawing:getTransitionAlpha()=3,1.0

getTransitionName()=4,null

drawing:getTranslationX()=3,0.0

drawing:getTranslationY()=3,0.0

drawing:getTranslationZ()=3,0.0

getVisibility()=7,VISIBLE

layout:getWidth()=3,324

drawing:getX()=3,0.0

drawing:getY()=3,0.0

drawing:getZ()=3,6.0

focus:hasFocus()=5,false

drawing:hasOverlappingRendering()=4,true

drawing:hasShadow()=4,true

layout:hasTransientState()=5,false

isActivated()=5,false

isClickable()=4,true

drawing:isDrawingCacheEnabled()=5,false

isEnabled()=4,true

focus:isFocusable()=4,true

isFocusableInTouchMode()=5,false

focus:isFocused()=5,false

isHapticFeedbackEnabled()=4,true

drawing:isHardwareAccelerated()=4,true

isHovered()=5,false

isInTouchMode()=4,true

layout:isLayoutRtl()=5,false

drawing:isOpaque()=5,false

isPressed()=5,false

isSelected()=5,false

isSoundEffectsEnabled()=4,true

drawing:willNotCacheDrawing()=5,false

drawing:willNotDraw()=5,false

不過這里缺了 absolute_x absolute_y，找了一下代碼

com.android.hierarchyviewer.ui.model.PropertiesTableModel

private void loadPrivateProperties(ViewNode node) {

int x = node.left;

int y = node.top;

ViewNode p = node.parent;

while (p != null) {

x += p.left - p.scrollX;

y += p.top - p.scrollY;

p = p.parent;

}

ViewNode.Property property = new ViewNode.Property();

property.name = "absolute_x";

property.value = String.valueOf(x);

privateProperties.add(property);

property = new ViewNode.Property();

property.name = "absolute_y";

property.value = String.valueOf(y);

privateProperties.add(property);

}

所以我們已經(jīng)知道了，上面抓到的數(shù)據(jù)是一個(gè)ViewNode，然后遞歸解析，算出絕對位置。

所以找到文字的位置就要，遞歸算到自己在根視圖的絕對位置，然后再修正下layout:getWidth(),layout:getHeight(),找到view的重點(diǎn)，最后再執(zhí)行點(diǎn)擊。

ok,下面談實(shí)現(xiàn)細(xì)節(jié)

hierarchyviewer其實(shí)是在android上開了一個(gè)ViewServer，然后把ViewServer的端口轉(zhuǎn)發(fā)到本地，然后在連本地的sockect，用sockect與ViewServer進(jìn)行通訊。

可以看到，dump出來的信息很全，而且是在用戶進(jìn)程中才有的信息。所以ViewServer會通過mWindowToken中的打印。調(diào)用鏈?zhǔn)沁@樣的：

WMS:viewServerWindowCommand

-> ViewRootImpl:executeCommand

-> ViewDebug:dispatchCommand

-> ViewDebug:dumpViewProperties

打印到目標(biāo)屬性用了注解+反射

private static void dumpViewProperties(Context context,Object view,

BufferedWriter out,String prefix) throws IOException {

if (view == null) {

out.write(prefix + "=4,null ");

return;

}

Class> klass = view.getClass();

do {

exportFields(context,view,out,klass,prefix);

exportMethods(context,prefix);

klass = klass.getSuperclass();

} while (klass != Object.class);

}

在每個(gè)view中，如果允許dump的話，就加入注解,比如說

@ViewDebug.ExportedProperty(flagMapping = {

@ViewDebug.FlagToString(mask = SYSTEM_UI_FLAG_LOW_PROFILE,

equals = SYSTEM_UI_FLAG_LOW_PROFILE,

name = "SYSTEM_UI_FLAG_LOW_PROFILE",

outputIf = true),

@ViewDebug.FlagToString(mask = SYSTEM_UI_FLAG_HIDE_NAVIGATION,

equals = SYSTEM_UI_FLAG_HIDE_NAVIGATION,

name = "SYSTEM_UI_FLAG_HIDE_NAVIGATION",

@ViewDebug.FlagToString(mask = PUBLIC_STATUS_BAR_VISIBILITY_MASK,

equals = SYSTEM_UI_FLAG_VISIBLE,

name = "SYSTEM_UI_FLAG_VISIBLE",

outputIf = true)

},formatToHexString = true)

int mSystemUiVisibility;

hierarchyviewer小結(jié)：

hierarchyviewer 就是通過在wms中開啟一個(gè)viewServer，把server的端口forward到本地端口上。socket連本地端口，發(fā)送 DUMP $(WindowHash),通過binder調(diào)用到具體的某個(gè)view，遞歸打印出有加注解的信息，回信息給socket，hierarchyviewer(gui)再對信息進(jìn)行整理，獲得絕對位置等信息。

小結(jié)

本文提供了四種獲取當(dāng)前顯示屏幕的文字信息的方法，簡單分析了下原理。

接下來準(zhǔn)備拓展下uiautomator，就可以支持文字查找與點(diǎn)擊了。實(shí)現(xiàn)看這篇文章:

Android抓取文字、文字位置的實(shí)現(xiàn)

如果對精確度有高要求，對速度有高要求，可以考慮下移植下hierarchyviewer部分的實(shí)現(xiàn)。這部分完全重寫需要不少時(shí)間，暫時(shí)偷個(gè)懶吧，不實(shí)現(xiàn)了：)

尾聲

用文字匹配搶微信紅包，查資料的時(shí)候發(fā)現(xiàn)網(wǎng)上不少這樣的文章。下面只是針對這個(gè)場景的個(gè)人猜想，沒有實(shí)際調(diào)研。

攻擊的話，看大多數(shù)的文章都提到用Accessibility,其實(shí)就是用第三種方法(uiautomator)。可以考慮用第四種方法(hierarchyviewer)。就不會受到Accessibility的限制了。

防守的話，新消息那一欄，用自定義view、非規(guī)則view，那么有新紅包這個(gè)文字信息就抓不到。

掃碼關(guān)注，實(shí)時(shí)互動

總結(jié)

以上是生活随笔為你收集整理的抓取android ui原理,Android抓取文字、文字位置的分析的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。