Hello all,
I’m looking to automate an app that’s been made in Unity and whilst I know that I cannot use any of the UI Automation, I’m looking to drive it by using tap commands using X Y co’ords. I’ve been able to do this via ADB shell but I need a solution that works for both iOS and Android
I’ve tried to use the Precise Tap but it doesn’t seem to work.I get the following output on the GUI window when I make an attempt
[details=Log Output][HTTP] --> POST /wd/hub/session/2f25d588-e918-49b6-a60c-7a69c8183ba9/execute {“script”:“mobile: tap”,“args”:[{“y”:1092,“touchCount”:1,“x”:602,“tapCount”:1,“duration”:1.14046556122449}]}
[MJSONWP] Calling AppiumDriver.execute() with args: [“mobile: tap”,[{“y”:1092,"…
[HTTP] <-- POST /wd/hub/session/2f25d588-e918-49b6-a60c-7a69c8183ba9/execute 501 3 ms - 122
[HTTP] --> GET /wd/hub/session/2f25d588-e918-49b6-a60c-7a69c8183ba9/source {}
[MJSONWP] Calling AppiumDriver.getPageSource() with args: ["2f25d588-e918-49b6-a60c-7…
[debug] [AndroidBootstrap] Sending command to android: {“cmd”:“action”,“action”:“source”,“params”:{}}
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got data from client: {“cmd”:“action”,“action”:“source”,“params”:{}}
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command of type ACTION
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command action: source
[debug] [AndroidBootstrap] [UIAUTO STDOUT] [APPIUM-UIAUTO] [debug] Returning result: {“status”:0,“value”:"<?xml version=\"1.0\" encoding=\"UTF-8\"?><hierarchy rotation=“0”><android.widget.FrameLayout index=“0” text="" class=“android.widget.FrameLayout” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[0,0][800,1216]" resource-id="" instance=“0”><android.widget.LinearLayout index=“0” text="" class=“android.widget.LinearLayout” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[0,0][800,1216]" resource-id="" instance=“0”><android.widget.FrameLayout index=“0” text="" class=“android.widget.FrameLayout” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[0,33][800,1216]" resource-id=“android:id/content” instance=“1”><android.widget.LinearLayout index=“0” text="" class=“android.widget.LinearLayout” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[0,33][800,1216]" resource-id=“uk.co.aifactory.chessfree:id/AIF_LinearLayout” instance=“1”><android.widget.FrameLayout index=“0” text="" class=“android.widget.FrameLayout” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused="fa
[debug] [AndroidBootstrap] Received command result from bootstrap
[debug] [AndroidBootstrap] Stream still not complete, waiting
[debug] [AndroidBootstrap] [UIAUTO STDOUT] lse" scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[67,33][733,172]" resource-id="" instance=“2”><android.widget.ImageView index=“0” text="" class=“android.widget.ImageView” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[67,33][733,147]" resource-id=“uk.co.aifactory.chessfree:id/ImageView01” instance=“0”/></android.widget.FrameLayout><android.view.View index=“1” text="" class=“android.view.View” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[0,172][800,868]" resource-id=“uk.co.aifactory.chessfree:id/chess” instance=“0”/><android.widget.LinearLayout index=“2” text="" class=“android.widget.LinearLayout” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[0,868][800,1216]" resource-id="" instance=“2”><android.widget.FrameLayout index=“0” text="" class=“android.widget.FrameLayout” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[0,868][400,1216]" resource-id="" instance=“3”><android.widget.LinearLayout index=“0” text="" class=“android.widget.LinearLayout” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[0,868][400,1216]" resource-id="" instance=“3”><android.widget.FrameLayout index=“0” text="" class=“android.widget.FrameLayout” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[0,1042][400,1216]" resource-id="" instance=“4”/></android.widget.LinearLayout><android.widget.ImageView index=“1” text="" class=“android.widget.ImageView” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[0,1088][185,1216]" resource-id=“uk.co.aifactory.chessfree:id/Logo” instance=“1”/></android.widget.FrameLayout><android.widget.LinearLayout index=“1” text="" class=“android.widget.LinearLayout” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[400,868][800,1216]" resource-id="" instance=“4”><android.widget.LinearLayout index=“0” text="" class=“android.widget.LinearLayout” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[400,868][800,1158]" resource-id=“uk.co.aifactory.chessfree:id/Controls1” instance=“5
“><android.widget.FrameLayout index=“0” text=”” class=“android.widget.FrameLayout” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[529,873][670,1013]" resource-id="" instance=“5”><android.widget.CheckBox index=“0” text=“Sound On” class=“android.widget.CheckBox” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“true” checked=“true” clickable=“true” enabled=“true” focusable=“true” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[529,949][670,1013]" resource-id=“uk.co.aifactory.chessfree:id/Intro_CheckBox” instance=“0”/></android.widget.FrameLayout><android.widget.FrameLayout index=“1” text="" class=“android.widget.FrameLayout” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused=
[debug] [AndroidBootstrap] [UIAUTO STDOUT] “false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[460,1018][740,1158]" resource-id="" instance=“6”><android.widget.Button index=“0” text=“PLAY!” class=“android.widget.Button” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“true” enabled=“true” focusable=“true” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[487,1018][713,1158]" resource-id=“uk.co.aifactory.chessfree:id/ButtonPlay” instance=“0”/></android.widget.FrameLayout></android.widget.LinearLayout><android.widget.TextView index=“1” text=“© 2010 AI Factory Ltd” class=“android.widget.TextView” package=“uk.co.aifactory.chessfree” content-desc="" checkable=“false” checked=“false” clickable=“false” enabled=“true” focusable=“false” focused=“false” scrollable=“false” long-clickable=“false” password=“false” selected=“false” bounds="[471,1158][800,1216]" resource-id=“uk.co.aifactory.chessfree:id/TextView01” instance=“0”/></android.widget.LinearLayout></android.widget.LinearLayout></android.widget.LinearLayout></android.widget.FrameLayout></android.widget.LinearLayout></android.widget.FrameLayout></hierarchy>"}[/APPIUM-UIAUTO]
[debug] [AndroidBootstrap] Received command result from bootstrap
[MJSONWP] Responding to client with driver.getPageSource() result: "<?xml version=“1.0” enco…
[HTTP] <-- GET /wd/hub/session/2f25d588-e918-49b6-a60c-7a69c8183ba9/source 200 53 ms - 8466
[HTTP] --> GET /wd/hub/session/2f25d588-e918-49b6-a60c-7a69c8183ba9/orientation {}
[MJSONWP] Calling AppiumDriver.getOrientation() with args: ["2f25d588-e918-49b6-a60c-7…
[debug] [AndroidBootstrap] Sending command to android: {“cmd”:“action”,“action”:“orientation”,“params”:{}}
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got data from client: {“cmd”:“action”,“action”:“orientation”,“params”:{}}
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command of type ACTION
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command action: orientation
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Current rotation: ROTATION_0
[debug] [AndroidBootstrap] Received command result from bootstrap
[MJSONWP] Responding to client with driver.getOrientation() result: “PORTRAIT”
[HTTP] <-- GET /wd/hub/session/2f25d588-e918-49b6-a60c-7a69c8183ba9/orientation 200 11 ms - 82
[AndroidBootstrap] [BOOTSTRAP LOG] [debug] Returning result: {“status”:0,“value”:“PORTRAIT”}
[HTTP] --> GET /wd/hub/session/2f25d588-e918-49b6-a60c-7a69c8183ba9/screenshot {}
[MJSONWP] Calling AppiumDriver.getScreenshot() with args: ["2f25d588-e918-49b6-a60c-7…
[debug] [ADB] Getting connected devices…
[debug] [ADB] 1 device(s) connected
[debug] [ADB] Running /Users/Bacon/Library/Android/sdk/platform-tools/adb with args: ["-P",5037,"-s",“emulator-5554”,“shell”,"/system/bin/rm","/data/local/tmp/screenshot.png;","/system/bin/screencap","-p","/data/local/tmp/screenshot.png"]
[debug] [ADB] Running /Users/Bacon/Library/Android/sdk/platform-tools/adb with args: ["-P",5037,"-s",“emulator-5554”,“pull”,"/data/local/tmp/screenshot.png","/var/folders/02/5wlsgm5j2d15wn76b4c83rvr08qkg5/T/appium11718-29170-1o58mf9.onely2e29.png"]
[MJSONWP] Responding to client with driver.getScreenshot() result: "iVBORw0KGgoAAAANSUhEUgAAAy…
[HTTP] <-- GET /wd/hub/session/2f25d588-e918-49b6-a60c-7a69c8183ba9/screenshot 200 974 ms - 1135290
[HTTP] --> GET /wd/hub/session/2f25d588-e918-49b6-a60c-7a69c8183ba9/contexts {}
[MJSONWP] Calling AppiumDriver.getContexts() with args: ["2f25d588-e918-49b6-a60c-7…
[debug] [AndroidDriver] Getting a list of available webviews
[debug] [ADB] Getting connected devices…
[debug] [ADB] 1 device(s) connected
[debug] [ADB] Running /Users/Bacon/Library/Android/sdk/platform-tools/adb with args: ["-P",5037,"-s",“emulator-5554”,“shell”,“cat”,"/proc/net/unix"]
[debug] [AndroidDriver] WEBVIEW_2632 mapped to pid 2632
[debug] [AndroidDriver] Getting process name for webview
[debug] [ADB] Getting connected devices…
[debug] [ADB] 1 device(s) connected
[debug] [ADB] Running /Users/Bacon/Library/Android/sdk/platform-tools/adb with args: ["-P",5037,"-s",“emulator-5554”,“shell”,“ps”]
[debug] [AndroidDriver] Parsed pid: 2632 pkg: uk.co.aifactory.chessfree!
[debug] [AndroidDriver] from: u0_a60,2632,1138,798852,86736,ffffffff,f771f405,R,uk.co.aifactory.chessfree
[debug] [AndroidDriver] returning process name: uk.co.aifactory.chessfree
[debug] [AndroidDriver] Found webviews: [“WEBVIEW_uk.co.aifactory.chessfree”]
[debug] [AndroidDriver] Available contexts: [“NATIVE_APP”,“WEBVIEW_uk.co.aifactory.chessfree”]
[MJSONWP] Responding to client with driver.getContexts() result: [“NATIVE_APP”,"WEBVIEW_uk.c…
[HTTP] <-- GET /wd/hub/session/2f25d588-e918-49b6-a60c-7a69c8183ba9/contexts 200 107 ms - 122
[HTTP] --> GET /wd/hub/status {}
[MJSONWP] Calling AppiumDriver.getStatus() with args: []
[MJSONWP] Responding to client with driver.getStatus() result: {“build”:{“version”:“1.5.3”…
[HTTP] <-- GET /wd/hub/status 200 32 ms - 83
[HTTP] --> GET /wd/hub/status {}
[MJSONWP] Calling AppiumDriver.getStatus() with args: []
[MJSONWP] Responding to client with driver.getStatus() result: {“build”:{“version”:“1.5.3”…
[HTTP] <-- GET /wd/hub/status 200 14 ms - 83
[/details]
I’m looking to make my automation scripts in Python. Any help on how to trigger simple X Y taps would be much appreciated.
Edit, I’m using Appium.app version 1.5.3