Unable to inspect app when enableMultiWindows is true with UiAutomator2

Support
1

I was trying to locate an overlay system alert of a native app. The alert is on a window that is not in original page source. But when I enable enableMultiWindows, appium inspector cannot capture device screenshot at all with java.lang.ClassCastException. Any suggestion of why or how should I use enableMultiWindows properly?

Device capability:
esired_caps = {
‘appWaitPackage’: package_name,
‘appWaitDuration’: ‘30000’,
‘adbExecTimeout’: ‘60000’,
‘platformName’: ‘Android’,
‘autoGrantPermissions’: ‘true’,
‘noReset’: ‘true’,
‘deviceName’: ‘Android’,
‘enableMultiWindows’: ‘true’,
‘automationName’: ‘UiAutomator2’,
‘udid’: os.environ.get(‘UDID’)
}

Appium desktop log:
[WD Proxy] Proxying [GET /] to [GET http://127.0.0.1:8200/wd/hub/session/7660821f-bdd1-4b97-b242-82d2420b2787] with no body
[WD Proxy] Got response with status 200: {“sessionId”:“7660821f-bdd1-4b97-b242-82d2420b2787”,“value”:{“lastScrollData”:null}}
[MJSONWP (38f007f0)] Responding to client with driver.getSession() result: {“platform”:“LINUX”,“webStorageEnabled”:false,“takesScreenshot”:true,“javascriptEnabled”:true,“databaseEnabled”:false,“networkConnectionEnabled”:true,“locationContextEnabled”:false,“warnings”:{},“desired”:{“appWaitActivity”:“gogolook.callgogolook2.intro.CallerIdIntroActivity”,“appWaitDuration”:30000,“appWaitPackage”:“gogolook.callgogolook2”,“automationName”:“UiAutomator2”,“deviceName”:“Pixel4xl-emu”,“enableMultiWindows”:“true”,“ensureWebviewsHavePages”:true,“platformName”:“Android”,“platformVersion”:“10”,“udid”:“emulator-5554”,“newCommandTimeout”:0,“connectHardwareKeyboard”:true},“appWaitActivity”:“gogolook.callgogolook2.intro.CallerIdIntroActivity”,“appWaitDuration”:30000,“appWaitPackage”:“gogolook.callgogolook2”,“automationName”:“UiAutomator2”,“deviceName”:“emulator-5554”,“enableMultiWindows”:“true”,“ensureWebviewsHavePages”:true,“platformName”:“Android”,“platformVersion”:“10”,“udid”:“emulator-5554”,“newCommandTimeout”:0,“connectHardwareKeyboard”:true,“deviceUDID”:“emulator-5554”,“deviceApiLevel”:29,"dev…
[HTTP] <-- GET /wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab 200 36 ms - 1341
[HTTP]
[HTTP] --> GET /wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab/window/current/size
[HTTP] {}
[MJSONWP (38f007f0)] Driver proxy active, passing request on via HTTP proxy
[WD Proxy] Matched ‘/wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab/window/current/size’ to command name ‘getWindowSize’
[WD Proxy] Proxying [GET /wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab/window/current/size] to [GET http://127.0.0.1:8200/wd/hub/session/7660821f-bdd1-4b97-b242-82d2420b2787/window/current/size] with no body
[WD Proxy] Got response with status 200: {“sessionId”:“7660821f-bdd1-4b97-b242-82d2420b2787”,“value”:{“height”:1977,“width”:1080}}
[WD Proxy] Replacing sessionId 7660821f-bdd1-4b97-b242-82d2420b2787 with 38f007f0-1f58-4316-b7c6-18cf244809ab
[HTTP] <-- GET /wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab/window/current/size 200 31 ms - 100
[HTTP]
[HTTP] --> GET /wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab/context
[HTTP] {}
[MJSONWP (38f007f0)] Calling AppiumDriver.getCurrentContext() with args: [“38f007f0-1f58-4316-b7c6-18cf244809ab”]
[MJSONWP (38f007f0)] Responding to client with driver.getCurrentContext() result: “NATIVE_APP”
[HTTP] <-- GET /wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab/context 200 3 ms - 84
[HTTP]
[HTTP] --> GET /wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab/contexts
[HTTP] {}
[MJSONWP (38f007f0)] Calling AppiumDriver.getContexts() with args: [“38f007f0-1f58-4316-b7c6-18cf244809ab”]
[AndroidDriver] Getting a list of available webviews
[ADB] Running ‘/Users/gogolook/Library/Android/sdk/platform-tools/adb -P 5037 -s emulator-5554 shell cat /proc/net/unix’
[AndroidDriver] Found no active devtools sockets
[AndroidDriver] Other sockets are: [
[AndroidDriver] “0000000000000000: 00000002 00000000 00010000 0001 01 13023 @com.android.internal.os.WebViewZygoteInit/d887e46d-911d-41eb-a94b-e07a3b75325b”,
[AndroidDriver] “0000000000000000: 00000002 00000000 00010000 0005 01 24990 @jdwp-control”,
[AndroidDriver] “0000000000000000: 00000003 00000000 00000000 0005 03 146379 @jdwp-control”,
[AndroidDriver] “0000000000000000: 00000003 00000000 00000000 0001 03 13025 @com.android.internal.os.WebViewZygoteInit/d887e46d-911d-41eb-a94b-e07a3b75325b”,
[AndroidDriver] “0000000000000000: 00000003 00000000 00000000 0005 03 143256 @jdwp-control”
[AndroidDriver] ]
[AndroidDriver] Found 0 webviews: []
[AndroidDriver] Available contexts: [“NATIVE_APP”]
[MJSONWP (38f007f0)] Responding to client with driver.getContexts() result: [“NATIVE_APP”]
[HTTP] <-- GET /wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab/contexts 200 120 ms - 86
[HTTP]
[HTTP] --> POST /wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab/context
[HTTP] {“name”:“NATIVE_APP”}
[MJSONWP (38f007f0)] Calling AppiumDriver.setContext() with args: [“NATIVE_APP”,“38f007f0-1f58-4316-b7c6-18cf244809ab”]
[MJSONWP (38f007f0)] Responding to client with driver.setContext() result: null
[HTTP] <-- POST /wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab/context 200 3 ms - 76
[HTTP]
[HTTP] --> GET /wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab/source
[HTTP] {}
[MJSONWP (38f007f0)] Driver proxy active, passing request on via HTTP proxy
[WD Proxy] Matched ‘/wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab/source’ to command name ‘getPageSource’
[WD Proxy] Proxying [GET /wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab/source] to [GET http://127.0.0.1:8200/wd/hub/session/7660821f-bdd1-4b97-b242-82d2420b2787/source] with no body
[WD Proxy] Got response with status 500: {“sessionId”:“7660821f-bdd1-4b97-b242-82d2420b2787”,“value”:{“error”:“unknown error”,“message”:“java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Boolean”,“stacktrace”:"java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Boolean\n\tat io.appium.uiautomator2.model.settings.EnableMultiWindows.getValue(EnableMultiWindows.java:33)\n\tat io.appium.uiautomator2.model.settings.EnableMultiWindows.getValue(EnableMultiWindows.java:21)\n\tat io.appium.uiautomator2.utils.AXWindowHelpers.getCachedWindowRoots(AXWindowHelpers.java:107)\n\tat io.appium.uiautomator2.core.AccessibilityNodeInfoDumper.toStream(AccessibilityNodeInfoDumper.java:163)\n\tat io.appium.uiautomator2.core.AccessibilityNodeInfoDumper.dumpToXml(AccessibilityNodeInfoDumper.java:179)\n\tat io.appium.uiautomator2.handler.Source.safeHandle(Source.java:54)\n\tat io.appium.uiautomator2.handler.request.SafeRequestHandler.handle(SafeRequestHandler.java:41)\n\tat io.appium.uiautomator2.server.AppiumServlet.handleR…
[W3C] Matched W3C error code ‘unknown error’ to UnknownError
[HTTP] <-- GET /wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab/source 500 135 ms - 253
[HTTP]
[HTTP] --> GET /wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab/screenshot
[HTTP] {}
[MJSONWP (38f007f0)] Calling AppiumDriver.getScreenshot() with args: [“38f007f0-1f58-4316-b7c6-18cf244809ab”]
[WD Proxy] Matched ‘/screenshot’ to command name ‘getScreenshot’
[WD Proxy] Proxying [GET /screenshot] to [GET http://127.0.0.1:8200/wd/hub/session/7660821f-bdd1-4b97-b242-82d2420b2787/screenshot] with no body
[WD Proxy] Got response with status 200: {“sessionId”:“7660821f-bdd1-4b97-b242-82d2420b2787”,“value”:"iVBORw0KGgoAAAANSUhEUgAABDgAAAjoCAYAAAA5jhrUAAAAAXNSR0IArs4c6QAAAARzQklUCAgI\nCHwIZIgAACAASURBVHic7N1J0CXHYSf2f9arb/9637E0GiBBLARIkARFkRJFSRxJFDURmgnPhGSH\nR2MfHOGTDz746AjffPHFFx99sGNi7HDYYS0UqfGIoijuJAhwwY4G0Gig9/3b3lbpQ72lXr3aK7Mq\nM+v/QzS+7b16uVdWVmaW+O//zX8je14PG+trWF9ZwzgIEARjLJAgIiJql2g7AERERERkMt/v9dDz\neuh5PnqeBwAQAhzUICIiIiIiIiJr+FvrG/C8HtZWVrDirwBSQnJwg4iIiIiIiIgs4m+srcPzPKz4\nK1jp+W2Hh4iIKBkH34mIiIgogz8ORgB6AHoQgr1HIiIyzPTUxD04iIiIiCiDPxyPISGwIjHZfIOI\niMggPDURERERUQHe7DtO3iAiIiIiIiIiS/kCAgLh5A0RGe8gIiIiIiIiIrKFL0Q4xCGEB8ElKvVw\nFgwRERERERFRKxYfm+LABTqHaIiIiIiIiIi6J1yi4s2Xqdis9eA7MEBEREREREREZCNfCAHIyTIV\ny/fgaH2Ao/UA0DKOOhERERERmY69dlJhPqJh+/QNokQs10REREREJuPgBqniA0C40SjgWXgtGK0M\nWoKfVdssTK9uEmCzSURERERE5LbJDA57L/60jjHkJYu9ydYxzCgiIiIiIiLXzR8TC1j5mFiJxUEO\nZTGIHziNfUnWMUUzkoiIiIiImsbeOqnkh0VKTkqVHXe6s0KpLAasZURERERERETW8KczN6b/iIiI\niIiIiIhs42ta4KGVHaHsBjvm/CjWyUgTERERURfUudZiN5na5s9mb3DYgCroZKnpZKSJiIiIiLKx\nm0xt82Z7cNhMgLWJiIiIiIiI…
[MJSONWP (38f007f0)] Responding to client with driver.getScreenshot() result: "iVBORw0KGgoAAAANSUhEUgAABDgAAAjoCAYAAAA5jhrUAAAAAXNSR0IArs4c6QAAAARzQklUCAgI\nCHwIZIgAACAASURBVHic7N1J0CXHYSf2f9arb/9637E0GiBBLARIkARFkRJFSRxJFDURmgnPhGSH\nR2MfHOGTDz746AjffPHFFx99sGNi7HDYYS0UqfGIoijuJAhwwY4G0Gig9/3b3lbpQ72lXr3aK7Mq\nM+v/QzS+7b16uVdWVmaW+O//zX8je14PG+trWF9ZwzgIEARjLJAgIiJql2g7AERERERkMt/v9dDz\neuh5PnqeBwAQAhzUICIiIiIiIiJr+FvrG/C8HtZWVrDirwBSQnJwg4iIiIiIiIgs4m+srcPzPKz4\nK1jp+W2Hh4iIKBkH34mIiIgogz8ORgB6AHoQgr1HIiIyzPTUxD04iIiIiCiDPxyPISGwIjHZfIOI\niMggPDURERERUQHe7DtO3iAiIiIiIiIiS/kCAgLh5A0RGe8gIiIiIiIiIrKFL0Q4xCGEB8ElKvVw\nFgwRERERERFRKxYfm+LABTqHaIiIiIiIiIi6J1yi4s2Xqdis9eA7MEBEREREREREZCNfCAHIyTIV\ny/fgaH2Ao/UA0DKOOhERERERmY69dlJhPqJh+/QNokQs10REREREJuPgBqniA0C40SjgWXgtGK0M\nWoKfVdssTK9uEmCzSURERERE5LbJDA57L/60jjHkJYu9ydYxzCgiIiIiIiLXzR8TC1j5mFiJxUEO\nZTGIHziNfUnWMUUzkoiIiIiImsbeOqnkh0VKTkqVHXe6s0KpLAasZURERERERETW8KczN6b/iIiI\niIiIiIhs42ta4KGVHaHsBjvm/CjWyUgTERERURfUudZiN5na5s9mb3DYgCroZKnpZKSJiIiIiLKx\nm0xt82Z7cNhMgLWJiIiIiIiIqMP8cN8NASHsfIoKAA5ukB6Wj/sRERERERmL13CkgQ8hIITnxiaj\nlgefDM…
[HTTP] <-- GET /wd/hub/session/38f007f0-1f58-4316-b7c6-18cf244809ab/screenshot 200 2053 ms - 842594

2

workaround: try “uiautomatorviewer” which is in android sdk path. e.g. on mac:
/Users/your_user_name/Library/Android/sdk/tools/bin

3

I tried uiautomatorviewer but the alert element is also not displayed in the hierarchy. That’s why I want to try enableMultiWindows. I can find the window of the overlay alert from adb shell dumpsys window which is a different window compared to original package window, here is the window info:

Window #2 Window{8be2bd5 u0 xxxxx.xxxxx}: // This is the alert I would like to find
mDisplayId=0 stackId=0 mSession=Session{5e8c84f 19318:u0a10138} mClient=android.os.BinderProxy@ec3b9bf
mOwnerUid=10138 mShowToOwnerOnly=true package=xxxxx.xxxxx appop=SYSTEM_ALERT_WINDOW
mAttrs={(0,556)(fillxwrap) gr=TOP CENTER sim={adjust=pan} ty=APPLICATION_OVERLAY fmt=TRANSPARENT wanim=0x1030000 or=SCREEN_ORIENTATION_BEHIND
fl=NOT_FOCUSABLE SHOW_WHEN_LOCKED TURN_SCREEN_ON DISMISS_KEYGUARD HARDWARE_ACCELERATED}

Window #5 Window{646cbba u0 xxxxx.xxxxx/xxxxx.xxxxx.main.LauncherActivity}: // This is the app
mDisplayId=0 stackId=18 mSession=Session{5e8c84f 19318:u0a10138} mClient=android.os.BinderProxy@8d3c4e5
mOwnerUid=10138 mShowToOwnerOnly=true package=xxxxx.xxxxx appop=NONE
mAttrs={(0,0)(fillxfill) sim={adjust=nothing forwardNavigation} ty=BASE_APPLICATION wanim=0x10302fe
fl=LAYOUT_IN_SCREEN LAYOUT_INSET_DECOR SPLIT_TOUCH HARDWARE_ACCELERATED DRAWS_SYSTEM_BAR_BACKGROUNDS
pfl=FORCE_DRAW_STATUS_BAR_BACKGROUND
vsysui=LAYOUT_STABLE LAYOUT_FULLSCREEN}

driver.switch_to.alert is also not an option which will get not implemented error. My test goal is to get one of the element text on the alert, so it’s not about accept it or not.

4

Your client library is probably erroneously passing the enableMultiWindows setting value as string rather than boolean (e.g. "true" instead of true).

5

Thank you, @mykola-mokhnach. I modified the value to be boolean (e.g. True on python code), the error is gone!

6

Thanks @mykola-mokhnach @Enpei_Chu and Aleksei for the suggestions. By following this post, I am able to inspect it through desktop inspector by using enableMultiWindows.

But when I am running the test, appium is unable to locate the element present on the other window. Any suggestions how can I handle this?