In my Android project, developed in compose, I have a simple Box which contains 3 Text views. Obviously since it’s a Box all the text views overlap each other (views on top of each other).
Box {
Text("text1", modifier = Modifier.semantics { contentDescription = "text1" })
Text("text2", modifier = Modifier.semantics { contentDescription = "text2" })
Text("text3", modifier = Modifier.semantics { contentDescription = "text3" })
}
The problem I’m facing is that appium inspector only detects the top most element (text3 in this case) and it is not possible to detect the views beneath the top most view (text1 and text2).
Is there any way for appium to detect all elements regardless if they are overlapping or not? I even tried adding enableMultiWindows and allowInvisibleElements but didn’t help.
My Appium settings
{
"platformName": "Android",
"appium:platformVersion": "13",
"appium:deviceName": "XXX",
"appium:automationName": "UIAutomator2",
"appium:appPackage": "XX",
"appium:appActivity": "XXX",
"appium:forceEspressoRebuild": true,
"appium:noReset": true,
"appium:settings[enableMultiWindows]": true,
"appium:settings[allowInvisibleElements]": true
}
can you see your elements with any other inspector e.g. with inspector in Android Studio?
Yes! all of them are visible in Android Studio Layout Inspector.
try enable debug logs and see that all settings applied correctly. i do not use Appium inspector but inspector from Android studio or pure page source instead. So cant suggest more …
Some logs I could find
[debug] [AppiumDriver@68b1] Calling AppiumDriver.createSession() with args: [{"platformName":"Android","appium:platformVersion":"13","appium:deviceName":"R5CW815KM6B","appium:automationName":"UIAutomator2","appium:appPackage":"xxxx","appium:appActivity":"xxxx.ui.activities.route.RouteActivity","appium:forceEspressoRebuild":true,"appium:noReset":true,"appium:settings[enableMultiWindows]":true,"appium:settings[allowInvisibleElements]":true,"appium:ensureWebviewsHavePages":true,"appium:nativeWebScreenshot":true,"appium:newCommandTimeout":3600,"appium:connectHardwareKeyboard":true},null,{"alwaysMatch":{"platformName":"Android","appium:platformVersion":"13","appium:deviceName":"R5CW815KM6B","appium:automationName":"UIAutomator2","appium:appPackage":"xxxx","appium:appActivity":"xxxx.ui.activities.route.RouteActivity","appium:forceEspressoRebuild":true,"appium:noReset":true,"appium:settings[enableMultiWindows]":true,"appium:settings[allowInvisibleElements]":true,"appium:ensureWebviewsHavePages":true,"appium:nativeWebScreenshot":true,"appium:newCommandTimeo...
[debug] [AppiumDriver@68b1] Event 'newSessionRequested' logged at 1696512767642 (15:32:47 GMT+0200 (Central European Summer Time))
[debug] [AndroidUiautomator2Driver@7487 (4e2b8df0)] Proxying [GET /window/current/size] to [GET http://127.0.0.1:8200/session/53d5db7b-403c-4199-80cb-02dd78355a73/window/current/size] with no body
[debug] [AndroidUiautomator2Driver@7487 (4e2b8df0)] Got response with status 200: {"sessionId":"53d5db7b-403c-4199-80cb-02dd78355a73","value":{"height":2214,"width":1080}}
[AppiumDriver@68b1] New AndroidUiautomator2Driver session created successfully, session 4e2b8df0-6116-414c-a3f5-703b29f139e3 added to master session list
[AppiumDriver@68b1] Applying the initial values to Appium settings parsed from W3C caps: {"enableMultiWindows":true,"allowInvisibleElements":true}
[debug] [AndroidUiautomator2Driver@7487 (4e2b8df0)] Matched '/appium/settings' to command name 'updateSettings'
[debug] [AndroidUiautomator2Driver@7487 (4e2b8df0)] Proxying [POST /appium/settings] to [POST http://127.0.0.1:8200/session/53d5db7b-403c-4199-80cb-02dd78355a73/appium/settings] with body: {"settings":{"enableMultiWindows":true,"allowInvisibleElements":true}}
From what I can see the settings should have been applied, right?
I added ignoreUnimportantViews and I can clearly see it works fine, so the rest of settings also working fine. Could this be a shortage of appium?
In UIA2 driver you can only see what is exposed to the accessibility. So check how compose handles this topic and if it is possible to change anything there from the source code perspective.
You could also try Espresso driver as a possible alternative. It has a special mode to natively interact with compose primitives. Although, this driver might be more complicated to setup.
You could also try Espresso driver as a possible alternative. It has a special mode to natively interact with compose primitives. Although, this driver might be more complicated to setup.
Espresso is not an option for us because it doesn’t support iOS and also the setup if fairly complicated as you mentioned
In UIA2 driver you can only see what is exposed to the accessibility. So check how compose handles this topic and if it is possible to change anything there from the source code perspective.
I couldn’t find any documentation that suggests if views are stack on each other, the accessibility will be affected in compose. Also, as I mentioned, all views are still shown in Android Studio layout inspector. So it should be really something with Appium/UIA2.
I’m clueless at this point.
I would try to ask at Compose developer community (e.g. Google) about how they interoperate with the Accessibility layer. Or try to read the Compose source code and figure that out yourself.
