Appium cannot detect elements with .clearAndSetSemantics?

Hello. I’m a QA. Currently we’re having trouble with our setup when we encountered elements using Jetpack Compose. I looked into our application code and noticed that elements that has .clearAndSetSemantics is not seen in Appium or in Appium Inspector, but we can see the elements that have .semantics. I looked into their difference (zero knowledge with Android code), it’s used if one element has different properties (one button, same placement, can be save or cancel depending on the interaction). So I’m confused as to how do we approach this in our automation. Other elements made with xml, are interacting smoothly. What’s your advice on this? How should we tweak our automation code? We’re using the normal .click , .send_keys

Same question is asked: https://stackoverflow.com/questions/75515201/appium-cannot-detect-elements-with-clearandsetsemantics

Here is a sample application code:

val classContainerScreen = stringResource(id = R.string.user_my_class_container_screen)
    SwipeRefresh(
        indicator = {state, trigger ->
            SwipeRefreshIndicator(
                state = state,
                refreshTriggerDistance = trigger,
                backgroundColor = commonColor.generalPullRefreshIndicator,
                contentColor = commonColor.primaryColor,
            )
        },
        modifier = Modifier.fillMaxSize()
            .clearAndSetSemantics {
                contentDescription = classContainerScreen
            },
        state = swipeRefreshState,
        onRefresh = {
            callbacks.refreshLoadMyClass()
        }
    )

Another sample application code:

val leaveClassDescription = stringResource(id = R.string.my_class_leave_class_button)
        val cancelRequestDescription = stringResource(id = R.string.my_class_cancel_request_button)
        Button(
            modifier = Modifier
                .constrainAs(button) {
                    top.linkTo(titleView.top)
                    bottom.linkTo(titleView.bottom)
                    end.linkTo(parent.end, margin = dimen.space16dp)
                    start.linkTo(titleView.end)
                }
                .clearAndSetSemantics {
                    contentDescription = if (itemState.isRequestedRemoval) {
                        cancelRequestDescription
                    } else {
                        leaveClassDescription
                    }
                },
            shape = RoundedCornerShape(dimen.space4dp),
            elevation = ButtonDefaults.elevation(dimen.space0dp),
            border = BorderStroke(
                width = dimen.space1dp,
                color = if (itemState.isRequestedRemoval) MaterialTheme.colors.Blue50 else MaterialTheme.colors.Gray30
            ),
            colors = ButtonDefaults.buttonColors(
                backgroundColor = Color.Transparent,
                contentColor = if (itemState.isRequestedRemoval) MaterialTheme.colors.Blue50 else MaterialTheme.colors.Gray60,
            ),
            onClick = onClick
        ) {
            Text(
                style = typography.subtitle2,
                text = if (itemState.isRequestedRemoval) stringResource(id = R.string.my_class_cancel_leave_class) else stringResource(
                    id = R.string.my_class_leave_class
                ),
            )

Automation setup when running locally (we also run in Saucelabs):

desired_caps = {
      "platformName": "Android",
      "appium:platformVersion": "11.0",
      "appium:deviceName": "emulator-5554",
      "appium:app": "/Users/faithberroya/Downloads/app-global-rc.apk",
      "appium:automationName": "UiAutomator2",
      "appium:appPackage": "com.test.school.assignment.rc",
      "appium:appActivity": "com.test.school.assignment.ui.SplashActivity"
}

# launch app
context.driver = webdriver.Remote("http://0.0.0.0:4723/wd/hub", desired_caps)
# add wait time
context.driver.implicitly_wait(20)
# app
context.app = Application(context.driver)

Sample look in Inspector (No handles for Leave Class, Class name, Class item):

What about Layout Inpector from Android Studio?

Hello Aleksei. Tried that one too but it only looks like this :confused:

On other pages (made with xml) it shows fine. We can see the content-description

ask dev help. hardly we can workaround.

possibly you need switch to Espresso driver but! in that case your test build should be builded with Espresso framework.

1 Like

Thank you Aleksei! Our dev investigated more and found out it was covered by another invisible element.