I’m trying to automate a native iOS application wit appium 1.4.13 (appium.dmg)
When i start the simulator (iOS 9.1), I see all elements, however in the inspector, some mobile elements are invisible, probably because the application has been optimized for blind people and some functionalities are hidden (e.g. a map). These elements are shown in the XML tree but i cannot click on them programmatically.
Am I missing something, or is Appium only able to automate “accessible” elements? If yes, is there any hint in the documentation about this?
Can you clarify your problem a bit more? So far, I’m understanding that there are one of two situations you are facing:
The Appium inspector is able to take a screenshot and display this screenshot. The XML tree representing the UI hierarchy is completely retrieved and displayed. However, some elements the XML tree states that are present are invisible.
The Appium inspector is able to take a screenshot and display this screenshot. The XML tree representing the UI hierarchy is either not completely retrieved, or only a partial representation of the UI hierarchy is available.
situation 1.
The XML Tree statest that they are invisible and they are invisible in the inspector, however i can see and interact with them in the simulator window.
You might be able to get a workaround by checking if the dimension and position of the UI element you want to interact with is non-zero. You can use the driver’s TouchAction API to try manually placing in some taps.
I’ve only seen cases where UI elements are reported invisible if they are in a list or are off screen. Can I see a sample output of the XML hierarchy and which element you want to interact with (assuming you’re not under some NDA from whichever project you’re on)?
When I try to tap on it I get (could not be tapped because the element is not visible):
info: [debug] Pushing command to appium work queue: "au.getElementsByIndexPaths([\"/0/0/12\"])"
info: [debug] Sending command to instruments: au.getElementsByIndexPaths(["/0/0/12"])
info: [debug] [INST] 2015-11-09 07:12:06 +0000 Debug: Got new command 23 from instruments: au.getElementsByIndexPaths(["/0/0/12"])
2015-11-09 07:12:06 +0000 Debug: evaluating au.getElementsByIndexPaths(["/0/0/12"])
info: [debug] [INST] 2015-11-09 07:12:06 +0000 Debug: evaluation finished
2015-11-09 07:12:06 +0000 Debug: Lookup returned [object UIAButton] with the name "OK" (id: 7).
info: [debug] [INST] 2015-11-09 07:12:06 +0000 Debug: responding with:
info: [debug] [INST] 2015-11-09 07:12:06 +0000 Debug: Running system command #24: /Applications/Appium.app/Contents/Resources/node/bin/node /Applications/Appium.app/Contents/Resources/node_modules/appium/submodules/appium-uiauto/bin/command-proxy-client.js /tmp/instruments_sock 2,{"status":0,"value":[{"ELEMENT":"7"}]}...
info: [debug] Socket data received (40 bytes)
info: [debug] Socket data being routed.
info: [debug] Got result from instruments: {"status":0,"value":[{"ELEMENT":"7"}]}
info: [debug] Responding to client with success: {"status":0,"value":[{"ELEMENT":"7"}],"sessionId":"8316f698-081c-4fb2-a28a-c5e9baab16ef"}
info: <-- POST /wd/hub/session/8316f698-081c-4fb2-a28a-c5e9baab16ef/elements 200 3999.759 ms - 89 {"status":0,"value":[{"ELEMENT":"7"}],"sessionId":"8316f698-081c-4fb2-a28a-c5e9baab16ef"}
info: --> POST /wd/hub/session/8316f698-081c-4fb2-a28a-c5e9baab16ef/timeouts {"type":"implicit","ms":1000}
info: [debug] Set iOS implicit wait to 1000ms
info: [debug] Responding to client with success: {"status":0,"value":null,"sessionId":"8316f698-081c-4fb2-a28a-c5e9baab16ef"}
info: <-- POST /wd/hub/session/8316f698-081c-4fb2-a28a-c5e9baab16ef/timeouts 200 2.310 ms - 76 {"status":0,"value":null,"sessionId":"8316f698-081c-4fb2-a28a-c5e9baab16ef"}
info: --> POST /wd/hub/session/8316f698-081c-4fb2-a28a-c5e9baab16ef/element/7/click {"id":"7"}
info: [debug] Pushing command to appium work queue: "au.tapById('7')"
info: [debug] Sending command to instruments: au.tapById('7')
info: [debug] [INST] 2015-11-09 07:12:07 +0000 Debug: Got new command 24 from instruments: au.tapById('7')
info: [debug] [INST] 2015-11-09 07:12:07 +0000 Debug: evaluating au.tapById('7')
2015-11-09 07:12:07 +0000 Debug: target.frontMostApp().elements()[0].elements()[12].tap()
2015-11-09 07:12:07 +0000 Debug: target.frontMostApp().elements()[0].elements()[12].scrollToVisible()
info: [debug] [INST] 2015-11-09 07:12:07 +0000 Debug: target.frontMostApp().elements()[0].elements()[12] - scrollToVisible cannot be used because this element has no scrollable ancestor.
info: [debug] [INST] 2015-11-09 07:12:07 +0000 Debug: target.frontMostApp().elements()[0].elements()[12] - scrollToVisible cannot be used because this element has no scrollable ancestor.
2015-11-09 07:12:07 +0000 Debug: target.frontMostApp().elements()[0].elements()[12] could not be tapped because the element is not visible
info: [debug] [INST] 2015-11-09 07:12:07 +0000 Error: VerboseError: elementId 7 could not be tapped
info: [debug] Socket data received (57 bytes)
info: [debug] Socket data being routed.
info: [debug] Got result from instruments: {"status":13,"value":"elementId 7 could not be tapped"}
info: [debug] Responding to client with error: {"status":13,"value":{"message":"An unknown server-side error occurred while processing the command.","origValue":"elementId 7 could not be tapped"},"sessionId":"8316f698-081c-4fb2-a28a-c5e9baab16ef"}
info: <-- POST /wd/hub/session/8316f698-081c-4fb2-a28a-c5e9baab16ef/element/7/click 500 2264.311 ms - 200
I tried again with appium 1.4.13.dmg and Xcode 6 and it works as well. So there’s a problem with Xcode7 / iOs 9.1.
This could also be a sideeffect of using the original uiAutomation Framework as iwd doesn’t work out of the box with iOS 9 yet.
I was with this same problem and after spend some hours trying to upgrade (and downgrade) my applications (Xcode and Appium) I resolved this problem changing the iPhone Simulator scale to 100%, and now it is working fine.
