Mobile: scrollTo fails in iOS hybrid webview

Issues/Bugs
1

Continuing my Appium adventure with the latest Appium.app 1.3.5, Appium-Python-Client 0.13, Xcode 6.1.1, iOS 8.x simulator.

Happily, when attempting to execute an element.click() against a native object that is out of view, Appium automatically scrolls it into view prior to clicking on it. No brain power or extra code required.

However, I’m finally venturing into hybrid webviews and am finding that the same simple element.click() fails horribly if the element physically exists, but is not visible / currently in view.

I’ve tried the following code to scroll to the element:

el = self.driver.find_element_by_xpath('//UIAWebView/UIALink[contains(@name,"Physical Security Services")]') self.driver.execute_script("mobile: scrollTo", {"element": el.id}) el.click()

I see the vertical scroll bar indicator flash in the right margin indicating something is attempting to scroll, but the webview itself never physically moves (is never scrolled).

What am I doing wrong?

What is the appropriate path to success?

Here is the pertinent logging which shows the successful element discovery, attempted scrollTo, then attempted and failed click (which appears to also automatically attempt a scrollTo).

2015-02-12 18:02:50:004 - info: [debug] [INST] 2015-02-12 18:02:49 +0000 Debug: Lookup returned [object UIALink] with the name "Physical Security Services" (id: 14). 2015-02-12 18:02:50:018 - info: [debug] [INST] 2015-02-12 18:02:49 +0000 Debug: responding with: 2015-02-12 18:02:50:031 - info: [debug] [INST] 2015-02-12 18:02:49 +0000 Debug: Running system command #59: /Applications/Appium.app/Contents/Resources/node/bin/node /Applications/Appium.app/Contents/Resources/node_modules/appium/node_modules/appium-uiauto/bin/command-proxy-client.js /tmp/instruments_sock 2,{"status":0,"value":{"ELEMENT":"14"}}... 2015-02-12 18:02:50:053 - info: [debug] Socket data received (39 bytes) 2015-02-12 18:02:50:053 - info: [debug] Socket data being routed. 2015-02-12 18:02:50:053 - info: [debug] Got result from instruments: {"status":0,"value":{"ELEMENT":"14"}} 2015-02-12 18:02:50:053 - info: [debug] Responding to client with success: {"status":0,"value":{"ELEMENT":"14"},"sessionId":"ced8e825-4fb5-4a04-b5aa-9762ae3a13ed"} 2015-02-12 18:02:50:054 - info: <-- POST /wd/hub/session/ced8e825-4fb5-4a04-b5aa-9762ae3a13ed/element 200 865.261 ms - 88 {"status":0,"value":{"ELEMENT":"14"},"sessionId":"ced8e825-4fb5-4a04-b5aa-9762ae3a13ed"} 2015-02-12 18:02:50:055 - info: --> POST /wd/hub/session/ced8e825-4fb5-4a04-b5aa-9762ae3a13ed/execute {"sessionId":"ced8e825-4fb5-4a04-b5aa-9762ae3a13ed","args":[{"element":"14"}],"script":"mobile: scrollTo"} 2015-02-12 18:02:50:057 - warn: scrollTo will be removed in a future version of appium 2015-02-12 18:02:50:057 - info: [debug] Pushing command to appium work queue: "au.getElement('14').scrollToVisible()" 2015-02-12 18:02:50:057 - info: [debug] Sending command to instruments: au.getElement('14').scrollToVisible() 2015-02-12 18:02:50:095 - info: [debug] [INST] 2015-02-12 18:02:50 +0000 Debug: Got new command 59 from instruments: au.getElement('14').scrollToVisible() 2015-02-12 18:02:50:109 - info: [debug] [INST] 2015-02-12 18:02:50 +0000 Debug: evaluating au.getElement('14').scrollToVisible() 2015-02-12 18:02:50:122 - info: [debug] [INST] 2015-02-12 18:02:50 +0000 Debug: target.frontMostApp().elements()[1].elements()[1].elements()[0].elements()[48].scrollToVisible() 2015-02-12 18:02:50:412 - info: [IOS_SYSLOG_ROW ] Feb 12 13:02:50 --- last message repeated 13 times --- 2015-02-12 18:02:50:412 - info: [IOS_SYSLOG_ROW ] Feb 12 13:02:50 CHRIGRAH-M-2038 lsd[68230]: LaunchServices: Currently 0 installed placeholders: ( 2015-02-12 18:02:50:412 - info: [IOS_SYSLOG_ROW ] ) 2015-02-12 18:02:50:779 - info: [IOS_SYSLOG_ROW ] Feb 12 13:02:50 CHRIGRAH-M-2038 assertiond[68218]: assertion failed: 14C109 12B411: assertiond + 11523 [3F572A0B-7E12-378D-AFEE-EA491BAF2C36]: 0x1 2015-02-12 18:02:52:708 - info: --> GET /wd/hub/status {} 2015-02-12 18:02:52:708 - info: [debug] Responding to client with success: {"status":0,"value":{"build":{"version":"1.3.5","revision":"a124a15677e26b33db16e81c4b3b34d9c6b8cac9"},"isShuttingDown":false},"sessionId":"ced8e825-4fb5-4a04-b5aa-9762ae3a13ed"} 2015-02-12 18:02:52:709 - info: <-- GET /wd/hub/status 200 1.108 ms - 178 {"status":0,"value":{"build":{"version":"1.3.5","revision":"a124a15677e26b33db16e81c4b3b34d9c6b8cac9"},"isShuttingDown":false},"sessionId":"ced8e825-4fb5-4a04-b5aa-9762ae3a13ed"} 2015-02-12 18:02:55:733 - info: [debug] Socket data received (39 bytes) 2015-02-12 18:02:55:734 - info: [debug] Socket data being routed. 2015-02-12 18:02:55:734 - info: [debug] Got result from instruments: {"status":0,"value":{"ELEMENT":"15"}} 2015-02-12 18:02:55:734 - info: [debug] Responding to client with success: {"status":0,"value":{"ELEMENT":"15"},"sessionId":"ced8e825-4fb5-4a04-b5aa-9762ae3a13ed"} 2015-02-12 18:02:55:734 - info: <-- POST /wd/hub/session/ced8e825-4fb5-4a04-b5aa-9762ae3a13ed/execute 200 5679.216 ms - 88 {"status":0,"value":{"ELEMENT":"15"},"sessionId":"ced8e825-4fb5-4a04-b5aa-9762ae3a13ed"} 2015-02-12 18:02:55:768 - info: [debug] [INST] 2015-02-12 18:02:55 +0000 Debug: evaluation finished 2015-02-12 18:02:55:783 - info: [debug] [INST] 2015-02-12 18:02:55 +0000 Debug: Lookup returned [object UIALink] with the name "Physical Security Services" (id: 15). 2015-02-12 18:02:55:796 - info: [debug] [INST] 2015-02-12 18:02:55 +0000 Debug: responding with: 2015-02-12 18:02:55:810 - info: [debug] [INST] 2015-02-12 18:02:55 +0000 Debug: Running system command #60: /Applications/Appium.app/Contents/Resources/node/bin/node /Applications/Appium.app/Contents/Resources/node_modules/appium/node_modules/appium-uiauto/bin/command-proxy-client.js /tmp/instruments_sock 2,{"status":0,"value":{"ELEMENT":"15"}}... 2015-02-12 18:03:01:323 - info: [IOS_SYSLOG_ROW ] Feb 12 13:03:01 --- last message repeated 15 times --- 2015-02-12 18:03:01:323 - info: [IOS_SYSLOG_ROW ] Feb 12 13:03:01 CHRIGRAH-M-2038 mstreamd[68209]: (Note ) PS: Media stream daemon stopping. 2015-02-12 18:03:01:324 - info: [IOS_SYSLOG_ROW ] Feb 12 13:03:01 CHRIGRAH-M-2038 mstreamd[68209]: (Note ) AS: <MSIOSAlbumSharingDaemon: 0x7f7f53b069b0>: Shared Streams daemon has shut down. 2015-02-12 18:03:01:324 - info: [IOS_SYSLOG_ROW ] Feb 12 13:03:01 CHRIGRAH-M-2038 mstreamd[68209]: (Warn ) mstreamd: mstreamd shutting down. 2015-02-12 18:03:01:994 - info: [IOS_SYSLOG_ROW ] Feb 12 13:03:01 CHRIGRAH-M-2038 assertiond[68218]: assertion failed: 14C109 12B411: assertiond + 11523 [3F572A0B-7E12-378D-AFEE-EA491BAF2C36]: 0x1 2015-02-12 18:03:05:739 - info: --> POST /wd/hub/session/ced8e825-4fb5-4a04-b5aa-9762ae3a13ed/element/14/click {"sessionId":"ced8e825-4fb5-4a04-b5aa-9762ae3a13ed","id":"14"} 2015-02-12 18:03:05:739 - info: [debug] Pushing command to appium work queue: "au.tapById('14')" 2015-02-12 18:03:05:739 - info: [debug] Sending command to instruments: au.tapById('14') 2015-02-12 18:03:05:866 - info: [debug] [INST] 2015-02-12 18:03:05 +0000 Debug: Got new command 60 from instruments: au.tapById('14') 2015-02-12 18:03:05:881 - info: [debug] [INST] 2015-02-12 18:03:05 +0000 Debug: evaluating au.tapById('14') 2015-02-12 18:03:05:896 - info: [debug] [INST] 2015-02-12 18:03:05 +0000 Debug: target.frontMostApp().elements()[1].elements()[1].elements()[0].elements()[48].tap() 2015-02-12 18:03:05:911 - info: [debug] [INST] 2015-02-12 18:03:05 +0000 Debug: target.frontMostApp().elements()[1].elements()[1].elements()[0].elements()[48].scrollToVisible() 2015-02-12 18:03:11:377 - info: [debug] [INST] 2015-02-12 18:03:11 +0000 Debug: target.frontMostApp().elements()[1].elements()[1].elements()[0].elements()[48] could not be tapped 2015-02-12 18:03:11:483 - info: [debug] [INST] 2015-02-12 18:03:11 +0000 Error: VerboseError: elementId 14 could not be tapped 2015-02-12 18:03:11:493 - info: [debug] Socket data received (58 bytes) 2015-02-12 18:03:11:494 - info: [debug] Socket data being routed. 2015-02-12 18:03:11:494 - info: [debug] Got result from instruments: {"status":13,"value":"elementId 14 could not be tapped"} 2015-02-12 18:03:11:494 - info: [debug] Responding to client with error: {"status":13,"value":{"message":"An unknown server-side error occurred while processing the command.","origValue":"elementId 14 could not be tapped"},"sessionId":"ced8e825-4fb5-4a04-b5aa-9762ae3a13ed"}

2

I’ve attempted switching contexts prior to attempting .click() or a scrollTo with no results.

self.driver.switch_to.context('WEBVIEW_1')

3

Bump - someone out there must have some suggestions for how to successfully scroll within iOS webviews within a native app.

I can’t imagine I need to manually construct a custom function to grab all visible elements, determine the last element on the page, grab it’s coordinates, scroll the page up, scan all visible elements again, and keep repeating until my element is in view. There has to be some successful built-in function that handles scrolling. Right? :smiley:

4

I finally had time to work on this some more and thought I’d share what I’ve found thus far in case anyone else has the same issue. Or reads this and offers something different.

Note I’m testing on the iOS8 simulator only at this time, not Android at all yet.

The solution appears to be to simply switch to the webview context self.driver.switch_to.context('WEBVIEW_1') prior to attempting to interact with the webview elements. Then remember to switch back to the native app context self.driver.switch_to.context('NATIVE_APP') prior to interacting with the native elements the webview is wrapped in.

Once you switch to the webview context the contents of which are then accessed just like they would using native Selenium. You can get the page source self.driver.page_source and see the full HTML.

So, instead of doing this in the native context:

self.driver.find_element_by_xpath('//UIAlink[@name="Hello, are you the text I'm looking for?"]').click()

You use this in the webview context:

self.driver.find_element_by_xpath('//a[text()="Hello, are you the text I'm looking for?"]').click()

And when invoking a click on the webview element Appium will now (just like it does for native elements) happily automatically scroll any out-of-view elements into view prior to clicking on it.

This is casually described here: http://appium.io/slate/en/v1.0.0/?ruby#automating-hybrid-apps Too bad I didn’t read that a couple of weeks ago. :smile:

1 Like
5

Hi Christopher,

You note above was helpful.

I load a document in my app and switch to webview. It works this far.
Trying to find an element in the webview for iOS fails.

Here is what i am trying to do:
def get_element_by_xpath(element_path)
begin
$appiumDriver.find_element(:xpath,element_path)
rescue
raise ElementNotFoundException, element_path
end
end

get_element_by_xpath(’//UIAWebView[text()=“This is an automation test”]’)
In my document this is the only line of text present.

I am not sure if it is even working right. I do see a context switch happen and this is what I see in the logs until timeout
info: [debug] [REMOTE] got applicationSentData response
info: [debug] Waited for 13280ms so far
info: [debug] [REMOTE] Executing ‘find_element’ atom in default context
info: [debug] [REMOTE] Sending javascript command
info: [debug] [REMOTE] Sending _rpc_forwardSocketData: message to remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] got applicationSentData response
info: [debug] Waited for 13792ms so far
info: [debug] [REMOTE] Executing ‘find_element’ atom in default context
info: [debug] [REMOTE] Sending javascript command
info: [debug] [REMOTE] Sending _rpc_forwardSocketData: message to remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] got applicationSentData response
info: [debug] Waited for 14301ms so far
info: [debug] [REMOTE] Executing ‘find_element’ atom in default context
info: [debug] [REMOTE] Sending javascript command
info: [debug] [REMOTE] Sending _rpc_forwardSocketData: message to remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] Receiving data from remote debugger
info: [debug] [REMOTE] got applicationSentData response

Thanks

6

In my message above I did not specifically note if you can use native locators like UIALink in find_by_xpath once you’ve switched contexts to the webview, so I cannot recall if that works or not and haven’t tried.

But I did specifically note that you can successfully use HTML friendly XPATH like you would when using native Selenium.

I mention this as I see in your code it looks like you’re attempting to locate an element in the webview using the native identifier //UIAWebView which I’m assuming is what’s causing the issue. Instead I’m guessing you need to compose an HTML friendly XPATH.

7

Hi Christopher,
I have used the code below, but it is throwing the error.
self.driver.switch_to.context(‘WEBVIEW_0’)

self.driver.switch_to.context(‘WEBVIEW_0’)
File “C:\Python27\lib\site-packages\appium\webdriver\switch_to.py”, line 31, i
n context
self._driver.execute(MobileCommand.SWITCH_TO_CONTEXT, {‘name’: context_name}
)
File “C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py”, l
ine 175, in execute
self.error_handler.check_response(response)
File “C:\Python27\lib\site-packages\appium\webdriver\errorhandler.py”, line 27
, in check_response
raise NoSuchContextException(wde.msg, wde.screen, wde.stacktrace)
NoSuchContextException: Message: No such context found.

kindly suggest.
thanks,

8

The error is clearly stated: “NoSuchContextException: Message: No such context found.”

You’re explicitly calling ‘WEBVIEW_0’ which may not exist. Possible cause are you simply don’t have a WEBVIEW context at all, or it has a different index such as WEBVIEW_2.

Here’s the generic functions I’m using. See if these help.

def switch_to_webview_context(self):
    # print "DEBUG: Contexts = ", self.driver.contexts
    # print "DEBUG: Current context = ", self.driver.current_context
    try:
        print "DEBUG: Switching to WEBVIEW context"
        # Webview context can change index sometimes, but there's only ever one (I think)
        contexts = self.driver.contexts  # Gets all contexts, should be NATIVE_APP and one WEBVIEW_[index]
        for context in contexts:
            if "WEBVIEW" in context:
                print "DEBUG: webview context = ", context
                self.driver.switch_to.context(context)
        return True
    except:
        print "WARNING! Could not switch to WEBVIEW context"
        return False

def switch_to_native_context(self):
    try:
        print "DEBUG: Switching to NATIVE_APP context"
        self.driver.switch_to.context('NATIVE_APP')
        return True
    except:
        print "WARNING! Could not switch back to NATIVE_APP"
        return False
9

Hi Christopher,
yes u r correct, I don’t have webview context. I tried this code. It is showing the following output.

DEBUG: Contexts = [u’NATIVE_APP’]

DEBUG: Switching to WEBVIEW context

WARNING! Could not switch to WEBVIEW context
ok

Ran 1 test in 43.161s

OK

but I need to have a webview context . I am new to appium. I am working with hybrid app and couldn’t find the elements in UI Automator. For native apps in UI Automator I can find web elements easily. so I searched in the net and found that I have to switch to webview app.
Kindly suggest.
Thanks,

10

@sweety1 You have a different problem you need to tackle first. You should start a new thread and make sure to post screenshots and the page source for your app. Use the Appium Inspector, then drive your app to the desired page, then click Refresh then then Copy XML in the Inspector to grab the page source. This will let people see exact what it is you’re dealing with so we can provide better help. Otherwise we have no idea if your app even has a webview or not, etc.

11

Hello @sweety1 did you managed to solve your problem?

I’m having the exact same thing, I can’t find or switch context to webview, although my hybrid app has to have a webview, but somehow appium doesn’t find it.

Any suggestions ?