Appium & Ruby - searching by resource ID is v. slow

sohaiba · September 30, 2016, 11:02am

I’m fairly new to automated testing and ruby. Basically, I am trying to find elements on a page and click on them. For android, when I run the code to find an element, it can sometimes take really long. Here’s an example of the code that runs the find element process:

 def find_element(element_name)
elements = nil
result = nil

if(is_iphone)
    element_name.gsub("'", '\\' * 4 + "'")
end

# Check if element_name is present in the lookup dictionary, if present, use value instead.
if(name_lookup(element_name, is_android == true ? "Android" : "iOS")) then
    element_name = name_lookup(element_name, is_android == true ? "Android" : "iOS")
end

# Search by name or exact text.
value = '//*[@name="' + element_name + '"]'
elements = $driver.find_elements(:xpath, value)
if (elements.size() > 0)
result = elements[0]
    return result
end

# Search by label.

label = '//*[@label="' + element_name + '"]'
elements = $driver.find_elements(:xpath, label)
if (elements.size() > 0)
    result = elements[0]
    return result
end


 if(is_android)
     # Search by resource id (Android only).
     elements = $driver.find_elements(:id, element_name)
     if (elements.size() > 0)
     result = elements[0]
         return result
     end
 end

# Search for element containing the text "element_name". Uses xpath.
# iOS searches by name, Android by text.
is_iphone ? (xpath = '//*[contains(@name, "' + element_name + '")]') : (xpath = '//*[contains(@text, "' + element_name + '")]')
elements = $driver.find_elements(:xpath, xpath)
if (elements.size() > 0)
    result = elements[0]
    return result
end

return result
end

Essentially, what’s happening is that searching by resource id is required to get past the login screen and doesn’t take long whatsoever - in fact it takes milliseconds. However, once I’m past the login screen, searching seems to take forever. Here’s an example of the log:

[HTTP] --> POST /wd/hub/session/4ee15b82-fcdb-4558-8e16-446fff65f34f/elements {"using":"id","value":"What's new"} [MJSONWP] Calling AppiumDriver.findElements() with args: ["id","What's new","4ee15b8... [debug] [BaseDriver] Waiting up to 0 ms for condition [debug] [AndroidBootstrap] Sending command to android {"cmd":"action","action":"find","params":{"strategy":"id","selector":"What's new","context":"","multiple":true}} [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got data from client: {"cmd":"action","action":"find","params":{"strategy":"id","selector":"What's new","context":"","multiple":true}} [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command of type ACTION [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Got command action: find [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Finding 'What's new' using 'ID' with the contextId: '' multiple: true [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Using: UiSelector[RESOURCE_ID=(**hidden**):id/What's new] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] getElements selector:UiSelector[RESOURCE_ID=(**hidden**):id/What's new] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Element[] is null: (0) [AndroidBootstrap] [BOOTSTRAP LOG] [debug] getElements tmp selector:UiSelector[INSTANCE=0, RESOURCE_ID=(**hidden**):id/What's new] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Using: UiSelector[RESOURCE_ID=android:id/What's new] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] getElements selector:UiSelector[RESOURCE_ID=android:id/What's new] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Element[] is null: (0) [AndroidBootstrap] [BOOTSTRAP LOG] [debug] getElements tmp selector:UiSelector[INSTANCE=0, RESOURCE_ID=android:id/What's new] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Using: UiSelector[DESCRIPTION=What's new] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] getElements selector:UiSelector[DESCRIPTION=What's new] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Element[] is null: (0) [AndroidBootstrap] [BOOTSTRAP LOG] [debug] getElements tmp selector:UiSelector[DESCRIPTION=What's new, INSTANCE=0] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Failed to locate element. Clearing Accessibility cache and retrying. [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Finding 'What's new' using 'ID' with the contextId: '' multiple: true [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Using: UiSelector[RESOURCE_ID=(**hidden)**:id/What's new] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] getElements selector:UiSelector[RESOURCE_ID=(**hidden**):id/What's new] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Element[] is null: (0) [AndroidBootstrap] [BOOTSTRAP LOG] [debug] getElements tmp selector:UiSelector[INSTANCE=0, RESOURCE_ID=(**hidden**):id/What's new] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Using: UiSelector[RESOURCE_ID=android:id/What's new] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] getElements selector:UiSelector[RESOURCE_ID=android:id/What's new] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Element[] is null: (0) [AndroidBootstrap] [BOOTSTRAP LOG] [debug] getElements tmp selector:UiSelector[INSTANCE=0, RESOURCE_ID=android:id/What's new] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Using: UiSelector[DESCRIPTION=What's new] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] getElements selector:UiSelector[DESCRIPTION=What's new] [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Element[] is null: (0) [AndroidBootstrap] [BOOTSTRAP LOG] [debug] getElements tmp selector:UiSelector[DESCRIPTION=What's new, INSTANCE=0] [debug] [AndroidBootstrap] Received command result from bootstrap [AndroidBootstrap] [BOOTSTRAP LOG] [debug] Returning result: {"status":0,"value":[]} [MJSONWP] Responding to client with driver.findElements() result: [] [HTTP] <-- POST /wd/hub/session/4ee15b82-fcdb-4558-8e16-446fff65f34f/elements 200 92230 ms - 74

Is there any way to speed this up? Will try to answer questions to clarify this if needed. Thanks!

Perhaps I should add, the screens where it seems to struggle are on screens which overlay the main screen. That is, once we get to the “home screen” then another, slightly see through screen, comes up to show a tutorial, which asks you to swipe or tap to dismiss. Secondly it struggles on the menu which drops down on top of the screen you were already on.

willosser · September 30, 2016, 6:35pm

@sohaiba finding by resource-id can take a long time if there are animations present. 10-20 extra seconds. I’ve run across suggestions to tell Appium to ignore unimportant views, but turns out much of what I was looking for was deemed as unimportant (Make Android tests run faster)

We had had problems with a seven second delay when searching by resource-id/name for an element that is not present, so we built a library to dump the page source and search through that hierarchy. This improved response times dramatically. We extended this to find the element and return it’s location. Now we can tap the location instead of press by resource-id. This has sped things up dramatically.

Given you made the prescient choice to use Ruby, I can share some of our Ruby code with you if you’d like.

sohaiba · October 3, 2016, 8:59am

Hi @willosser, thanks for your help. It would be very helpful if you could share some of that code. Unfortunately I wasn’t the one who made the code, I’m taking over for someone who left the company.

I would be very grateful if you could share how you go through that process you mention above.

Thanks!

willosser · October 4, 2016, 4:45pm

The core of our solution is built around get_page_source(). We use Nokogiri to parse it – you can choose to omit it if you don’t want to add nokogiri to your project.

# Dump page source and convert into string without html characters
#
# Appium can block waiting for page source, so we wrap it in a timer
# and try multiple times for up to two minutes
#
# @return [String] page source
def get_page_source
  appium_driver.get_source
  doc = Nokogiri::XML(source).to_s
  # Converting to string converts <, >, and & to
  doc.gsub(/&lt;/, '<').gsub(/&gt;/, '>').gsub(/&amp;/, '&')
end

Once you have the page source, you can search for specific elements, or multiple elements. This method allows us to search for one or more elements, returning the text/resource-id/element name that was passed in. As Ruby interprets this as true, we can make a decision based on whether it exists or not

# Grab screen elements to determine if element exists
# Faster than find_elements as it won't delay if the
# element does not exist
#
# field can be any type of field stored on the Android page in XML
#
# Example: elements_exist? 'resource-id', ['com.android.vending:id/uninstall_button',
#                                          'com.android.vending:id/launch_button']
#
# Note: The raw form of the field is used, i.e. with hyphens.
# Note: 'values' can be an array of regular expressions and/or strings
#
# @return [String, Boolean] with first found element value
#                           or 'false' in case no element found.
def elements_exist?(field, values)
  values = [*values]
  doc = get_page_source
  values.each { |value| return value if doc =~ /#{field}=\"#{value}\"/ }
  false
end

Once we knew that animations slow down our automation, we created the following method to grab the coordinates from the page source:

# Find the coordinates of the first matching element
#
# @param [<String>] field can be any type of field stored on the Android
#                   page in XML
# @param [<String>, <Array>, <Regex>] values contains strings to verify
#                                     for existence
# @raise [AutomationExceptions::InvalidResultsError] element not found
def elements_coordinates(field, values)
  values = [*values]

  doc = get_page_source
  results =
      doc.split('<android').select do |object|
        matches = values.select { |value| object =~ /#{field}=\"#{value}\"/ }
        !matches.empty?
      end

  raise AutomationExceptions::InvalidResultsError,
        'Could not find expected element on screen ' if results.empty?

  matches = results[0].match(/bounds="([^\\]*?)"/)
  raise AutomationExceptions::InvalidResultsError,
        "No boundaries found in #{results[0]}" if matches.nil?
  matches.captures[0]
end

I hope this helps

sohaiba · October 5, 2016, 3:44pm

Hi @willosser, thanks for your help! TBH once you mentioned animations in the above comment I just shut those under developer options on the phone and the test speed has shot up.

Thanks for all your help!

willosser · October 5, 2016, 5:33pm

Well, if you have that freedom in testing, that’s just wonderful.

haskida · November 29, 2016, 11:54pm

Is this exactly why my test significantly slows down at locating element when it enters a page with a seekbar that has a knob moving across it?
Every other page it is fast but on this page the time it takes to locate stuff becomes random and can take up to like 30 seconds to find something.

willosser · November 30, 2016, 6:12am

The behavior sounds similar. Note that this is very likely a problem with uiautomator and not appium. I haven’t upgraded to Appium 1.6 and uiautomator 2.0 yet. I wonder if the behavior is any different using that.

haskida · November 30, 2016, 6:35am

I am already on 1.6.0 and it has the same issue as when I was on 1.5.X. I also tried Genymotion, Android emulator, and real devices and all have the same result.

If I pause the player which in turn stops the knob from moving on the seek bar, then the element locating is back to normal…

I am trying your method of calling the driver page source and parse the XML to click on the coordinates of the element instead of using FindElement…

I am using C# and I wonder if there’s a done-for-me parser somewhere =/

slipy12 · November 30, 2016, 8:43am

I have the same problem in waiting for some elements when an animation is present but are you really sure it matters so much??

Just wait some extra seconds… I will search through the library trying to improve this part. @willosser ty for your workaround. I will also check with uiautomator2