Ruby Android scroll_to and scroll_to_exact methods blow up on resourceId

mario_bianco · July 15, 2017, 7:25pm

Hi all,

Apologies if this is a known bug, I’ve been digging into Ruby and Appium in the past week in an effort to setup Android automation and I came across an issue with both the scroll_to and scroll_to_exact Ruby methods. Disclaimer, I learned Ruby just this week but this does indeed look like a valid bug from what I can tell.

Issue 1
The issue is that these methods attempt to scroll by 3 different scenarios. They first scroll by text, then description, then resourceId. If I was to enter a valid resource id of “android:id/text1” then it would first scroll by text, then description, then eventually resourceId and then throw the following error:

Selenium::WebDriver::Error::UnknownCommandError: The requested resource could not be found, or a request was received using an HTTP method that is not supported by the mapped resource.
from /usr/local/lib/ruby/gems/2.4.0/gems/selenium-webdriver-3.4.0/lib/selenium/webdriver/remote/response.rb:69:in `assert_ok’

The Easy Fix
The reason why this happens is because there is a semicolon in the following code:
scroll_uiselector(resource_id(text, "new UiSelector().resourceId(#{text});"), scrollable_index)

When I reverse engineer both the scroll_to methods in arc, I find that if I remove the semicolon then I’m able to successfully scroll to an element by its resourceId.

The code referencing the semicolon locations:

github.com

appium/ruby_lib/blob/c705d40f89117b7dea37ff6365c6d4f418e51ce9/lib/appium_lib/android/element/generic.rb#L45


      
          
          # Scroll to the first element containing target text or description.
          # @param text [String] the text or resourceId to search for in the text value and content description
          # @param scrollable_index [Integer] the index for scrollable views.
          # @return [Element] the element scrolled to
          def scroll_to(text, scrollable_index = 0)
            text = %("#{text}")
          
            args = scroll_uiselector("new UiSelector().textContains(#{text})", scrollable_index) +
                   scroll_uiselector("new UiSelector().descriptionContains(#{text})", scrollable_index) +
                   scroll_uiselector(resource_id(text, "new UiSelector().resourceId(#{text});"), scrollable_index)
          
            find_element :uiautomator, args
          end
          
          # Scroll to the first element with the exact target text or description.
          # @param text [String] the text or resourceId to search for in the text value and content description
          # @param scrollable_index [Integer] the index for scrollable views.
          # @return [Element] the element scrolled to
          def scroll_to_exact(text, scrollable_index = 0)
            text = %("#{text}")

and

github.com

appium/ruby_lib/blob/c705d40f89117b7dea37ff6365c6d4f418e51ce9/lib/appium_lib/android/element/generic.rb#L59


      
          
              # Scroll to the first element with the exact target text or description.
              # @param text [String] the text or resourceId to search for in the text value and content description
              # @param scrollable_index [Integer] the index for scrollable views.
              # @return [Element] the element scrolled to
              def scroll_to_exact(text, scrollable_index = 0)
                text = %("#{text}")
          
                args = scroll_uiselector("new UiSelector().text(#{text})", scrollable_index) +
                       scroll_uiselector("new UiSelector().description(#{text})", scrollable_index) +
                       scroll_uiselector(resource_id(text, "new UiSelector().resourceId(#{text});"), scrollable_index)
          
                find_element :uiautomator, args
              end
            end # module Android
          end # module Appium

Repro Steps:

Connect to emulator via arc for easy debugging then paste the following commands:
scrollable_index = 0
resource_id = “android:id/text1” # Note, use a valid resource-id, this is just an example one
resource_id = %(“#{resource_id}”)
content = “new UiSelector().resourceId(#{resource_id});”
find_element :uiautomator, “new UiScrollable(new UiSelector().scrollable(true).instance(#{scrollable_index})).scrollIntoView(#{content}.instance(0))”
An error of “Selenium::WebDriver::Error::UnknownCommandError” will be thrown
Hit the up arrow twice and remove the semicolon then hit Enter
Hit the up arrow twice and hit Enter (it should be on the find_element statement)
Emulator correctly scrolls to the element

Issue 2
When scrolling by text, I notice that the scrolling will first go to the element, but then scroll away from it (most likely because the scroll by text gets executed and then the scroll by description does). The issue is that when you attempt to scroll to an element and then click on it, you are left with an exception being thrown because the element is no longer in view since the second scrolling script (description) executed. When I break out the 3 separate args script statements into their own action, then the scrolling scrolls correctly to the element and stays there.

Thus I believe it would be more ideal to break out the 3 separate arg script statements into their own methods imo.

Here is some sample code of the methods I stubbed out in case anyone else in the community would like to benefit from this:

def scroll_to_element_by_text(text, scrollable_index = 0)
  text = %("#{text}")

  content = "new UiSelector().textContains(#{text})"

  find_element :uiautomator, "new UiScrollable(new UiSelector().scrollable(true).instance(#{scrollable_index})).scrollIntoView(#{content}.instance(0));"
end


def scroll_to_element_by_exact_text(exact_text, scrollable_index = 0)
  exact_text = %("#{exact_text}")

  content = "new UiSelector().text(#{exact_text})"

  find_element :uiautomator, "new UiScrollable(new UiSelector().scrollable(true).instance(#{scrollable_index})).scrollIntoView(#{content}.instance(0));"
end

def scroll_to_element_by_content_description(content_desc_text, scrollable_index = 0)
  content_desc_text = %("#{content_desc_text}")

  content = "new UiSelector().description(#{content_desc_text})"

  find_element :uiautomator, "new UiScrollable(new UiSelector().scrollable(true).instance(#{scrollable_index})).scrollIntoView(#{content}.instance(0));"
end

def scroll_to_element_by_resource_id(resource_id, scrollable_index = 0)
  # Pass the full resource-id into the param e.g.:
  # "com.example.Test:id/enter"

  resource_id = %("#{resource_id}")

  content = "new UiSelector().resourceId(#{resource_id})"

  find_element :uiautomator, "new UiScrollable(new UiSelector().scrollable(true).instance(#{scrollable_index})).scrollIntoView(#{content}.instance(0));"
end

willosser · July 16, 2017, 4:06pm

There have been a number of bugs in scroll_to and scroll_to_exact in the Ruby implementation over the years. We implemented our own local version of those two methods as you described, and each time I’ve tested the product to see if it’s working, it hasn’t. I recommend filing a ticket on github – with your work, it looks like it should be simple to fix. Because I’m cautious, I’d also recommend implementing your own routines so that (a) you know they will work now, and (b) they will be faster.

mario_bianco · July 17, 2017, 11:13pm

Hi @willosser thanks for the speedy reply!

I’ll try and post a Merge Request or write up a bug issue for the actual github repository sometime early this week.

thuy · December 13, 2017, 11:54am

@mario_bianco I tried to scroll_to, scroll_to_exact with resource_id but always get a null result. And I don’t think the semicolon is a problem, I removed it same with your suggestions, but still not work. I think about the format of resource id, maybe I use incorrect, doesn’t it?
my_package_name:id/resource_id

armate · April 21, 2021, 5:34pm

I tried all the solutions everywhere mentioned… nothing works for me FOr me it keeps scrolling up and down for all the solutions. Not sure what to do.

Aleksei · April 21, 2021, 7:02pm

read tutorial -> http://appium.io/docs/en/writing-running-appium/tutorial/swipe-tutorial/
it is java but uiScrollable commands are same.

armate · April 27, 2021, 9:09pm

Hi Aleksei,

i want to scroll from middle of the screen from an frame layout. Nothing is working for me. As you see in the screenshot of my app, this instructor menu i need to drag up. can you please help?

Aleksei · April 28, 2021, 3:44am

Inside “Instructor”? I see above screen is greyed out and not scrollable. correct?

armate · May 11, 2021, 5:31pm

yes, it got worked now. Thanks.