REQUEST: Android find_element and scroll tips

[Christopher_Graham]

I have a decent suite of Python based iOS tests working well, including scrolling, locating elements, working with large tables, etc. Bingo!

I’m venturing into Android automation and am hoping someone can save me hours of research and point me to some concise documentation on how to deal with screens with large ListView containers that have lots of LinearLayout rows, most of which will be out of the viewable area?

Some problem areas I need to tackle:

• Android find_element functions fail to locate the element if it is out of view since the page XML only includes visible elements. Surprise! iOS doesn’t have this same issue, the full page XML is available so you can locate items 10 page scrolls out of view.
• So I’ll have to scroll the page in hopes of finding the element. Which is probably fine if I know exactly which element I’m looking for. But that’s not always the case as a lot of my data is dynamic and unknown.
• I have a lot of tests that require knowledge of how many rows of data are displayed. On iOS I can simply: cells = self.driver.find_elements_by_xpath('//UIATableView/UIATableCell') and have everything I need. Since the Android page XML doesn’t include items out of view I need to come up with some clever function that can scroll the page and count up the rows. And know when I’ve scrolled to the end of the list. There could be 10, 34, or 300. I’m assuming 100 people have already figured this out and can throw some code my way.
• Many of my lists are composed of dynamic data. The row will physically be rendered, but one or more may have “Loading…” in the row until the APIs respond with all of the data. On iOS for simplicity’s sake I simply poll all of the UIATableCell until the ActivitySpinner disappears, repeat for every row. This can take up to 120 seconds max. This ensures the whole page has loaded so none of the tests proceed until all of the potentially dependent data is available. Once I crack the case on scrolling this should be easy to do on Android, too.

[shermaneric]

Hi Christopher!
First of all, congratulations on your work so far it sounds like you’re well on your way!

I have a similar problem as you and I have made some common methods used in the solution below. There may be a more efficient way to do this, but so far it’s been so good—quite reliable.

I’ll write this up as pseudocode. If it’s not what you were looking for that’s totally cool. If you want to find out more, I can upload you some snippets that I am using.

Method to get all table cells in a table view on android:

1. Get the current elements on screen . For conversations sake, let’s say there are 5 table cells visible .
2. Store the contents in 1) as an array .
3. I do a fuzzy, relative swipe down . I use Percentages to account for various screen sizes on android.
4. I swipe down far enough where I’ll guarantee an overlap with the first set in 1)
5. More explanation:
   -get initial rows 1-5
   -fuzzy swipe down a given %.
   -I compare the last row in my previous data (pre swipe) with the last row of my new data (Post swipe)
   -if they are the same, I know that I am done.

• if they are not the same (say now rows 3-8 are onscreen (give or take a row) -add to the array

6. Repeat this logic until we reach the Last row.
   

7. So Ruby has a nice command called Uniq, which removes all duplicates from your array . After running at through uniq, I have the correct table cells in a neat little array which I can reference at any time.

I know that this process Can be slow but it is quite reliable . With roughly 150 cells, this takes roughly 3 minutes

Again let me know if you want to know more. Or, if you find a better way to do this let me know I might use that idea

Thanks!
Eric

[RamS]

Very nicely explained how to do things on iOS…

Not sure why such limitation in Android …Something, Android client can address in future releases?

[Christopher_Graham]

@shermaneric That sounds completely logical and is generally what I was thinking. Can you explain in more specific detail about the calls you’re making to scroll, and how you’re determining the percent? Sharing your experience there would be extremely helpful.

At this point the simplest way to scroll using the Python client that I’ve found seems to be:

element_to_tap = self.driver.find_element_by_xpath(<xpath_to_element_near_bottom_of_screen>) element_to_drag_to = self.driver.find_element_by_xpath(<xpath_to_element_near_top_of_screen>) self.driver.scroll(element_to_tap, element_to_drag_to)

All you need to do is feed it two webdriverobjects, it seems to do the rest. But I haven’t fully validated it works in all of my scenarios, that it can scroll down as well as up, that it doesn’t over-scroll, that it works in hybrid webviews, if it can work within smaller frames/scrollable menus/whatever, etc.

Nor have I tried accumulating an intelligent, curated array of the actual rows yet. I already have a function to purge duplicates from lists/arrays, so good tip there. I’m not immediately aware of any of my lists where there may be valid dupes (shouldn’t ever be any off the top of my head), but that’s a potential concern if using a function that blindly purges each and every dupe found.

As far as execution time is concerned, that’s at the very bottom of my list as compared to reliability. Everything I’ve done uses explicit waits, so it’s often slower than I’d really want. But so much more reliable than blind time.sleep() or implicit waits.

I forgot to include this in my OP but here’s the link to the Python Scrolling thread where I’ve contributed all of my iOS experience related to that: How to scroll in Appium using python

[shermaneric]

Hi @Christopher_Graham
Thanks for the detailed response.

I’m going to send you 2 files:

1. CommonPages.rb //it’s a set of common page objects I have specifically for android. (I have a separate one for iOS but it probably doesn’t apply here).
2. bbStudentCommonApi.rb //it’s a set of common page objects I use for android AND ios.

Notable methods:

1. bbStudentCommonApi.rb //you can find all of the swipe methods i’m using. I’m using the appium TouchAction library described here: https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/touch-actions.md

2. CommonPages.rb

• getTableCellsFromUI // gets the list of table cells on a given table view. You pass in the table view element and it returns a list of table cells start to finish.

*is_table_cell_populated // basically checks the first row "in my case “OUTLINE” that it’s populated before continuing. it’s similar to your first thought about checking for your “Loading” indicators going away before continuing.

*clickThisItem // given a text string and table view - click on the item. could be modified to click the “nth item”.

I’m attaching the 2 files and the screen shot of my table view for context. Note that this example scrolls to roughly 100 cells fyi.

About your comments:

1. I haven’t tried scroll, the alternative here is to use Android UI Automator’s UIScrollable methods. Syntax like this roughly: new UiScrollable(new UiSelector().scrollable(true).instance(0)).scrollIntoView(new UiSelector().textContains(“00.jpg”).instance(0))

1a. While some people show promise with this. a) i’m not sure if it can scroll on table cells offscreen. b) i’ve heard this has broken sometime around 1.3. If you search on this site, you can find out more. Maybe there’s a workaround, who knows!

1b. I hear you about the potential concern about duplicates - and my solution has holes with that. my probability is so low for that, and if there was — there’s probably a way to put some sort of marker to identify these duplicates. I’d further say that to avoid this potential duplicates, I’d talk to my developers about adding unique resource identifiers for each table cell on the page (maybe an appended counter).

I hope you find this helpful. I have to sign out now, but please feel free to ask any follow up questions and for sure share anything out you find that may be easier!
much thanks!
eric

bbStudentCommonApi.rb.txt (6.5 KB) commonPages.rb.txt (5.9 KB)

[Christopher_Graham]

Note that self.driver.scroll(element_to_tap, element_to_drag_to) performs the simulated movement so fast Android treats it like a flick instead of a scroll. So you cannot use it to programmatically scroll a single row at a time. I’ll hopefully have time to revisit this tomorrow.

[Christopher_Graham]

Attempting to locate elements in a ListView in Android is THE MOST annoying thing ever.

It is fairly easy if:

• Your list only contains a single, common row type.
• Your list is so short it all fits on one screen (thus the XML remains unchanged if you try to scroll)
• Your list has no table groups (separators)

However, if your list is long, and has table groups (separators) it starts making it extremely difficult.

For example, I have a dynamic ListView similar to this. Any of the Separators may or may not be there. If any Separator group does exist, it will have one or more (effectively unlimited) Items.

SEPARATOR 1
    Item 1
    Item 2
    Etc.
SEPARATOR 2
    Item 1
    Item 2
    Etc.
SEPARATOR 3
    Item 1
    Item 2
    Etc.

If I simply want to get all of the Items without regard to if they belong to a particular Separator I can just use
@shermaneric logic - start at the top, gather all items on screen, scroll-gather-repeat until your last row data doesn’t change. Works great.

However, if you want to find ONLY the items within a particular Separator things get extremely complicated.

For example, if I want to locate the Items for Separator 2 but Separator 1 has 50 items, the page will be so long the initial XML will not even include Separator 2. Not a huge issue, I’ll simply have to scroll from the top of the list until I find Separator 2. That’s easy using @shermaneric logic. Just to clarify, finding the Separator is easy. Gather its items is where the complication arrives.

To find specific Items that are related to each Separator I would normally use XPATH AXES which work perfectly… until you scroll the page and your Separator(s) no longer exists in the XML.

For example, I want to locate all Items android.widget.LinearLayout that exists under Separator 1 android.widget.TextView[@text='Separator 1'] you could use the following XPATH:

//android.widget.LinearLayout[preceding-sibling::android.widget.TextView[1]/@text='Separator 1']

This works perfectly. But only on the first page/screen. In this example, assuming there’s 100 Items under Separator 1 which will require a few screen scrolls to find them all. Once you scroll the page to find more Items under Separator 1 then the XML will no longer include Separator 1 and your XPATH AXES no longer function.

My current path is to craft some clever logic that scans the page for existence of my target Separator, and any other Separator, and dynamically composes an appropriate XPATH using various AXES. But my brain is starting to melt.

This would all be very much easier if each Item (table row) had a unique (but recurrent) resource-id or some other unique attribute I could search for such as @resource-id=‘separator1_row’ and then @resource-id=‘separator2_row’ but they don’t, they’re all effectively identical, and only differentiated by the Separator they fall under.

XPATH AXES work perfectly in iOS since the entire page XML is always provided regardless of how much data is out of view.

Soliciting your [hopefully better] ideas.

[Tarun_gupta]

@Christopher_Graham

d

can u guys help me with the code snippet.since i am facing the similar issue.
i have dynamic list view items. java code repferably

[Tarun_gupta]

@shermaneric : i need java code for the same logic… i am stuck in dynamic list view data.
unable to handle scroll

[shermaneric]

Hi @Tarun_gupta Sorry, I do not know Java. I can walk you through any logic questions you have, though.

[Christopher_Graham]

Here’s some humor for you. I haven’t worked on Android automation in a while and originally did all of my Appium + Python development on a Nexus tablet, including the table management discussed here.

Fast forward to today, I happened to hook up Nexus phone and my table code is failing to find all of the unique table rows, apparently due to some scrolling differences related to the screen size. Watching the scrolling it appears that it sometimes drags too far and some table rows are being truncated (not fully visible) when attempting to scroll the bottom most row in the current view up to be the top most row. I did an internet search to hopefully find some new information on this… and found my way here.

Here’s the code I was apparently happily using to manage the scrolling while gather all table rows from the current screen.

# Tap on the last (bottom) visible row on screen
element_to_tap = all_rows[-1]
# Drag to the first (top) visible row on screen
element_to_drag_to = all_rows[0]  # Fails sometimes on small screens, drags top row slightly out of view
# Use driver.swipe, note it needs a long, slow duration to ensure a scroll vs flick is performed
self.driver.swipe(element_to_tap.location["x"], element_to_tap.location["y"],
                   element_to_drag_to.location["x"], element_to_drag_to.location["y"], 3000)
# Let screen settle after the scroll, otherwise elements may still be moving from swipe
time.sleep(4)

I think I need to update the element_to_drag_to coordinates to include add the element’s size to the x+y coordinates, so it doesn’t overshoot the drag_to action.