Android - using chrome embedded (hybrid app)
I’m able to click on and enter text on most elements fine.
At one point there is a java pop-up menu so I’m now trying to click on the menu, but I really need to use an Action not click, so I find the XY coordinates and they are near the top-left of the screen. Eventually it dawns on me that this bit of page is probably in an iframe and I need to work out the co-ords of the frame to get element co-ords… or switch to the iframe.
- I cannot/find see the iframe in appium inspector it’s all just android.widget.framelayouts all stacked
- how do I find the iframe? if indeed my coordinates for the element being wrong are due to iframe pain. click() on the element works, but it clicks too slow or something and interacts with the parent element instead.
Appium inspector shows the “bounds” of the element correctly. but when I query element.location, it’s in the top-left corner. location[‘x’] + size[‘width’] are waaay to small compared to the “bounds” I see in the inspector. Why is that? I have the right element clicks on it work, taps dont!
Do I need to do some kind of query to return all the iframes? if so what does the query look like? When i dump the page source there are no iframes, since the page is having a lot of JS in it.
you could probably switch between iframes being in webview context: https://www.guru99.com/handling-iframes-selenium.html
I actually am just guessing that I’m in an iframe with this one based on the way appium is returning small values for x and y of an element that is half way down on a tablet screen at around 1200,1300 it’s telling me the element is at 150, 250 . So that’s why I thought it’s an iframe.
And yes i’m in webview already, driving an Embedded browser on Android devices was going to leave me with terrible confusion and unable to build good locators - yes the webpage is not one I own.
and appium reports the element is sitting in a different place. the inspector app is very “kludgey” to use sadly, not a great advertisement for good app design for testers to be comparing experience against, but here you can see the locator is not really useful.
The screenshot above is from native context. Coordinates in native context are returned by UiAutomator, so the only place to complain about these is Google.
yes - in web context i am unable to “tap” that element at all, only to “click” it. I had to resolve an issue where appium-gui-server was unable to install the chrome driver on windows due to file permissions. And then in the inspector I notice that a load of other elements overlap the vertical “…” ellipsis.
I have had endless distraction issues with context switching in the test code, a load of tests were just failing to switch context… so that co-ordinate system mapping now makes a bit more sense. I did not know that. I think my real question is this. How do other people build SSO integrated apps, specifically Azure AD integrated apps and then test them? I’m thinking of going an API route instead of using Appium to drive some of my test.
If you don’t test SSO itself then it is always better to do it via API. Such change would make tests much more stable and faster: https://appiumpro.com/editions/23-making-your-appium-tests-fast-and-reliable-part-5-setting-up-app-state
That does help a bit, yes. So I think the biggest hurdle for me with SSO is, that very few people seem to have described their mobile testing problem too well. A lot of threads where people ask for help, and never update a thread later to explain what they did or how they solved in the end. Example: https://stackoverflow.com/questions/71943814/appium-delete-sso-cookies-in-android-and-ios-apps-with-appium . I have not got a lot of experience as to how the SSO library works and how the browser was cacheing a login cookie, this was my big hurdle.
For example: If someone opens a browser on the phone and forgets to close it, the switch-to-context to go to webview seems to randomly attach to the browser instead of the embedded browser. I only noticed something was hinky because on some phones the selenium driver seems to connect very quickly and on some phones it takes about 10 seconds, then I realized the faster phones to handle the context-switch (which were the only ones failing my tests) had a leftover browser from a technician.
It would have helped if I had know that the context switch can sometimes do that. Basically appium was controlling the wrong browser. Easy to not spot that error when you are distracted.
The biggest problem with things like application back doors is that you already have to build 2 packages, and now we are asking teams to build in security features, but not give them tools to do so safely. I’m using a pattern to set up app state already, since we use a hidden IPC connection that only builds into the debug package. We have a lot of strict security review here. But we do have to pre-load a configuration file anyway, so I’ve added code to close any leftover browsers to that code now.
The biggest issue I have had in previous jobs with bypassing things like interactive login in a test, is that you risk an out-of-box failure going undetected if you never have a stable fully interactive login test case in your smoke suite. We also have to test a good few scenarios as we have to test all the error messages are correct for interactive login failures. The longest time waster in my test is not the SSO login though, it’s a lot of downloading that I have to do, and appium taking 20-30 seconds just to start the server since we don’t use a “grid”. That might be my next optimization, and to upgrade to appium 2. But just copying files onto and off the device still adds a lot of test time. Which is not helped when some of my engineers are working from home over VPN, LOL.
