Not able to identify element in WEBVIEW in iOS hyprid app(build in Ionic) automation using Appium on real device

Hi All, did anyone get the solution for the above issue? I have everything latest in place. Appium, ios_webkit_debug_proxy, appium drivers, etc. After I start ios_webkit_debug_proxy, am able to see the device in the session list and if I try to open the app, I am not able to see the webview part in the session to inspect. Instead I am seeing the elements as XCUITestInputField, XCUITestInputSecureField (for password), etc. Is this the expected behaviour or will I be able to see the html part of the code? Please help me to solve this issue, if I am really missing anything important.

here’s my capabilities

IOSDriver driver;
DesiredCapabilities caps = new DesiredCapabilities();
caps.SetCapabilitiy(“app”, appPath);
caps.SetCapabilitiy(“bundleid”, “com..*”);
caps.SetCapability(“autoWebView”, “true”);
caps.SetCapability(“browserName”, “iOS”);
caps.SetCapability (“platformVersion”, Constants.PlatformVersion);
caps.SetCapability(“deviceName”, deviceName);
caps.SetCapability(“startIWDP”,true);

driver = new IOSDriver(new URL(“http://127.0.0.1:4723/wd/hub”),caps, TimeUnit.FromSeconds(10000));

In the appium log I am able to see only NATIVE_APP Context but not the WEBVIEW. Please help.

Hi SandySandy7,

Since your app is in native view after clicking the native element i suppose app is moving to Webview. If yes/you belive so,
then try to check the contexts in ruby. If both the contexts are there then swich to WEBVIEW and try to perform the required action

Hi Isaias_Silva, In pop-up what are the ionic tags for elements(which you want to identify).
Also is it visible in Appium Doctor??

Hey csharsha,
This link should pretty much resolve your problem

hi dhanu,

driver.switchTo.frame(frameid or index); and then try. If not check both page and popup dialogue are in same contexts or not. If they are in different contexts say page in “NATIVE” and dialogue in “WEBVIEW” then switch the contexts accordingly
like
driver.context(“NATIVE”);

Making a short summary.

  1. For Android, I can identify the elements through the ID (I contacted the DEVs to enter the ID in the fields) … Unfortunately I did not succeed in searching for class, class_name, xpath etc … just by ID.

  2. It worked in env.rb by configuring as follows for Android:
    NOTE: the most important is: autoWebview: true.

Before do
def caps
{
caps: {
browserName: ‘’, # Deve usar uma string vazia se estiver automatizando um aplicativo
deviceName: ‘0029179798’, # ‘0029179798’ Motorola Moto Z Play
platformName: ‘Android’,
app: (File.join(File.dirname(FILE), ‘build/apk/baseline09_new.apk’)),
appPackage: ‘io.ionic.starter’,
appActivity: ‘io.ionic.starter.MainActivity’,
newCommandTimeout: 1800,
platformVersion: ‘8.0.0’,
autoWebview: true,
androidScreenshotPath: ‘results/screenshots’,
nativeWebScreenshot: true,
automationName: ‘uiautomator2’,
chromedriverExecutable: (File.join(File.dirname(FILE), ‘drivers/chromedriver.exe’))
},
appium_lib: { server_url:SERVER_URL, port:PORT }
}
end
end

  1. I have not tried it on IOS yet - I’m thinking I’ll have problems … but keep this post open for us to discuss.

If anyone knows of any GEM that helps, or something related to the protractor with ruby, or method or GEM that helps identify the other elements of IONIC.
also search hints for xpath element, class with ruby + appium.

  1. Below is some more information on going to the NATIVE context and WEBVIEW Context with ruby.
    in the hooks.rb file, where I do the encoding of the screenshot for each scenario I do the following

Before do
@driver = Appium::Driver.new(caps, true)
Appium.promote_appium_methods Object
@driver.start_driver
#Espera IMPLÍCITA - Define um tempo limite de espera para todos os elementos
@driver.set_wait(5)

#contexts = @driver.available_contexts
#puts contexts
# @driver.set_context('WEBVIEW_io.ionic.starter')
# Contextos disponívels: ["NATIVE_APP", "WEBVIEW_io.ionic.starter", "WEBVIEW_chrome"]
# Os contextos WEBVIEW_io.ionic.starter e 
# WEBVIEW_chrome são similares e estão setados no env.rb = autoWebview: true

end

Notice that the context switch is commented out, this is because when I put it in env.rb that “autowebview” = true, I’m already saying that I’m going to work with IONIC + Angular in the WEBVIEW context.

In some situations, I need to go back to the NATIVE context … for example, to make clicks on the screen in some coordinate, I need to go to the native context.

I do as follows.

In a method on my page.

def limpar_campo
    # CLICAR NO BOTÃO CONFIRMAR (Coordenadas do MOTO Z Play - Android)
    @driver.set_context('NATIVE_APP')
    1.times{ Appium::TouchAction.new.tap( x: 945, y:1545, count: 1).release.perform }
    @driver.set_context('WEBVIEW_io.ionic.starter')
end

So when I call the cleanup_field method, I click on the Android X to delete the contents of a certain field. For example, to delete the word “Apple” I need to clear the method field 5 times, ie, press 5 times in the Android X to delete the content.

Well, that’s what I know for now … I hope to help them and I hope they help me when I go to IOS.

Hugs
Isaias Silva - Tester Enginner - [email protected]

I am able to work find elements using Webview, to find elements I am downloading entire source code from mobile and using in local browser (our ionic app won’t open in Web) and its working same as selenium.

Sorry to bring this thread back from the dead, but I’ve got a solution that works for me, and I’d like to share it.

Like the others, everything seems like it should work, but when I try to set the context to the webview, it fails, and nothing can be found in the view. BUT if I background and foreground the app, THEN IT WORKS. My elements are found from then on for the test.

I haven’t figured out WHY yet, but at least I have a path forward. Here’s a code snippet (Jest with async javascript):

  it('should do something', async () => {
     await driver.backgroundApp(2);  //put app in background for 10 seconds
     const searchElement = await driver.waitForElementById('Skip', wd.asserters.isDisplayed && wd.asserters.isEnabled, 10000, 100)
     await searchElement.click()
  })

In my case, there is a native splash screen displayed before the webview is intialized. I think this is related to the failure. also, this only works if I set autoWebview to FALSE.