Clicking on webview element executed but nothing happens (iOS Hybrid Application)

sarthak.garg · January 27, 2016, 1:43pm

I have read previous threads about this issue, but either the threads are old or do not work in my case.

I am using Appium with the following configurations:
Appium.app: v1.4.13
XCode: 7.1
iOS Version: 9.1
Device: iPhone 6 (Simulator and Real device both)

I have a Hybrid iOS application with native views and webviews. The ‘click’ WebElement method works fine for native views. However, on execution upon webviews (in the correct webview context) ‘click’ is performed but nothing happens.

I have tried the following solutions:

Switching to native context and performing click
Verdict: the element cannot be recognised in the native context
Getting the element location and performing click on that location
Verdict: the location coordinates and the actual coordinates are different
Executing JavascriptExecutor queries (mobile:tap, arguments[0].click() etc)
Verdict: nothing happens
Set nativeWebTap to true
Verdict: no change
Using Selenium TouchActions
Verdict: no change even after implementing ‘HasTouch’ method

Here is my code:
public class RealDevice {

	public AppiumDriver driver;
	public WebDriverWait wait;
	
	@Before
	public void setUp() throws Exception{
		DesiredCapabilities capabilities = new DesiredCapabilities();
		capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "iOS");
		capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION, "9.1");
                //udid and device name capabiliies
		capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, "Selendroid");
		capabilities.setCapability("nativeWebTap", true);
		capabilities.setCapability(MobileCapabilityType.APP, "path...");		
		driver = new IOSDriver(new URL("http://0.0.0.0:4723/wd/hub"), capabilities);
		wait = new WebDriverWait(driver, 600);
	}
	
	@After
	public void tearDown() throws Exception{
		driver.quit();
	}
	
	@Test
	public void testingRealDevice() throws Exception{	
		System.out.println("PRE EXECUTION");
		
		//Wait for the 'view' button to appear then open the issue
		String viewName = "View";
		wait.until(ExpectedConditions.presenceOfElementLocated(By.name(viewName)));
		driver.findElement(By.name(viewName)).click();	
		
		Thread.sleep(3000);
		
		Set<String> availableContexts = driver.getContextHandles();
		for(String context: availableContexts){
			System.out.println(context);
			if(context.contains("WEBVIEW")) driver.context(context);
		}
		
		System.out.println(driver.getContext());
		
		Thread.sleep(3000);
		
		WebElement element = driver.findElement(By.xpath("//*[@id='box31']"));
		element.click();
		
		System.out.println("CHECKPOINT");
		
		Thread.sleep(5000);
	}
}

Note that the element is detected as I’ve fetched it’s attributes and they are correct. However the click operation is not performed on it.

Here is the Appium log for ‘click’

info: [debug] [REMOTE] Executing 'click' atom in default context
info: [debug] [REMOTE] Sending javascript command
info: [debug] [REMOTE] Remote debugger data sent [{"method":"Runtime.evaluate","params":{"objectGroup":"console","includeCommandLineAPI":true,"doNotPauseOnExceptionsAndMuteConsole":true,"expression":"(function(){return function(){function h(a){return...]

Suraj786 · May 20, 2016, 1:10pm

@sarthak.garg Did you find solution for this? I am also facing the same issue while automation of iOS hybrid app build in ionic…

sarthak.garg · May 20, 2016, 5:40pm

Hi yes @Suraj786.

The Problem:
Clicking on web element but nothing happens because the click happens at the wrong location. I tested my click method on an Android device with touch pointer location ON and the co-ordinates were all wrong.

The Cause:
My app is designed in such a way that it stretches the content to fit the screen with the content. Selenium/Appium expects the content to be placed in the un-stretched location and therefore performs click at the wrong location.

The Solution:
The solution is VERY simple. Many people have tried to explain it but have failed to present it in a good way. I hope my explanation makes sense to you and to anyone who reads it in the future. Here it is -

Due to the stretching of content, there is a difference between the device pixels and the css pixels. Device pixels are native screen pixels which are different from css pixels on web. We can get the css pixels from element.getLocation(). All we need to do is multiply it with the right ratio and voila, we get the correct location.

Let us say that the location that we get when we use element.getLocation() be (x1, y1).
Switch to web context (can be any webview just switch to it).
Get the element location (x1, y1)
Get the element size
Get the window size via driver.manage().window().getSize();
Run 2 javascript queries to get screen.availHeight and screen.availWidth
Switch to native context
Your ratio for correct x-coordinate will be availWidth/windowWidth
Your ratio for correct y-coordinate will be availHeight/windowHeight
Find the center of the element you want to tap on using it’s location and dimensions.
Now multiply this central point with the ratio you just calculated.
Sorted.

My code in Java looks as follows:

                switchContext(true);

		Point elementLocation = element.getLocation();
		Dimension elementSize = element.getSize();
		Dimension windowSize = driver.manage().window().getSize();

                JavscriptExecutor js = (JavascriptExecutor) driver;
                long webRootWidth = js.executeScript("return screen.availWidth");
                long webRootHeight = js.executeScript("return screen.availHeight");

		switchContext(false);

		double xRatio = (double) (webRootWidth * 1.0 / windowSize.getWidth());
		double yRatio = (double) (webRootHeight * 1.0 / windowSize.getHeight());

                int pointX += (int) ((elementLocation.getX() + elementSize.getWidth() / 2.0));
	       int pointY += (int) ((elementLocation.getY() + elementSize.getHeight() / 2.0));
       	 return new Point((int) (pointX * xRatio), (int) (pointY * yRatio));

Let me know if you still cannot find any solution. If this doesn’t help you, let me know, I had a lot of problem with finding the right algorithm and making it my own and I would be happy to help everyone else as well.

Also, I’ve made a framework of my own using Appium and Selenium to run generic test scripts across all Android, iOS and Web platforms for certain apps. So I’ve had my fair share of problems with Appium. But in the end it all works out.

Going to start a blog soon as well. Again, hope I helped someone. Over and out.

Suraj786 · July 13, 2016, 11:36am

Hi Sarthak, I was trying to click using Appium but not able to do so. ALso tried your solution and in vain.

Nothing worked for me.
My app is built in ionic framework. Is this the issue?

Ionic Hybrid app has ionic specific tags such as … If I create the xpath’s using these tags and try to perform action… Nothing happens and control just passes to next step.

Is appium clicks not compatible on Ionic apps??

NaveenKS · May 3, 2017, 4:45am

Hi Sarthak,

Even i am facing the same issue, where the click action happens but does nothing.

I am using Appium1.6.3, in this

WebElement.click, performs action but does nothing on the real device.

Then, I used TouchActions, but this errors out “… tap not defined” and “method not implemented yet”

any help on how to make click action work will be a great help.

sangmai350 · May 24, 2018, 3:06pm

Hey, anyupdate in 1.8.0 ? I’m facing the same issue with appium 1.8.0 and ionic application

hm2286 · August 6, 2018, 7:25am

Hi Sarthak, I am also facing issue in clicking on login button. I have used webelement to inspect the element as it comes under Webcontext & the element identified using uiautomator is not working.

Problem is element is identified but when we click on it, it seems to be clicked but in reality no action is taken. Please suggest.

WebElement loginbtn = driver.findElement(By.xpath("//button[@aria-label=‘LoginButton’]"));
loginbtn.click();

deepikathiru · December 27, 2018, 12:15pm

Hi @sarthak.garg Ur reply relates to the issue i have now. I was also facing the same issue the element location provided by getLocation() is different when checking with pointer manually.

I tried the same that you provided but still then it doesn’t work. Have you got any other workaround for this. It will be highly helpful.

Thanks in advance

atalip · March 27, 2023, 7:46am

I had same issue on Appium 8.0.0 and iOS version 16.3

TouchActions solved my issue. Hope helping with you as well.

WebElement element = driver.findElement(By.xpath("//XCUIElementTypeButton[@label='Close']"));
TouchActions action = new TouchActions(driver);
action.click(element).perform();

Use Actions instead of TouchActions on Appium 8.3.0

 Actions action = new Actions(driver);
 action.click(element).perform();

Aniket_Gulhane · August 18, 2023, 11:05am

"Hi, I was also facing the same issue in clicking a field on an iOS hybrid app. I tried different approaches like TouchAction, Action, and so on.

After carefully debugging the code, I found that the context change was done successfully, but the click action was not performing. The execution then proceeded to the next step. So, I tried changing the locator for the WebElement. Although the previous locator was working for Android and mWeb, I found that one of my CSS locators worked for clicking the field.

Subsequently, I tried adding CSS locators for other WebElements, and my script worked fine.

Note: try to use relative CSS locators, e.g., tagName[attribute=‘value’], div[class=‘bold-text’].

I hope this might help all of you."