How to get absolute coordinates on iOS?

Support
1

I need to automate tapping on a specific bar in a bar chart that’s stored inside of a canvas in a Safari browser on iOS. There’s no useful information in the html to get this information unfortunately so I need to rely on hard coded coordinates in order to tap where I need to. On Android I can easily do this by turning on Pointer location in the developer settings - I can’t seem to find the equivalent on iOS.

I found some javascript code snippets on the net that I could inject into the web inspector on safari to give me coordinates based on user touch but it seems like the coordinates I get there are relative to the web screen, I need absolute coordinates of the entire screen (if resolution is 1920x1080 I need to select which pixel to tap on).

$( document ).click(
        function( event ){

            // Client variables.
            console.log(
                "clientX/Y: " +
                event.clientX + ", " +
                event.clientY
            );

            // Page variables. NOTE: These are the ones
            // that are officially supported by the
            // jQuery Event object.
            console.log(
                "pageX/Y: " +
                event.pageX + ", " +
                event.pageY
            );

            // Hrule.
            console.log( "......" );

        }
    );

Does anyone have any suggestions how to accomplish this? Here is the method I intend to use (and use for Android with success):

public void touchFunction(AppiumDriver driver, int x, int y) throws InterruptedException{
		
	//Gets webview context
	String originalContext = driver.getContext();
	
	//Need to switch to NATIVE_APP in order to use native commands on the devices
	driver.context("NATIVE_APP");
	
	//taps a coordinate
	TouchAction touchAction = new TouchAction(driver)
		.tap(x,y)
		.release();
	touchAction.perform();
        Thread.sleep(1000);
	
	//Switch back to webview context
	driver.context(originalContext);
}

Thanks!

2

You can try to discover the percentage of where your button is. Say your total width is 1 and total height is 1. Your button is at x = 0.4, y = 0.6.

Now just send to the method your screen max width * 0.4 and same for height. This way, independently of the screen size your code will work.

In order to know the original %, I advice you to install appium_lib and ruby and use the console to do some trial and error. If not I think with inspector you may be able to send commands very quickly (never used it that way). Finally you can even use the old ruler :smiley: and calculate the %

1 Like
3

Take a screenshot and open your favourite image editing software.

2 Likes