Recalculate WebElement location and dimensions when using scaled template and screenshot

I have done a little bit of testing with original size screenshots and templates vs half-sized screenshots and templates. In my testing, finding a template in a screenshot with size 1080x1920 (case A) takes on average 500 ms, whereas the same case with a screenshot size 540x960 (case B) takes between 100 and 150 ms in most cases.

Taking this into account, a test suite that makes 2000 find operations will spend about 5 minutes (2000 x 0.150 / 60) in case B versus 16 minutes, 40 seconds (2000 x 0.500 / 60) in case A. Find operations where half-sized screenshots and templates are resized to match the screen size (1080x1920 in my tests) take longer than case A, which is counterproductive in terms of performance.

The problem with case B is the click() method will never work, because the resulting location and dimensions of the find operation are reduced by 50%. For instance, if the actual center of a button is at point (500,800), the click() method will be performed at point (250, 400) in that case.

Given that screenshot testing is becoming popular due to its benefit when verifying app screen layouts, an additional feature that would be very useful in Appium would be a setting to indicate all the resulting coordinates of an image find operation must be recalculated when the screenshot size does not match the actual screen size.

So, in the case above, on a 1080x1920 screen, if Appium gets a 540x960 screenshot it will determine that given screenshot must be resized by a factor of 2. This factor, in the feature requested, should not be applied to the screenshot but to all coordinate-related values of the WebElement resulting from a find operation; meaning if the center of an element found by image in case B is 400x200, the factored center will be 800x400, which will match the actual center on the screen.

Have you tried to play with image matching settings?

Yes. None of the image matching settings achieve the scenario I mentioned; that’s why I’m requesting this feature.

When the screenshot doen’t match the screen size, the only options allowed by the settings are to resize the screenshot so they match or leave the screenshot the size you’re getting. The problem with the latter is the click() method of the resulting WebElement you get from a find method will not work.