Is there a way to compare an image with appium

UD · March 16, 2016, 5:46pm

Once we launch any application, we first need to verify the application logo/image.

Say i have my expected image in SDCard/images/mypic.jpg.

This image i want to compare with the application logo.

How can i achieve through appium?

afwang · March 16, 2016, 6:19pm

Let me suggest a possible solution. You would take a screenshot with Appium, and then use Appium’s file pull/push feature to retrieve the expected image’s file. Your test would then crop the relevant area from the screenshot and then compare the processed image to the expected image you retrieved with Appium. You would then have to resort to some image comparison library to compare the images and verify that they are close enough to each other.

A better question to ask yourself is, “Why do I have to verify that our application logo is even visible on the screen?” What requirements do you have for such an image? If you need to verify that the image simply exists and is visible on the screen, then instead of comparing a processed screenshot to the expected image, you should retrieve the image just like any other UI element, and then check its dimensions to make sure that it’s on the screen.

UD · March 17, 2016, 4:18am

Thanks for your reply afwang.

I am not sure why you are least bothered about an application image’s comparison? Any application or website belongs to one brand and which is depicted in the application/company logo.

In my career, i see couple of projects having test cases which verifies this logo.

And i know in QTP, there is an option called Bitmap checkpoint which just performs this operation.

So checking the same can be done through appium?

Ankit_Jain · March 17, 2016, 5:02am

I have one more approach for comparing say two image
1.Ask your dev team to add url of image in content-desc attribute on image so that you can see the url in content-desc attribute in uiautomatorviewer
2.Now find the image element say driver.findElement(By.id("")).getAttribute(“name”); and store this url into some string variable
3.Now use assertion and check if my string.contains(“abcd”) in url or not
Here “abcd” would be your expected url(not the complete one but some part which is constant)

afwang · March 17, 2016, 5:41pm

I’m not quite sure what you mean by pointing out I’m not bothered by your particular problems. I was suggesting a simpler solution to a rather rare problem.

daluu · March 19, 2016, 6:35am

FYI, you might want to take a look at these relevant info:

http://dzaiats.github.io/appium-sikuli-driver/

artkoshelev · March 23, 2016, 12:55pm

Is there any reason to keep image on SD card? Usually you should keep it your tests ‘resources’ folder. Once you take screenshot with appium, you can use any library to compare image with expected (take a look at https://github.com/yandex-qatools/ashot in case you use Java).

deadmoto · April 29, 2016, 9:22pm

We take screenshots using Appium and use ImageMagick to compare them against the baseline.
If your tests are written in Java (or any other JVM-compatible language), check it out: https://github.com/HotwireDotCom/image-assert

rgonzo · January 25, 2017, 12:38am

To start with, I am using java.

Here is the screenshot method:

public void screenshot(String path_screenshot) throws IOException{
    File srcFile=driver.getScreenshotAs(OutputType.FILE);
    String filename=UUID.randomUUID().toString();
    File targetFile=new File(path_screenshot + filename +".jpg");
    FileUtils.copyFile(srcFile,targetFile);
}

The method above basically takes a screenshot of the entire screen. Feel free to modify it to do what you want…

I have yet to figure out how to take a screenshot of a MobileElement object only and not the entire screen…,
the way I used to be able to take a screenshot of a View object in Robotium…

But assuming you are able to take a jpg screenshot of your entire screen in your appium test, you can then use these snippets of code to do an image comparison in java:

import android.graphics.Bitmap;
import java.nio.ByteBuffer;
import java.nio.ByteBuffer;
import java.util.Arrays;

public static boolean imagesEquals(Bitmap bitmap1, Bitmap bitmap2) {
    ByteBuffer buffer1 = ByteBuffer.allocate(bitmap1.getHeight() * bitmap1.getRowBytes());
    bitmap1.copyPixelsToBuffer(buffer1);

    ByteBuffer buffer2 = ByteBuffer.allocate(bitmap2.getHeight() * bitmap2.getRowBytes());
    bitmap2.copyPixelsToBuffer(buffer2);

    return Arrays.equals(buffer1.array(), buffer2.array());
}

The method above compares 2 bitmaps and checks to see if they are equal. Java magic

But now you ask, how do I create bitmaps in my code?

Here is an example that shows you how to create 2 bitmaps, one of the screenshot and one of the image that will be compared:

import android.graphics.BitmapFactory;

Bitmap bitmap1 = BitmapFactory.decodeFile("/path/to/screeshot/screenshot.jpg");
Bitmap bitmap2 = BitmapFactory.decodeFile("/path/to/image_that_will_be_compared/assertionImage.jpg");

then once you have the bitmap objects created, simply use the imagesEquals method above to return a boolean value:

//Compare the two images and return the result
boolean areImagesEqual = MobilUtils.imagesEquals(mBitmap1, mBitmap2);

In this case I created a static method and called the class + the method, passed in the two bitmap objects into the method and viola, you get a true or false value.

Anto_Rodrigo · August 16, 2018, 12:33pm

@rgonzo Is this possible to implement Bitmap in normal java appium automation code.
I’m receiving the following error.

java.lang.RuntimeException: Stub!
at android.graphics.BitmapFactory.decodeFile(BitmapFactory.java:29)

mykola-mokhnach · August 16, 2018, 8:07pm

github.com

appium/appium/blob/master/docs/en/writing-running-appium/image-comparison.md

# Image Comparison Features

This article describes the set of image comparison features available in Appium. These features are available in all drivers and require OpenCV 3 native libs. Also, each feature is able to visualize the comparison result, so you can always track what is going on under the hood to select optimal matching parameters to achieve the best comparison results.


## Prerequisites

- OpenCV 3+ native libraries
- [opencv4nodejs](https://github.com/justadudewhohacks/opencv4nodejs) npm module: `npm i -g opencv4nodejs`. By default the preinstall script of this module also downloads and makes all the required OpenCV libs from source, but this requires developer tools to be available on the host system.
- Appium Server 1.8.0+


## Purpose

Image comparison might be handy for many automation tasks. For example:
- It is necessary to figure out whether the given picture example is present on the screen
- It is necessary to calculate coordinates of some predefined on-screen object
- It is necessary to verify whether the current on-screen object state is similar to the expected state

This file has been truncated. show original

Anto_Rodrigo · August 18, 2018, 9:55am

@mykola-mokhnach I’ve tested with the below code. But the test always getting pass for the same and different images. How to validate this, It seems like the image comparison not working properly. Any solution?

byte screenshot = Base64.encodeBase64(driver.getScreenshotAs(OutputType.BYTES));
FeaturesMatchingResult result = driver
.matchImagesFeatures(screenshot, originalImg, new FeaturesMatchingOptions()
.withDetectorName(FeatureDetector.ORB)
.withGoodMatchesFactor(40)
.withMatchFunc(MatchingFunction.BRUTE_FORCE_HAMMING)
.withEnabledVisualization());
assertThat(result.getVisualization().length, is(greaterThan(0)));
assertThat(result.getCount(), is(greaterThan(0)));
assertThat(result.getTotalCount(), is(greaterThan(0)));
assertFalse(result.getPoints1().isEmpty());
assertNotNull(result.getRect1());
assertFalse(result.getPoints2().isEmpty());
assertNotNull(result.getRect2());

mykola-mokhnach · August 18, 2018, 12:47pm

Appium just invokes the corresponding OpenCV primitives under the hood. If the current set of parameters does not work for your images then try something different or a different matching method.

Anto_Rodrigo · August 20, 2018, 5:29am

Yep thanks for your reply @mykola-mokhnach. I guess Similarity Calculation method is the one I expected which works fine for me.

Exactly matches the size(resolution) of the two images
Returns the score for image comparison, maximum score is 1.0

CODE:

byte screenshot1 = Base64.encodeBase64(driver.getScreenshotAs(OutputType.BYTES));
byte screenshot2 = Base64.encodeBase64(driver.getScreenshotAs(OutputType.BYTES));
SimilarityMatchingResult result = driver
.getImagesSimilarity(screenshot1, screenshot2, new SimilarityMatchingOptions()
.withEnabledVisualization());
assertThat(result.getVisualization().length, is(greaterThan(0)));
assertThat(result.getScore(), is(greaterThan(0.0)));

Anto_Rodrigo · September 4, 2018, 9:39am

@mykola-mokhnach I couldn’t compare the images with little different resolutions (varied based on device resolution). So the expected image size has differed with screenshot size.
I’m getting the following exception:

org.openqa.selenium.WebDriverException: An unknown server-side error occurred while processing the command. Original error: Both images are expected to have the same size in order to calculate the similarity score

Is this possible to neglect the resolution while comparing the images in the following appium image comparison method

SimilarityMatchingResult result = driver
.getImagesSimilarity(screenshot1, screenshot2, new SimilarityMatchingOptions()
.withEnabledVisualization());

mykola-mokhnach · September 4, 2018, 10:14am

No it is not possible, at least for this particular comparison method. You must resize the images on your own before comparison

Anto_Rodrigo · September 4, 2018, 1:13pm

Yeah thanks @mykola-mokhnach for your reply ! Yes I did image resize on the screenshot for get rid of this issue.