Compare_images with appium and openCV using WindowsDriver


#1

I find that I am able to utilize the findImageOccurrence function of an appium driver when that driver is only defined as an AppiumDriver and driving a mobile device.
(found here: https://appium.readthedocs.io/en/latest/en/writing-running-appium/image-comparison/)

When trying image comparison as above with a WindowsDriver connected to windows 10, I get a proxy error:

[HTTP] --> POST /wd/hub/session/6e67d3f6-01b1-4391-ae45-fcb0c8cbcfb8/appium/compare_images
[HTTP] {"mode":"matchTemplate","firstImage":"iVBORw0KGgoAAAANSUhEUgAACfYAAAzkCAIAAACYgxr+AAAJJmlDQ1BpY2MAAEiJlZVnUJNZF8fv8zzphUASQodQQ5EqJYCUEFoo0quoQOidUEVsiLgCK4qINEWQRQEXXJUia0UUC4uCAhZ0gywCyrpxFVFBWXDfGZ33HT+8/5l7z2/+c+bec8/5cAEgiINlwct7YlK6wNvJjhkYFMwE3yiMn5bC8fR0A9/VuxEArcR7ut/P+a4IEZFp/OW4uLxy+SmCdACg7GXWzEpPWeGjy0wPj//CZ1dYsFzgMt9Y4eh/eexLzr8s+pLj681dfhUKABwp+hsO/4b/c++KVDiC9NioyGymT3JUelaYIJKZttIJHpfL9BQkR8UmRH5T8P+V/B2lR2anr0RucsomQWx0TDrzfw41MjA0BF9n8cbrS48hRv9/z2dFX73kegDYcwAg+7564ZUAdO4CQPrRV09tua+UfAA67vAzBJn/eqiVDQ0IgALoQAYoAlWgCXSBETADlsAWOAAX4AF8QRDYAPggBiQCAcgCuWAHKABFYB84CKpALWgATaAVnAad4Dy4Aq6D2+AuGAaPgRBMgpdABN6BBQiCsBAZokEykBKkDulARhAbsoYcIDfIGwqCQqFoKAnKgHKhnVARVApVQXVQE/QLdA66At2EBqGH0Dg0A/0NfYQRmATTYQVYA9aH2TAHdoV94fVwNJwK58D58F64Aq6HT8Id8BX4NjwMC+GX8BwCECLCQJQRXYSNcBEPJBiJQgTIVqQQKUfqkVakG+lD7iFCZBb5gMKgaCgmShdliXJG+aH4qFTUVlQxqgp1AtWB6kXdQ42jRKjPaDJaHq2DtkDz0IHoaHQWugBdjm5Et6OvoYfRk+h3GAyGgWFhzDDOmCBMHGYzphhzGNOGuYwZxExg5rBYrAxWB2uF9cCGYdOxBdhK7EnsJewQdhL7HkfEKeGMcI
[MJSONWP (6e67d3f6)] Driver proxy active, passing request on via HTTP proxy
[debug] [WD Proxy] Matched '/wd/hub/session/6e67d3f6-01b1-4391-ae45-fcb0c8cbcfb8/appium/compare_images' to command name 'compareImages'
[debug] [WD Proxy] Proxying [POST /wd/hub/session/6e67d3f6-01b1-4391-ae45-fcb0c8cbcfb8/appium/compare_images] to [POST http://127.0.0.1:4724/wd/hub/session/AA32AB2F-4A13-4412-86DE-2A0F8800BF6E/appium/compare_images] with body: {"mode":"matchTemplate","firstImage":"iVBORw0KGgoAAAANSUhEUgAACfYAAAzkCAIAAACYgxr+AAAJJmlDQ1BpY2MAAEiJlZVnUJNZF8fv8zzphUASQodQQ5EqJYCUEFoo0quoQOidUEVsiLgCK4qINEWQRQEXXJUia0UUC4uCAhZ0gywCyrpxFVFBWXDfGZ33HT+8/5l7z2/+c+bec8/5cAEgiINlwct7YlK6wNvJjhkYFMwE3yiMn5bC8fR0A9/VuxEArcR7ut/P+a4IEZFp/OW4uLxy+SmCdACg7GXWzEpPWeGjy0wPj//CZ1dYsFzgMt9Y4eh/eexLzr8s+pLj681dfhUKABwp+hsO/4b/c++KVDiC9NioyGymT3JUelaYIJKZttIJHpfL9BQkR8UmRH5T8P+V/B2lR2anr0RucsomQWx0TDrzfw41MjA0BF9n8cbrS48hRv9/z2dFX73kegDYcwAg+7564ZUAdO4CQPrRV09tua+UfAA67vAzBJn/eqiVDQ0IgALoQAYoAlWgCXSBETADlsAWOAAX4AF8QRDYAPggBiQCAcgCuWAHKABFYB84CKpALWgATaAVnAad4Dy4Aq6D2+AuGAaPgRBMgpdABN6BBQiCsBAZokEykBKkDulARhAbsoYcIDfIGwqCQqFoKAnKgHKhnVARVApVQXVQE/QLdA66At2EBqGH0Dg0A/0NfYQRmATTYQVYA9aH2TAHdoV94fVwNJwK58D58F64Aq6HT8Id8BX4NjwMC+GX8BwCECLCQJQRXYSNcBEPJBiJQgTIVqQQKUfqkVakG+lD7iFCZBb5gMKgaCgmShdliXJG+aH4qFTUVlQxqgp1AtWB6kXdQ42jRKjPaDJaHq2DtkDz0IHoaHQWugBdjm5Et6OvoYfRk+h3GAyGgWFhzDDOmCBMHGYzphhzGNOGuYwZxExg5rBYrAxWB2uF9cCGYdOxBdhK7EnsJewQdhL7HkfEKeGMcI64YFwSLg9XjmvGXcQN4aZ...
[WinAppDriver] [STDOUT]
[WinAppDriver] [STDOUT]
[WinAppDriver] [STDOUT] ==========================================
[WinAppDriver] [STDOUT] POST /wd/hub/session/AA32AB2F-4A13-4412-86DE-2A0F8800BF6E/appium/compare_images HTTP/1.1
[WinAppDriver] [STDOUT] Accept: application/json, */*
[WinAppDriver] [STDOUT] Connection: close
[WinAppDriver] [STDOUT] Content-Length: 251754
[WinAppDriver] [STDOUT] Content-Type: application/json; charset=utf-8
[WinAppDriver] [STDOUT] Host: 127.0.0.1:4724
[WinAppDriver] [STDOUT] User-Agent: appium
[WinAppDriver] [STDOUT]
[WinAppDriver] [STDOUT]
[WinAppDriver] [STDOUT] ### WinAppDriver::CommandHandler::Appium::HandleCommandAsMidPoint - Command  is not recognized
[WinAppDriver] [STDOUT] HTTP/1.1 404 Not Found
[WinAppDriver] [STDOUT] Content-Length: 158
[WinAppDriver] [STDOUT] Content-Type: application/json
[WinAppDriver] [STDOUT]
[WinAppDriver] [STDOUT] {"status":9,"value":{"error":"unknown command","message":"Command not recognized: POST: /session/AA32AB2F-4A13-4412-86DE-2A0F8800BF6E/appium/compare_images"}}
[WD Proxy] Got an unexpected response: {"status":9,"value":{"error":"unknown command","message":"Command not recognized: POST: /session/AA32AB2F-4A13-4412-86DE-2A0F8800BF6E/appium/compare_images"}}
[debug] [MJSONWP (6e67d3f6)] Encountered internal error running command: ProxyRequestError: Could not proxy command to remote server. Original error: 404 - {"status":9,"value":{"error":"unknown command","message":"Command not recognized: POST: /session/AA32AB2F-4A13-4412-86DE-2A0F8800BF6E/appium/compare_images"}}
[debug] [MJSONWP (6e67d3f6)]     at JWProxy.proxy (C:\Users\bousia\AppData\Roaming\npm\node_modules\appium\node_modules\appium-base-driver\lib\jsonwp-proxy\proxy.js:180:13)
[debug] [W3C] Matched W3C error code 'unknown command' to UnknownCommandError
[HTTP] <-- POST /wd/hub/session/6e67d3f6-01b1-4391-ae45-fcb0c8cbcfb8/appium/compare_images 500 28 ms - 183
[HTTP]
[HTTP] --> DELETE /wd/hub/session/6e67d3f6-01b1-4391-ae45-fcb0c8cbcfb8
[HTTP] {}
[debug] [MJSONWP (6e67d3f6)] Calling AppiumDriver.deleteSession() with args: ["6e67d3f6-01b1-4391-ae45-fcb0c8cbcfb8"]
[debug] [BaseDriver] Event 'quitSessionRequested' logged at 1552403561236 (11:12:41 GMT-0400 (Eastern Daylight Time))
[Appium] Removing session 6e67d3f6-01b1-4391-ae45-fcb0c8cbcfb8 from our master session list
[debug] [WinAppDriver] Deleting WinAppDriver session
[debug] [WinAppDriver] Deleting WinAppDriver server session

Not sure if this is an Appium issue or WinAppDriver issue.
Does image comparison work while driving a Windows device?

Appium Java Client 7.0.0


#2

This is a bug in Windows driver. Please report it to Microsoft, so they add the endpoint to the list of endpoints, which should not be proxied


#3

Hi @mykola-mokhnach,

I am getting some kickback from my winappdriver issue:

Can you give me some guidance?