I’ve spent the week trying to get Windows phone working and I’m nearly there, but it’s proving to be even more roundabout than our iOS implementation.
1. Microsoft’s test automation API’s cost money
They require Visual Studio Premium edition which is very expensive. We could integrate this with Appium, but it would require that users have the Premium or Ultimate editions of VS2013.
2. The Windows Phone Sandbox is Hard to Penetrate
We cannot run shell commands from the automation framework as we could with iOS. The only viable entry point I have found is through the file system. I haven’t confirmed this yet, but it appears you can drop files onto the device from a Windows desktop program that you can then read with the application through the mobile APIs.
3. It Will Likely Require 2-3 Servers to Get all the Work Done
The Coded UI test stuff runs on the device, which means we will likely need to start a server on the device that reads file that another server drops on the device.T hat other server will have to be in C# and then will likely need to get commands from the regular node appium server. I have not found a way to call HTTP requests from Coded UI as there’s no way to give it web permissions in the app manifest (which it will throw exceptions over repeatedly) since there is no App manifest for Coded UI tests.
We at 2GIS have also faced problem of Windows Phone automation and tried to come up with a solution.
First we developed Selenium-based driver for Windows Phone 8.1 automation that required modification of app under test (it is already working driver that we use in our company, it can be found here https://github.com/2gis/Winium.StoreApps).
But what we really wanted was to develop driver that will allow testing of any application without any modification, just like Appium does. Recently we were working on a prototype of Appium like driver. I’ve recently discovered your repository https://github.com/appium/windows-phone-8-bootstrap.
Our solution is based on ideas similar to yours. But instead of compiling test project on each command use coded ui test with infinite loop and running http server (based on socket server).
You can find repository here https://github.com/2gis/Winium.StoreApps
It is in early prototype stage, requiring you to manually deploy tested app to emulator or device and determine ip address of emulator or device. But we already have code base from our previous project that can help us automate it.
The solution consits of two main parts: Winium.StoreApps.CodedUITestProject and Winium.StoreApps.Driver.
Winium.StoreApps.CodedUITestProject is main worker, it is looped coded ui test with http server.
Winium.StoreApps.Driver is driver that listens for JsonWireProtocol commands and forwards most of them to Winium.StoreApps.CodedUITestProject, also it would be responsible for deploying app under test to emulator or device, determining ip address of coded ui test server, and other stuff that can only be done outside windows phone.
To clarify process:
Winium.StoreApps.Driver (Driver) starts to listen for JsonWireProtocol commands
When NewSession command is accepted
2.1. Driver deploys app to emulator or device (starting emulatr if needed). Not implemented in prototype at the moment
2.2. Driver runs Winium.StoreApps.CodedUITestProject (which contains HTTP server) using vs.test.console
2.3. CodedUITestProject sarts to listen for commands on predefined port
2.3. Driver discovers emulator or device ip adress Now we have driver running on a Windows desktop, and your app under test and CodedUITestProject running on Windows Phone. Back to driver.
Driver receives some command, like FindElement or ClickElement
3.1. Driver forwards this command using http request to the ip address
3.2. CodedUITestProject receives command
3.3. CodedUITestProject processes command
3.4. CodedUITestProject returns command result in HTTP response
3.5 Driver forwards response back to test client/user
… Step 3 repeats as many times as needed/defined by tests
…
Driver receive Close command
4.1. Driver kills vs.test.console process, essentially killing running CodedUITestProject server on Windows Phone
A small update, on September 8-9 I gave a couple talks about Windows Phone automation using Winium project (we have to implementation of driver, one requires you to add small test server to your app under test, this one has a lot of features and stable, also I demoed CodedUI version, which does not require app modification, but is an early prototype and requires you to own CodedUI enabled Visual Studio).
slides can be found here http://www.slideshare.net/NickAbalov/windows-phone-and-desktop-test-automation-london-september-2015
The video will be up some time later.
It is not actively developed at the moment, most of needed functionality was done, so we do bug fixes and add new features from time to time. Winpohnedriver and Winium.StoreApps should be ready for use, CodedUI version is still a prototype.
One more update. Thanks to help from Alexander Bayandin from Badoo we recently merged our Silverlight (winphonedriver) and StoreApps (Winium.StoreApps) driver implementations into one single project – Winium.Mobile https://github.com/2gis/Winium.Mobile.
I also started implementing Web/Hybrid apps support based on the work done by Jim Evans on WindowsPhoneDriver. But it will still require embedding automation server, so depending on your framework/toolset) and actual process and doablity of server embedding might vary on frameworks/tools you use. We are currently developing and testing it for native StoreApps applications with embedded WebViews.
Please let us know about automating UWP app on Windows 10 mobile.
I have a UWP app installed in my Windows 10 mobile phone. Please give some insights on setting the desired capabilities for Windows 10 mobile phone.
I have automated the same UWP app on Windows 10 desktop using Appium. The same is to be automated in Windows 10 mobile. I am stuck as to how to set the desired capabilities so that it understands that the automation has to be started on mobile.
For Windows 10 desktop automation of the UWP app, I have WinAppDriver installed on the desktop. Is there any other driver that I need to install for doing the same on Windows mobile or the same WinAppDriver is required?
How do I know the Package name of the UWP app which is installed on Mobile?
It would be of great help if you take some time out and reply to my queries.
Looking forward for your support and cooperation.