IOS WebDriverAgent won't install in Azure DevOps Pipeline

Issues/Bugs
1

Azure Devops MacOS 11 pipeline is failing to install the appium-webdriveragent onto an iPhone simulator.

Background: This test project works properly on my local box for both Android and iphone emulator/simulators. This process worked properly in the pipeline until a few weeks ago. (Note that Android is still running properly.)

Error: Xcode builds the webdriveragent as expected but seems unable to install to the simulator.

Steps taken: We have gone as far as to call to the xcode build process outside of appium testing and using the derivedDataPath capability to be sure the .app is built and available. This has not made a difference.

Platform: IOS
Appium Version: 1.22 (Fresh install each time pipeline is run.)
Simulator: iPhone 13 Pro Max. (Have also tried with iPhone 12 Pro Max)
Simulator runtime version: 15.0
Language: Java
OS: MacOS 11 (Azure Devops Pipeline Image)

Capabilities:
DesiredCapabilities caps = new DesiredCapabilities();

	caps.setCapability(MobileCapabilityType.PLATFORM_VERSION, getCapabilityInfo(a, "platform.version")); //version of ios on the device
	caps.setCapability(MobileCapabilityType.PLATFORM_NAME, getCapabilityInfo(a, "platform.name")); //ios
	caps.setCapability(MobileCapabilityType.DEVICE_NAME, getCapabilityInfo(a, "device.name")); //human readable name of the simulator
	caps.setCapability(MobileCapabilityType.AUTOMATION_NAME, getCapabilityInfo(a, "automation.name")); //the appium test interface to use 'XCUITest'
	caps.setCapability(MobileCapabilityType.UDID, getCapabilityInfo(a, "udid")); //the unique id of the simulator
	caps.setCapability(MobileCapabilityType.NO_RESET, true);
	caps.setCapability(IOSMobileCapabilityType.BUNDLE_ID, getCapabilityInfo(a, "bundleId")); //the bundle name of pular app
	caps.setCapability(IOSMobileCapabilityType.ACCEPT_INSECURE_CERTS, "true");
	caps.setCapability("autoGrantPermissions", true);
	caps.setCapability(IOSMobileCapabilityType.AUTO_ACCEPT_ALERTS, true);
	caps.setCapability(MobileCapabilityType.APP, getCapabilityInfo(a, "application.path"));  //sets the local path to the bundle, this allows appium to install the app on the simulator
	caps.setCapability(IOSMobileCapabilityType.SHOW_IOS_LOG, true); //outputs a lot of IOS logging
	caps.setCapability(IOSMobileCapabilityType.SHOW_XCODE_LOG, true); //outputs a lot of xcode logging			
	caps.setCapability(IOSMobileCapabilityType.WDA_STARTUP_RETRIES, 2); //how many times do we try to start the WDA on the device?
	caps.setCapability(IOSMobileCapabilityType.WDA_LAUNCH_TIMEOUT, 240000);
	caps.setCapability(IOSMobileCapabilityType.USE_PREBUILT_WDA, true); //Don't build the WDA in appium_webdriveragent file via xcode.  Use the one we built in a hack in the pipeline.
	caps.setCapability("derivedDataPath", "/usr/local/lib/node_modules/appium/node_modules/appium-webdriveragent/Build/Products/Debug-iphonesimulator/"); //the folder where the appium service will look for the wda .app file.
	driver = new IOSDriver<MobileElement>(serverUrl, caps);
	driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
	return driver;

Error Messages:

In the Logs, we can see the connection to the simulator is successful:

[DevCon Factory] Requesting connection for device 2C167F04-934E-459C-8506-7A1D5AC59FAC on local port 8100

[debug] [DevCon Factory] Cached connections count: 0

[DevCon Factory] Successfully requested the connection for 2C167F04-934E-459C-8506-7A1D5AC59FAC:8100

[debug] [XCUITest] Starting WebDriverAgent initialization with the synchronization key ‘XCUITestDriver’

[debug] [WD Proxy] Matched ‘/status’ to command name ‘getStatus’

[debug] [WD Proxy] Proxying [GET /status] to [GET //127.0.0.1:8100/status] with no body
[WD Proxy] connect ECONNREFUSED 127.0.0.1:8100
[debug] [WebDriverAgent] WDA is not listening at ’ 127.0.0.1:8100/’
[debug] [WebDriverAgent] WDA is currently not running. There is nothing to cache
[debug] [XCUITest] Trying to start WebDriverAgent 2 times with 10000ms interval
[debug] [XCUITest] These values can be customized by changing wdaStartupRetries/wdaStartupRetryInterval capabilities
[debug] [BaseDriver] Event ‘wdaStartAttempted’ logged at 1635862145012 (14:09:05 GMT+0000 (Coordinated Universal Time))
[debug] [WebDriverAgent] WDA is not listening at ‘/127.0.0.1:8100/’
[debug] [WebDriverAgent] WDA is currently not running. There is nothing to cache
[debug] [XCUITest] Trying to start WebDriverAgent 2 times with 10000ms interval
[debug] [XCUITest] These values can be customized by changing wdaStartupRetries/wdaStartupRetryInterval capabilities

Eventually we get:

[debug] [WD Proxy] Proxying [GET /status] to [GET /127.0.0.1:8100/status] with no body
[WD Proxy] connect ECONNREFUSED 127.0.0.1:8100
[WD Proxy] connect ECONNREFUSED 127.0.0.1:8100
[Xcode] 2021-10-28 22:42:57.183 xcodebuild[5122:56653] [MT] IDETestOperationsObserverDebug: 2.863 elapsed – Testing started completed.
[Xcode]
[Xcode] 2021-10-28 22:42:57.183 xcodebuild[5122:56653] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec – start
[Xcode] 2021-10-28 22:42:57.183 xcodebuild[5122:56653] [MT] IDETestOperationsObserverDebug: 2.863 sec, +2.863 sec – end
[Xcode]
[debug] [WD Proxy] Matched ‘/status’ to command name ‘getStatus’
[debug] [WD Proxy] Proxying [GET /status] to [GET http: //127.0.0.1:8100/status] with no body

[Xcode] Testing failed:
[Xcode] WebDriverAgentRunner:
[Xcode] WebDriverAgentRunner-Runner encountered an error (Failed to install or launch the test runner. If you believe this error represents a bug, please attach the result bundle at /usr/local/lib/node_modules/appium/node_modules/appium-webdriveragent/Logs/Test/Test-WebDriverAgentRunner-2021.10.28_22-42-54-+0000.xcresult. (Underlying Error: Cannot launch simulated executable: no file found at /usr/local/lib/node_modules/appium/node_modules/appium-webdriveragent/Build/Products/Debug-iphonesimulator/WebDriverAgentRunner-Runner.app))

[Xcode]
[Xcode] ** TEST EXECUTE FAILED **

Note that the ‘missing’ .app directory is available and has the build files inside. This is being confirmed in another step before running the test:

*Starting: Hack:  Check WebDriverAgent exists*
*==============================================================================*
*Task         : Bash*
*Description  : Run a Bash script on macOS, Linux, or Windows*
*Version      : 3.189.0*
*Author       : Microsoft Corporation*
*Help         : h.  ttps://.  docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash*
*==============================================================================*
*Generating script.*
*========================== Starting Command Output ===========================*
*/bin/bash --noprofile --norc /Users/runner/work/_temp/df19e31f-d482-40c5-a576-f0583a88eb41.sh*
*/usr/local/lib/node_modules/appium/node_modules/appium-webdriveragent*
*Target Dir = /usr/local/lib/node_modules/appium/node_modules/appium-webdriveragent/Build/Products/Debug-iphonesimulator*
*appium .app dir = /usr/local/lib/node_modules/appium/node_modules/appium-webdriveragent/Build/Products/Debug-iphonesimulator/WebDriverAgentRunner-Runner.app*
*Moving to the target directory to check for WebDriverAgentRunner-Runner.app*
*Listing the contents of /usr/local/lib/node_modules/appium/node_modules/appium-webdriveragent/Build/Products/Debug-iphonesimulator*
*total 0*
*drwxr-xr-x  8 runner  admin  256 Nov  2 14:05 WebDriverAgentRunner-Runner.app*
*Listing the contents of /usr/local/lib/node_modules/appium/node_modules/appium-webdriveragent/Build/Products/Debug-iphonesimulator/WebDriverAgentRunner-Runner.app .*
*total 408*
*drwxr-xr-x  8 runner  admin     256 Nov  2 14:05 Frameworks*
*-rw-r--r--  1 runner  admin    7916 Nov  2 14:05 Info.plist*
*-rw-r--r--  1 runner  admin       8 Aug 21 22:42 PkgInfo*
*drwxr-xr-x  3 runner  admin      96 Nov  2 14:02 PlugIns*
*-rwxr-xr-x  1 runner  admin  192800 Nov  2 14:05 WebDriverAgentRunner-Runner*
*drwxr-xr-x  3 runner  admin      96 Nov  2 14:05 _CodeSignature*
*done hacking this thing.*
*Finishing: Hack:  Check WebDriverAgent exists*

As far as I can tell after several weeks of digging, there is something wrong with the interaction between the webdriveragent and the latest IOS runtime 15.0. We’ve seen this issue with past appium releases but it looks like the issue was resolved in 1.15 or 1.16 last year.

Any help with this would be appreciated as we are unable to automate our testing while this issue continues.

2

I assume the error is expected then. Or check if the hack above is working properly

3

No, the USE_PREBUILT_WDA is set to true so we can build the WDA outside of the appium testing run. We only did this because appium itself was unable to install the WDA to the simulator. Building ourselves was a desparate attempt to get around the issue. I would prefer to remove USE_PREBUILT_WDA and just let the framework deal with the WDA as it should.

4

Also, the hack is building and creating the .app as expected. Again, we are only doing this build process ourselves as a way to check if XCode is properly building the WDA. What I can’t figure out is why the WDA won’t install on the simulator.

5

Was the hack done with current Xcode? I’ve had problems in the past where I updated Xcode and tried to run with WebDriverAgent that was built with previous Xcode. Maybe update the hack? I’ve found it useful to reinstall Appium and rebuild WebDriverAgent whenever I update Xcode. It’s not stated but you should be using Xcode 13 with iOS 15 runtime. HTH

1 Like
6

Yes, the Pipeline spins up a MacOS 11 image provided by Microsoft. So appium is installed fresh each time we run the pipeline. I believe xcode 13 was added to the image back in October. And that’s when this trouble started.

When you say you rebuild WebDriverAgent… why? Appium builds and installs WDA when testing against IOS, right?

7

I have always had trouble with consistently building WebDriverAgent. At some point I realized that if I changed the bundle ID to my company’s name, I could just sign it with our Enterprise certificate. So I do that every time. Works like a charm.

1 Like
8

Here is another reason I ask about the Xcode version. When testing on iOS 15, I discovered that debug versions of our app had to be signed in a particular way. I found this document from Apple very informative, and it helped me solve the problem:

https://developer.apple.com/documentation/xcode/using-the-latest-code-signature-format

1 Like