Cannot get WebDriverAgent running on iPhone (signing appears good, no obvious errors)

I recently got a new iPhone7 for automated testing, and got it working with our on my macbook perfectly fine. I then moved the iPhone 7 to a mac mini (for CI) that has the same appium server version, and xcode version. I tried using AppiumDesktop with the same capabilities to no avail. I’ve exported the certs and mobileprovision setup from xcode9.2 from my laptop to the mac mini (the signing appears good), but just cannot get WebDriverAgent running on the phone correctly (also tried raw appium 1.7.2 server with npm and the java-client I had working with the phone and my laptop.

Environment:
Appium Desktop 1.4.1 (server 1.7.2)
XCode 9.2
IOS 11.2.6 (iPhone 7)

capabilities:

{
  "platformName": "IOS",
  "platformVersion": "11.2.6",
  "deviceName": "HL's iPhone",
  "noReset": true,
  "app": "/Users/jenkins/Downloads/company-artifact-1.0.0b.ipa",
  "automationName": "XCuiTest",
  "udid": "578cf34a0da3baf526826e632faea2b4678bb486",
  "showXcodeLog": true,
  "showIOSLog": false,
  "xcodeOrgId": "XXXXXXXXXX",
  "xcodeSigningId": "iPhone Developer",
  "updatedWDABundleId": "com.test.bundleId.WebDriverAgentRunner",
  "useNewWDA": false
}

Lines before the error:

[Xcode] 15:09:04.029 xcodebuild[44904:635805] Uploading test configuration to app container.
[Xcode] 15:09:05.766 xcodebuild[44904:635805] Updating test process environment variables.
[Xcode] 15:09:05.767 xcodebuild[44904:635805] Updated testing environment variables {
[Xcode]     "DYLD_FRAMEWORK_PATH" = "/private/var/containers/Bundle/Application/06650CDC-4ACB-4DE3-A1F7-82B49F87AB5C/WebDriverAgentRunner-Runner.app/Frameworks:/Users/jenkins/Library/Developer/Xcode/DerivedData/WebDriverAgent-hlaluljvfxayrsckuptlltxvitpl/Build/Products/Debug-iphoneos:__PLATFORMS__/iPhoneOS.platform/Developer/Library/Frameworks:/Developer/Library/PrivateFrameworks";
[Xcode]     "DYLD_INSERT_LIBRARIES" = "/Developer/usr/lib/libMainThreadChecker.dylib";
[Xcode]     "DYLD_LIBRARY_PATH" = "/private/var/containers/Bundle/Application/06650CDC-4ACB-4DE3-A1F7-82B49F87AB5C/WebDriverAgentRunner-Runner.app/Frameworks:/Users/jenkins/Library/Developer/Xcode/DerivedData/WebDriverAgent-hlaluljvfxayrsckuptlltxvitpl/Build/Products/Debug-iphoneos:__PLATFORMS__/iPhoneOS.platform/Developer/Library/Frameworks:/Developer/Library/PrivateFrameworks";
[Xcode]     "MTC_CRASH_ON_REPORT" = 1;
[Xcode]     "XCODE_DBG_XPC_EXCLUSIONS" = "com.apple.dt.xctestSymbolicator";
[Xcode]     XCTestConfigurationFilePath = "/private/var/mobile/Containers/Data/Application/D6DACDD9-1ECE-40FC-BCD3-ABB2DF2A1AA4/tmp/WebDriverAgentRunner-903637F7-62CF-4D35-8320-1AE67A3C89FA.xctestconfiguration";
[Xcode] }
[Xcode] 15:09:05.994 xcodebuild[44904:635584] Adding console adaptor  34 stderr: (null) 0> for test session 903637F7-62CF-4D35-8320-1AE67A3C89FA.
[Xcode] 15:09:06.298 xcodebuild[44904:635584] Test process runnable PID is 392.
[Xcode] 15:09:06.341 xcodebuild[44904:635584] ############## DID EXECUTE #################
[Xcode] 15:09:06.342 xcodebuild[44904:635584] : parseConsoleOutputFromOriginalOutput: called, 912 bytes written out to StandardOutputAndStandardError
[Xcode] 15:09:06.343 xcodebuild[44904:635584]  finished receiving data from 
[Xcode] 15:09:06.343 xcodebuild[44904:635584] : _considerFlushingDelegateBlockQueue - nothing to flush
[Xcode] 15:09:06.344 xcodebuild[44904:635584] Test operation failure: Launch session expired before checking in.
[Xcode] 15:09:06.344 xcodebuild[44904:635584] _finishWithError:Error Domain=IDETestOperationsObserverErrorDomain Code=5 "Launch session expired before checking in." UserInfo={NSLocalizedDescription=Launch session expired before checking in.}
[Xcode] 
[BaseDriver] Event 'wdaStartFailed' logged at 1522091346402 (15:09:06 GMT-0400 (EDT))
[XCUITest] Unable to launch WebDriverAgent because of xcodebuild failure: "xcodebuild failed with code 65". Make sure you follow the tutorial at https://github.com/appium/appium-xcuitest-driver/blob/master/docs/real-device-config.md. Try to remove the WebDriverAgentRunner application from the device if it is installed and reboot the device.

Try
npm uninstall -g appium

and then install (npm install -g appium) it again and build webdriver agent project.

Hi Singh,

I actually don’t have appium installed globally. I’m currently trying to use it from the Appium Desktop, which has appium as a node module pegged to version 1.7.2

vr-qa03:app jenkins$ npm ls appium
[email protected] /Applications/Appium-1.4.1.app/Contents/Resources/app
└── [email protected]

vr-qa03:app jenkins$

I installed that app yesterday (March 26th), so the contents should be very fresh. I could remove the node_modules directory and re-install based on the package.json values.

vr-qa03:~ jenkins$ npm uninstall -g appium
up to date in 0.117s
vr-qa03:~ jenkins$

Update from my work today. I switched to a “newer” mac mini (from mid-2012 to mid-2014 with better cpu and graphics) and the Appium Desktop 1.4.1 worked out of the box. My old Appium 1.7.2 server is still not working complaining about signing even on the same machine with the same capabilities! Still working it out. I may need to re-install appium 1.7.2 on the machine (in the directory I’m executing from… not globally)

you would still need to build Webdriver Project to make Xcuitest work for iOS client.
The error code 65 is usually thrown when you have not build web driver Project.

So here’s the question then… What’s the difference between how Appium Desktop 1.4.1 (which runs appium server 1.7.2 as well) and basic appium server startup works? I was under the impression that desktop just runs the appium server in a similar way to how I’m starting it up (never mind signing etc…). Though when looking at the Appium Desktop logs, I don’t see any signing currently (and yes, the server 1.7.2 even with a new fresh appium install is failing to sign like I have hit a while ago (ie… why does Desktop work, and the server does not?!) My signing identity is found, and I have a mobile provision profile for this on the machine.

Signing Identity: “iPhone Developer: Benjamin Douglas (YYYYYYYYYY)”

/Users/jenkins/Library/Developer/Xcode/DerivedData/WebDriverAgent-dypkdtwipcubgsehtkwtmqpgrtvf/Build/Products/Debug-iphoneos/WebDriverAgentLib.framework/Frameworks/RoutingHTTPServer.framework: unknown error -1=ffffffffffffffff

For instance on differences. Here’s the output from building WebDriverAgentLib and WEbDriverAgentRunner from Appium Desktop 1.4.1

[Xcode] === BUILD TARGET WebDriverAgentLib OF PROJECT WebDriverAgent WITH CONFIGURATION Debug ===
[Xcode]
[Xcode]
[Xcode]
[Xcode] Check dependencies
[Xcode]
[JSONWP Proxy] Proxying [GET /status] to [GET http://localhost:8100/status] with no body
[JSONWP Proxy] Proxying [GET /status] to [GET http://localhost:8100/status] with no body
[Xcode]
[Xcode]
[Xcode] === BUILD TARGET WebDriverAgentRunner OF PROJECT WebDriverAgent WITH CONFIGURATION Debug ===
[Xcode]
[Xcode]
[Xcode] Check dependencies
[Xcode]
[Xcode]
[Xcode]
[Xcode] ** TEST BUILD SUCCEEDED **

Here’s the output from building the same with just Appium Server (I do startup with the call so I know that’s why I have the nice timestamps :: ‘node_modules/appium/build/lib/main.js --port 4858 --callback-port 4859 --log-timestamp --debug-log-spacing’)

2018-03-27 15:56:50:924 - [Xcode] === BUILD TARGET WebDriverAgentLib OF PROJECT WebDriverAgent WITH CONFIGURATION Debug ===
2018-03-27 15:56:50:925 - [Xcode]
2018-03-27 15:56:50:926 - [Xcode]
2018-03-27 15:56:50:935 - [Xcode]
2018-03-27 15:56:50:936 - [Xcode] Check dependencies
2018-03-27 15:56:50:936 - [Xcode]
2018-03-27 15:56:51:219 - [debug] [JSONWP Proxy] Proxying [GET /status] to [GET http://localhost:8100/status] with no body
2018-03-27 15:56:51:800 - [debug] [JSONWP Proxy] Proxying [GET /status] to [GET http://localhost:8100/status] with no body
2018-03-27 15:56:51:804 - [Xcode]
2018-03-27 15:56:51:804 - [Xcode]
2018-03-27 15:56:51:965 - [Xcode] CodeSign /Users/jenkins/Library/Developer/Xcode/DerivedData/WebDriverAgent-dypkdtwipcubgsehtkwtmqpgrtvf/Build/Products/Debug-iphoneos/WebDriverAgentLib.framework/Frameworks/RoutingHTTPServer.framework
2018-03-27 15:56:51:966 - [Xcode] cd /Users/jenkins/build_tools/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent
2018-03-27 15:56:51:966 - [Xcode] export CODESIGN_ALLOCATE=/Applications/Xcode-9.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/codesign_allocate
2018-03-27 15:56:51:968 - [Xcode] export PATH=“/Applications/Xcode-9.2.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin:/Applications/Xcode-9.2.app/Contents/Developer/usr/bin”
2018-03-27 15:56:51:972 - [Xcode]
2018-03-27 15:56:51:974 - [Xcode] Signing Identity: “iPhone Developer: Benjamin Douglas (4DA9X54L5R)”
2018-03-27 15:56:51:980 - [Xcode]
2018-03-27 15:56:51:980 - [Xcode] /usr/bin/codesign --force --sign 572E419A85926BB577981FE12B0DE72BAC79B693 --preserve-metadata=identifier,entitlements,flags --timestamp=none /Users/jenkins/Library/Developer/Xcode/DerivedData/WebDriverAgent-dypkdtwipcubgsehtkwtmqpgrtvf/Build/Products/Debug-iphoneos/WebDriverAgentLib.framework/Frameworks/RoutingHTTPServer.framework
2018-03-27 15:56:51:981 - [Xcode]
2018-03-27 15:56:52:297 - [Xcode] /Users/jenkins/Library/Developer/Xcode/DerivedData/WebDriverAgent-dypkdtwipcubgsehtkwtmqpgrtvf/Build/Products/Debug-iphoneos/WebDriverAgentLib.framework/Frameworks/RoutingHTTPServer.framework: unknown error -1=ffffffffffffffff
2018-03-27 15:56:52:297 - [Xcode]
2018-03-27 15:56:52:298 - [Xcode] Command /usr/bin/codesign failed with exit code 1
2018-03-27 15:56:52:301 - [Xcode]
2018-03-27 15:56:52:303 - [Xcode]
2018-03-27 15:56:52:305 - [Xcode]
2018-03-27 15:56:52:318 - [debug] [JSONWP Proxy] Proxying [GET /status] to [GET http://localhost:8100/status] with no body
2018-03-27 15:56:52:349 - [Xcode] ** TEST BUILD FAILED **

Interestingly enough… starting up the appium server in the same fashion with the terminal using the Appium Desktop appium 1.7.2 server worked, where using the npm install [email protected] did not from a different directory… so something different between those two setups… still investigating

Some very interesting things that really just probably boil down to how the Appium Desktop is distributed… but still… the smaller Appium Desktop 1.4.1 node_mobiles/appium is much smaller (and has fewer node_modules in it) than the recently installed npm install [email protected]

vr-qa04:build_tools jenkins$ ls -l node_modules/appium/node_modules/* | wc -l
7919
vr-qa04:build_tools jenkins$ ls -l /Applications/Appium.app/Contents/Resources/app/node_modules/appium/node_modules/ | wc -l
369

vr-qa04:build_tools jenkins$ du -hs node_modules/appium/node_modules/
655M node_modules/appium/node_modules/
vr-qa04:build_tools jenkins$ du -hs /Applications/Appium.app/Contents/Resources/app/node_modules/appium/node_modules/
159M /Applications/Appium.app/Contents/Resources/app/node_modules/appium/node_modules/

I tried node 9.9.0 as well as 8.10.0 and 6.13.1

Of course… when trying to use the Appium Desktop install as a grid node… back to square 1… problem signing…

Is this the actual error, or a red herring?

2018-03-27 19:41:32:768 - [Xcode]
2018-03-27 19:41:32:804 - [Xcode] CodeSign /Users/jenkins/Library/Developer/Xcode/DerivedData/WebDriverAgent-dypkdtwipcubgsehtkwtmqpgrtvf/Build/Products/Debug-iphoneos/WebDriverAgentLib.framework/Frameworks/RoutingHTTPServer.framework
2018-03-27 19:41:32:804 - [Xcode] cd /Users/jenkins/build_tools/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent
2018-03-27 19:41:32:804 - [Xcode] export CODESIGN_ALLOCATE=/Applications/Xcode-9.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/codesign_allocate
2018-03-27 19:41:32:804 - [Xcode] export PATH=“/Applications/Xcode-9.2.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin:/Applications/Xcode-9.2.app/Contents/Developer/usr/bin”
2018-03-27 19:41:32:804 - [Xcode]
2018-03-27 19:41:32:804 - [Xcode] Signing Identity: “iPhone Developer: Benjamin Douglas (XXXXXXXXXXX)”
2018-03-27 19:41:32:804 - [Xcode]
2018-03-27 19:41:32:804 - [Xcode] /usr/bin/codesign --force --sign CODE_SIGN_STRING --preserve-metadata=identifier,entitlements,flags --timestamp=none /Users/jenkins/Library/Developer/Xcode/DerivedData/WebDriverAgent-dypkdtwipcubgsehtkwtmqpgrtvf/Build/Products/Debug-iphoneos/WebDriverAgentLib.framework/Frameworks/RoutingHTTPServer.framework
2018-03-27 19:41:32:804 - [Xcode]
2018-03-27 19:41:32:900 - [Xcode] /Users/jenkins/Library/Developer/Xcode/DerivedData/WebDriverAgent-dypkdtwipcubgsehtkwtmqpgrtvf/Build/Products/Debug-iphoneos/WebDriverAgentLib.framework/Frameworks/RoutingHTTPServer.framework: unknown error -1=ffffffffffffffff
2018-03-27 19:41:32:900 - [Xcode]
2018-03-27 19:41:32:901 - [Xcode] Command /usr/bin/codesign failed with exit code 1
2018-03-27 19:41:32:901 - [Xcode]
2018-03-27 19:41:32:902 - [Xcode]
2018-03-27 19:41:32:902 - [Xcode]
2018-03-27 19:41:32:920 - [Xcode] ** TEST BUILD FAILED **
2018-03-27 19:41:32:920 - [Xcode]
2018-03-27 19:41:32:920 - [Xcode]
2018-03-27 19:41:32:921 - [Xcode]
2018-03-27 19:41:32:921 - [Xcode] The following build commands failed:
2018-03-27 19:41:32:921 - [Xcode] CodeSign /Users/jenkins/Library/Developer/Xcode/DerivedData/WebDriverAgent-dypkdtwipcubgsehtkwtmqpgrtvf/Build/Products/Debug-iphoneos/WebDriverAgentLib.framework/Frameworks/RoutingHTTPServer.framework
2018-03-27 19:41:32:921 - [Xcode] (1 failure)

This exact same command works on my laptop (all certs and keys are on both, there are no duplicate keys or certs, and access control is granted to code sign and set to “System Defaults”)

It’e been a week I’m stuck with this problem. Not finding any ways to resolve it. Can anyone please help me with it.
My system info:
Appium Desktop: 1.15.1
Xode: 11
Mac OS: Mojave
iOS : 13+

After I run the script, simulator opens, app gets installed and even the webdriver agent gets installed, but webdriver agent is not starting after that.