Error 500 with WDA when running XCUITests on iOS Github Actions self-hosted (Xcode 15.4)

Issues/Bugs
1

Configuration
Appium Version: 2.10.3
Appium Driver Version: XCUITest 7.18.0
Simulator/Emulator or Real Device: Simulator
The language you are using for writing tests: Javascript / Typescript (via webdriver.io)
MacOS 14.5

Issue
Hello! Running into an odd issue when attempting to run some Appium XCUITests against an iOS simulator on a self-hosted Github Actions runner where we try to initiate a connection after the app / sim is stood up and we eventually time out. What’s frustrating here is that the Github-hosted M1 (macos-latest) works fine. The only difference I can spot is that GHA builder is on Xcode 15.0.1 and self-hosted builder is on 15.4 but locally, I have no issues with 15.4 either, very perplexing. Happy to submit this over on the Appium Github if that is a more appropriate spot. Just trying to figure out what I may be missing on my self-hosted builder in regards to WDA.

Here is the high-level error we get:

[0-1] 2024-06-24T17:55:52.074Z INFO webdriver: DATA {
[0-1]   capabilities: {
[0-1]     alwaysMatch: {
[0-1]       platformName: 'iOS',
[0-1]       'appium:deviceName': 'iPhone 15 Pro',
[0-1]       'appium:platformVersion': '17.5',
[0-1]       'appium:automationName': 'XCUITest',
[0-1]       'appium:appWaitActivity': 'tv.plex.rn.app',
[0-1]       'appium:showXcodeLog': true
[0-1]     },
[0-1]     firstMatch: [ {} ]
[0-1]   },
[0-1]   desiredCapabilities: {
[0-1]     platformName: 'iOS',
[0-1]     'appium:deviceName': 'iPhone 15 Pro',
[0-1]     'appium:platformVersion': '17.5',
[0-1]     'appium:automationName': 'XCUITest',
[0-1]     'appium:appWaitActivity': 'tv.plex.rn.app',
[0-1]     'appium:showXcodeLog': true
[0-1]   }
[0-1] }
[0-1] 2024-06-24T18:02:22.455Z DEBUG webdriver: request failed due to response error: unknown error
[0-1] 2024-06-24T18:02:22.457Z INFO webdriver: Retrying 1/3
[0-1] 2024-06-24T18:02:22.457Z INFO webdriver: [POST] http://127.0.0.1:4723/session
[0-1] 2024-06-24T18:02:22.456Z WARN webdriver: Request failed with status 500 due to An unknown server-side error occurred while processing the command. Original error: Unable to launch WebDriverAgent. Original error: xcodebuild failed with code 65. This usually indicates an issue with the local Xcode setup or WebDriverAgent project configuration or the driver-to-platform version mismatch.

Here is a gist of the entire Appium log:

Here are the capabilities we are using which is working fine locally (MacOS) and on latest-macOS builder (Github Actions):

      platformName: 'iOS',
      'appium:deviceName': process.env.IOS_DEVICE_NAME,
      'appium:platformVersion': process.env.IOS_PLATFORM_VERSION,
      'appium:automationName': 'XCUITest',
      'appium:appWaitActivity': 'tv.plex.rn.app',
      'appium:showXcodeLog': true,
2

Failed call to AXDisableAccessibilityOnTermination: kAXErrorCannotComplete" UserInfo={NSLocalizedDescription=Failed call to AXDisableAccessibilityOnTermination: kAXErrorCannotComplete

this looks like XCTest issue, maybe a bug in Xcode/Simulator itself.

3

Yeah looks like it :expressionless: Can you think of any other ideas to isolate this issue?

  1. I can try rolling back the self-hosted builder Xcode to match what is on Github’s builder
  2. I can dig around more to see if there is something about the session created on self-hosted that perhaps is missing a permission or other functionality. This builder has had some other complications due to being headless. I know that Github’s M1 builders are presumably headless, but I’m sure they have applied fixes and other steps to make them function “reliably” for xcode builds.