Instruments Crash Running Parallel Tests in Physical Devices

Support
1

Hey everyone. I’m stucked with this issue for almost a week, hope you can give me some insight. I’m running parallel tests in physical apple devices through Jenkins, more specifically one iPhone, one iPad and one iPod. I start an independent Appium server for each device programmatically, each instance with its own port and directory (otherwise the tests won’t run in parallel). After each test suite executed on a specific device, I kill the appium server process (related to that device) and launch the server and driver again for that device port.

If instruments behave well, the tests run normally. However, if instruments gives an error on launching in any appium instance, even though each instance can try to launch instruments again (and they do), when an instance that had a launch error finally launches instruments, all other appium instances that are already running instruments crash with connection refused or session terminated.

I do not know if this is an instruments issue (killing its instances on retry) or an appium issue (related to -r parameter). If this is an instruments issue, I can’t think of a way of handling this (accept ideas if this is the case). Bellow follows how I start the appium server and driver for each device, I’m using ruby:

Constants

DEVICES = [
  {id: 'IPHONE6S_9_3_1', udid: '???', ios: '9.3.1', appium_port: 5700},
  {id: 'IPADMINI2_9_2', udid: '???', ios: '9.2', appium_port: 5701},
  {id: 'IPOD6GEN_9_1', udid: '???', ios: '9.1', appium_port: 5702}]

Appium Server Start

    Process.fork { system "appium --full-reset -r 20 --port #{$device[:appium_port]} --log-level debug -U #{$device[:udid]} --tmp tmp/#{$device[:appium_port]}/ > tmp/#{$device[:appium_port]}/appium.log" }

Apprium Driver Start

opts = {
    caps: {
        platformName: :ios,
        deviceName: $device[:id],
        platformVersion: $device[:ios],
        udid: $device[:udid],
        app: APP_ADDRESS,
        launchTimeout: 90000,
        newCommandTimeout: 90000,
        autoAcceptAlerts: false,
        fullReset: true,
        noReset: false
    },
    appium_lib: {
        port: $device[:appium_port],
        wait_timeout: 120
    }
}
Appium::Driver.new(opts).start_driver
Appium.promote_appium_methods RSpec::Core::ExampleGroup

Errors that make instruments launch retry. After retrying and reconnecting, all other instances crash

[Instruments] Error launching instruments: Instruments never checked in
OR
[Instruments] Error launching instruments: Instruments crashed on startup

Crashes found in other instances after instruments launches during retries

Selenium::WebDriver::Error::NoSuchDriverError: A session is either terminated or not started
OR
Errno::ECONNREFUSED:Failed to open TCP connection to 127.0.0.1:5700 (Connection refused - connect(2) for "127.0.0.1" port 5700)

Thanks in advance and let me know if any more information is needed here.

2

Hey guys, I found a workaround, even though I don’t like it, hope this can help anyone with similar issues. I dug a little into appium (v1.5.3) source code and found out that when instruments don’t work as expected, a function called “killAllInstruments” is called. Well, if you are running a single appium instance, there is nothing to worry about, however if you are running several appium instances in parallel, call this function is the worst case scenario ever.

I found out that it was possible to recover the PID of the current instruments process being executed and changed the line where the “killAllInstruments” function is being called and requested the process with the instruments PID to be killed, this way only the problematic instruments instance will be killed.

LINE 311 in FILE: /Users/username/Desktop/node_modules/appium-ios-driver/node_modules/appium-instruments/build/lib/instruments.js

// COMMENT LINE 311 // return _regeneratorRuntime.awrap((0, _utils.killAllInstruments)());

Then replace for the task to kill the instruments instance by PID:

_logger2['default'].debug('Killing instruments on port ' + this.proc.pid); return _regeneratorRuntime.awrap((0, _teen_process.exec)('kill', ['-9', this.proc.pid]));
The first line here is just a log entrance to help identifying in appium log when the instruments instances (and which) were killed. The second line explicitly kills the instruments instance by its PID. I strongly discourage anyone to edit appium source code by themselves. In this specific case I saw myself in need to do so, since I’m using an old appium version (because of instruments) and running tests in parallel (which is not exactly what appium was intended for).