appium:shutdownOtherSimulators is not working

Issues/Bugs
1

The problem

appium:shutdownOtherSimulators flag doesn’t work for me as it used to.
Have two tests running on (Test1) iPhone 15, (Test2) iPhone 15 Pro simulators. Tests are executed one by one. After Test1 was completed Test2 started, but simulator with iPhone 15 was still running.It is expected that after the Test1 iPhone 15 simulator should be closed.

Previously we used appium v1.22.2 and every new test was started by closing previous simulator and I had no situations when more than one simulator were launched at the same time. Appium server is running with --relaxed-security flag so this is not the reason.

Environment

[email protected]
[email protected] [installed (npm)]
MacOS Sonoma 14.1(23B74) Apple M2(Virtual) Memory 16GB
[email protected]
[email protected]
Real device or emulator/simulator: emulator
XCode v15.4 (installed via Xcodes)
Simulator v17.5
io.appium.java-client v9.2.2
org.testng.testng v7.9.0

Details

image
image1810×999 173 KB

To reproduce

@Test
  public static void safariScenario_Sim17_iPhone15_Test() throws MalformedURLException {
    DesiredCapabilities capabilities = new DesiredCapabilities();
    capabilities.setCapability("appium:automationName", "XCUITest");
    capabilities.setCapability("platformName", "iOS");
    capabilities.setCapability("appium:autoGrantPermissions", true);
    capabilities.setCapability("appium:showXcodeLog", true);
    capabilities.setCapability("appium:shutdownOtherSimulators", true);
    capabilities.setCapability("browserName", "Safari");
    capabilities.setCapability("appium:platformVersion", "17.5");
    capabilities.setCapability("appium:deviceName", "iPhone 15");

    WebDriver driver = new RemoteWebDriver(new URL("http://10.221.188.32:5555/wd/hub"), capabilities);
    try {
      driver.get("https://google/com");
      Thread.sleep(5000);
    } catch (Exception e){
      e.printStackTrace();
    } finally {
      driver.quit();
    }
  }

  @Test
  public static void safariScenario_Sim17_iPhone15Pro_Test() throws MalformedURLException {

    DesiredCapabilities capabilities = new DesiredCapabilities();
    capabilities.setCapability("appium:automationName", "XCUITest");
    capabilities.setCapability("platformName", "iOS");
    capabilities.setCapability("appium:autoGrantPermissions", true);
    capabilities.setCapability("appium:showXcodeLog", true);
    capabilities.setCapability("appium:shutdownOtherSimulators", true);
    capabilities.setCapability("browserName", "Safari");
    capabilities.setCapability("appium:platformVersion", "17.5");
    capabilities.setCapability("appium:deviceName", "iPhone 15 Pro");

    WebDriver driver = new RemoteWebDriver(new URL("http://10.221.188.32:5555/wd/hub"), capabilities);
    try {
      driver.get("https://google/com");
      Thread.sleep(5000);
    } catch (Exception e){
      e.printStackTrace();
    } finally {
      driver.quit();
    }
  }

xcrun simctl list devices

== Devices ==
-- iOS 16.4 --
    iPhone SE (3rd generation) (F4A260A9-33F6-422F-BD15-1B4EFCE523EC) (Shutdown) 
    iPhone 14 (EF08AF58-0DF1-491D-AFAA-C66BE78C9DAC) (Shutdown) 
    iPhone 14 Plus (90B9D3A1-5A18-4F44-915D-1DF798CA902A) (Shutdown) 
    iPhone 14 Pro (CCE91409-2D5A-410C-A021-905A749985C8) (Shutdown) 
    iPhone 14 Pro Max (5B086459-351E-4EE7-A1EB-072B82B1227F) (Shutdown) 
    iPad Air (5th generation) (FD5A3252-61B6-4D95-9E6E-73604471C3A5) (Shutdown) 
    iPad (10th generation) (397DE18F-856A-4AE9-AF46-58466E507B4D) (Shutdown) 
    iPad mini (6th generation) (C265B2D1-438A-450E-BD4F-A6363B1D21C8) (Shutdown) 
    iPad Pro (11-inch) (4th generation) (E1954D25-7A2A-41D4-AD24-2FA4B469763A) (Shutdown) 
    iPad Pro (12.9-inch) (6th generation) (DE6D38A7-D1E0-4653-B5E8-35423A309E8E) (Shutdown) 
-- iOS 17.0 --
    iPhone SE (3rd generation) (EBB413FE-9DE1-4C36-8332-BEC0CF995E02) (Shutdown) 
    iPhone 15 (B17A7A8A-9FFC-406E-90DF-2A6B38072371) (Shutdown) 
    iPhone 15 Plus (426F866B-B29C-4B66-A17D-90A22D65D26D) (Shutdown) 
    iPhone 15 Pro (1414A3ED-0FFA-465F-B9B3-FD783032E4DF) (Shutdown) 
    iPhone 15 Pro Max (B43AEC80-D6F0-4C21-8636-EFB69E367B66) (Shutdown) 
    iPad Air (5th generation) (0B4F3ED9-EADB-4F9D-9FB6-8D684A62ED89) (Shutdown) 
    iPad (10th generation) (22115A06-7750-4E67-9F5E-F7EFF4544FB2) (Shutdown) 
    iPad mini (6th generation) (497FAB70-FC4D-4652-B33F-D679A2AEF556) (Shutdown) 
    iPad Pro (11-inch) (4th generation) (B88BAE04-BAED-49E7-AE2E-65D719857CA5) (Shutdown) 
    iPad Pro (12.9-inch) (6th generation) (CCB430AF-0189-4D58-8083-AB08F46EA9D1) (Shutdown) 
-- iOS 17.5 --
    iPhone SE (3rd generation) (3B0531D3-857B-4ED0-A540-9A11C2733C5B) (Shutdown) 
    iPhone 15 (A4B9DF84-D6B6-42F5-A149-E43405BE8BAB) (Booted) 
    iPhone 15 Plus (3C49480A-29FA-460F-8267-14BECC814914) (Shutdown) 
    iPhone 15 Pro (9ECC6F2A-3B11-4734-9EE1-EA754F0647AC) (Booted) 
    iPhone 15 Pro Max (F1AEA7FB-C9A6-40D8-A628-6515ADAB0499) (Shutdown) 
    iPad (10th generation) (DAF35B81-5D57-4C7A-B8C6-EA1663F81D9B) (Shutdown) 
    iPad mini (6th generation) (928011A7-1E91-4A3D-BC53-42F7E50A8487) (Shutdown) 
    iPad Air 11-inch (M2) (A29E3D86-389D-4769-A471-567B14BA581E) (Shutdown) 
    iPad Air 13-inch (M2) (D6C113A3-239A-44E6-9AE3-16003C8A4991) (Shutdown) 
    iPad Pro 11-inch (M4) (DEFEC6A1-5DEB-4339-AEA6-E914ACB984E5) (Shutdown) 
    iPad Pro 13-inch (M4) (FBAB4FB6-4C7F-4479-9868-811FD0556A7B) (Shutdown) 
-- Unavailable: com.apple.CoreSimulator.SimRuntime.iOS-15-5 --
    iPhone 13 Pro (0857D218-0BBE-45D5-B4A1-BFDCB06C045A) (Shutdown) (unavailable, runtime profile not found using "System" match policy)
    iPhone 13 Pro Max (8AAB9264-21F4-482B-BFC7-F5810C810DD6) (Shutdown) (unavailable, runtime profile not found using "System" match policy)
    iPhone 13 mini (5CD09BC3-EC0A-4C19-9F35-ADBA3C0E0F72) (Shutdown) (unavailable, runtime profile not found using "System" match policy)
    iPhone 13 (B315E734-4513-4B14-BBAA-3D96EA5E14A8) (Shutdown) (unavailable, runtime profile not found using "System" match policy)
    iPhone SE (3rd generation) (D6853346-32FF-4A3E-84D3-5B109B64AC08) (Shutdown) (unavailable, runtime profile not found using "System" match policy)
    iPod touch (7th generation) (EF8A7F00-08B6-4050-B0AB-F72178BFF127) (Shutdown) (unavailable, runtime profile not found using "System" match policy)
    iPad Pro (9.7-inch) (16E928FB-052E-4923-A616-012CF351D9C9) (Shutdown) (unavailable, runtime profile not found using "System" match policy)
    iPad (9th generation) (0401007C-5FC5-4846-B46B-1ADA5E501915) (Shutdown) (unavailable, runtime profile not found using "System" match policy)
    iPad Pro (11-inch) (3rd generation) (65E88A00-A08D-424D-992F-943CFAD5CD60) (Shutdown) (unavailable, runtime profile not found using "System" match policy)
    iPad Pro (12.9-inch) (5th generation) (EC5CFB46-CF58-4BB8-B9CB-7B22D524C0BD) (Shutdown) (unavailable, runtime profile not found using "System" match policy)
    iPad Air (5th generation) (05BFF48B-D4EF-425E-A43A-D165F50BA95D) (Shutdown) (unavailable, runtime profile not found using "System" match policy)
    iPad mini (6th generation) (964F4233-327A-4ED2-A8A1-FA4E111A718D) (Shutdown) (unavailable, runtime profile not found using "System" match policy)
2

It might be the code does not detect other simulators are in Booted state while the test is starting.
Such assumption is confirmed by the No other running simulators have been detected log line.

You could still help us debugging the issue by running the [do not merge] debug simulators shutdown by mykola-mokhnach · Pull Request #2428 · appium/appium-xcuitest-driver · GitHub branch locally and providing the updated logs from it. Follow appium/packages/appium/docs/en/guides/branch-testing.md at master · appium/appium · GitHub for more details on how to test a driver from a GitHub branch

3

And again two tests one by one.
iPhone 15 first
iPhone 15 Pro second

appium.out.log - https://discuss.appium.io/t/appium-shutdownothersimulators-is-not-working/43005 · GitHub

Actually what we need is here:

2024-07-12 15:12:13:740 - e[38;5;24m[e1532b1e]e[0me[38;5;196m[XCUITestDriver@7211]e[0m >>> allDevices: [
  {
    "sdk": "17.5",
    "dataPath": "/Users/admin/Library/Developer/CoreSimulator/Devices/3B0531D3-857B-4ED0-A540-9A11C2733C5B/data",
    "dataPathSize": 18337792,
    "logPath": "/Users/admin/Library/Logs/CoreSimulator/3B0531D3-857B-4ED0-A540-9A11C2733C5B",
    "udid": "3B0531D3-857B-4ED0-A540-9A11C2733C5B",
    "isAvailable": true,
    "deviceTypeIdentifier": "com.apple.CoreSimulator.SimDeviceType.iPhone-SE-3rd-generation",
    "state": "Shutdown",
    "name": "iPhone SE (3rd generation)",
    "platform": "iOS"
  },
  {
    "sdk": "17.5",
    "lastBootedAt": "2024-07-12T15:10:47Z",
    "dataPath": "/Users/admin/Library/Developer/CoreSimulator/Devices/A4B9DF84-D6B6-42F5-A149-E43405BE8BAB/data",
    "dataPathSize": 873152512,
    "logPath": "/Users/admin/Library/Logs/CoreSimulator/A4B9DF84-D6B6-42F5-A149-E43405BE8BAB",
    "udid": "A4B9DF84-D6B6-42F5-A149-E43405BE8BAB",
    "isAvailable": true,
    "logPathSize": 540672,
    "deviceTypeIdentifier": "com.apple.CoreSimulator.SimDeviceType.iPhone-15",
    "state": "Booted",
    "name": "iPhone 15",
    "platform": "iOS"
  },

and here:

2024-07-12 15:12:13:741 - e[38;5;24m[e1532b1e]e[0me[38;5;196m[XCUITestDriver@7211]e[0m >>> otherBootedDevices: []

This condition always return false: device.udid !== device.udid.
Why is it needed?

4

thanks for checking it. Please try if fix: Filter out other simulators properly for shutdown by mykola-mokhnach · Pull Request #2429 · appium/appium-xcuitest-driver · GitHub fixes your issue

5

Yes, thanks for your fast response and help. Now it works