Parallel execution with list of devices

Feature
1

We are writing a mobile phone automation framework, running in parallel on the selenium grid. We have an idea - having our own device farm, launch tests without being tied to specific devices, but make it so that the framework itself knows which devices (for example, ios) are now free and runs tests on them. Has anyone done this?

2

I did it. One macMini (16Gb, SSD) ables to run 20 phones (10 iPhones and 10 Android phones). Android runs a bit faster (20-30%). Total output we have about 700 our tests per hour. All tests record video (20 video where 10 ios streams cording on mac side, while on Android it does on device). CPU load about 80%. Memory consumption about 85-90%. We select devices to run in jenkins but programatically check on start what device actually connected and eliminate absent.

TestNG and Java.

1 Like
3

Wow… Sounds great! Can you show the part of your framework where you do parallel launches ?

4

no. as always we write code for some company we are working. but I can point or share details how to make some particular part. start from something. have some problem - ask here…

5

Ok. As I mentioned above we want to run tests and not to be tied to specific devices. I can get list of UDIDs and device names in gradle task, like that:

def stdout = new ByteArrayOutputStream()
exec {
    commandLine('zsh', './script.zsh')
    standardOutput = stdout;
}
String udids = stdout.toString()

where script is pretty simple

#!/bin/zsh

xcrun simctl list devices | grep “(Booted)” | grep -E -o -i “([0-9a-f]{8}-([0-9a-f]{4}-){3}[0-9a-f]{12})”

This data is needed for Desired caps.
As I saw in articles and videos about selenium grid, specific driver instance should be created in test class. It means that I should specify Desired caps in that class. How to avoid it ? How to create DriverFactory that will produce driver instances with different devices for different classes ?

6

our approach a bit different but you can adapt. we know! devices we have.

  1. grab online devices uuid with:
instruments -s devices
  1. have file with known devices which have associations: device and appium port. in our case:
  "iphone8_1": {
    "platformName": "iOS",
    "platformVersion": "14.1",
    "deviceName": "iPhone QA1",
    "udid": "xxxx",
    "appiumPort": 6001
  },
  1. now first before running tests we start appium servers with needed ports: 6001, 6002 …
  2. now we assign number of test threads = number of available devices
  3. now starting open driver instances. each against needed appium server with needed port. Java and TestNG:
    @Parameters({"device"})
    @Step("beforeMethod")
    @BeforeMethod(alwaysRun = true)
    public void beforeMethod(
            @Optional("") String device) {
             driver.setDriver(new AppiumDriverFactory().getDriver(getDevice());
    }

where getDevice() is synchronized function which takes one device from list of available devices. only ONE thread can take device from it at the same time and return when test completes in “afterMethod”.

7

Could you please show work with threads ? Cause it’s not clear to me. How do you assign thread to test class ? How thread takes and give back device from list ?

8

Thread is test code. Just add number of threads into testNG xml generates needed amount of them.

Simple nice example from google search

9

Hello. I am trying to parallelize my appium tests now. I also use TesnNg and have a couple of questions. In which way do you parelellize your tests in testng xml by parallel=“methods” or parallel=“tests”.
I faced some problems when trying to use parallel=“methods” because in a such case I need to pass device parameters for each method in xml (or probably I miss something)
And in which format do you pass device parameter in testng xml file?

10

hi Sofia,

I use it as parameter which we can send also in command line and thus we have flexibility to choose devices right in jenkins job.

example. let’s name parameter as ‘devices’ and set by default 3 phones: ‘iphone11_1, iphone11_2,iPhone8_1’.
in pom.xml we adding property:

<properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>11</maven.compiler.source>
        <maven.compiler.target>11</maven.compiler.target>
        ....
        <devices>iphone11_1, iphone11_2,iPhone8_1</devices>
    </properties>
   ...
<build>
        ...
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.0</version>
                <configuration>
                    <source>11</source>
                    <target>11</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.22.1</version>
                <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>${suiteName}</suiteXmlFile>
                    </suiteXmlFiles>
                   ...
                    <systemPropertyVariables>
                       ....
                        <devices>${devices}</devices>
                    </systemPropertyVariables>
                </configuration>
                ...
            </plugin>
        </plugins>
    </build>

in command to override this value as possibly you know:

mvn test -DsuiteName=suites/smoke/mySuite.xml -Ddevices=iPhone10_1,iPhone7

now! you should read and store this value in ‘beforeSuite’ let say devicesList as list of Strings.
where is magic? magic in ‘beforeMethod’ when you take and remove one phone from this list with synchronization! per ‘devicesList’ variable to avoid problems with multi thread.
when test ended you need return device into list in ‘afterMethod’.