Appium Terminating Unexpectedly/Not Receiving Commands

cdubwisdom · November 9, 2023, 9:24pm

I am not sure what is going on but I am getting a org.openqa.selenium.NoSuchSessionException: A session is either terminated or not started error. This just suddenly started happening. I made no code changes or updates.The server just started to time out one day.

Android 14
iOS 17
Mac Sonoma 14
Appium 2.2.1
uiautomator2 2.34.1
xcuitest 5.8.2
java 11
node 21.1.0

appium_server_log_iOS.log (621.2 KB)
appium_server_log_Android.log (865.8 KB)

Looking at the logs it seems that the time between appium commands keeps getting longer. Almost seems like it is freezing up then the newCommandTimeout is reached and the session in terminated. I have newCommandTimeout set to 3 minutes and have tried setting it to 10 minutes with the same result.

I am using java 11 and Cucumber TestNG to run my automation tests on real Android and iOS devices. I am locating the app elements using PageFactory and AppiumFieldDecorators. A quick example code is

public class AppPage{
    public AppPage(AppiumDriver driver){
        PageFactory.initElements(new AppiumFieldDecorator(driver, Duration.ofSeconds(15L)), this);
    }

    @iOSXCUITFindBy(accessibility = "Welcome")
    @AndroidFindBy(accessibility = "Welcome")
    private WebElement welcomeTxt;

    public bolean isPageOpen(){
        return welcomeTxt.isDisplayed();
    }
}

Even the simple WebElement.isDisplayed() method will sometimes just never reach the appium server and it will just sit silent until it times out.

I have tried adding both implicit and explicit waits but eventually the times between commands just exceeds the newCommandTimeout. Even scenarios that identical except for the users will take longer and longer to execute as the test run progresses.

Aleksei · November 9, 2023, 9:47pm

What selenium version?
What Java client version?

Met similar while trying to increase above versions in mine code. Will investigate more next week.

PS Look at java memory while test running. It is constantly increasing and this cause slowness.

mykola-mokhnach · November 10, 2023, 7:43am

Try to add some debug logs to your client code or just run the code in the debug mode to figure out where exactly the delay happens.

cdubwisdom · November 10, 2023, 1:56pm

Java 11, don’t have selenium installed just Appium 2.2.1

cdubwisdom · November 10, 2023, 4:50pm

So I believe my issue may be Cucumber and not Appium. my client code seems to take a considerable amount of time to connect my scenarios steps with their corresponding step definition. I guess this can be closed for now as not an appium issue but if I find additional details I will still update this thread.

Aleksei · November 13, 2023, 2:02pm

initial setup

Java 11
org.seleniumhq.selenium : selenium-java : 4.9.0
java-client : 8.4.0

test 1:

org.seleniumhq.selenium : selenium-java :  -> 4.12.1

= no slowness, maximum memory of largest java thread was about 2Gb 

first test:  [purchaseOver50]   BaseTest:                     | afterMethod():   Execution time: 92sec

test 2:

org.seleniumhq.selenium : selenium-java :  -> 4.12.1
java-client : -> 8.6.0

= slowness starts quite fast after 5-10min. memory of largest java thread reaching 3.5-4Gb.

first test: [purchaseOver50]   BaseTest:                     | afterMethod():   Execution time: 281sec - already slower in 3 times!

after 30min everything dead, java reaching 5Gb and only `A session is either terminated or not started`

In appium logs no changes, appium server did not change. All command executing fast. Problem looks like in slowness of java-client itself.
I found that the slowness starts from 8.5.1 which give not many commits between 8.5.0.

Reproduced on two independent machines.

@mykola-mokhnach any ideas? I bet this is - refactor: Replace cglib with bytebuddy (#1923 )

PS i catched a dump with jstack + Spindump and can share you privately if it can help

mykola-mokhnach · November 13, 2023, 8:05pm

cglib is pretty much abandoned. We cannot rely on it anymore.

Regarding the slowness. Try to get the client from the recent master snapshot and reproduce the issue with it. Also in this case it would probably be hard find anything in the heap snapshot as it must be linked to your actual framework code. the best way to do anything about it would be to let the code run for a while, then connect a remote debugger to it and check which objects are piling up in the memory and are not getting garbage collected.

Aleksei · November 13, 2023, 8:29pm

Need still to make some changes with migrate to latest client. Do not believe it will help but …

cdubwisdom · November 14, 2023, 2:42pm

Thanks @Aleksei and @mykola-mokhnach for continuing to look into this. I downgraded my appium java-client dependency and it looks to be running without issue now. Its still running so I will update once/if it completes.

Aleksei · November 14, 2023, 3:46pm

@mykola-mokhnach we need to investigate… sooner or later it will be blocker to use appium Java client.

cdubwisdom · November 14, 2023, 4:46pm

Test Suite can to completion without any unexpected errors. So it does seem to be an issue with the new versions of java-client.

For reference
pom.xml with Timeout/Slowness issue

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>App</groupId>
<artifactId>Mobile-Automation</artifactId>
<version>1.0-SNAPSHOT</version>

<properties>
    <maven.compiler.release>11</maven.compiler.release>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <mobileOS>Android,iOS</mobileOS>
</properties>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.0</version>
            <configuration>
                <release>11</release>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0</version>
            <configuration>
                <suiteXmlFiles>
                    <suiteXmlFile>testng.xml</suiteXmlFile>
                </suiteXmlFiles>
                <properties>
                    <testnames>${mobileOS}</testnames>
                </properties>
            </configuration>
        </plugin>
    </plugins>
</build>

<dependencies>
    <dependency>
        <groupId>io.appium</groupId>
        <artifactId>java-client</artifactId>
        <version>8.6.0</version> #Note I also got the error with v9.0.0
    </dependency>
    <dependency>
        <groupId>org.hamcrest</groupId>
        <artifactId>hamcrest</artifactId>
        <version>2.2</version>
    </dependency>
    <dependency>
        <groupId>com.googlecode.json-simple</groupId>
        <artifactId>json-simple</artifactId>
        <version>1.1.1</version>
    </dependency>
    <dependency>
        <groupId>io.cucumber</groupId>
        <artifactId>cucumber-java</artifactId>
        <version>7.14.0</version>
    </dependency>
    <dependency>
        <groupId>io.cucumber</groupId>
        <artifactId>cucumber-core</artifactId>
        <version>7.14.0</version>
    </dependency>
    <dependency>
        <groupId>io.cucumber</groupId>
        <artifactId>gherkin</artifactId>
        <version>27.0.0</version>
    </dependency>
    <dependency>
        <groupId>com.testvagrant.mdb</groupId>
        <artifactId>mdb</artifactId>
        <version>2.2.1</version>
    </dependency>
    <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>7.8.0</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>io.cucumber</groupId>
        <artifactId>cucumber-testng</artifactId>
        <version>7.14.0</version>
    </dependency>
    <dependency>
        <groupId>io.netty</groupId>
        <artifactId>netty-common</artifactId>
        <version>4.1.100.Final</version>
    </dependency>
</dependencies>

pom.xml with no issues

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>App</groupId>
<artifactId>Mobile-Automation</artifactId>
<version>1.0-SNAPSHOT</version>

<properties>
    <maven.compiler.release>11</maven.compiler.release>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <mobileOS>Android,iOS</mobileOS>
</properties>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.0</version>
            <configuration>
                <release>11</release>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0</version>
            <configuration>
                <suiteXmlFiles>
                    <suiteXmlFile>testng.xml</suiteXmlFile>
                </suiteXmlFiles>
                <properties>
                    <testnames>${mobileOS}</testnames>
                </properties>
            </configuration>
        </plugin>
    </plugins>
</build>

<dependencies>
    <dependency>
        <groupId>io.appium</groupId>
        <artifactId>java-client</artifactId>
        <version>8.5.0</version>
    </dependency>
    <dependency>
        <groupId>org.hamcrest</groupId>
        <artifactId>hamcrest</artifactId>
        <version>2.2</version>
    </dependency>
    <dependency>
        <groupId>com.googlecode.json-simple</groupId>
        <artifactId>json-simple</artifactId>
        <version>1.1.1</version>
    </dependency>
    <dependency>
        <groupId>io.cucumber</groupId>
        <artifactId>cucumber-java</artifactId>
        <version>7.14.0</version>
    </dependency>
    <dependency>
        <groupId>io.cucumber</groupId>
        <artifactId>cucumber-core</artifactId>
        <version>7.14.0</version>
    </dependency>
    <dependency>
        <groupId>io.cucumber</groupId>
        <artifactId>gherkin</artifactId>
        <version>27.0.0</version>
    </dependency>
    <dependency>
        <groupId>com.testvagrant.mdb</groupId>
        <artifactId>mdb</artifactId>
        <version>2.2.1</version>
    </dependency>
    <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>7.8.0</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>io.cucumber</groupId>
        <artifactId>cucumber-testng</artifactId>
        <version>7.14.0</version>
    </dependency>
    <dependency>
        <groupId>io.netty</groupId>
        <artifactId>netty-common</artifactId>
        <version>4.1.100.Final</version>
    </dependency>
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-remote-driver</artifactId>
        <version>4.13.0</version>
    </dependency>
</dependencies>

Aleksei · November 22, 2023, 1:10pm

@mykola-mokhnach
tested with latest java-client 9.0.0

            <artifactId>selenium-java</artifactId>
            <version>4.15.0</version> (Appium requires this ! version)

            <artifactId>java-client</artifactId>
            <version>9.0.0</version>

Two machines with iOS phones Java17(12 iPhones), with Android phones Java11(11 Nokia phones) installed on macMinis where tests starting.

Both give huge slowness and memory consumption screen below (from top iOS, bottom Android)

example of memory usage with reverted changed on iOS machine:

            <artifactId>selenium-java</artifactId>
            <version>4.13.0</version>

            <artifactId>java-client</artifactId>
            <version>8.5.0</version>

A bit info of dump

 Biggest Objects 

Class Name	Shallow Heap	Retained Heap

jdk.internal.loader.ClassLoaders$AppClassLoader @ 0x700363ca8 JNI Global 96	347,570,936
class org.aspectj.apache.bcel.util.ClassLoaderRepository @ 0x7006a8400 24	73,134,152
jdk.internal.loader.ClassLoaders$PlatformClassLoader @ 0x700358aa8 JNI Global 96	48,694,840

Biggest Top-Level Dominator Classes 

Label	Number of Objects	Used Heap Size	Retained Heap Size	Retained Heap, %
jdk.internal.loader.ClassLoaders$AppClassLoader	1	96	347,570,936	46.82%
java.lang.String	868,689	20,848,536	107,604,128	14.50%
java.lang.Class	7,395	178,864	86,629,840	11.67%
io.netty.buffer.PoolChunk	40	3,200	84,101,576	11.33%
jdk.internal.loader.ClassLoaders$PlatformClassLoader	1	96	48,694,840	6.56%
org.aspectj.weaver.tools.WeavingAdaptor$WeavingAdaptorMessageHolder	279	13,392	7,593,264	1.02%

Total: 6 entries
876,405	21,044,184	682,194,584

Rafael_Murer · January 10, 2024, 12:31pm

Hi I think is similar to this following issue right?

Aleksei · January 31, 2024, 12:48pm

java-client: 9.1.0 -> still reproducing

Rafael_Murer · March 14, 2024, 7:54am

Hi, is there a way we can follow up this issue? I feel like I seeing other issues that might resolve if I update the java client which I cannot due to this issue.

Aleksei · March 14, 2024, 8:00am

issue fixed → bug: AppiumFieldDecorator: Memory leak after moving from java-client 8.5.0 to 8.5.1 .... 9.1.0 · Issue #2119 · appium/java-client · GitHub
just need to wait next version 9.1.0+

Aleksei · March 20, 2024, 12:54pm

java-client 9.2.0
Two macMinis memory usage screenshot after 15 min run. Almost zero memory increase from start.

Each runs 12 phones: one iPhones, other Android phones.

@mykola-mokhnach thanks for fix!