How to handle test failure in appium ios/android

Support
1

Hi,

I am using appium 1.4.13 and node js client to write tests for android and ios.

I am looking for a way to make my tests independent of each other. Right now, if the test fails, on ios it goes to home screen and closes the simulator and all my subsequent tests are failing. I want to somehow handle the test failure so that even if one test fails, it does not affect the next test in the process.

I have thought of again launching the app if the test fails but its opening the simulator and the app from the scratch which is not helpful.

Anyone knows how to achieve this thing with appium?

Thanks a lot for your help. :slight_smile:

2

First, this is not an Appium problem, it is a testing framework problem. Your problem is that you are not using a testing framework. A testing framework will allow you to identify tests, give you enhanced logging/reporting, and above all give you a way to fail a test without stopping the test runs. I did a quick google search which you can reference here:

https://www.google.com/search?q=testing+framework+for+javascript&ie=utf-8&oe=utf-8

Looking through these I thought Jasmine looked pretty good. Here is the release before ‘bleeding edge’:

http://jasmine.github.io/2.3/introduction.html

I would encourage you to take a look at these and find one you are comfortable with.

CAVEAT: I do my testing in Ruby, and use Rspec as the testing framework. Jasmine seems a bit Rspec-like.

3

I’ve given this a lot of thought but personally haven’t found any good way to overcome the flakiness of the iOS tests. I tend to think that it’s due to relative youth of the platform and I hope this will only get better with time and with more smart developers.Right now, it’s pretty stressful, to say the least

4

Thanks a lot for your reply. Yes, I agree with you regarding use of test framework. I did not mention about the test framework which I am using. I am using mocha framework for running appium tests and I dont know how to handle test failure. I can identify tests , get logging/reporting but do not have an idea on how to fail a test without stopping test runs.

The workaround I am thinking is changing the way I am writing my tests, I am thinking to start every test from the same point in the app and then executing it from that point.

Thanks for your inputs.

5

Ok, fair enough. Take a look at the mocha assertions:

https://mochajs.org/#assertions

You need to use those in your tests if you want mocha to report a failure and continue on. I’m choosing ‘expect’ because I’m a bit familiar with that:

So my test might look like this:

expect(5).to.be.a(‘number’);
expect(‘i’).to.be.a(‘number’);
expect(6).to.be.a(‘number’);

So the above should give 2 passes and one failure. The last pass should be run in spite of the failure above it.

If you translate that to an Appium test it might look like this:

current_screen=LoginScreen.new;
expect(current_screen).to.be.a(LoginScreen);
expect(current_screen).to.be.a(SettingsScreen);
expect(current_screen).to.be.a(LoginScreen);

Once again we have 2 passes and a failure. That last test should still run given the failure on test number 2. Does this make sense?

6

Yes totally. I got the idea of what I need to do and how to write tests using mocha and expect js.

I will implement this in my code now. Thanks. :smile: