I am using the Cucumber (JAVA) framework for my automation testing and have carefully designed my scenarios so that after each scenario the user is logged out. That means each scenario starts with a clean slate with logging in, and is independent of any scenario. I did this mostly so that if one scenario fails…the tests will continue to run with the next scenario; so that every scenario does not fail.
However even with my current approach if one test fails the rest fail. For e.g. If User tries to compose a message and that compose button is corrupted and for some reason the test fails…it moves to the next test but since the next test starts with logging into the app…that next test also fails since the current screen is the compose button screen, not the log in screen.
How can I, if it is even possible, make sure that if one test fails the rest do not fail ? Please advise.
I don’t have experience with the Cucumber framework, but you could call the “resetApp” method of AppiumDriver. For Android tests, I believe this method clears the application directory for the app under test. You could also do this manually by spawning a process that runs “adb shell pm clear application_ID” (see https://developer.android.com/tools/help/shell.html#pm ). This should make the app appear as if it was just freshy installed to each of your test cases, provided you have some sort of cleanup method between test case to do the reset process.
Thanks for your response. Nothing seems to be working for me. I cannot run a logout scenario as part of my @After because the logout is done through a particular screen…and if the tests are failing at some other screen…it will never be able to go to the screen where the user needs to log out so the @After would never be doable. So I have to have an @After hook that will do a driver.quit or driver.resetApp.
Problem is that if I do @After driver.quit and then a @Before for a new driver I get an error saying “remote browser may have died” or something to that effect. If I use @After driver.closeApp and @Before driver.launchApp() that does not work either as it says “another instruments session is around”. If I do a @After of driver.closeApp() and driver.launchApp()…with no @Before hook then I get an error saying “after hook failure”. If I do a @After of driver.quit()…with no @Before hook then I get an error saying “Session ID is null. Using WebDriver after calling quit()?”.
I have no clue how to handle this. FYI. I have a class which contains a method containing the driver: protected IOSDriver getDriver(). All the other classes then just declare ‘IOSDriver driver = getDriver();’.
Hey there:
I keep thinking about your original problem and ask the question of why your compose button is corrupted in the first place.
If it’s due to the app under test, it’s truly a blocking bug that needs to be fixed - end of story.
If it’s environment specific, then I would recommend working with your ops team to truly insure a deterministic testing environment.
If either of those are not feasible I have some thoughts:
Your corrupt button test case is tagged as @wip (work in progress) and is skipped
You can create what I call an “alternate_find” method - where it’s input is an array of possible element locators to look for. If it finds one, it returns and acts on it. The goal here is to prepend this to your logout after hook insuring successful logout.
Driver.quit and restart (or launchApp I forget) can work - but it’s quite slow. I’ve found over time the underlying instrumentation doesn’t like to be restarted so frequently.
@shermaneric
the Compose button is not really corrupted but this is a useful case for me to work on. If any of y tests fail…the remaining ones fail as well…since the first line of the next scenario is about logging in to the app…and I cannot be logging in unless am already logged out.
Hi Styris: thanks for the response! I clearly understand your point. I know that ruby cucumber has an after if scenario.failed hook. I’m sure Java cucumber must have something similar.
I haven’t seen your app, but I’ll take a wild guess that the screen that has the compose button also has a cancel button, and tapping cancel can lead to a series of tap(s) that can get you to log out.
As I mentioned earlier, my cohorts came up with an alternate_find method where you would pass in a series of possible elements to find…the first one it finds returns that element in which you could perform further actions (like tapping cancel and logging out).
(You could make this even more elegant and do this at a screen level using asserts and create the proper page object methods to call…)
This alternate find could be used in the scenario.failed logic and achieve your goal of logging out after a failed scenario without the expensive operations of quitting and restarting the driver (not to mention uninstalling and reinstalling your app if on a real iOS device)
Alas I respect your decision if you wish to continue the driver.quit/restart approach. It seems like there is success with this and ultimately you need to be comfortable with your test harness.
I wish you all the best of luck, please let me know if you’d like me to help further.
Thanks!
@shermaneric Thanks. I like your idea of having various different tapping points (if you will)…but here is the deal. The compose button was just 1 example…there could some other similar screens in the app where if it fails…there is no way to log out. So if I go that route is that not going to become too huge ? Also…if I were to use something like a scenario.failed logic…again the app should be at a screen where it can get to the logout screen.
Either way it is a great suggestion…but for now if I could somehow get my driver.reset or driver.quit (btw I am on a real device and both these methods do not unistall/reinstall…they just close the app and relaunch it) to work that would be ideal for me. Maybe its something with how am declaring my driver etc. I’m just not sure and its something that I need to figure out fast.
Thanks for the reply.
Since you’re on a time crunch, I’ll give you a quick suggestion. Again, this is in Cucumber-Ruby but the logic I imagine is similar. You may have tried it. In that case, I would be all out of suggestions:
After do |s|
time_for_driver_ready=7
if s.failed?
#If you are on an iOS Device
$driver.quit
#Reinstall iOS app using ideviceinstaller.
start_driver #this is a ruby method. not sure if there is one in Java.
sleep(time_for_driver_ready)
#else
reset
end
end
I’m not a fan at all of waits for the sake of waiting, but increasing the time would get you most likely out of the pickle you are in.
Alternatively you could explicitly wait while combing the logs for this if on iOS device : info: Instruments is ready to receive commands
As for the longer term solution of logging out gracefully, I don’t think it would be too bad. If you have a page object model (and additionally the page factory model - which I don’t have yet but I’d like to), you could quickfly find out what screen you were on, call the method to get back to your main screen and the log out – all in the after hook.
@shermaneric thanks a lot for your suggestion. Actually right before I read this post of yours I had tried something like driver.resetApp(); and then Thread.sleep(5000) in my @After hook. So now it passes. However there is a catch.
The app closes and relaunches about 5-8 times before the first scenario and then again 5-8 after the first scenario.
So I am trying various things and if I get close to what I need I will definitely let you know as well!
1 step better. The problem was I had the @After in 5 classes so it was picking each one up. Now it does it only once. I just need to put the Thread.sleep in a better place now…instead of just for that one method referencing the Compose page object.
