More detailed reports of failed tests

Support
1

Hello,
I have managed to configure appium tests running on CI(TeamCity).
Everything is great and working. One thing that I have not managed to do is to create some reports of failed tests and information which tells why they failed.

I am using gradle so default junit report is generated with time of execution and logs which leads me to line of failed test. But the most important information for mobile developer or QA is what really happend. Test had issues with finding element or maybe it was real bug in application.

As for me the most important stuff will be:

  • Appium server log (not whole but regarding execution of failed test)
  • Android/IOS application log (filtered by level or package) from device
  • Maybe some screenshots or video with process of running test.

Is there any way to enable more detailed reports of failure only on client side?

I am not asking about everything, but maybe there is way(I am pretty sure, there is) to make reports richer.

I am really sorry if there was such discussion earlier.

Will appreciate any comment.

2

Are you making use of junit RunListener ?

3

No, I do not use RunListener. Does it have such functionality. Simple googling just show its official JUNIT listener.

4

it is all depends who is reading reports and logs.
i made in following way:

  1. when test failed we have some exception. And mostly it is like assertTrue(“failed to open Login screen”, somePage.isThisPageLoaded())
  2. when test failed we do screenshot of device state
  3. we have retry logic and when test failed retry will start. if retry succeded fail will not be reported in report.
  4. when all completed i create Excel report with “org.apache.poi”

All retries, screenshots and logs are for automation QA.

5

Both developers and QA will read those reports.
I understand that you are calling screenshot on each failed test.
Can you please describe information which you export to excel?

6

@bazted example of beginning - https://screencast.com/t/0R8vd7pVioH2

7

@bazted , We too use kind of similar approach as @Aleksei does . Difference is we are using listener to monitor test execution and take screenshots/logs on failure and keep track of failed tests for future analysis and run .

Once the runlistener is added to your code , you can define the logic “what to do?” on failure , success and exception. .

you may take a look at below link for a start .

8

@harigovind @Aleksei Thank you very much. I will take closer look to RunListener and will try to implement it.