Request: method to record the original stream

Feature
1

The problem

Input frame rate changes over time (i.e. when testing a resource heavy app like a mobile game). Specifying a frame rate for recordings that are higher than the input frame rate leads to dropped frames (the recording output video is cut short).

Details

If I start a recording with 25 fps and the input drops to 20 fps for periods of time (which happens often when testing resource heavy apps) then 1-2 mins of the video output will get cut. The workaround is to use 10 fps and then afterwards to convert the video in order to change the speed again using ffmpeg setpts argument. So I am forced to use 10 fps resulting in low quality videos.

Would be nice if the output video matches the fps of the input dynamically if possible, so I don’t have to do any manual conversion and so I am not forced to have low quality videos.

Example log where the input framerate dropped to average of 20 fps 2022-01-13 21:23:50:350 [ffmpeg] encoded 21241 frames in 1043.45s (20.36 fps), 30.76 kb/s, Avg QP:34.32: https://gist.github.com/sSpockderpantss/309ca920e68a500cb972cbd2b4f1fbef

specified frame rate was 25 fps 2022-01-13 21:06:26:120 [W3C (4639e98a)] Calling AppiumDriver.startRecordingScreen() with args: [{"forceRestart":true,"timeLimit":1800,"videoType":"libx265","videoFps":25,"videoScale":"320:-2","pixelFormat":"yuv420p"},"4639e98a-d3e6-4ee0-b704-98dfbebedffe"]

leading to a video being cut short by 1-2 mins

Request

A method that allows me to record the original stream, so that I can do the post-encoding myself. Please make this available for the Appium-Python-Client.

2

AFAIK this is a common issue when recording from a live MJPEG stream, which does not have a constant framerate. The only possible way to solve that would be to record the original stream and than do the post-encoding based on the input stream parameters, which are not known in runtime.

3

Is this possible using appium? To record the original stream? If so I would like to do so, I don’t mind doing the post-encoding myself. If not, then I request this as a feature.

4

xcuitest driver does record the resolting video as mjpeg if you don’t explicitly set a different encoder: https://github.com/appium/appium-xcuitest-driver/blob/6614d50cd8208b17af8ceb6e9668aa07f2e57974/lib/commands/recordscreen.js#L220

5

I just tested not explicitly setting a different encoder, the fps is still not matching the source fps, resulting in cut videos, I think this is because the fps is hard coded to be 10fps? https://github.com/appium/appium-xcuitest-driver/blob/6614d50cd8208b17af8ceb6e9668aa07f2e57974/lib/commands/recordscreen.js#L223

I cannot fix cut videos with post-encoding.

6

Seems like I will be forced to use 4fps videos if I want to use Appium for game development (since the server log shows the average frame rate to be about 4.x fps frequently, if I use any fps higher than 4 I end up with videos that are cut that I can do nothing about from my end), its a bit weird that this cannot be addressed. I will probably be exploring other frameworks because of this, like IDB recording or maybe even using third party apps.

Would love to use Appium though, the other features are quite nice, but recordings are quite crucial in order to identify bugs and such. Thanks for the help either way.

7

You could probably try https://github.com/danielpaulus/quicktime_video_hack for more seamless video stream. Unfortunately XCTest itself does not provide any better possibilities for screen recording

1 Like
8

Anyway i like iOS video better for smaller size. While Android I do post process with variable bitrate using ffmpeg to have nice 3min video with size of few hundred kilobytes.

9

Any update for iOS device is less than iOS 17 ( that I know we could use appium-xcuitest-driver/docs/reference/execute-methods.md at master · appium/appium-xcuitest-driver · GitHub)?

Is it possible that we could use Variable Frame Rate to make the video speed is normal?