Decoding artifacts on Sony using ExoPlayer

[kareljuricka]

PROBLEM DESCRIPTION

There appears to show artifacts on video played by ExoPlayer during some playback.

Since update v6.7140, artifacts were shown for H.264 and H.265 codecs.

After update, playback for H.264 is fixed but problem still occurs on H.265.

PREVIEW

H.264: https://www.youtube.com/watch?v=lYwh2mDMW3w - no artifacts at all, thats OK from update to v6.7140

H.265: https://www.youtube.com/watch?v=V28J-KIC6ys - see artifacts appearing, BUG persists

DEMO FILES
https://drive.google.com/drive/folders/1SkGPRf5ObVtV8_pOFuT6pUhYOtleFOoV?usp=sharing

GITHUB ISSUE

https://github.com/google/ExoPlayer/issues/7009

FINAL NOTES

Problems is only if using exo player from google. Using VLC or MX player for same data is OK for both codecs.

Problem doesn't exist on other android tv devices, like Nvidia Shield or Mi box.

[Kuschelmonschter]

Is there a recent ExoPlayer installer (APK) available somewhere?

[kareljuricka]

I build exo player demo apk with our content files. You can download it here:

https://drive.google.com/file/d/16z86sIhwWnt65l2NKEladHSKm64xLyPm/view?usp=sharing

In app there are two demo examples:

TS h264 - works WRONG before v6.7140 update, CORRECT after v6.7140 update

TS h265 - works WRONG before and also after v6.7140 update

[Kuschelmonschter]

I would add this to the initial posting. The easier it is to reproduce, the better. I will try the sample with Plex which also uses ExoPlayer2. Sony has fixed Plex issues in the past...

[kareljuricka]

We tested our ts files on PLEX and I can confirm that problem occurs in the same way:

After patch v6.7140, problem persist on h265, but h264 is fixed.

[Kuschelmonschter]

Also Kodi seems to be stumbling over something in the sample. There is some video stuttering and audio dropouts, no artifacting though. This is what the log reveals:

2020-12-09 19:25:56.972 T:25118 WARNING <general>: ActiveAE - large audio sync error: 1150.575362

When I look at video and audio PTS, the offset between the two is non-existent. Audio is typically a few hundred ms ahead of video as video takes longer to process. So this might lead to timing issues.

[Kuschelmonschter]

So I do believe that it depends on the performance of the video pipeline within a given app/playback framework for a given video format. With respect to the discrepancy between H.264 and H.265, the decoding of H.264 might just be faster than H.265 and might have improved with Pie firmware 6.7140 just enough. It might help to schedule presentation more into future which might result in more frames being buffered. Is there something like a latency value one can set for ExoPlayer? Or probably enable async MediaCodec or Tunneled playback to gain some video decoding performance.

I am quite confident though that this is not a problem of the TV itself. The timings of the TS might lead to problems which devices with better performance (also depending on the format) might be able to handle or not.

[kareljuricka]

We've inspected possibility of performance issues and we've tried both of your tips (MediaCodec async mode and Tunneled playback).
Unfortunately it didn't help. We find out that VPFO value during problematic playback is in sane range for smooth playback - above 44000. We see similar values on high-end/high-performance devices so we think it might not be performance problem after all.
You can see all debug values in delivered demo youtube videos in the left top corner (links are in issue description). There are no dropped frames, only rb values are increasing (https://github.com/google/ExoPlayer/blob/8a0d52be4b5402af0987fc7d009bc49977c72b67/library/ui/src/mai...) whis looks like processed frames, increasing at rate 25 per second (which corresponds to 25 FPS of the video)