1080p video playback on cubox-i
I have been experimenting with hardware acceleration for video playback on i.MX6 (cubox-i), that came with Freescale BSP. The most recent released for Freescale BSP is version 3.10.17, for their kernel GA 3.10.17.Hardware acceleration of anything is a fickle thing, because there are non-standard ways of doing things (one may disagree and says that there are too many standards to choose from: vdpau, vaapi, openmax, gstreamer, and others). In the end, different hardware vendors supports their hardware with different libraries.
Freescale chooses to use gstreamer to support video playback acceleration. It comes are a set of open-source gstreamer plugins, which links to closed-source VPU* and IPU* libraries, for gstreamer 0.10. Yeah that's the old version but it still works. Freescale has a newer (semi-official?) "community"(?) gstreamer 1.0 libraries but the support is not identical with 0.10 libraries.
[*Note: VPU is Freescale's hardware video processing unit, and IPU is Freescale's hardware Image Processing Unit]
The 0.10 libraries are purely VPU/IPU based and does not depend on a working GPU. It has a few plugins: a demux plugin ("aiurdemux") that recognises certain common formats like AVI, MKV, WEBM, MP4 and others, a VPU-assisted video decoder ("vpudec") and a video renderer (sink) that is IPU-assisted (mfw_v4lsink), among others.
If this sounds like alien language, let me translate it for you: "demux" is the one that splits a media file into its video and audio component (so they can be routed and processed by different components). "video decoder" is the one that decompress the video file and outputs raw video data, and "video renderer" is the component that takes that raw video data and display it on screen (possibly converting the formats in between).
Together they form a hardware accelerated video playback pipeline that can play 1080p video effortlessly with less than 10% CPU usage. As a comparison, even the quad-core i.MX6Q cannot play 1080p video smoothly using only its ARM cores alone.
There are just a few hiccups. The plugins apparently can't rotate the playback when you launch it using gst-launch (e.g see here - although if you use the gplay tool it would work), and if you try to use the plugin to play video on a rotated display (rotated using xrandr etc), then video playback gets totally wrong. This is mainly because certain parts of plugin follows X coordinate orientation, while other parts of it are stuck with real CRTC coordinates (that is, they aren't rotated).
I've done some fixes on that and the gstreamer plugins can now play video in any orientation (even from gst-launch) as well as in any display orientation. As a bonus, I have the code that makes video playback rotated following the display rotation automatically. Of course, because the final rendering is done by IPU, this is only possible if the size of the video output is not more than 1024x1024 pixels - this is an IPU limitation which isn't easily bypassed.
Combined with this: gstreamer npapi plugin, one can make an alternative video player plugin that provides accelerated video playback (compared to xine-plugin or vlc-plugin, for example); and when built with the "fixed" version, the video will play correctly on any display orientation. I know that, I've built one myself. It runs nicely on FatdogArm :)
Edit - Delete
Comments:
Posted on 27 Aug 2015, 16:07 by Haihong"big fan of your Fatdog works"
Can I contact you? I some imx6 based project work on video rotated/portrait, but so far stuck very badly that I need some help with...
my email: luohaihong@gmail.com
Delete
