Hardware-accelerated video playback on Odroid U2/U3 - part 2

Near the end of my previous post, I said that while the video playback was decent, it was not as smooth as expected, and that "memeka" from the odroid forum suggested to use cluttersink instead of glimagesink.

I have now built and tested cluttersink, and indeed cluttersink shows smoother playback than glimagesink. There is one small difference: cluttersink doesn't respect aspect ratio, it will just scale up everything to its window size; while glimagesink does honour aspect ratio (but you can disable it if you don't need it).

Here is an example pipeline that plays an mp4 file, both audio and video, with full video acceleration using cluttersink:
gst-launch-1.0 filesrc location=/path/to/media.mp4 ! qtdemux name=m m. ! queue ! h264parse ! video/x-h264, stream-format=byte-stream, alignment=au ! v4l2video8dec ! v4l2video10convert ! cluttersink m. ! queue ! faad ! audioconvert  ! alsasink

The same pipeline, simplified using decodebin:
gst-launch-1.0 filesrc location=/path/to/media.mp4 ! decodebin name=m m. ! queue ! v4l2video10convert ! cluttersink m. ! queue ! audioconvert ! autoaudiosink

cluttersink is available in "gst2-clutter" package in the repository.

I also re-uploaded Odroid U2/U3 kernel package for FatdogArm Beta2, now with MFC firmware (which is needed for hardware acceleration). Thanks to forum member "mories" who informed me that I forgot to do that. Beta2 sfs is also re-uploaded, now with updated glib (2.38.2 now, it was 2.36.3 previously) which is needed for cluttersink and its libraries, among other small fixes.

Posted on 4 Oct 2014, 16:48 - Categories: FatdogArm Linux Arm
No comments - Edit - Delete

Hardware-accelerated video playback on Odroid U2/U3

Thanks to the effort of many people, I have uploaded gstreamer 1.x libraries (1.4.3) (gstreamer2 and gst2-plugins-* in the repository) which provide support for video playback acceleration on Odroid U2/U3 on FatdogArm.

The accelerated pipeline in Odroid U2/U3 goes like this:
- hardware decoding (Samsung S5P MFC codec) - use v4l2video8dec
- hardware colourspace conversion (Samsung FIMC) - use v4l2video10convert
- GPU-assisted rendering - use glimagesink

There were a few hassles to finally get to this.

The hardware decoding was quite a breeze. The kernel has supported the MFC codec for a while. The userspace support was done by a gstreamer plugin called mfcdec, but this was finally deprecated and replaced with v4l2-based decoder in the most recent gstreamer version.

Hardware colourspace conversion didn't work due to a bug in libv4l2, as found out by forum member "memeka" in this post. The library basically assumes that all v4l2 devices can use DMABUF interface to v4l2 devices, but Samsung FIMC kernel drivers don't support that, only normal mmap-ed access. Disabling libv4l2 forces gstreamer to use its own internal code which works with either. So, following his advice, FatdogArm gstreamer libraries are built without libv4l2 dependency. However, the default colourspace converter for gst-play and playbin is unchanged and is still the software decoder (videoconvert) - because forcing the default to Samsung converter results in colour artifacts. If you need to use hardware colourspace converter, you will need to build the pipeline yourself (and if you do this yourself, there is no artifact - the artifact is only seen when using gst-playback/playbin with hardware converter, which is odd).

GPU-assisted rendering is done by glimagesink. This component replaces eglglessink in earlier gstreamer version. Due to a long-standing bug in Mali driver (documented here, found in r3p2 and still exist in r4p0), it doesn't always work (well - it *mostly* doesn't work on my tests). To make it work, I patched the binary driver (it's also in the repo as libmail-odroid).

The result of the acceleration is decent, but it is not as smooth as I hope. "memeka" in another forum post said that the best sink is cluttersink (better than glimagesink). We'll see about that later.

To test that it is all working, try this pipeline (it is one long line):
gst-launch-1.0 filesrc location=/path/to/media.mp4 typefind=true ! qtdemux name=m m.video_0 ! h264parse ! video/x-h264, stream-format=byte-stream, alignment=au ! v4l2video8dec ! v4l2video10convert ! glimagesink

This pipeline will only play the video and there is no buffering, but you get the idea.

Posted on 3 Oct 2014, 6:41 - Categories: FatdogArm Linux Arm
No comments - Edit - Delete

FatdogArm Beta2 release

FatdogArm Beta2 is a minor release, mainly:
- bugfixes and upstream update from Fatdog64 700
- updated kernel package for Odroid U2/U3 with r4p0 Mali driver
- introducing support for cubox-i (Freescale i.MX6DL/i.MX6Q SoC)

Release note here.
Get it from here or any of ibiblio mirrors.

Posted on 1 Oct 2014, 19:04 - Categories: FatdogArm Linux Arm
No comments - Edit - Delete

New article - Linux multi-seat

It has been a while since I wrote an article, so here is a new one: Demystifying Linux multi-seat.

Posted on 1 Oct 2014, 15:29 - Categories: Linux General
No comments - Edit - Delete

1080p video playback on cubox-i

I have been experimenting with hardware acceleration for video playback on i.MX6 (cubox-i), that came with Freescale BSP. The most recent released for Freescale BSP is version 3.10.17, for their kernel GA 3.10.17.

Hardware acceleration of anything is a fickle thing, because there are non-standard ways of doing things (one may disagree and says that there are too many standards to choose from: vdpau, vaapi, openmax, gstreamer, and others). In the end, different hardware vendors supports their hardware with different libraries.

Freescale chooses to use gstreamer to support video playback acceleration. It comes are a set of open-source gstreamer plugins, which links to closed-source VPU* and IPU* libraries, for gstreamer 0.10. Yeah that's the old version but it still works. Freescale has a newer (semi-official?) "community"(?) gstreamer 1.0 libraries but the support is not identical with 0.10 libraries.

[*Note: VPU is Freescale's hardware video processing unit, and IPU is Freescale's hardware Image Processing Unit]

The 0.10 libraries are purely VPU/IPU based and does not depend on a working GPU. It has a few plugins: a demux plugin ("aiurdemux") that recognises certain common formats like AVI, MKV, WEBM, MP4 and others, a VPU-assisted video decoder ("vpudec") and a video renderer (sink) that is IPU-assisted (mfw_v4lsink), among others.

If this sounds like alien language, let me translate it for you: "demux" is the one that splits a media file into its video and audio component (so they can be routed and processed by different components). "video decoder" is the one that decompress the video file and outputs raw video data, and "video renderer" is the component that takes that raw video data and display it on screen (possibly converting the formats in between).

Together they form a hardware accelerated video playback pipeline that can play 1080p video effortlessly with less than 10% CPU usage. As a comparison, even the quad-core i.MX6Q cannot play 1080p video smoothly using only its ARM cores alone.

There are just a few hiccups. The plugins apparently can't rotate the playback when you launch it using gst-launch (e.g see here - although if you use the gplay tool it would work), and if you try to use the plugin to play video on a rotated display (rotated using xrandr etc), then video playback gets totally wrong. This is mainly because certain parts of plugin follows X coordinate orientation, while other parts of it are stuck with real CRTC coordinates (that is, they aren't rotated).

I've done some fixes on that and the gstreamer plugins can now play video in any orientation (even from gst-launch) as well as in any display orientation. As a bonus, I have the code that makes video playback rotated following the display rotation automatically. Of course, because the final rendering is done by IPU, this is only possible if the size of the video output is not more than 1024x1024 pixels - this is an IPU limitation which isn't easily bypassed.

Combined with this: gstreamer npapi plugin, one can make an alternative video player plugin that provides accelerated video playback (compared to xine-plugin or vlc-plugin, for example); and when built with the "fixed" version, the video will play correctly on any display orientation. I know that, I've built one myself. It runs nicely on FatdogArm :)

Posted on 27 Sep 2014, 4:51 - Categories: FatdogArm Linux Arm
1 Comment - Edit - Delete

Fatdog64 700 beta1 is released.

This release is mostly for bug fixes and some package updates.

Forum announcement: http://murga-linux.com/puppy/viewtopic.php?p=799326.
Release notes: http://distro.ibiblio.org/fatdog/web/700b1.html.

Get it as usual from ibiblio or one of its mirrors: aarnet, uoc.gr, and nluug.nl.

Posted on 15 Sep 2014, 21:12 - Categories: Fatdog64 Linux
No comments - Edit - Delete

Xscreenshot updated

Add "delayed" snapshot function to Xscreenshot accessed by Shift-hotkey). This is useful when you want to capture open menus etc. The usual way of pressing hotkey to capture won't do because as soon as press a key, the menu is closed.

"Delayed" snapshot avoid this by you pressing the hotkey first (to trigger the capture), and it giving you the chance to open the menu, and after a while it will do the capture behind the program's back.

Get it from here.

Posted on 12 Sep 2014, 20:31 - Categories: Linux General
No comments - Edit - Delete

New URL for this blog

This blog and its corresponding wiki is now accessible from http://www.lightofdawn.org/. So it's time to update your bookmark or your RSS reader . The original URL will remain for a while, but will eventually be removed.

Posted on 25 Aug 2014, 0:01 - Categories: General
No comments - Edit - Delete

aufs - please support Mr. Okajima

Aufs filesystem is a full-featured, full-fledged layered filesystem for Linux, created by Mr. Junjiro Okajima. It is an alternative to "unionfs", Unlike unionfs, however, aufs is actively developed, and actively supported. Unlike other alternative Linux layered filesystems like "overlayfs", aufs is feature rich and has a lot of options. Aufs is mature and stable software, it is about 10 years old by now.

I don't know how extensive aufs is used in the field, but I have a suspicion that many "live" OS systems (with support for persistence) will use aufs in form or another. I also think that aufs would be widely deployed in the embedded world, where one needs to have "writable" rootfs while still having the base OS in ROM. What I *do* know is that aufs is used in Fatdog64, FatdogArm, Puppy Linux and its multiplicity of derivatives, Slax, AntiX, and perhaps many others.

I recently had the pleasure of working directly Mr. Okajima.

I had problems with kernel oops with FatdogArm on cubox-i. Researching the problems, I found that kernel oops was a known common problem in the official cubox-i kernel and there were some solutions - but they didn't seem to work for me. My oops symptoms were quite different, too - the kernel didn't immediately crashed; but all filesystem operation failed, which lead me to suspect that the problem may have had to do with aufs (the other popularly Linux distros on cubox-i - Arch, Ubuntu - they all don't use aufs as far as I'm aware).

I escalated the problem to Mr. Okajima through the public mailing list and to my pleasant surprise, found that he was all courtesy, and very responsive. Mr. Okajima is very knowledgeable person - not only he helped me to troubleshoot the problem, but he actually solved it. This is despite the fact this was his first time looking at problems in ARM systems (and the problem required looking at ARM code dissasembly to find the cause); and the fact that he didn't have any ARM boxes whatsoever to reproduce the problem. One rarely encounters this kind of support and talent - even in the commercial world.

I'd encourage those who beneftis from aufs - especially commercially - to support the work of Mr. Okajima in whatever means that they can. Donation link is here (this goes straight to Mr. Okajima - not to me).

Posted on 17 Aug 2014, 5:27 - Categories: Linux General
No comments - Edit - Delete

Skype - please re-enable ALSA support!

I use Skype. Not because I'm a fan of it (after all, I wrote PSIP, but by necessity.

Recently I've got an email from Skype that they are terminating access from all Skype clients prior to 4.3 - so if I haven't upgraded to the latest and greatest, I'd better do it, or else no connection.

Which is generally fine and dandy except that Skype 4.3 drops support for ALSA. ALSA is the *native* audio system in Linux. Skype does, however, chooses to support PulseAudio instead. I don't have a problem with them supporting PulseAudio, but why the need to drop support for ALSA, which has been working for years and years?

I don't want to discuss the merits (of the lack thereof) of PulseAudio here - I'd just point out that not all of us Linux users run (or can run, or want to run) PulseAudio.

Since the ALSA back-end used to work beautifully (ALSA was supported in version 4.2 - one version before 4.3), and since PulseAudio usage is clearly optional (one can run Skype even without PulseAudio installed - but you don't get voice services (call/talk) - you can still send text and files) - I don't think it's that difficult to get ALSA support back in the next updates.

Skype - if you're listening, please, enable back ALSA support in your next update.

Posted on 17 Aug 2014, 4:51 - Categories: General
2 Comments - Edit - Delete

Pages: ... [11] [12] [13] [14] [15] [16] ...