Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
How does hardware assisted video decoding work?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Multimedia
View previous topic :: View next topic  
Author Message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2177
Location: $TERM

PostPosted: Sat Sep 29, 2012 3:44 pm    Post subject: How does hardware assisted video decoding work? Reply with quote

If you're doing hardware assisted video decoding, the GPU should assist decoding the video; but the GPU is limited to decoding a few set of videos formats, commonly mpeg4, h264 and mpeg2.

There're various libraries available to make GPU capabilities available as an easy API which video players (and libraries) can implement. e.gs include vaapi, vdapu, xvmc, and libxv

In most players (mplayer, VLC, xine, Gstreamer), you've to specify the 'output driver' which specifies the way in which the video will be decoded; some of these output drivers are implemented using the libraries that are hardware accelerated, e.g. the xv driver of VLC, mplayer and gstreamer, the vdapu driver of mplayer etc...

Is all this true? Cause I've not read any of this anywhere; this's just what I think, and I also thing it's wrong cause --

If I use a hardware accelerated (e.g. vaapi) driver for a video which the hardware can't decode, what will happen in mplayer atleast?

What's gl and gl2 output in mplayer? Hardware decode the video using openGL GPU capabilities? On my hardware, this doesn't appear to reduce any CPU usage compared to XV, instead it increases by 1 or 2%.

In VLC (which supports VAAPI and should use hardware to decode h264 on my i3 with HD2000), I cant figure it out if VLC is using the hardware or not. In codecs section, if I enable hardware h264 decoding, then the CPU usage increases instead, and there's no vaapi option in the video output drivers list.

Another question about VLC and mplayer is that, if it's the video output driver's responsibility to do hardware decoding, then what are those 'enable hardware acceleration' buttons doing?
_________________
Buy from companies supporting opensource -- IBM, Dell, HP, Hitachi, Google etc...
Disfavor companies supporting only Win -- Logitech, Epson, Adobe, Autodesk, Pioneer, Kingston, WD, Yahoo, MSI, XFX
My blog
Back to top
View user's profile Send private message
Dorsai!
Apprentice
Apprentice


Joined: 27 Jul 2008
Posts: 255
Location: Bavaria

PostPosted: Mon Oct 01, 2012 1:45 pm    Post subject: Reply with quote

On Mplayer, most of the video output options just decide the way the video is drawn on the screen. xv should also be accelerated if you have a gl compatible driver active, but is the more compatible solution as gl works only if a capable driver is loaded. But the actual decoding is fully done by the CPU.

vdpau instead does something else. the full video stream, together with the place the final video should end up on the screen is handed to the graphics card which does all the decoding and displaying by itself.

vaapi is just a wrapper around different concepts, as vdpau and its AMD equivalent share different apis.

VLC uses a different approach by only letting the card decode the video and getting back the raw video stream for further processing. It does not work directly with vdpau but uses some sort of shader language based decoding if I am not mistaken (but I might be, I dont use VLC myself).
Back to top
View user's profile Send private message
Gusar
Advocate
Advocate


Joined: 09 Apr 2005
Posts: 2643
Location: Slovenia

PostPosted: Mon Oct 01, 2012 2:04 pm    Post subject: Reply with quote

Dorsai! wrote:
vaapi is just a wrapper around different concepts

vaapi is not a wrapper, it's a full API just like VDPAU and XvBA are.

@dE_logics: What VLC does is it sends the bitstream to the graphic card to be decoded and then copies decoded frames back into system memory. MPlayer doesn't do such copying, the graphic card displays the video directly. That's why VLC is less efficient.

Also, decoding is *not* done on the GPU, but on a dedicated ASIC that's part of the graphic card. This dedicated ASIC is single-purpose - it decodes specific formats (mpeg2, h264, VC1), and that's it. The reason this was done is that GPUs suck at video decoding.

The gl and gl2 in mplayer outputs are just that - outputs. They are means to display the picture on the screen. Their primary job is colorspace conversion (the video is YV12 while displays are RGB, so you need to convert the video in order to display it) and scaling (so that you can watch the video in fullscreen).

MPlayer and other players have several such means of displaying the picture. The most common one is Xv. It's an X extension that in the past provided access to a graphic card's hardware overlay. This overlay was a dedicated part of the graphic chip responsible for colorspace conversion and scaling. Modern graphic cards don't have such overlays anymore, the 3D engine is used to perform colorspace conversion and scaling.
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2177
Location: $TERM

PostPosted: Mon Oct 01, 2012 2:48 pm    Post subject: Reply with quote

Dorsai! wrote:
On Mplayer, most of the video output options just decide the way the video is drawn on the screen. xv should also be accelerated if you have a gl compatible driver active, but is the more compatible solution as gl works only if a capable driver is loaded. But the actual decoding is fully done by the CPU.

vdpau instead does something else. the full video stream, together with the place the final video should end up on the screen is handed to the graphics card which does all the decoding and displaying by itself.

vaapi is just a wrapper around different concepts, as vdpau and its AMD equivalent share different apis.

VLC uses a different approach by only letting the card decode the video and getting back the raw video stream for further processing. It does not work directly with vdpau but uses some sort of shader language based decoding if I am not mistaken (but I might be, I dont use VLC myself).


Ok, so, in mplayer, the driver actually changes the way the video gets decoded and displayed on the screen. So in vdapu, it may happen that the actual video after being decoded by the GPU gets displayed on the screen using Xv or gl/gl2; or maybe we can set options for the driver and customize it.
_________________
Buy from companies supporting opensource -- IBM, Dell, HP, Hitachi, Google etc...
Disfavor companies supporting only Win -- Logitech, Epson, Adobe, Autodesk, Pioneer, Kingston, WD, Yahoo, MSI, XFX
My blog
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2177
Location: $TERM

PostPosted: Mon Oct 01, 2012 2:52 pm    Post subject: Reply with quote

Gusar wrote:
Dorsai! wrote:
vaapi is just a wrapper around different concepts

vaapi is not a wrapper, it's a full API just like VDPAU and XvBA are.

@dE_logics: What VLC does is it sends the bitstream to the graphic card to be decoded and then copies decoded frames back into system memory. MPlayer doesn't do such copying, the graphic card displays the video directly. That's why VLC is less efficient.

Also, decoding is *not* done on the GPU, but on a dedicated ASIC that's part of the graphic card. This dedicated ASIC is single-purpose - it decodes specific formats (mpeg2, h264, VC1), and that's it. The reason this was done is that GPUs suck at video decoding.

The gl and gl2 in mplayer outputs are just that - outputs. They are means to display the picture on the screen. Their primary job is colorspace conversion (the video is YV12 while displays are RGB, so you need to convert the video in order to display it) and scaling (so that you can watch the video in fullscreen).

MPlayer and other players have several such means of displaying the picture. The most common one is Xv. It's an X extension that in the past provided access to a graphic card's hardware overlay. This overlay was a dedicated part of the graphic chip responsible for colorspace conversion and scaling. Modern graphic cards don't have such overlays anymore, the 3D engine is used to perform colorspace conversion and scaling.


But what happens if I change the output driver to vdapu? If the output driver handles only displaying the video on screen, then how can vdapu modify the decoding technique?
_________________
Buy from companies supporting opensource -- IBM, Dell, HP, Hitachi, Google etc...
Disfavor companies supporting only Win -- Logitech, Epson, Adobe, Autodesk, Pioneer, Kingston, WD, Yahoo, MSI, XFX
My blog
Back to top
View user's profile Send private message
pigeon768
l33t
l33t


Joined: 02 Jan 2006
Posts: 669

PostPosted: Mon Oct 01, 2012 3:23 pm    Post subject: Re: How does hardware assisted video decoding work? Reply with quote

There are two different things to talk about here. There is the output driver, and the codec. You need both to display a video, and both can have dramatic effects on performance.

xv, gl, gl2 are output drivers. Once a video is decoded, the output driver will take care of transporting the raw video frames (at this point it's basically a sequence of bitmaps) from your media player's process memory to your display. xv uses a system designed for playing video, gl and gl2 will use the more general 3d graphics system.

vdpau and vaapi will both decode the compressed video and perform the function of the output driver. Since when the video is decoded to a raw frame, the raw frame is already in the video card's memory, it wouldn't make sense to use the same driver to display it to the screen. They've also gone and made vdpau and vaapi available as generalized output drivers, so you can use that functionality even if the compressed video can't be decoded by the video card.

In mplayer, with vdpau, you configure mplayer to decode the video with vdpau with the command line argument `-vc ffh264vdpau,ffodivxvdpau,ffmpeg12vdpau,ffwmv3vdpau,ffvc1vdpau`. It will attempt to use one of those codecs, all of which utilize vdpau, in that order, and if none of them work it will fail. If you use `-vc ffh264vdpau,ffodivxvdpau,ffmpeg12vdpau,ffwmv3vdpau,ffvc1vdpau,` (note the extra comma) if none of those codecs work it will fall back to the default libavcodec codecs.

You configure mplayer to use the vdpau output driver with `-vo vdpau`. This will work regardless of whether or not mplayer falls back to the libavcodec codec.

I don't use vlc, and mainline mplayer does not yet support vaapi, so I can't help you with your other questions.
_________________
My political bias.
Back to top
View user's profile Send private message
Gusar
Advocate
Advocate


Joined: 09 Apr 2005
Posts: 2643
Location: Slovenia

PostPosted: Tue Oct 02, 2012 10:51 am    Post subject: Reply with quote

dE_logics wrote:
So in vdapu, it may happen that the actual video after being decoded by the GPU gets displayed on the screen using Xv or gl/gl2; or maybe we can set options for the driver and customize it.

That's possible in theory, but MPlayer doesn't support it. If you decode using VDPAU, you need to display using VDPAU. That's simply how MPlayer has been coded. Using other output methods would require copying decoded frames back into system memory, mplayer doesn't support that. But, like I said, VLC does.

dE_logics wrote:
But what happens if I change the output driver to vdapu? If the output driver handles only displaying the video on screen, then how can vdapu modify the decoding technique?

I don't get what you mean here. VDPAU, also VAAPI, have two parts: decoding and presentation (displaying the video). You can use VDPAU presentation regardless of how the video was decoded (you can decode the video in software, then use VDPAU to display it). But if you decode using VDPAU, then it depends how a particular player was coded.
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2177
Location: $TERM

PostPosted: Tue Oct 02, 2012 12:39 pm    Post subject: Reply with quote

Thanks everyone!

Things are 100% clear now and this post comes up on the first page of Google search result for "how hardware decoding works".
_________________
Buy from companies supporting opensource -- IBM, Dell, HP, Hitachi, Google etc...
Disfavor companies supporting only Win -- Logitech, Epson, Adobe, Autodesk, Pioneer, Kingston, WD, Yahoo, MSI, XFX
My blog
Back to top
View user's profile Send private message
pd1986
Guru
Guru


Joined: 19 Feb 2012
Posts: 303
Location: Paris

PostPosted: Wed Oct 03, 2012 7:36 pm    Post subject: Reply with quote

dE_logics wrote:
Thanks everyone!

Things are 100% clear now and this post comes up on the first page of Google search result for "how hardware decoding works".


good news, good for spreading the knowledge and GENTOO, haha :)

In fact, I am interested in this too. as I have ATI card, but I often use mplayer. In mplayer of official version, vaapi isn't included, while it is in vlc. but I really don't find a large difference of cpu use between mplayer and vlc when I play a movie of mkv in 1080p. 15%-20% by vlc vs 20%-25% by mplayer.

One time I heard somewhere, if I choose xv, I could still get the hardware acceleration. can't remember where it is from.
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2177
Location: $TERM

PostPosted: Thu Oct 04, 2012 3:46 am    Post subject: Reply with quote

pd1986 wrote:
dE_logics wrote:
Thanks everyone!

Things are 100% clear now and this post comes up on the first page of Google search result for "how hardware decoding works".


good news, good for spreading the knowledge and GENTOO, haha :)

In fact, I am interested in this too. as I have ATI card, but I often use mplayer. In mplayer of official version, vaapi isn't included, while it is in vlc. but I really don't find a large difference of cpu use between mplayer and vlc when I play a movie of mkv in 1080p. 15%-20% by vlc vs 20%-25% by mplayer.

One time I heard somewhere, if I choose xv, I could still get the hardware acceleration. can't remember where it is from.


Join the discussion --

http://forum.videolan.org/viewtopic.php?f=13&t=104718
_________________
Buy from companies supporting opensource -- IBM, Dell, HP, Hitachi, Google etc...
Disfavor companies supporting only Win -- Logitech, Epson, Adobe, Autodesk, Pioneer, Kingston, WD, Yahoo, MSI, XFX
My blog
Back to top
View user's profile Send private message
Gusar
Advocate
Advocate


Joined: 09 Apr 2005
Posts: 2643
Location: Slovenia

PostPosted: Thu Oct 04, 2012 11:57 am    Post subject: Reply with quote

pd1986 wrote:
One time I heard somewhere, if I choose xv, I could still get the hardware acceleration.

Only in VLC, which copies frames into system memory. Which takes CPU time, so maybe that's why you don't see much difference between hardware and software decoding.

MPlayer doesn't do any copying, it displays the frames directly, which makes it more efficient than VLC. That's why you need to use a specific video output. For VDPAU you also need to specify the appropriate hardware codec (one of ffh264vdpau/ffmpeg12vdpau/ffvc1vdpau/ffodivxvdpau depending on the file you want to play).

To sum up regarding MPlayer:
Hardware decoding with VDPAU
Code:
-vo vdpau -vc ffh264vdpau

Hardware decoding with VAAPI (requires a patched mplayer)
Code:
-vo vaapi -va vaapi
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2177
Location: $TERM

PostPosted: Fri Oct 05, 2012 6:19 pm    Post subject: Reply with quote

I tried VLC on 1st generation mobile i3 and the CPU usage doped to half.

Anyway, XBMC is currently the only easily available alternative that works. Also, those mplayer-vaapi ebuilds don't work.
_________________
Buy from companies supporting opensource -- IBM, Dell, HP, Hitachi, Google etc...
Disfavor companies supporting only Win -- Logitech, Epson, Adobe, Autodesk, Pioneer, Kingston, WD, Yahoo, MSI, XFX
My blog
Back to top
View user's profile Send private message
BrandonKByrne
n00b
n00b


Joined: 05 Oct 2012
Posts: 2

PostPosted: Sat Oct 06, 2012 6:50 am    Post subject: Reply with quote

hello everyone

In fact, I am interested in this too. as I have ATI card, but I often use mplayer. In mplayer of official version, vaapi isn't included, while it is in vlc. but I really don't find a large difference of cpu use between mplayer and vlc when I play a movie of mkv in 1080p. 15%-20% by vlc vs 20%-25% by mplayer. thanks for sharing such a nice question
Back to top
View user's profile Send private message
Gusar
Advocate
Advocate


Joined: 09 Apr 2005
Posts: 2643
Location: Slovenia

PostPosted: Sat Oct 06, 2012 11:16 am    Post subject: Reply with quote

@BrandonKByrne:

Which driver are you using? The open source radeon driver only decodes mpeg1/2 in hardware using the Gallium3D VDPAU implementation. And even that is not done using the dedicated hardware decoder, but with shaders.

The closed source Catalyst/fglrx driver uses XvBA to access the hardware decoder. Only a separate branch of XBMC has direct support for XvBA. Though there's also xvba-video, a wrapper that allows using hardware decoding in VAAPI players.
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2177
Location: $TERM

PostPosted: Sat Oct 06, 2012 12:46 pm    Post subject: Reply with quote

I tried VLC on 1st generation Intel i3, and the CPU usage was halved; but not in my or anyone else's case.
_________________
Buy from companies supporting opensource -- IBM, Dell, HP, Hitachi, Google etc...
Disfavor companies supporting only Win -- Logitech, Epson, Adobe, Autodesk, Pioneer, Kingston, WD, Yahoo, MSI, XFX
My blog
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Multimedia All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum