Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
the libav ffmpeg clusterf*ck
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Multimedia
View previous topic :: View next topic  
Author Message
darkphader
Veteran
Veteran


Joined: 09 May 2002
Posts: 1217
Location: Motown

PostPosted: Thu Jul 31, 2014 2:28 am    Post subject: the libav ffmpeg clusterf*ck Reply with quote

Jeez has this been going on for a time... some things really want libav and others really want ffmpeg.

Sometime ago was running libav and then I wanted XBMC so I switched to ffmpeg, and then recently some things didn't like that (dragon for one). So WTF, I went back to libav, but then, of course, XBMC didn't like that.

If only I could use libav for qall of the other proggies and still get XBMC to work... but hey XBMC ships with it's own ffmpeg libraries and in fact they recommend you use them, but for some reason the Gentoo gods do not provide the opportunity to do so.

So here's a diff to the xbmc-13.1.ebuild that worked for me (libav installed). Don't tell me about vdpau (if you use it make sure libvdpau is installed and a modern nvidia driver) as this was a quick test patch. But before I go any further it would be nice if some others got their feet wet and reported back as I don't use many of XBMC's features. It doesn't try to bring in ffmpeg, it did build and the features I tested worked fine.
Code:
100d99
<       || ( >=media-video/ffmpeg-1.2.1:0=[encode] ( media-libs/libpostproc >=media-video/libav-10_alpha:=[encode] ) )
124,127d122
<       vdpau? (
<               || ( x11-libs/libvdpau >=x11-drivers/nvidia-drivers-180.51 )
<               || ( >=media-video/ffmpeg-1.2.1:0=[vdpau] >=media-video/libav-10_alpha:=[vdpau] )
<       )
228,229d222
<               --enable-external-libraries \
<               $(has_version 'media-video/libav' && echo "--enable-libav-compat") \
261c254,256
<               $(use_enable xrandr)
---
>               $(use_enable xrandr) \
>               --disable-external-libraries \
>               --disable-libav-compat

Basically the patch disables external libraries (ffmpeg or libav), and also disables the libav-compat feature (as we're using xbmc's internal ffmpeg libraries it wont build unless we explicitly disable this).

Chris
_________________
WYSIWYG - What You See Is What You Grep
Back to top
View user's profile Send private message
SamuliSuominen
Retired Dev
Retired Dev


Joined: 30 Sep 2005
Posts: 2133
Location: Finland

PostPosted: Thu Jul 31, 2014 10:55 am    Post subject: Reply with quote

We don't have this covered in http://wiki.gentoo.org/ but the reasons are same for every distribution,

http://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries#Why_no_Bundled_Libraries
http://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries#Security

Providing a way to use bundled libraries, specially ones like FFmpeg that are known to have had massive amount of vulnerabilities in every release, would be totally irresponsible

And if user really wants to do this, knowingly enable the vulnerable libraries, he can do that by using EXTRA_ECONF="--disable-external-libraries" from package.env file,
no ebuild editing required
Back to top
View user's profile Send private message
darkphader
Veteran
Veteran


Joined: 09 May 2002
Posts: 1217
Location: Motown

PostPosted: Thu Jul 31, 2014 3:10 pm    Post subject: Reply with quote

ssuominen wrote:
And if user really wants to do this, knowingly enable the vulnerable libraries, he can do that by using EXTRA_ECONF="--disable-external-libraries" from package.env file,
no ebuild editing required


Ahha, thanks, that's a simpler solution for sure.

As to the bundled libraries, I think they are only used by the specific program and therefore don't present a system wide attack vector. XBMC in this case will use its bundled ffmpeg, but everything in my system uses the Gentoo version of libav. So if the XBMC crew is lapse with an ffmpeg update I may be at risk when running XBMC, if Gentoo errs with libav updates or patches then I have a larger attack vector with any program relying on the libav libs able to put me at risk. Would this be more or less correct?
_________________
WYSIWYG - What You See Is What You Grep
Back to top
View user's profile Send private message
darkphader
Veteran
Veteran


Joined: 09 May 2002
Posts: 1217
Location: Motown

PostPosted: Thu Jul 31, 2014 4:26 pm    Post subject: Reply with quote

ssuominen wrote:
And if user really wants to do this, knowingly enable the vulnerable libraries, he can do that by using EXTRA_ECONF="--disable-external-libraries" from package.env file,
no ebuild editing required


Unfortunately this is not enough as the ebuild still tries to pull in ffmpeg (unless one is using a masked version of libav, which doesn't work for many things). Don't quite see how to fix that with EXTRA_ECONF.
_________________
WYSIWYG - What You See Is What You Grep
Back to top
View user's profile Send private message
SamuliSuominen
Retired Dev
Retired Dev


Joined: 30 Sep 2005
Posts: 2133
Location: Finland

PostPosted: Thu Jul 31, 2014 5:21 pm    Post subject: Reply with quote

darkphader wrote:
ssuominen wrote:
And if user really wants to do this, knowingly enable the vulnerable libraries, he can do that by using EXTRA_ECONF="--disable-external-libraries" from package.env file,
no ebuild editing required


Unfortunately this is not enough as the ebuild still tries to pull in ffmpeg (unless one is using a masked version of libav, which doesn't work for many things). Don't quite see how to fix that with EXTRA_ECONF.


You can use /etc/portage/profile/package.provided file to tell Portage you won't be needing external FFmpeg

So, whole setup should be:

Code:

$ cat /etc/portage/env/xbmc.conf
EXTRA_ECONF="--disable-external-libraries"
$ cat /etc/portage/package.env
media-tv/xbmc xbmc.conf
$ cat /etc/portage/profile/package.provided
virtual/ffmpeg-9
media-video/ffmpeg-2.3
media-video/libav-10.2


Be sure to use /etc/portage/profile, not /etc/portage for package.provided file, and note that the file format is different from, for example, package.keywords, and it doesn't have the = char as first char on the line
If a directory, like /etc/portage/env or /etc/portage/profile is missing, just create it

"man portage" and pressing / key and entering search word like package.env or package.provided will tell more, and 'n' key gets you to next search result, 'N' key gets you backward

I hope I got everything right out of memory. Good luck!
Back to top
View user's profile Send private message
darkphader
Veteran
Veteran


Joined: 09 May 2002
Posts: 1217
Location: Motown

PostPosted: Thu Jul 31, 2014 6:02 pm    Post subject: Reply with quote

ssuominen wrote:
You can use /etc/portage/profile/package.provided file to tell Portage you won't be needing external FFmpeg

So, whole setup should be:

Code:
$ cat /etc/portage/env/xbmc.conf
EXTRA_ECONF="--disable-external-libraries"
$ cat /etc/portage/package.env
media-tv/xbmc xbmc.conf
$ cat /etc/portage/profile/package.provided
virtual/ffmpeg-9
media-video/ffmpeg-2.3
media-video/libav-10.2



That helps a lot. The workaround without using an ebuild edit is:
Code:
$ cat /etc/portage/env/xbmc.conf
EXTRA_ECONF="--disable-external-libraries --disable-libav-compat"
$ cat /etc/portage/package.env
media-tv/xbmc xbmc.conf
$ cat /etc/portage/profile/package.provided
media-video/ffmpeg-1.2.1

The additional --disable-libav-compat is needed to override the ebuild setting --enable-libav-compat when libav is installed, which causes a build failure when using the internal libs.
Note: man portage states to not use virtuals in package.provided and my first test just adding media-video/ffmpeg-1.2.1 to package.provided was enough to have it not try and pull in ffmpeg.

Thanks again. Although not sure it was that much simpler than a quick ebuild edit :-)

Chris
_________________
WYSIWYG - What You See Is What You Grep
Back to top
View user's profile Send private message
SamuliSuominen
Retired Dev
Retired Dev


Joined: 30 Sep 2005
Posts: 2133
Location: Finland

PostPosted: Thu Jul 31, 2014 7:03 pm    Post subject: Reply with quote

darkphader wrote:
Thanks again. Although not sure it was that much simpler than a quick ebuild edit :-)


It isn't for a *one time* deal, but this way you can get version bump upgrades without editing the new ebuilds again
Back to top
View user's profile Send private message
darkphader
Veteran
Veteran


Joined: 09 May 2002
Posts: 1217
Location: Motown

PostPosted: Thu Jul 31, 2014 7:26 pm    Post subject: Reply with quote

ssuominen wrote:
It isn't for a *one time* deal, but this way you can get version bump upgrades without editing the new ebuilds again

Yes, but many times a newer ebuild and/or version solves the issue that I made a custom ebuild for and I need think about it no further. Whereas the portage workarounds continue to be in play for future versions whether needed or not. I'm guessing I could reduce the package.env atom to a single version and then just edit that if a newer version fails. Also package.provided appears to be system wide whereas using a local ebuild only affects that one package instance. So sure using the portage workarounds is nice and automatic for future versions but may create its own set of issues.

Chris
_________________
WYSIWYG - What You See Is What You Grep
Back to top
View user's profile Send private message
gcyoung
Apprentice
Apprentice


Joined: 04 Jul 2007
Posts: 170
Location: England

PostPosted: Wed Sep 03, 2014 6:26 pm    Post subject: Libav v ffmpeg Reply with quote

I have a similar problem, in that I recently installed mpv which requires libav. since I find the program excellent for dvb video progs etc., I'm keen to keep it, and have removed mplayer, xine et allia.

An upgrade to libav-9999 forced me to clean out former unwanted programs including ffmpeg. However, it would appear that k3b, which I wish to to keep, needs transcode, which I find will not compile without ffmpeg. There are several other programs which will not complle, as well, but I think I can live without them.

Any advice as to how I can get k3b up and working with libav present would be appreciated, since I'd be reluctant to give up mpv.

Later...
Apologies!

The problem with transcode would not seem to have anything to do with ffmpeg. On a closer look, I find it seems to have a problem with lame and is now reporting an error:

"/usr/include/lame/lame.h:856:11: note: expected 'unsigned char *' but argument is of type 'char *'
/bin/sh ../libtool --tag=CC --mode=compile x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I.. -D_REENTRANT -DMOD_PATH=\"/usr/lib64/transcode\" -I.. -I../src -I/usr/include -Wall -Wstrict-prototypes -Wmissing-prototypes -march=native -O2 -pipe -c -o export_divx5_la-divx4_vbr.lo `test -f 'divx4_vbr.c' || echo './'`divx4_vbr.c
make[2]: *** [export_divx5_la-aud_aux.lo] Error 1
make[2]: *** Waiting for unfinished jobs....
libtool: compile: x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I.. -D_REENTRANT -DMOD_PATH=\"/usr/lib64/transcode\" -I.. -I../src -I/usr/include -Wall -Wstrict-prototypes -Wmissing-prototypes -march=native -O2 -pipe -c divx4_vbr.c -fPIC -DPIC -o .libs/export_divx5_la-divx4_vbr.o
divx4_vbr.c: In function 'VbrControl_init_2pass_vbr_encoding':
divx4_vbr.c:151:10: warning: ignoring return value of 'fread', declared with attribute warn_unused_result [-Wunused-result]
divx4_vbr.c:157:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
divx4_vbr.c:158:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
divx4_vbr.c:204:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
divx4_vbr.c:236:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
make[2]: Leaving directory `/var/tmp/portage/media-video/transcode-1.1.7-r3/work/transcode-1.1.7/export'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/var/tmp/portage/media-video/transcode-1.1.7-r3/work/transcode-1.1.7'
make: *** [all] Error 2
* ERROR: media-video/transcode-1.1.7-r3::gentoo failed (compile phase):
* emake failed "

I assumed it was to do with ffmpeg since other programs reported a block with ffmpeg and libav when I tried to get them back.
Back to top
View user's profile Send private message
ct85711
Veteran
Veteran


Joined: 27 Sep 2005
Posts: 1791

PostPosted: Wed Sep 03, 2014 8:13 pm    Post subject: Reply with quote

Strange that transcode won't compile with libav, as that's what I have on my computer with transcode installed. I'm recompiling it again, just to be sure; but I don't recall any issue last time I compiled it. Below is the use flags I have set for libav and transcode, maybe it's an issue with your use flags gcyoung.

*note* I just finished compiling transcode without any issues popping up besides the usual qa warning stuff that you can ignore.

Code:

[ebuild   R    ] media-video/libav-9.16:0/9  USE="X aac alsa bzip2 encode gpl hardcoded-tables mmx mp3 network openssl pulseaudio sdl speex ssl theora threads truetype vaapi vorbis vpx x264 xvid zlib -3dnow -3dnowext (-altivec) -amr -avx -bindist -cdio -cpudetection -custom-cflags -debug -doc -faac -fdk -frei0r -gsm -ieee1394 -jack -jpeg2k -mmxext (-neon) -opus -oss -pic -rtmp -schroedinger -ssse3 -static-libs {-test} -tools -v4l -vdpau (-vis)" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild   R    ] media-video/transcode-1.1.7-r3  USE="X a52 aac alsa dvd iconv jpeg mmx mp3 mpeg ogg postproc sdl sse sse2 theora truetype vorbis x264 xml xvid -3dnow (-altivec) -dv -imagemagick -lzo -mjpeg (-nuv) -oss (-pic) -quicktime -v4l" 0 KiB
[ebuild   R    ] media-sound/lame-3.99.5-r1  USE="-debug (-mmx) -mp3rtp -sndfile -static-libs" ABI_X86="(64) -32 (-x32)" 1,412 KiB
Back to top
View user's profile Send private message
gcyoung
Apprentice
Apprentice


Joined: 04 Jul 2007
Posts: 170
Location: England

PostPosted: Thu Sep 04, 2014 6:07 pm    Post subject: libav v ffmpeg Reply with quote

I thought you might be right, c185771 in that libav might be taking over some of the USE flags of transcode. So I had a close look at the compilation, and although there is a great deal I don't follow, I saw that there were a rather large number of warnings and some errors througout the compilation. More than my limited programming knowledge could follow.

As this has only occurred since I "keyworded" libav-9999 and removed ffmpeg I suspect is has to do with errors present in libav. I am so impressed with the versatility and quality of video produced using mpv that I think I can live without transcode until libav and mpv are a bit more stable.

Thanks anyway for responding.
Back to top
View user's profile Send private message
ct85711
Veteran
Veteran


Joined: 27 Sep 2005
Posts: 1791

PostPosted: Thu Sep 04, 2014 6:22 pm    Post subject: Reply with quote

The thing that alerts me, as a possible source of issue, is using a -9999 version package. Those packages are development packages, and typicaly are not stable in any fashion (typicaly will break more things than not). A general rule that I've always followed is advoid any -9999 packages at all costs, and only use if absolutely necessary (expect for the world to explode when using them too). Sadly, right now at this moment I'm not in linux to try compiling mpv on my system. But you may want to try the regular media-video/libav-9.16 with it, and see what happens. I don't recall for sure, but I'm pretty sure it's only marked as unstable, and from my experience, I rarely have too much problem with an unstable package. A package that hasn't been keyworded though, generaly are really broken; so your looking for some trouble there.

Most likely, in a couple days I'll be back in linux for my usual weekly update; and I can try compiling mpv out then if your issue hasn't been fixed.
Back to top
View user's profile Send private message
gcyoung
Apprentice
Apprentice


Joined: 04 Jul 2007
Posts: 170
Location: England

PostPosted: Thu Sep 04, 2014 8:00 pm    Post subject: libav V ffmpeg Reply with quote

You 're probably right, although "nothing ventured nothing gained". I went there because >=mpv-4.0.0 needs it and it's now at version 5.0.1.
I'd been thinking of dropping ffmpeg and moving to libav anyway, and this upgrade simply forced the issue. I think earlier issues of libav do not block ffmpeg, but --9999 does.
Back to top
View user's profile Send private message
ct85711
Veteran
Veteran


Joined: 27 Sep 2005
Posts: 1791

PostPosted: Sat Sep 06, 2014 9:01 pm    Post subject: Reply with quote

Well, I took a look into mpv, and it seems for my system I won't be able to test it out. For me, it's more of gst-plugins-libav is preventing me from updating libav to >10 which is required for mpv. The fun part is, gst-plugins-libav is out of date (the current available from their website is
Quote:
gst-libav-1.4.1.tar.xz 27-Aug-2014 04:57 5.0M
). So till, we get gst updated in portage to use the new version; my system is locked out from trying mpv with libav. I checked our bugzilla, and there's a bug already open reguarding this and seems the maintainer nows of this =media-libs/gstreamer-1.4.1: version bump. So it's more of waiting, till gst plugins gets updated to test this out on my system with libav. Only other way is use ffmeg instead of libav.
Back to top
View user's profile Send private message
franzf
Advocate
Advocate


Joined: 29 Mar 2005
Posts: 4565

PostPosted: Sun Sep 07, 2014 8:57 am    Post subject: Reply with quote

Just some notes:
* mpv (at least in the past) preferred ffmpeg over libav (so "compatible only with libav" is wrong). Have a look at the ebuild, it allows both, ffmpeg and libav. I use mpv-9999 and rebuilt it just yesterday - everything went fine :) (using ffmpeg)
* gst-plugins-libav compiled fine here with ffmpeg
* Even transcode had no issue here - just installed current stable version (1.1.7-r3)

I have media-video/ffmpeg-2.3.3 installed, if that matters. I tried libav some time ago but it only caused me trouble. Went back to ffmpeg and run it on several computers without any issue.
Back to top
View user's profile Send private message
ct85711
Veteran
Veteran


Joined: 27 Sep 2005
Posts: 1791

PostPosted: Sun Sep 07, 2014 4:26 pm    Post subject: Reply with quote

Yep, I knew mpv perfers ffmpeg, sadly my system is using libav; and not really feeling like breaking my system to convert over to ffmpeg again just for some testing. I had a feeling gcyoung's issue is more of using -9999 level packages with regular packages.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Multimedia All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum