Piping Out From ffmpeg Suddenly Stopped Working

Message

Xanadu · Post by **Xanadu** » Sat Jul 11, 2009 12:53 am

Hey!

I've been doing (essentially) the following example in a script WEEKLY for the past 2 years and this weeks it has stopped working:

Code: Select all

ffmpeg -i Pop.mp3 -f wav - | lame - Pop2.mp3

and the output .mp3 file is empty. Here is what happens on the screen:

Code: Select all

$ ffmpeg -i Pop.mp3 -f wav - | lame - Pop2.mp3 ; midentify Pop2.mp3
FFmpeg version 0.5, Copyright (c) 2000-2009 Fabrice Bellard, et al.
  configuration: --prefix=/usr --libdir=/usr/lib --shlibdir=/usr/lib --mandir=/usr/share/man 
  --enable-static --enable-shared --cc=i686-pc-linux-gnu-gcc --disable-debug --disable-optimizations 
  --enable-libfaac --enable-libmp3lame --enable-libvorbis --enable-libtheora --enable-libx264
  --enable-libxvid --disable-demuxer=oss --disable-muxer=oss --enable-x11grab --enable-pthreads 
  --enable-libfaad --enable-libspeex --enable-libopenjpeg --enable-libamr-nb --enable-libamr-wb 
  --enable-nonfree --disable-altivec --cpu=prescott --disable-vhook --enable-gpl --enable-postproc 
  --enable-avfilter --enable-avfilter-lavf --enable-swscale --disable-stripping --enable-hardcoded-tables
  libavutil     49.15. 0 / 49.15. 0
  libavcodec    52.20. 0 / 52.20. 0
  libavformat   52.31. 0 / 52.31. 0
  libavdevice   52. 1. 0 / 52. 1. 0
  libavfilter    0. 4. 0 /  0. 4. 0
  libswscale     0. 7. 1 /  0. 7. 1
  libpostproc   51. 2. 0 / 51. 2. 0
  built on Jul 10 2009 06:58:55, gcc: 4.3.2
Input #0, mp3, from 'Pop.mp3':
  Duration: 00:00:00.20, start: 0.000000, bitrate: 71 kb/s
    Stream #0.0: Audio: mp3, 22050 Hz, stereo, s16, 64 kb/s
Output #0, wav, to 'pipe:':
    Stream #0.0: Audio: pcm_s16le, 22050 Hz, stereo, s16, 705 kb/s
Stream mapping:
  Stream #0.0 -> #0.0
Press [q] to stop encoding
size=      18kB time=0.21 bitrate= 707.3kbits/s
video:0kB audio:18kB global headers:0kB muxing overhead 0.238715%
LAME 3.98.2 32bits (http://www.mp3dev.org/)
CPU features: MMX (ASM used), SSE (ASM used), SSE2
Using polyphase lowpass filter, transition band: 10758 Hz - 11025 Hz
Encoding <stdin> to Pop2.mp3
Encoding as 22.05 kHz j-stereo MPEG-2 Layer III (11x)  64 kbps qval=3
ID_EXIT=EOF

Here is a cat of the output mp3:

Code: Select all

$ cat Pop2.mp3
ÿódInfo@@@@@@@@@@@@@@@@@@@@@@@@ÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ9LAME3.98rn
þ@$À.@&¸i*ÿód¤HLAME3.98.2UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUULAME3.98.2UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
UUÿód»¤HUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUULAME3.98.2UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUÿ
ódÿ¤HUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
UUUUUUUUUUUUUUUUUUUUUUUUUUUULAME3.98.2UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUÿódÿ¤
HUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU

(I put in line breaks so the browser won't side scroll).

Lame is working fine and ffmpeg works fine, as long as I don't use a pipe. I can:

Code: Select all

ffmpeg -i Pop.mp3 -f wav Pop.wav && lame Pop.wav Pop2.mp3

and that works fine. I can't pipe it out to lame. I guess there's the chance that it's lame that's not taking the pipe in any more, but, with the cat of the mp3 above you can see that lame is trying to do something but has no idea what is being thrown at it.

Any ideas? Anyone having the same problem?

Xanadu · Post by **Xanadu** » Sat Jul 11, 2009 12:23 pm

I thought a super simple test was in order to make sure bash wasn't hosed or something:

Code: Select all

$ echo "2+2" | bc
4

I'm starting to think that maybe it's lame not getting the piped info and not ffmpeg sending out garbage.

If someone knows how to do this with something other than lame, I'm all ears:

Code: Select all

        ffmpeg -i "$INFILE" -v 2 -vn -f wav -ac 2 -ar "$ID_AUDIO_RATE" -map 0.1 - | \
        lame -q0 -V5 --vbr-new --resample "$ID_AUDIO_RATE" - "$TMP_DIR/$AUDIO_FILE"

(I'm aware that the variables there are unknown to anyone but me, but, it's fairly obvious what I'm doing, anyway...)

***EDIT***

Just as a test, I've recompiled both ffmpeg and lame. The problem still exists.

Xanadu · Post by **Xanadu** » Sun Jul 12, 2009 12:06 am

More news.

It's definitely something wrong with ffmpeg. If I convert a flac to an mp3:

Code: Select all

flac -dc "$FLAC" | lame -ms -q0 -V0 --vbr-new -ms "$FLAC" "$MP3"

everything works fine.

Any ideas? I've already recompiled ffmpeg. I haven't actually updated it in a good while now. I do not use ~x86 for it.

Code: Select all

     Mon Jan  5 21:12:46 2009 >>> media-video/ffmpeg-0.4.9_p20081014
       merge time: 5 minutes and 15 seconds.

     Mon Jan 26 21:48:39 2009 >>> media-video/ffmpeg-0.4.9_p20081219
       merge time: 7 minutes and 14 seconds.

     Fri Feb  6 06:19:14 2009 >>> media-video/ffmpeg-0.4.9_p20090201
       merge time: 4 minutes and 44 seconds.

     Wed May 13 21:39:19 2009 >>> media-video/ffmpeg-0.5-r1
       merge time: 5 minutes and 8 seconds.

     Tue Jun 30 18:43:10 2009 >>> media-video/ffmpeg-0.5-r1
       merge time: 6 minutes and 46 seconds.

     Fri Jul 10 06:59:34 2009 >>> media-video/ffmpeg-0.5-r1
       merge time: 5 minutes and 27 seconds.

(that's only from this year - I didn't think it was important to paste merge history back to July of '07

)

Anyway, any ideas?

DooBeDooBeDo · Post by **DooBeDooBeDo** » Sun Jul 12, 2009 10:07 am

I'd say this is a bug in ffmpeg.

To test: I took a random mp3 and copied it as pop.mp3

Code: Select all

ffmpeg -i pop.mp3 -f wav pop.wav

this produced a playable wav file.

Code: Select all

ffmpeg -i pop.mp3 -f wav - >test.pop.wav

this produced an unplayable file of the same size.

I then checked the file headers in a hex editor and they were different! So I replaced the header of the unplayable file with the header from the playable file and lo and behold it played!

So when ffmpeg produces files to stdout it produces a different header than when writing to a file - hence ffmpeg bug.

Xanadu · Post by **Xanadu** » Sun Jul 12, 2009 12:05 pm

DooBeDooBeDo wrote:So when ffmpeg produces files to stdout it produces a different header than when writing to a file - hence ffmpeg bug.

Thank you VERY much for testing! It's good to know it's not just my machine doing this!

All of a sudden, though? You can see from my genlop output that the version hasn't changed since May.

Code: Select all

     Wed May 13 21:39:19 2009 >>> media-video/ffmpeg-0.5-r1
       merge time: 5 minutes and 8 seconds.

     Tue Jun 30 18:43:10 2009 >>> media-video/ffmpeg-0.5-r1
       merge time: 6 minutes and 46 seconds.

     Fri Jul 10 06:59:34 2009 >>> media-video/ffmpeg-0.5-r1
       merge time: 5 minutes and 27 seconds.

I wish there was a way to see past USE flags in case there was something that changed that way that I hadn't noticed (I update with "emerge -qDNu @system @world"). Still if the version hadn't changed, then there shouldn't be an issue, right? I've been doing it this way on a (near) weekly basis for a couple years now. I just haven't had to for a month or so, so I'm just noticing this now.

I just looked at the Changelog and there's nothing really there that would change any major behavior (meaning no patches from upstream that got thrown in or something).

Could it be something else on our systems that's been updated that changed the way stdout behaves?

DooBeDooBeDo · Post by **DooBeDooBeDo** » Sun Jul 12, 2009 1:28 pm

maybe not a bug, see here: http://archives.free.net.ph/message/200 ... 4c.pt.html

Xanadu · Post by **Xanadu** » Sun Jul 12, 2009 1:40 pm

DooBeDooBeDo wrote:maybe not a bug, see here: http://archives.free.net.ph/message/200 ... 4c.pt.html

OK, that answers some, sort of. I've not changed my script in just about 2 years and was doing it this way just a few weeks ago. Taking a .vob's audio as a .wav (-f wav) and piping it out to lame to re-encode as an mp3.

I just ran a test and tried to tell lame that it was a raw PCM input (-r) and gave it the sample rate and stereo type (stereo, joint, mono) and it still fails to write a valid file like it has been doing for years.