HOWTO conveniently convert videos using mencoder, x264

[fangorn]

I don't know.

The Format of the pages was adjusted to this width when the posts contained the code. After deleting the code from the pages the width stayed as it was.

[fangorn]

I just released the first version of bluray-info (version 0.0.1, so don't expect too much stability or features )

It is a perl program based on a parser for .mpls Playlists of BluRay directory structures from bdtools (unreleased AFAIK and therefore reimplemented in Perl).
It can print Clip information as the original mpls_dump, but additionally it can export tsMuxeR .meta files or even use it to output .ts files directly.

Please use it with care, as it is widely untested.

[ExElNeT]

I really like your scripts, they work nice.

Still there are some things I would like to see in your scripts. I am looking for a nice solution to encode a dvd into a mkv container. Your solution is perfect if you have the computing power. Unfortunately I dont. So I would like to use your scripts with a different codec or maybe no codec at all. (keep the mpeg2).

Where would I start if i wanted to extend your scripts?

thanks, greetz
exelnet

[fangorn]

I just updated a few helper programs.

dejunk now supports optional file- and directory names as command line parameters and only processes them if given.

bluray-info now has the feature export chaptermarks to seperate file when writing tsMuxeR .meta file(s) and optionally demuxing Sup subtitles instead of copying it to .ts container and optionally convert them to VobSub (DVD) subtitle format (using BDSup2Sub). Optionally it only processes all titles longer than <min> minutes, be it for display or export functions.

choose_x264_encoding_parameters.sh is reimplemented in Perl and renamed to choose_video_parameters. No new features atm, but it should now run in any environment with a standard Perl environment.

The released versions of my other scripts use the old (bash) version at the moment! I will change the used helper script with the next releases.

[fangorn]

I really like your scripts, they work nice.

Still there are some things I would like to see in your scripts. I am looking for a nice solution to encode a dvd into a mkv container. Your solution is perfect if you have the computing power. Unfortunately I dont. So I would like to use your scripts with a different codec or maybe no codec at all. (keep the mpeg2).

Where would I start if i wanted to extend your scripts?

thanks, greetz
exelnet

you could replace everything after
# first pass
in function encode_video by this:

Code: Select all

   nice -n 10    $mencoder $DEVICE $AUDIO $nosubs \
        -ovc copy  $audiocodec \
        $INPUT_FRAMERATE $OUTPUT_FRAMERATE \
        -o "${TMP}/""$moviename""_tmp.avi" 2>> "${TMP}/""$moviename""_mencoder.log"

This copies video and the first audio stream to an avi. normally the rest of the calls (mainly to avi2mkv) should work es usual, but I have not tested it.

You could also replace the -ovc copy part by -ovc lavc -lavcopts ...
There are quite a few howtos for converting DVD content to MPEG4 (DIVX/XVID). I'd start with a 1pass setting for testing what -lavcopts suit your needs.

[fangorn]

There are some news (after quite a while of not saying anything, but I was busy).

I just released bluray-info-0.2.0. It has some new features, but mainly is easier in handling and more tested. It is dependent on a new program though. I created a script for all the tools I plan to use in perl versions of all my scripts. toolbox_fangorn is available from the list of programs and has it's own post.

As already stated I am in the process of rewriting all my scripts in Perl and in the process clean them up a lot and centralize functions used in all of them. Here is a little update on the progress I make.

Atm. I am testing avi2mkv (the more advanced features like automatic audio recoding) on both Linux and Windows. Basics are already working.

blu2mkv is rewritten and I just have to test it. riptoh264 is a little stubborn atm. as some dependencies are not available for windows and I have to think of how to replace them either in Perl or with available external programs.

Also I set up a little project page for all my tools. It is reachable under http://fangornsrealm.eu
(don't complain about the design, please. I know it is ugly as hell. But I do neither have the time nor the skills to design a prettier one.)

[Brad1213]

I'm having trouble with blu2mkv

I have built x264,ffmpeg and mplayer from svn/git and configured them using the options from earlier in this thread.
When I run blu2mkv, I get this:

Code: Select all

blu2mkv -b 3000 -p hq casino-test.mkv 

Video Bitrate set to 3000
x264 encoding preset set to hq
Source file is of Type  AVI
Width = 1920
Height = 1080
Aspect = 1.77777777777778
Aspect should be 1.77777777777778
Source Framerate: 23.976
Input Framerate used: 24000/1001
Output Framerate used: 24000/1001
No audio stream specified. Trying to use specified languages if available in source file. If not, default stream will be used.
Resolution = 1920:1080
Do not scale is yes
video is interlaced: no
dynamic noise reduction is active: no

using audio stream: default
calling external program choose_x264_encoding_parameters.sh
external call was successfull
using seleted preset hq
geometryfilteroptions expand=1920:1080
video filters used for first pass:  -vf expand=1920:1080,hqdn3d=2:1:2,harddup
AUDIO channel used
second audio
MEncoder SVN-r30985-4.3.4 (C) 2000-2010 MPlayer Team
success: format: 0  data: 0x0 - 0x17a79b54
AVI file format detected.
[aviheader] Video stream found, -vid 0
[aviheader] Audio stream found, -aid 1
VIDEO:  [avc1]  1920x1080  24bpp  23.976 fps  25996.1 kbps (3173.3 kbyte/s)
[V] filefmt:3  fourcc:0x31637661  size:1920x1080  fps:23.976  ftime:=0.0417
Input fps will be interpreted as 23.976 instead.
Opening video filter: [expand osd=1]
Expand: -1 x -1, -1 ; -1, osd: 1, aspect: 0.000000, round: 1
Opening video filter: [harddup]
Opening video filter: [hqdn3d=2:1:2]
Opening video filter: [expand w=1920 h=1080]
Expand: 1920 x 1080, -1 ; -1, osd: 0, aspect: 0.000000, round: 1
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264)
==========================================================================
audiocodec: framecopy (format=2000 chans=2 rate=48000 bits=16 B/s=56000 sample-1)
[h264 @ 0xb7cf00]concealing 8160 DC, 8160 AC, 8160 MV errors
[h264 @ 0xb7cf00]concealing 6921 DC, 6921 AC, 6921 MV errors.000 [0:0]
Movie-Aspect is 1.78:1 - prescaling to correct movie aspect.
[h264 @ 0xb7cf00]concealing 7448 DC, 7448 AC, 7448 MV errors.004 [0:0]
[h264 @ 0xb7cf00]concealing 6970 DC, 6970 AC, 6970 MV errors.008 [0:0]
[h264 @ 0xb7cf00]concealing 8063 DC, 8063 AC, 8063 MV errors.011 [0:0]
[h264 @ 0xb7cf00]concealing 6805 DC, 6805 AC, 6805 MV errors.014 [0:0]
[h264 @ 0xb7cf00]concealing 8034 DC, 8034 AC, 8034 MV errors.017 [0:0]
[h264 @ 0xb7cf00]concealing 7001 DC, 7001 AC, 7001 MV errors.020 [0:0]
[h264 @ 0xb7cf00]concealing 7899 DC, 7899 AC, 7899 MV errors.022 [0:0]
[h264 @ 0xb7cf00]concealing 7268 DC, 7268 AC, 7268 MV errors.024 [0:0]
[h264 @ 0xb7cf00]concealing 8033 DC, 8033 AC, 8033 MV errors.026 [0:0]
[h264 @ 0xb7cf00]concealing 7035 DC, 7035 AC, 7035 MV errors.027 [0:0]
[h264 @ 0xb7cf00]concealing 8101 DC, 8101 AC, 8101 MV errors.029 [0:0]
[h264 @ 0xb7cf00]concealing 6543 DC, 6543 AC, 6543 MV errors.030 [0:0]
[h264 @ 0xb7cf00]concealing 7921 DC, 7921 AC, 7921 MV errors.031 [0:0]
[h264 @ 0xb7cf00]concealing 6535 DC, 6535 AC, 6535 MV errors.032 [0:0]
[h264 @ 0xb7cf00]concealing 8101 DC, 8101 AC, 8101 MV errors.033 [0:0]
[h264 @ 0xb7cf00]concealing 6068 DC, 6068 AC, 6068 MV errors.034 [0:0]
[h264 @ 0xb7cf00]concealing 8160 DC, 8160 AC, 8160 MV errors.035 [0:0]
Writing header...2f ( 1%) 10.75fps Trem:   4min   0mb  A-V:0.042 [0:0]
ODML: vprp aspect is 16:9.
Setting audio delay to 0.083s.
Writing header...
ODML: vprp aspect is 16:9.
Setting audio delay to 0.083s.
Pos: 117.9s   2878f (99%) 14.61fps Trem:   0min  14mb  A-V:0.042 [598:448]
Flushing video frames.
Writing index...
Writing header...
ODML: vprp aspect is 16:9.
Setting audio delay to 0.083s.

Video stream:  598.467 kbit/s  (74808 B/s)  size: 8979742 bytes  120.037 secs  2878 frames

Audio stream:  448.000 kbit/s  (56000 B/s)  size: 6636000 bytes  118.500 secs
video filters used for second pass:  -vf spp,expand=1920:1080,hqdn3d=2:1:2,harddup
MEncoder SVN-r30985-4.3.4 (C) 2000-2010 MPlayer Team
success: format: 0  data: 0x0 - 0x17a79b54
AVI file format detected.
[aviheader] Video stream found, -vid 0
[aviheader] Audio stream found, -aid 1
VIDEO:  [avc1]  1920x1080  24bpp  23.976 fps  25996.1 kbps (3173.3 kbyte/s)
[V] filefmt:3  fourcc:0x31637661  size:1920x1080  fps:23.976  ftime:=0.0417
Input fps will be interpreted as 23.976 instead.
Opening video filter: [expand osd=1]
Expand: -1 x -1, -1 ; -1, osd: 1, aspect: 0.000000, round: 1
Opening video filter: [harddup]
Opening video filter: [hqdn3d=2:1:2]
Opening video filter: [expand w=1920 h=1080]
Expand: 1920 x 1080, -1 ; -1, osd: 0, aspect: 0.000000, round: 1
Opening video filter: [spp]
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264)
==========================================================================
audiocodec: framecopy (format=2000 chans=2 rate=48000 bits=16 B/s=56000 sample-1)
[h264 @ 0xb7cf00]concealing 8160 DC, 8160 AC, 8160 MV errors
[h264 @ 0xb7cf00]concealing 6921 DC, 6921 AC, 6921 MV errors.000 [0:0]
[PP] Using external postprocessing filter, max q = 6.
Movie-Aspect is 1.78:1 - prescaling to correct movie aspect.

Exiting...
muxing to Matroska container
mkvmerge v2.9.9 ('Tutu') built on Apr  2 2010 15:14:22
Error: The file 'casino-test.avi' has unknown type. Please have a look at the supported 
file types ('mkvmerge --list-types') and contact the author Moritz Bunkus <moritz@bunkus.org> 
if your file type is supported but not recognized properly.

It fails on the second pass, I checked the command.log and ran the second pass again.

Code: Select all

/usr/local/bin/mencoder casino-test.mkv -oac copy -nosub -ovc x264 -x264encopts me=umh:me_range=16:nodct_decimate:8x8dct:
nofast_pskip:trellis=1:partitions=p8x8,b8x8,i8x8,i4x4:mixed_refs:keyint=240:psy_rd=0.8,0.2:frameref=3:bframes=4:b_adapt=2:
b_pyramid=normal:weight_b:weightp=1:direct_pred=auto:subq=6:mbtree:chroma_me:cabac:aq_mode=1:deblock:vbv_maxrate=9500:
vbv_bufsize=12000:level_idc=41:threads=auto:ssim:psnr:bitrate=3000:pass=2 -passlogfile casino-test_2pass.log 
-vf spp,expand=1920:1080,hqdn3d=2:1:2,harddup -fps 24000/1001 -ofps 24000/1001 -o casino-test.avi

MEncoder SVN-r30985-4.3.4 (C) 2000-2010 MPlayer Team
success: format: 0  data: 0x0 - 0x17a79b54
AVI file format detected.
[aviheader] Video stream found, -vid 0
[aviheader] Audio stream found, -aid 1
VIDEO:  [avc1]  1920x1080  24bpp  23.976 fps  25996.1 kbps (3173.3 kbyte/s)
[V] filefmt:3  fourcc:0x31637661  size:1920x1080  fps:23.976  ftime:=0.0417
Input fps will be interpreted as 23.976 instead.
Opening video filter: [expand osd=1]
Expand: -1 x -1, -1 ; -1, osd: 1, aspect: 0.000000, round: 1
Opening video filter: [harddup]
Opening video filter: [hqdn3d=2:1:2]
Opening video filter: [expand w=1920 h=1080]
Expand: 1920 x 1080, -1 ; -1, osd: 0, aspect: 0.000000, round: 1
Opening video filter: [spp]
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264)
==========================================================================
audiocodec: framecopy (format=2000 chans=2 rate=48000 bits=16 B/s=56000 sample-1)
[h264 @ 0xb7cf00]Missing reference picture
[h264 @ 0xb7cf00]decode_slice_header error
[h264 @ 0xb7cf00]Missing reference picture
[h264 @ 0xb7cf00]decode_slice_header error
[h264 @ 0xb7cf00]Missing reference picture
[h264 @ 0xb7cf00]decode_slice_header error
[h264 @ 0xb7cf00]Missing reference picture
[h264 @ 0xb7cf00]decode_slice_header error
[h264 @ 0xb7cf00]concealing 8160 DC, 8160 AC, 8160 MV errors
[h264 @ 0xb7cf00]top block unavailable for requested intra4x4 mode -1 at 0 50
[h264 @ 0xb7cf00]error while decoding MB 0 50, bytestream (15385)
[h264 @ 0xb7cf00]concealing 6921 DC, 6921 AC, 6921 MV errors
[PP] Using external postprocessing filter, max q = 6.
Movie-Aspect is 1.78:1 - prescaling to correct movie aspect.
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast FastShuffle SSEMisalign LZCNT
x264 [error]: requested bitrate is too low. estimated minimum is 78 kbps
x264_encoder_open failed.
FATAL: Cannot initialize video driver.

Exiting...

I've tried changing the bitrate but it doesn't make any difference.
Any ideas?

Brad

[fangorn]

I had similar problems once (when it was a very young feature) with mbtree. Now it is only a young feature and I have not had any problems with it lately, but who knows?

You could try if changing the :mbtree part in the encoding options to :nombtree makes this work. I honestly don't know if this option affects the normal twopass log too or if it is sufficient to just run the second pass without mbtree support.

Also I recognized that the audio stream detection did not find anything in your case. Does the MKV have not language specifications (you can change that on the fly with mmg header editor) or do the language settings in blu2mkv do not match your preferred languages (you have to change them by hand at the top of the program).

[Brad1213]

Hi

I realised I didn't have mmg, so I remerged media-video/mkvtoolnix with wxwidgets to get mmg.
Opened the test file in mmg and it said it was not a valid mkv (xbmc was playing this test clip with no problems).
I had used, mencoder -ss 600 -endpos 120 movie.mkv -oac copy -ovc copy -o movie-test-clip.mkv which appears to create an invalid mkv.

Guess that was part one of the problem , so I used to mmg to export some test clips as valid mkvs.

I tried changing the mbtree to nombtree but still the same error. I had already spent a couple days trying to get this working and was convinced it was something to do with my version or config of mplayer and x264.

So I decided to try the ~amd64 versions from portage. Which pulled in x264-0.0.20100118, ffmpeg-0.5_p21602 and mplayer-1.0_rc4_p20100213-r1
x264 is now at libx264.so.83 rather than the libx264.so.92 that I had from built from git.

I set the path to /usr/bin/mencoder and tried blu2mkv again and it worked, almost.
It did the second pass, but failed on the mkvmerge.

Code: Select all

Setting audio delay to 0.083s.

Video stream: 3001.905 kbit/s  (375238 B/s)  size: 48594983 bytes  129.504 secs  3106 frames

Audio stream:  448.000 kbit/s  (56000 B/s)  size: 7252000 bytes  129.500 secs
muxing to Matroska container
mkvmerge v2.9.9 ('Tutu') built on Apr  6 2010 18:51:09
Error: No language specified in '--language 0:'.

I re-ran mkvmerge from the command.log but changed the '--language 0:' to '--language 0:eng' and it worked fine.

I had set the lang in avi2mkv

Code: Select all

first="eng"
second= ""

and blu2mkv

Code: Select all

mkv_firstlanguage="eng"
mkv_firstlanguage=""

I set them all to eng and re-ran blu2mkv, it worked fine. Not sure if that is a bug.

What would be good bitrate to use to encode from a bluray source? Xbmc says the bluray disk (also h264) is playing at 24-26Mbits, I re-encoded the test clip with -b 3000 and the bitrate when playing is about 1-1.2Mbits. The file size went from 350mb to 75mb, I couldn't see any loss of quality (2min clip is a small sample though).
So I set -b 6000 -p vdhq for the whole movie, but am still waiting on that..

Thanks for the scripts and help.
Brad

[fangorn]

Yes, with these settings mencoder produces "wrong" mkvs. It actually produces an AVI that is named .mkv

The language problem is most likely a bug. I have this one on the agenda for the next version of avi2mkv anyway (which I am testing thoroughly atm.)

For bluray reencodings I use -b 6000 (default of blu2mkv for 1080p) and nq, which give - for my opinion - the best tradeoff between quality and reencoding time.

[fangorn]

I fixed a bug in bluray-info. New version available

[fangorn]

I just released some new versions with some new features. Main change nevertheless is the rewrite in Perl, so theoretically it should work on any Operating system that has Perl (and the respective dependencies, which is quite a problem with avi2mkv). I am not that used to actually using (as in not only running a program on it) Windows any more - as in the nineties with Windows NT 4.0 - so I have some problems setting up a testing environment that is capable of running avi2mkv. Therefore I decided to not longer postpone the release of version 2.0.0, even if one of the main features (windows) is not tested at all.

I added configuration file support to toolbox_fangorn and therefore all following Perl versions, for the moment bluray-info_0.2.1 and avi2mkv_2.0.0 or later.

I added handling of multiple languages and audio streams to avi2mkv. It handles as many audio streams as languages are specified (from script, config file or on the command line) be it internal, external or both.

Edit: Also there is a new version of choose_video_parameters where I fixed some bugs.

[fangorn]

And another brand new program to announce: riptoh264 is now available in version 2.0.0 as brand new Perl rewrite with all the shiny new features like configuration file and multiple languages and audio stream support.

Also by default subtitles are now only exported when they match a preferred language and all subtitles can be exported by command line switch.

there is now normally no need for the -d option (when -n is used to specify a title, it is advisable to use the -d option too)


I also fixed a few bugs in avi2mkv subtitle handling and added some functions to toolbox-fangorn. New versions available.

[fangorn]

I have just released version 3.0.0 of menc.
In the process of rewriting the program in Perl I added some neat features, fixed some bugs and
cleaned up a little.

Also I fixed some bugs in avi2mkv, added some features to toolbox_fangorn. Also riptoh264 has
seen updates. New versions available.

[CrazyTerabyte]

Have you thought about adding an ebuild for your tools to the Sunrise overlay?

[fangorn]

I don't have any experience with writing ebuilds (just a little editing) so it would take me quite some time. I still have blu2mkv in queue for a rewrite and thorough testing.

I don't see an ebuild or multiple ebuilds in the near future from my side.

But if someone feels capable of writing (and maintaining) ebuilds, I am the last one to complain. At least the dependencies of the Perl versions are easier to satisfy than for the old bash versions.

At the moment I am thinking about a mailing list to announce releases and bugfixes.

Also, if someone has suggestions, requests, bug reports, ... PM me here or send me mail at progs @ fangornsrealm.eu

[CrazyTerabyte]

I don't have any experience with writing ebuilds (just a little editing) so it would take me quite some time.

Come on... You wrote your previous scripts in bash, you say you've edited a few ebuilds... And you can't make one? I'm pretty sure you can! It's not that difficult. And since your scripts are quite "stand-alone", the ebuild will just run "dobin" at the src_install() to install them, and basically nothing else.

Check this: http://devmanual.gentoo.org/
Most pages are not updated to EAPI=2, but it still gives you a good idea about how to write an ebuild.

At the moment I am thinking about a mailing list to announce releases and bugfixes.

I think a newsfeed (RSS or Atom) at your blog or project homepage might be the best solution. It's easy to implement, and it's easy for everyone to keep updated.


BTW, if you want to parse a program output, try exporting LC_ALL=C before running that program. It will instruct the program to use the built-in, non-translated strings.

[fangorn]

[ And since your scripts are quite "stand-alone.

Have you read the dependency list of avi2mkv? And I have reduced it already!

Yes, I know my ways in bash, but with "edited" I meant "edit the arch" "delete a problematic patch", ...

And my project page is static html once designed by a "homepage builder" (therefore the ugly look) and now updated by hand. I don't know how to implement RSS in this.

[CrazyTerabyte]

If you use some kind of version control system, I'd suggest you to host your project at either http://github.com/ (if you use Git) or http://bitbucket.org/ (if you use Hg). I personally prefer Hg (AKA Mercurial), and I think bitbucket has a bit more features than github; but after all git or hg is just a personal preference.

If you don't use any version control system... Well... You should.

[fangorn]

Thank you, but for me saved versions work great. If I would have co-developers, I would use something, but for me alone it is just overkill.

[CrazyTerabyte]

Thank you, but for me saved versions work great. If I would have co-developers, I would use something, but for me alone it is just overkill.

I used to think like that... (really, that's true!) Until I finally realized how good is to have a VCS, and then I started using one on every project, even the smaller ones.

Notice that Hg and Git (and Bzr) are distributed, and you can use them even without an external server. Heck, they are trivial to setup, just one command! I agree that SVN is an overkill, but maybe you should try one of the distributed VCS.

(don't worry, this is the last time I'll suggest this - I won't annoy you any more)

[Korhul]

avi2mkv fails for me with this message:

Code: Select all

syntax error at /usr/local/bin/avi2mkv line 920, near ") {"
Global symbol "$AUDIOID" requires explicit package name at /usr/local/bin/avi2mkv line 922.
Global symbol "$AUDIOID" requires explicit package name at /usr/local/bin/avi2mkv line 923.
syntax error at /usr/local/bin/avi2mkv line 924, near "}"
Execution of /usr/local/bin/avi2mkv aborted due to compilation errors.

[fangorn]

Do you use the most recent version?

If yes, try an older version (sorry, no link yet, but the directory is openly accessible).

Anyway, this should not happen as I use avi2mkv all the time, so at least it should start.

[Brad1213]

Hi

The latest version of avi2mkv fails, for me aswell.

Code: Select all

avi2mkv -p vdhq -c title02_chapters.xml -i -t title02 title02.avi title02_first.dts title02_eng.ac3
syntax error at /usr/local/bin/avi2mkv line 920, near ") {"
syntax error at /usr/local/bin/avi2mkv line 924, near "}"
Execution of /usr/local/bin/avi2mkv aborted due to compilation errors.

Version http://progs.fangornsrealm.eu/avi2mkv/avi2mkv_1.9.99 works fine.
Version 2.0.0, 2.0.1 and 2.0.3 all fail with the same error.

Brad

[bigdweeb]

I'm getting the same error. I started tracing this back to try and fix it.

I am running the following command:

Code: Select all

riptoh264 -p uhq -n Bolt

and I get the same error as above.

The problem is happening in the sub avi2mkv_audio_test_codec () subroutine. The problem is that $AUDCODEC being used in the switch statement on line 920 is NULL. $AUDCODEC is being set on line 918 as the lowercase version of $temp, but that variable is also NULL. $temp is being set in the if/else statement above this, and I found the if was being triggered in my case. In that if statement, $temp is being set by grep'ing through the the @ret array, but when I printed out the contents of the array I found this:

Code: Select all

The aid option must be an integer: -identify
 Error parsing option on the command line: -aid
 MPlayer SVN-r30554-4.3.4 (C) 2000-2010 MPlayer Team

I went back one line to 904 where @ret gets set by running the contents of $command. The contents of $command are:

Code: Select all

/usr/bin/mplayer  Bolt.avi  -aid   -identify -frames 0 -vc null -vo null -ao null -nocache 2>&1

So it looks like the contents of $AUDIOSRC that gets passed to this function needs to include "-aid" plus an integer. I'm not sure what is calling this function yet though.