Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Script for turning DVD into mkv's
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Multimedia
View previous topic :: View next topic  
Author Message
Nicias
Guru
Guru


Joined: 06 Dec 2005
Posts: 446

PostPosted: Wed Jul 03, 2013 2:47 pm    Post subject: Script for turning DVD into mkv's Reply with quote

Hello,

I'm in the process of moving my dvd library to harddisk. It has TV shows and movies. I have worked out what I think is a good script and I wanted to see if anyone had any suggestions for improvements. The script is run automatically up insertion of a DVD. /var/tmp/dvd-import is on one the system drive. /var/srv/media is a second shared drive and /tmp is tmpfs (~2G)
Code:
#!/bin/bash

TMPDIR=/var/tmp/dvd-import/
SUBDIR=/tmp/sub-import/

DISC=`/usr/bin/lsdvd | /bin/grep Disc | /bin/sed 's/Disc\ Title:\ //'`

DESTDIR=/var/srv/media/videos/holding/ripping/`/bin/date +%F-%a-%R| /bin/sed 's/:/-/'`-${DISC}

/bin/date

/bin/mkdir -p ${DESTDIR}
/bin/mkdir -p ${SUBDIR}
/bin/mkdir -p ${TMPDIR}

rawtitles=`/usr/bin/lsdvd -q | /bin/grep -v 00:0[0-9]:[0-9][0-9] | /bin/grep Length | /usr/bin/sort -k4,2 `
/bin/echo "${rawtitles}"
titles=`/bin/echo "${rawtitles}" | /usr/bin/uniq -f3 | /usr/bin/cut -b 8-9  | /bin/sed 's/^[0]*//'`

for title in $titles
   do
   /bin/echo title = $title
                listing=`/usr/bin/lsdvd -qast ${title}`
                /usr/bin/dvdxchap -t ${title} /dev/dvd > ${SUBDIR}/${DISC}-${title}.chap
                VOBNAME=${TMPDIR}/${DISC}-${title}.vob
      /usr/bin/mplayer dvd://${title} -dumpfile ${VOBNAME} -dumpstream
      SUBOPTS=" "
      subtracks=`/bin/echo "${listing}" | /bin/grep Subtitle | /usr/bin/cut -d":" -f2,3 |\
            /bin/sed "s:,::g" | /usr/bin/cut -d" " -f2,4  | /bin/sed "s: :-:" `
      for subtrack in ${subtracks}
      do
         set -- `/bin/echo $subtrack | /bin/sed -e "s:-: :" -e "s:xx:en:"`
         N=$(($1-1))
         L=`/bin/echo $2 | /bin/sed "s:xx:en:"`
         /bin/echo "Extracting subtitle track ${N} of language ${L}"
         /usr/bin/mencoder -really-quiet ${VOBNAME} \
            -nosound -ovc copy -o /dev/null \
            -vobsubout ${SUBDIR}/${DISC}-${title}-SUBS-${N} -sid ${N} \
            -vobsuboutindex ${N} -vobsuboutid ${L}
         if [ -s ${SUBDIR}/${DISC}-${title}-SUBS-${N}.sub ]
         then
            SUBOPTS=" ${SUBOPTS} --default-track 0:0 --language 0:${2} \
               =${SUBDIR}/${DISC}-${title}-SUBS-${N}.idx \
                =${SUBDIR}/${DISC}-${title}-SUBS-${N}.sub "
         fi
      done
      AUDOPTS=" "
      audiotracks=`/bin/echo "${listing}" | /bin/grep Frequency | \
                                        /usr/bin/cut -d":" -f2,3 | /bin/sed "s:,::g" | \
                                        /usr/bin/cut -d" " -f2,4  | /bin/sed "s: :-:" `
      for audiotrack in ${audiotracks}
      do
         set -- `/bin/echo $audiotrack | /bin/sed -e "s:-: :" -e "s:xx:en:"`
         L=`/bin/echo $2 | /bin/sed "s:xx:en:"`
         AUDOPTS="${AUDOPTS} --language ${1}:${L}"
      done
      AUDOPTS=" -a `/bin/echo "${audiotracks}" | /usr/bin/cut -d"-" -f1 \
                    | /usr/bin/tr "\n" "," | /bin/sed -e  's/\(.*\),/\1/g'` ${AUDOPTS} "
      /usr/bin/mkvmerge --default-language en -o ${DESTDIR}/${DISC}-${title}.mkv \
         --engage no_simpleblocks -S ${AUDOPTS} =${VOBNAME} ${SUBOPTS} \
                        --chapters ${SUBDIR}/${DISC}-${title}.chap
done

/bin/rm -rf ${TMPDIR}/*.vob
/bin/rm -rf ${TMPDIR}/*.mkv
/bin/rm -rf ${SUBDIR}/*.sub
/bin/rm -rf ${SUBDIR}/*.idx


/bin/chown -R me:mine ${DESTDIR}
/usr/bin/eject
/bin/date

Any thoughts?


Last edited by Nicias on Tue Jul 30, 2013 10:41 pm; edited 5 times in total
Back to top
View user's profile Send private message
DawgG
l33t
l33t


Joined: 17 Sep 2003
Posts: 866

PostPosted: Thu Jul 04, 2013 9:37 am    Post subject: Reply with quote

Nice script!
But i think there might be problems with the subtitles in relation to the video-format: afaik mkv-files cannot contain subs in vobsub-format and if you dump a video-stream (as vob) with mplayer there is no way to (de)activate the subs in it.
also, if there is no transcoding of the streams then there is no great advantage in this conversion over just decrypting and dumping the main titles of the dvds as isos or dvd-structures.
have you checked out media-video/shrip? in some cases i found it quite useful.
GOOD LUCK!
_________________
DUMM KLICKT GUT.
Back to top
View user's profile Send private message
Nicias
Guru
Guru


Joined: 06 Dec 2005
Posts: 446

PostPosted: Thu Jul 04, 2013 3:21 pm    Post subject: Reply with quote

Thanks for the feedback.

All the references I saw did want me to do OCR on the vobsubs, but that seemed like too much work, plus according to mkvmerge's website it does support vobsubs.

In terms of not being able to separate out the subs from the vobs, it does actually seem to work.

I have a separate script that does the transcoding to x264 but I run that in batches.

The advantages are:
  • This breaks up the episodes. If it is a DVD of a TV show it will typically have multiple episodes on a disk, this breaks them out as separate mkv's.
  • This removes any trailers you would be forced to watch.
  • The resulting files are more compatible, my set-top-box sometimes chokes on dvd iso's.


I'll have a look at shrip.
Back to top
View user's profile Send private message
HolgerB
n00b
n00b


Joined: 07 Sep 2011
Posts: 49

PostPosted: Fri Jul 05, 2013 11:44 am    Post subject: Reply with quote

DawgG wrote:
Nice script!
But i think there might be problems with the subtitles in relation to the video-format: afaik mkv-files cannot contain subs in vobsub-format and if you dump a video-stream (as vob) with mplayer there is no way to (de)activate the subs in it.

I can only second it..nice script.

MKV-files can include native vobsub files just fine. Simply check out the manpage of mkvmerge :)
Back to top
View user's profile Send private message
Nicias
Guru
Guru


Joined: 06 Dec 2005
Posts: 446

PostPosted: Fri Jul 05, 2013 5:00 pm    Post subject: Reply with quote

HolgerB wrote:
DawgG wrote:
Nice script!
But i think there might be problems with the subtitles in relation to the video-format: afaik mkv-files cannot contain subs in vobsub-format and if you dump a video-stream (as vob) with mplayer there is no way to (de)activate the subs in it.

I can only second it..nice script.

MKV-files can include native vobsub files just fine. Simply check out the manpage of mkvmerge :)


Thanks, I updated it slightly to deal with a problem I was having with mkvmerge, ffmpeg, and lsdvd disagreeing about the order of some audio tracks.
Back to top
View user's profile Send private message
The Doctor
Moderator
Moderator


Joined: 27 Jul 2010
Posts: 2678

PostPosted: Sat Jul 06, 2013 12:53 am    Post subject: Reply with quote

Looks very interesting, however I keep getting an error when I try to use it. line 36:
Code:
syntax error near unexpected token `<'


The line is
Code:
done < <( /usr/bin/lsdvd -q -s -t ${title} | /bin/grep Subtitle | /usr/bin/cut -d":" -f2,3

_________________
First things first, but not necessarily in that order.

Apologies if I take a while to respond. I'm currently working on the dematerialization circuit for my blue box.
Back to top
View user's profile Send private message
Nicias
Guru
Guru


Joined: 06 Dec 2005
Posts: 446

PostPosted: Sat Jul 06, 2013 1:58 pm    Post subject: Reply with quote

The Doctor wrote:
Looks very interesting, however I keep getting an error when I try to use it. line 36:
Code:
syntax error near unexpected token `<'


The line is
Code:
done < <( /usr/bin/lsdvd -q -s -t ${title} | /bin/grep Subtitle | /usr/bin/cut -d":" -f2,3


Some of the lines are too long for the forum. That should be one line until the next ).
Back to top
View user's profile Send private message
The Doctor
Moderator
Moderator


Joined: 27 Jul 2010
Posts: 2678

PostPosted: Sat Jul 06, 2013 8:43 pm    Post subject: Reply with quote

Strange. It still gives me the same error.
_________________
First things first, but not necessarily in that order.

Apologies if I take a while to respond. I'm currently working on the dematerialization circuit for my blue box.
Back to top
View user's profile Send private message
Nicias
Guru
Guru


Joined: 06 Dec 2005
Posts: 446

PostPosted: Sat Jul 06, 2013 10:13 pm    Post subject: Reply with quote

I don't know what to say, look carefully at the script making sure you understand all the steps. If you do you should be able to see if a line is broken incorrectly or something like that.
Back to top
View user's profile Send private message
The Doctor
Moderator
Moderator


Joined: 27 Jul 2010
Posts: 2678

PostPosted: Sat Jul 06, 2013 10:57 pm    Post subject: Reply with quote

Thanks for the help. It looks like line 32 had a break in it. I suppose I should have seen it earlier, but my bash is fairly rudimentary. The script now runs, except for the snag of a few missing tools (currently compiling).

Thanks for posting your script and helping me to understand it. Its a an excellent solution and I know many people will appreciate having such a well thought out script on-line.

EDIT: The script does work perfectly once the line breaks are removed.
_________________
First things first, but not necessarily in that order.

Apologies if I take a while to respond. I'm currently working on the dematerialization circuit for my blue box.


Last edited by The Doctor on Sun Jul 07, 2013 8:31 am; edited 1 time in total
Back to top
View user's profile Send private message
Nicias
Guru
Guru


Joined: 06 Dec 2005
Posts: 446

PostPosted: Sat Jul 06, 2013 11:37 pm    Post subject: Reply with quote

No problem.

If you have any other question I'll try and help, but I'm not an expert so I might not be able to have a pat answer. I also have a script that I'm finishing up that transcodes to x264 and strips non-english subtitles and audio tracks I'll put that up when I am happy with it. I wish there were a better way to post a script than in a [code] [/code] block since that messes up the linebreaks etc.
Back to top
View user's profile Send private message
The Doctor
Moderator
Moderator


Joined: 27 Jul 2010
Posts: 2678

PostPosted: Sat Jul 06, 2013 11:43 pm    Post subject: Reply with quote

Nicias wrote:
I wish there were a better way to post a script than in a [ code ] [ /code ] block since that messes up the linebreaks etc.


I think that code blocks and a pastebin may be a good way.
_________________
First things first, but not necessarily in that order.

Apologies if I take a while to respond. I'm currently working on the dematerialization circuit for my blue box.
Back to top
View user's profile Send private message
Nicias
Guru
Guru


Joined: 06 Dec 2005
Posts: 446

PostPosted: Sun Jul 07, 2013 12:23 am    Post subject: Reply with quote

Well, if pastebin would help http://pastebin.com/jT2jsK5d (updated)

Last edited by Nicias on Tue Jul 09, 2013 3:56 pm; edited 1 time in total
Back to top
View user's profile Send private message
Nicias
Guru
Guru


Joined: 06 Dec 2005
Posts: 446

PostPosted: Tue Jul 09, 2013 3:56 pm    Post subject: Reply with quote

I was having trouble with a couple of things, and so updated the script.

  • "unknown" audio/subtitle tracks were causing trouble, so I labeled them as "en", so that they would be automatically kept by a later script that strips the unneeded audio and subtitle tracks.
  • Sometimes it would fail to loop over all of the titles. I tracked this down to something about how the 'while read' structure treats stdin. I got rid of that structure and used only for loops.
  • I wasn't initializing the SUBOPTS and AUDOPTS properly. Now I do.


The second and third problems weren't showing up when I was testing the script mostly on movie disks which only have one worthwhile title. Sorry for any trouble that anyone had sorting through my wibbly-wobbly script.
Back to top
View user's profile Send private message
matrix47
n00b
n00b


Joined: 20 Jan 2008
Posts: 21
Location: Moscow, Russia

PostPosted: Thu Jul 11, 2013 6:41 pm    Post subject: Reply with quote

There's a pretty cool tool for exactly that purpose: media-video/makemkv
_________________
May the force be with you!
Back to top
View user's profile Send private message
Nicias
Guru
Guru


Joined: 06 Dec 2005
Posts: 446

PostPosted: Thu Jul 11, 2013 7:53 pm    Post subject: Reply with quote

It seems like that isn't a command line utility. This machine is headless.
Back to top
View user's profile Send private message
HolgerB
n00b
n00b


Joined: 07 Sep 2011
Posts: 49

PostPosted: Sun Jul 21, 2013 9:56 pm    Post subject: Reply with quote

makemkv sports both a GUI and a commandline version.

The commandline version is called makemkvcon

Have a look here:
http://www.makemkv.com/developers/usage.txt
Back to top
View user's profile Send private message
Nicias
Guru
Guru


Joined: 06 Dec 2005
Posts: 446

PostPosted: Sun Jul 21, 2013 11:45 pm    Post subject: Reply with quote

It still requires a lot of gui packages. Maybe I'll look into it later.
Back to top
View user's profile Send private message
HolgerB
n00b
n00b


Joined: 07 Sep 2011
Posts: 49

PostPosted: Mon Jul 22, 2013 7:55 am    Post subject: Reply with quote

The question that remains for me is if you really gain any benefits from this.

Your shell script is more transparent and has less external dependencies.
Back to top
View user's profile Send private message
Nicias
Guru
Guru


Joined: 06 Dec 2005
Posts: 446

PostPosted: Tue Jul 30, 2013 10:47 pm    Post subject: Reply with quote

I tried out makemkv and it can do what I want. Looks like it would be a pain to just rip what I want, but I think I could get it to work. However, I like my script and it's working for me. The packages for makemkv account for about 10% of my installed packages, so I think I'll stick with what I have now.

I update the script above and on pastebin with some further improvements.
  • Only run lsdvd twice (versus like 5 times or so)
  • A better method of keeping the audio tracks in order. If you just copy them all they end up in the wrong order. If you specify them all with, for example -a 1,2,3, then they end up in the right order. Go figure.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Multimedia All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum