Blu-Ray (on gentoo) how-to [solved with MakeMKV, WIP]

[Tariella]

Edited for better overview

There is a method for automated ripping. Look at the part about MakeMKV.

The manual version:

What you need for BluRay on gentoo (or any other linux):
[*] A BluRay drive for which the patch here is usable (my LG GGW-H20L works great).
[*] The newest version of the software AACSkeys from the doom9 forums.
[*] The newest version of the software dumpHD from the doom9 forums.
[*] Or instead of the first two points: The VolumeUniqueKey (VUK) of the disc you want to play
[*] Your kernel should be version 2.6.25 and higher for UDF 2.5 support, else you won't be able to read the disc's filesystem.

UPDATE:
BD+ can now be automatically decoded using dumpHD.
Please notice that it won't work for all discs. More info about the newest dumpHD release here.
This also works quite fine in streaming mode.

What doesn't work right now:
For the newest discs check the doom9 forums. It takes some time until the latest MKB version keys are out.

Here is what to do:

Note: You can skip point one and two if you already have the VolumeUniqueKey (VUK) of the disc you want to dump/watch. Also you'll need the VUK if your drive is not patched.

First of all you should mount your BD drive Make sure you can see the files, otherwise check your kernel version and configuration (File Systems -> UDF).

On the mounted disc look for a BDSVM directory to verify if is a BD+ disc. Chances are great that it won't work then.

1) Patch your drive with the AACS patch linked above. It should work with wine, so no Win is necessary.

2) Download AACSkeys and extract into a folder of your choice. It contains the program for both 32 and 64 bit and a library file that can be used with dumpHD directly - no need to run AACS manually any more, just put the .so file into your dumpHD directory. There also is a file named ProcessingDeviceKeysSimple.txt. It contains the processing keys needed to decrypt the media. There are different MKB Version and each has its own key. Newer keys can be found on the internet (the doom9 forums are the place to go, or look here). Put additional/newer keys into this file, if needed.

3) Download and extract dumpHD into a folder of your choice. Since its in java you dont need to install it.
There are now two possibilities:
If you followed step one and two make sure that the AACSkeys library files and the ProcessingDeviceKeysSimple.txt is in the dumpHD folder.
If you skipped step one and two edit the file keydb.cfg where all known keys are stored. You can find keys for HD and BluRay media in the doom9 forums (won't link there for legal reasons, but it should be fairly easy to find anyway).

So then you can start dumpHD (use the shell-script for gui mode).
You can either dump your disc to a folder of your choice to edit it further or you can use dumphd in commandline mode to stream the movie directly to your media player (might give performance issues - since there are no drivers yet to support GPU based decoding).
example for streaming to mplayer:

Code: Select all

./dumphd.sh /media/cdrom1 | mplayer -cache 8192 -demuxer lavf -aid 1 -channels 6 -

Playback of the dumped files (works also with streaming):

You'll need the newest SVN version of mplayer and ffmpeg (they can be emerged easily using the berkano-overlay).
The mplayer options should work for both Blu-Ray and HD-DVD.

The following mplayer-options are originally from a guide for mythtv:

Code: Select all

mplayer -fs -vo xv -demuxer lavf -ac <audiocodec> -aid <audiotrackid> -fps 24000/1001 <moviefile>

-fs fullscreen
-vo xv to use Xv as for video-output
-demuxer must always be lavf
-fps 24000/1001 this is for the 24p movie format of all blurays

to find out audiocodec and track run

Code: Select all

ffmpeg -i <moviefile>

for liba52 use -ac ffac3 (this is [e]ac3 format)
for mlp use -ac mlp (this is MLP/trueHD format)

If mlp is not recognised copy /etc/mplayer/codecs.conf to ~/.mplayer/codecs.conf and add the following:

Code: Select all

audiocodec mlp
  info "FFMpeg mlp"
  status buggy
  format 0x20504C4D
  driver ffmpeg
  dll "mlp"

When experiencing slow playback, you can experiment with one or more of the following options:

Code: Select all

-lavdopts threads=2:fast:skiploopfilter=all -sws 0 -framedrop -cache 8192 -autosync 30

-autosync probably won't help here because there is no sync information in the most movies, ymmv
Also you can try and nice your mplayer to -20.

If you encounter audio/video sync issues:
When mplayer is struggling, do you get "Too many video packets in the buffer" console output? If so, in libmpdemux/demuxer.h change:

Code: Select all

#define MAX_PACK_BYTES 0x800000

to:

Code: Select all

#define MAX_PACK_BYTES 0x8000000

and recompile mplayer.

Thanks for this tip goes to grant123 who found this info on the doom9 forums.

Recoding your dumped movie:
This part of the guide is using information provided by Fangorn.

mencoder does AFAIK still NOT recognise .m2ts containers. So a direct handling will not happen!

For use with mencoder you have to do (after ripping the to .m2ts files)
1. identify video (easy, usually its the only one) and audio streams (not so easy, I had best results with plain AC3, if there is only eac3 strip it down to ac3 with tsMuxer, the same is possible for DTS HD)
2. copy the necessary streams into a MPEG transport stream (.ts) using tsMuxer or any other capable app.
3. run the script
4. mux the .avi (contains video and first audio stream) with additional audio into .mkv container using mmg

ad 4. If you don't need a second audio stream you could use the .avi file directly. But I found it less difficult to handle (fastforward, pause, ...) and more stable (audio sync) when in a .mkv container.

Here is a link to fangorn's transcoding scripts. It is now a seperate thread.

alternate method (more automated) ripping - MakeMKV:
MakeMKV is a program that creates MKV files from bluray movie discs. The ebuild can be found in the multimedia overlay.
It doesn't need a patched drive and works with the latest BD+ versions.

A note on subtitles:
There is a java program that can turn .sup subtitle format into .sub subtitles (which you can easily use with your player): BDSup2Sub. It is able to run on Linux.

Thanks to all who helped out with research and providing solutions (especially Fangorn for transcoding and beandog for the MakeMKV ebuild).

Moderator note: Stickied! -- 27 Feb 2009, beandog

[beandog]

I havent looked at the links yet, but I'm pretty sure there's no software to actually rip a Blu-ray disc for Linux right now. Windows, yes.

As far as playback for the actual movie (once ripped), just install MPlayer on a fast box (at least dual-core, 3ghz or so) with the right use flags and you should be able to play it back without problem.

If you do figure something out though, let us know, I'm equally curious about getting it working.

[Gusar]

1) To read the filesystem on the discs I seem to need the UDF 2.5 patch for the kernel. Can I use it to patch the gentoo-sources or should I use the vanilla-sources? Would it be useful to use the already patched zen-sources?

Either of these is fine I'd say. Gentoo-sources doesn't contain many patches, so the UDF patch should apply without conflicts.

2) Since there are loads of overlays: Which one ist containing the newest svn version of mplayer and ffmpeg? Are the needed versions even in portage now?

The berkano overlay has the newest ebuilds. Make sure you activate the eac3 and truehd USE flags.

3) Anybody already tried the dumpHD program from the doom9 forums? Is it usable in gentoo? Is there an ebuild anywhere?

No idea about ebuilds, but it should work on gentoo provided you have java installed. It's a self contained package anyway, so no real need for an ebuild.


That all said, you may still have a problem: BD+. It hasn't been cracked yet by the open-source community. So the only discs you'll be able to rip are those without BD+ "protection" and then only if a key is available for them or 'aacskeys' can obtain one.


@beandog: DumpHD is cross-platform and works with both hd-dvd and blu-ray, with the caveats mentioned above.

[Tariella]

Thanks for your answers!

I'll report back as soon as there are positive results or new questions.

@beandog:
My second link is about a tool named dumpHD made by someone in the doom9 forums.
In the last link of my previous post there is a little "guide" on using it in linux.
I'ts claiming to rip (even copy protected?) hd and blu-ray media.

It will take some time until I'm able to check it out since I'll have to patch my kernel first and install svn-mplayer and ffmpeg.

[yngwin]

add udf 2.5 support to my box for reading and writing data (by patching the kernel?)
So here are my questions:
1) To read the filesystem on the discs I seem to need the UDF 2.5 patch for the kernel. Can I use it to patch the gentoo-sources or should I use the vanilla-sources? Would it be useful to use the already patched zen-sources?

I am not sure if zen-sources contains the udf-2.50 patch, but hitchhiker-sources definitely do. The hitchhiker-sources are the latest gentoo-sources patched with reiser4 and udf-2.50 support, available, just like the mplayer and ffmpeg subversion ebuilds from the berkano overlay.

[beandog]

As of today I'm owner of a Blu-ray device capable of reading HD-DVD and Blu-ray discs and burning Blu-ray media.

btw, what kind of hardware did you get?

[jesso]

You could try nerolinux. it supports blu-ray and hddvd on linux.

[Tariella]

btw, what kind of hardware did you get?

My device is a LG GGW-H20L. It can read hd-dvd and blu-ray and write all blu-ray formats (SL/DL, BD-RE).

[Tariella]

Update:
I emerged hitchhiker-sources and now I can read the filesystem of my first hd-dvd.

Next step ist to experiment with dump-hd.
Mplayer svn version refuses to merge from berkano overlay. I'll patch it manually.

[yngwin]

The mplayer-2008.0103 snapshot from berkano overlay should work.

[Tariella]

It won't compile:

Code: Select all

{standard input}: Assembler messages:
{standard input}:6853: Error: no such instruction: `pabsw %xmm0,%xmm0'
{standard input}:6854: Error: no such instruction: `pabsw %xmm1,%xmm1'
{standard input}:6855: Error: no such instruction: `pabsw %xmm2,%xmm2'
{standard input}:6857: Error: no such instruction: `pabsw %xmm3,%xmm3'
{standard input}:6859: Error: no such instruction: `pabsw %xmm4,%xmm4'
{standard input}:6861: Error: no such instruction: `pabsw %xmm5,%xmm5'
{standard input}:6863: Error: no such instruction: `pabsw %xmm6,%xmm6'
{standard input}:6865: Error: no such instruction: `pabsw %xmm7,%xmm7'
{standard input}:7239: Error: no such instruction: `pabsw %xmm2,%xmm2'
{standard input}:7241: Error: no such instruction: `pabsw %xmm3,%xmm3'
{standard input}:7243: Error: no such instruction: `pabsw %xmm4,%xmm4'
{standard input}:7245: Error: no such instruction: `pabsw %xmm5,%xmm5'
{standard input}:7251: Error: no such instruction: `pabsw %xmm2,%xmm2'
{standard input}:7253: Error: no such instruction: `pabsw %xmm3,%xmm3'
{standard input}:7255: Error: no such instruction: `pabsw %xmm4,%xmm4'
{standard input}:7257: Error: no such instruction: `pabsw %xmm5,%xmm5'
{standard input}:21598: Error: no such instruction: `pmulhrsw %mm5,%mm0'
{standard input}:21599: Error: no such instruction: `pmulhrsw %mm5,%mm1'
{standard input}:21641: Error: no such instruction: `pmulhrsw %mm5,%mm0'
{standard input}:21642: Error: no such instruction: `pmulhrsw %mm5,%mm1'
make[1]: *** [i386/dsputil_mmx.o] Fehler 1
make[1]: Leaving directory `/var/tmp/portage/media-video/mplayer-2008.0103/work/
mplayer/libavcodec'
make: *** [libavcodec/libavcodec.a] Fehler 2

I already posted about this in the berkano-overlay thread.

[nutbar21]

Seems that the ebuild disagrees with your hardware setup or something. Have you tried just manually building mplayer straight out of svn? It's not 100% desirable since you can't just unmerge it later but if you write down where you install it there won't be a problem getting rid of it later if you want to.

I'm _very_ interested in seeing how far you get with this. I was wondering about this process myself about two weeks ago (can't afford a Blu-Ray drive yet though or I would be trying the same process as you right now).

[Tariella]

Ok, status update:
Someone over in the berkano-overlay thread posted his use-flags and I was able to compile with them. I'll analyze the use-flags later to find out which one was the troublemaker.

Now the following points are solved:
- kernel with udf 2.5 support (hitchhiker sources from bekano-overlay)
- filesystem of the test hd-disk can be read now
- newest mplayer and ffmpeg emerged (eac3 use-flag is not usable atm)
- searching keys for the test-hd-dvd using aacskeys

next step:
- decrypting the hd-content with dumphd


I did a long research of the doom9 forums and it seems that the following steps will be needed:
(see also the second link in my first post) - Edited for correctness

1) aacskeys package -> for reading the discs keys (HD & blu ray)
1) dumphd package -> for dumping decrypted contend to hdd

Both packages exist in a linux version. (aacskeys worked better for me in the windows version with wine though - see step 1 below).


The whole process of decrypting is working like this (not yet verified, slightly corrected -please correct me if I'm wrong!):

The player software connects to the drive using the mkb* file on the media-disc (which contains some encrypted key data) and the player's certification key and thus acquires the authorisation to talk to the drive (the player must be licensed - New media will supply automatic firmware updates which will eventually revoke the authorisation of the player software if its known to have leaks).
Since there are only 3 different versions of mkb filetypes until now and the processing keys (general keys for all discs of each mkb version) were found, the players certification key is not needed atm.

1)Using the processing keys the aacskeys program/library can get the discs volume id which gives the vuk (volume unique key) needed for decryption.

1)Dumphd will use the aacskeys library or the keydb.cfg file and decrypt the medias contents to the hdd.

A new and patched mplayer should be able to play it from there (svn version with USE-flags eac3 truehd, look at the berkano-overlay).


Here is what I figured out about how it sould work:

Note: Step 1 might be unnecessary if the key of your volume is already in the keydb.cfg file provided by dumphd. Best try dumphd first.

1) aacskeys will be able to read the discs contents with the file ProcessingDeviceKeysSimple.txt which is providedin the package. The latest version contains a 32bit linux version of the program and the library as well as the source code.

you should be able to compile it from source the following way:
forum post explaining compilation

Note: I wasn't able to create a usable 64bit library for the use with dumphd from the provided sources. Maybe someone can post the correct command for this.
Since I wasn't able to use the linux binary (claimed to have a wrong processing key), I tried the win32 exe in wine and it gave me a lot of keys (for the disc and each chapter)

2) Dumphd is written in java and the newest version comes with a shellscript (you'll have to set it executable first).
A file named keydb.cfg comes with it containing already the newest volume keys (the newest version can be found at the doom9 forums - see links)

If your disc's key is not already in the keybd.cfg, you'll have to find it yourself - on the internet or using aacskeys from step 1.

If you have the libaacskeys.so in the dumphd directory it will try to find the key to your disk automatically.

If you use the aacskeys program manually, you'll have to edit the database file named keybd.cfg containing the key, so that dumphd can use it.
A program capable of doing this for you can be found here. Its written in java and should run on linux too.

Once dumphd is provided with the key it will dump the decrypted contents of your hd- or blu-ray-media to the hdd. The program is using a gui frontend which should be self-explanatory.

ATTENTION: Not all of these steps are yet verified by me - If this kills your drive, your computer, your pets or your relatives I won't be responsible

Other useful links (not checked out by me yet):
Blu ray region code remover
DecryptHD - claims to decrypt on-the fly without dumping (needs fuse suport)

If someone feels like playing without it before I have tried it - feel free to experiment. I'm interested in results and problems.
Detailed questions about the programs linked here should be posted on the doom9-forums though, since I don't know more about it than I wrote here.

[x1um1n]

Hi,
I'm also trying to sort this out, I'm not bothered about playing BluRay (I've got a PS3 for that) or burning them, but I would like to be able to rip them.

So far, I've got the hitchhiker-sources kernel installed, works perfectly. (I'm running Gentoo 2008.0 btw)

aacskeys-0.2.9:

Code: Select all

x1um1n@omnius:~$ Install/aacskeys_0.2.9/bin/linux/aacskeys /mnt/bdrom
Install/aacskeys_0.2.9/bin/linux/aacskeys: /usr/lib32/libcrypto.so.0.9.8: no version information available (required by Install/aacskeys_0.2.9/bin/linux/aacskeys)
aacskeys v0.2.9

It looks like the given Host Certficate/Private Key has been revoked (by your drive).


ERROR: SENDHOSTCHAL errnr: -1

I'm not sure if the "no version information available" is an issue, but from a little googling, it appears that I need to use dumpVID to "hammer" my drive.
Under wine, this sits like this:

Code: Select all

DumpVID 0.3 by KenD00 (adapted for bluray testing)

Drive type is recognised as CDROM/DVD.

Sending SPC1 Test Unit CDB6 command..done.
Returned good status.

Press ENTER to start hammering

Hammering drive...

and doesn't appear to do anything, I'm leaving it going now in case it gets there eventually. (hopefully without killing either my drive or SATA bus)

Thats where I'm stuck for now. If you can see I've done sumat wrong, please gimme a shout..

[Tariella]

As far as I know dumpvid needs a player-program that can talk to the drive. It then intercepts the communication between the software and the device. That is why I never used it.

I had problems with aacskeys too (under linux) so I used the windows version with wine and it gave me a whole list of keys. Unfortunately media file ripped with this keys is unusable. I checked the found keys with validatevuk (win programm also) and it returned me a "not valid" message.

So I'll wait for my american HD discs - they shoud arrive any day now. Some of them are in the keydb.cfg file of the dumphd program and can be dumped without finding the key first. Then I'll see if I can get it to work (I'll have to get the keys for my other discs somewhere else though).

[x1um1n]

That explains why dumpvid didn't appear to do anything

Did your keydb.cfg come with keys already in it? Mines empty.
Is there somewhere I can find an up-to-date list of keys?

I think my next step (quick fix) will be to try and get xp working under virtualbox/vmware and install AnyDVD HD, but I'm prob not gonna get chance to play with that till next week.

It's not a particularly good option, as it means I've got to run winsucks and also increase the size of my ~ to accomodate at least 1 25Gb BluRay.

I'll let you know how it goes.

Thanks

[Tariella]

I got my dumphd keydb.cfg file from this doom9 forums thread (I found the link to this thread in the Ubuntu guide linked in my first post).

I'm glad that I'm not the only one fiddling around with it. Keep your results incoming - even if there is no success, we can still work on this together. The more brains involved the better

My next step will be to try to get the aacskeys linux version running. I will have to compile the included source-code to a library that can be used by dumphd or to a binary that actually gives some results.
The big problem is that I can compile the code with the included makefile, but its not 64bit. Or maybe it is and dumphd is not? Anyways it complains about wrong ELF class/length or something like this.
Helpful ideas are welcome

[x1um1n]

I couldn't get aacskeys to compile either, but the prebuilt binary worked for me.

Are you using a multilib profile, or straight 64bit?
I'm using 2008.0_beta2 multilib if that helps any..

[Gusar]

aacskeys won't be much help, because the host key it uses has been revoked. So you need to provide it a VID. For that, you need DumpVID, which requires a licensed player, which in turn requires Windows (and you also need to buy said player). If the key isn't in the doom9.org thread, you're pretty much screwed.

Welcome to the wonderful world of DRM.

[x1um1n]

Where's dvdJon when you need him???

[Tariella]

I'm using the profile default-linux/amd64/2007.0, afaik it supports multilib.
Since I'm always using portage I'm not very firm in compiling by hand

I thought aacskeys works with the processing key that was found some time before (a processing key is a "general"-key for all devices afaik). Is it already revoked?
I couldn't find any comments on it in the aacskeys doom9 forums-thread.

Where did you hear/read that it is revoked? (And how could my drive already know about it when the disc I inserted should be some months old)?

Well the next key will surely be found soon. Until then I'll have to get the keys of my discs from my win-using friends

I'll still try to get it to work

Where's dvdJon when you need him???

I guess they sent him to prison for cracking css

[Gusar]

I thought aacskeys works with the processing key that was found some time before (a processing key is a "general"-key for all devices afaik). Is it already revoked?

There are many keys involved in AACS. Each file on the disc is encrypted with a Title Key. To get to the Title Keys, you need the VUK (Volume Unique Key). To get the VUK, you need the Processing Key and the VID (VolumeID). You can hammer a licensed player for the VID or you calculate it with a Host Key. Processing Keys are also limited to a specific MKB (Media Key Block). We only have Processing Keys for MKBv1 and MKBv3, but not for MKBv4 and later. Each movie has a specific MKB and the version gets upped at each revocation round (the first revocation round was April 21st 2007, so all movies released before have MKBv1 and movies released after have MKBv3. Don't know when exactly the next round that brought MKBv4 was, sometime early this year, and there was another round a short while ago).

Aacskeys is using Processing Keys for MKBv1 and MKBv3 and the Host Key from PowerDVD 7.x. It is this Host Key that gets revoked by newer discs. So, you can use aacskeys normally, but as soon as you put in a disc new enough to have the PowerDVD 7.x Host key on it's revocation list (that would be MKBv4 discs I think), aacskeys looses the ability to calculate the VID forever. Your only chance now is to provide the VID directly - which we can't do, because we need a licensed player running on Windows to get it. There are people who have gotten VUKs directly from some players by other means and posted VUKs for MKBv4 discs on doom9.org, but they haven't shared that info with the public.

Now if all that seemed complicated... that's just the tip of the iceberg and it's quite possible that I've made a mistake somewhere. There's the whole revocation process with Subset-Difference and a bunch of stuff that goes way over my head. Then there are Segment Keys which aren't in use yet.



AND (yes, big and) this only takes care of AACS. Newer Blu-ray discs (MKBv4 and MKBv7 - yes, they're skipping numbers) also have BD+. BD+ is code running in a Virtual Machine that runs on the player. The only ones to crack that so far are Slysoft. They could do it, because their AnyDVD is a commercial product and they use the money they charge for it to buy lots of drives and movies to test on.

I can only repeat: Welcome to the wonderful world of DRM.


Oh, and how can your drive know about revocations? They're stored in the MKB and this info is added to the firmware of your drive. So, if you want to use aacskeys, beware of MKBv4 and later discs - for x1um1n it's too late already.

[x1um1n]

Looks like the best way at the mo is to use VirtualBox +xp +AnyDVD HD.

I've setup VirtualBox last night, easy enuff.
I'm going away for a few days, but hopefully by this time next week I'll have AnyDVD running on it and I'll post back with the results.

Its a pain, because it means having Windoze installed and either getting a cracked copy of AnyDVD or paying £70 ish.

[Tariella]

Thanks for the clarification.

As far as I understand it is "only" the host key that seems to be revoked for x1um1n and not the device key of the drive itself?

So with a new host key or processing key it would be possible to use aacskeys again?
If there is no new host key or processing key found it would still be possible to use the VUK to dump the media's contents - did I get this right?

I'm still a bit confused about the different sorts of keys but I guess I'm beginning to grab the concept.

I'll try to get my disc's VUKs from other sources than aacskeys and check the steps of dumping and playing. Maybe that will be more successful.

[Gusar]

As far as I understand it is "only" the host key that seems to be revoked for x1um1n and not the device key of the drive itself?

Computer drives don't have device keys, only standalone (hardware) players do. On the other hand, standalones don't have host keys. So: standalone -> device key; software player -> host key

So with a new host key or processing key it would be possible to use aacskeys again?

A new host key alone would be enough for MKBv1 and MKBv3. For other MKBs we also need new processing keys (one for each MKB).

If there is no new host key or processing key found it would still be possible to use the VUK to dump the media's contents - did I get this right?

Yes, the VUK is sort of the holy grail. All other keys serve the purpose of getting to the VUK. With it, you can decrypt the movie.

I'm still a bit confused about the different sorts of keys but I guess I'm beginning to grab the concept.

It's a convoluted scheme they've come up with. They just forgot the basic rule of DRM - at some point the key is available somewhere (the first methods simply scanned a memory dump of the machine while a movie is playing). And in the end, this does nothing to thwart piracy. Only us linux users get screwed over.

I'll try to get my disc's VUKs from other sources than aacskeys and check the steps of dumping and playing. Maybe that will be more successful.

The huge cracking effort at doom9.org seems to have stopped or at least slowed down a lot. There are two reasons: 1.) Being more user friendly, most of the doom9.org community went for HD-DVD, specifically the Xbox360 drive. But now HD-DVD is dead. And 2.) Most of the doom9.org community runs Windows and they seem to be perfectly happy with AnyDVD. Without AnyDVD, there'd be much more motivation to crack BD+