Archives September 2009

New server!

I finally got around to upgrading the server that serves this very page you’re reading to Ubuntu Jaunty today, up from Ubuntu Hardy. Yes I know, maybe I should have waited for Karmic, or even Lucid, but the biggest reason why I did this was that I’ve migrated this server from the little Pentium 4 Shuttle XPC that was in use before onto a Virtualbox 3.0.6 headless VM hosted on an Ubuntu Jaunty box running on top of an Intel E5200 CPU.

You’re probably wondering why I’d use an E5200 when it doesn’t have hardware virtualisation features built in? Well, the server consumes very little juice compared to the Pentium 4 (26% less in fact), it’s more powerful, it’s cheap, cheerful, produces less ambient heat, is a heck of a lot quieter and there’s loads of CPU time left over to do other things outside of the VM on the host side.

CPU wise, when the server gets really busy I’ve seen spikes as high as 50%, but it never exceeds that, so as far as I’m concerned, it’s fine. If I ever need this box to do anything more significant, I’ll upgrade the CPU to something that does have VT-x later on.

If you see anything unusual/missing/dead from today onwards, please let me know in a comment!

HowTo: Configure Ubuntu to be able to use and respond to NetBIOS hostname queries like Windows does

Users in the Windows world are very used to referencing PC’s via their NetBIOS names instead of their IP address. If your PC has a dynamic IP address (DHCP-assigned) of 192.168.0.12 and its hostname (computer name) is “gordon”, Windows users can happily jump into a command line or an Explorer window and ping the name “gordon” which will magically resolve to 192.168.0.12.

If your host is not configured with a Hosts file entry on your local PC or a DNS entry to associate a name with an IP address, Ubuntu can only use the IP address of that PC to communicate with it which means you have to remember what that IP address is with your feeble grey-matter in your head. Likewise, Ubuntu will not respond to a Windows PC pinging its NetBIOS name because Ubuntu does not use NetBIOS at all by default and so it will ignore such requests.

So how do we get Ubuntu to resolve NetBIOS names like Windows? And how can we allow Windows to ping Ubuntu like another Windows PC? Read on…

Let’s illustrate the problem first. You’ll need a Windows PC on your network to test this. For this article, the Ubuntu PC will be called “gordon” and the Windows PC will be called “alyx”.

On either PC, if you open a terminal or Command Line window and ping the opposing machine, eg:

$ ping alyx

or

C:\> ping gordon

You get an error stating that the host cannot be found. Now in the case of Windows, if you were to ping another Windows PC instead of an Ubuntu PC, you can ping its name with no problem.

Let’s sort this out, shall we?

Allowing Ubuntu to ping Windows NetBIOS names

Ubuntu is setup for Linux use, not Windows use, so we need to install a package that will allow Ubuntu to more readily mix in with Windows networks and use NetBIOS. This package is called “winbind”.

  1. Open a terminal and type in the following at the terminal prompt:

    $ sudo apt-get install winbind
  2. Once installed, we need to tell Ubuntu to use WINS (as provided by winbind) to resolve host names. Type in:

    $ sudo gedit /etc/nsswitch.conf

    …which will open the file into the Gnome Editor.
  3. Scroll down to the line that starts with “hosts:”. In Ubuntu Jaunty, it looks similar to this:

    hosts:          files mdns4_minimal [NOTFOUND=return] dns mdns4
  4. Add the word “wins” on the end of this line such that is now looks like:

    hosts:          files mdns4_minimal [NOTFOUND=return] dns mdns4 wins
  5. Save and exit the editor.
  6. Now let’s ping the name of our Windows box again.

    $ ping alyx

    …and it now resolves!
  7. Pat yourself on the back.

Allowing Windows to ping Ubuntu NetBIOS names

This is just one half of the equation. We now need to allow Windows to be able to ping Ubuntu PC’s using its NetBIOS name. This requires Ubuntu to recognise and respond to that request. We need to setup a server daemon to do this. In Ubuntu, this particular server daemon is called Samba.

  1. Installing Samba is simplicity itself. Open a terminal and type in:

    $ sudo apt-get install samba
  2. Once that has finished, your Ubuntu PC will automagically respond to all NetBIOS queries for its hostname straight away, and that’s not just from Windows machines, but other Ubuntu machines (configured with the “winbind” package) as well.
  3. Pat yourself on the back again. Smilie: :)

HowTo: Deal with BD+ copy protection when ripping Blu-ray titles using Ubuntu

A fair while back now, I wrote an article detailing how to decode Blu-ray titles using Ubuntu and an LG GGC-H20L Blu-ray optical drive.

This article detailed how to decrypt just about every movie under the sun except for a newer type of protection called “BD+” which I never got around to supplementing my original article with.

What is “BD+” protection? Well in short, it’s the deliberate corruption of random parts of the video track of the movie (well, OK – that is a highly simplified definition as BD+ protection can do a lot more than that, but the end result is the same – to prevent unauthorised playback which includes ripping). The idea BD+ is that when you rip the title, you can still watch the movie, but with some or all of the screen corrupt at various stages in the movie which well and truely ruins the movie-watching experience, especially since you paid good money for it and should not be forced to buy a dedicated consumer Blu-ray player when you’ve got a perfectly good PC that can do the same task.

But hang on, if the movie is deliberately corrupt, then how come it plays fine in a stand-alone consumer Blu-ray player or PlayStation3 console?

Well, let me tell you about that and how to get around it yourself.

I have to give credit to the movie studios for this one. It’s a simple, and annoying, method of protection. But as with anything, it was eventually reverse-engineered and broken, and neat little tools were developed to allow us consumer types to backup, or watch in our preferred way, our movies bought with our hard-earned cash.

So what’s this BD+ thing all about? Basically after the movie is mastered and just before being pressed to discs, an extra step is taken where by random parts of the movie data stream are deliberately exchanged with random data or removed altogether, thus corrupting the video stream. A record is kept, however, of what parts of the movie have been changed – a table listing where, when and what data needs to be put back into the movie stream in order to watch the movie back in its original uncorrupted format. This table is called a “conversion table”, and it is processed by your Blu-ray player while you watch the movie, with the correct data substituted back into the video stream before the image hits your screen, thus resulting in a proper uncorrupted picture.

An example of a corrupted video stream showing the BD+ Protection in full effect.
An example of the repaired video stream using the Conversion Table.

So how do we get around BD+? Well, all we have to do is follow this conversion table ourselves and correct the corrupted data as the title is decrypted.

As I showed in my previous article, the DumpHD application is brilliant and it has been extended by the author KenD00 to allow the “plugging in” of another program called the “BD VM Debugger”. What this program does is simple – it executes the Java Virtual Machine that runs the conversion table in concert with the normal decrypting process which happens when the disc is played in your normal BD player, patching up the stream as it goes. The end result is a clean decryption with no corrupt video stream.

This tutorial was written using Ubuntu Jaunty but should work with Intrepid and should definitely work with Karmic and beyond as well.

DISCLAIMER: This article describes decrypting BD titles using an Intel or AMD based PC with Ubuntu Linux. At this time of writing you cannot use Ubuntu installed on a PlayStation3 console to deal with BD+ copy protection because the BD VM Debugger and AACS Keys applications are not available for the PPC processor used by the PS3.

So let’s set this up, but first – since my last article, DumpHD has been updated to 0.61 so let’s upgrade this first. Go and download yourself a copy.

  1. Extract the archive out by either double-clicking on it or via the terminal. You should get a “dumphd-0.61″ directory.
  2. If you are upgrading from an older version of DumpHD, copy over the “KEYDB.cfg” file, overwriting the archive copy. No point losing your collection of keys accumulated thus far. Smilie: :)
  3. You’re done for this bit.

The AACSKeys program (which extracts the decryption key for the Blu-ray title and can automatically update your “KEYDB.cfg” file for you when you insert a new Blu-ray title) has also been updated to 0.4.0c since my last article, so go download yourself a copy of that as well.

  1. Extract the archive out by either double-clicking on it or via a terminal. You should get a “aacskeys-0.4.0c”.
  2. Copy the “ProcessingDeviceKeysSimple.txt” and “HostKeyCertificate.txt” into the “dumphd-0.61″ directory.
  3. Copy over the “libaacskeys.so” file located in the “/lib/linux32/” OR “/lib/linux64/” directories (depending on which architecture you’re using) to the “dumphd-0.61″ directory. Do NOT copy or create the “/lib/linux32″ or “/lib/linux64″ directories themselves. Copy the library file only.
  4. You’re done for this bit.

Right, let’s get the BD VM Debugger installed. As of this writing, the current version is 0.1.5. Go and download yourself a copy.

  1. This archive is provided as a 7zip file. Ubuntu does not have out-of-the-box support for this archive format, so install it first with:

    $ sudo apt-get install p7zip-full
  2. Once installed, extract the archive either by double-clicking on it like any normal archive, or via the terminal as follows:

    $ 7z e bdvmdbg-0.1.5.7z
  3. Copy over the everything into the “dumphd-0.61″ directory except the “changelog.txt”, “readme.txt” and “debugger.sh” files since you don’t really need them, but there’s no harm copying them anyway.
  4. That’s it!

You should now have a total of at least of 17 files and two directories inside the “dumphd-0.61″ directory (if you are setting up these tools for the first time, you will only have 15 files instead, as two of them  – conv_tab.bin & hash_db.bin – are generated by DumpHD in conjunction with the BD VM Debugger).

The prepared DumpHD folder with the tools we need.

Now let’s try decrypting a BD+ protected Blu-ray title. In this example, I will use the Australian release of “Day Watch”, the sequel to the Russian epic “Night Watch”.

The BD+ Protected “Day Watch” Blu-ray title I am ripping.

NOTE: Your ability to decrypt a given Blu-ray title, BD+ protected or not, will ultimately depend on the MKB version of the disc. As of this writing, DumpHD can only decrypt up to MKB version 10. Newer discs using version 11 or later can only be decrypted once suitable decryption keys are uncovered and added to the “ProcessingDeviceKeysSimple.txt” file in the “dumphd-0.61″ directory.

The obtaining of the decryption key of the Blu-ray title also requires the player authentication mechanism of your Blu-ray drive to be bypassed, or through use of a drive that deliberately does not have this feature such as some imported drives from China. In the case of my LG GGC-H20L drive, I used a modified firmware so that the drive always gave up the disc’s decryption key regardless of what player certificate I used – blacklisted or not.

  1. Start the DumpHD program by double-clicking on the “dumphd.sh” icon. You will be asked if you want to run the script file. Click on the “Run” button.
Starting the DumpHD application.
  1. When the DumpHD GUI appears, make a note of the messages in the bottom pane to ensure that AACSKeys and the BD VM Debugger was found and loaded OK. You should see the following information:

    DumpHD 0.61 by KenD00
    Opening Key Data File… OK
    Initializing AACS… OK
    Loading aacskeys library… OK
    aacskeys library 0.4.0 by arnezami, KenD00
    Loading BDVM… OK
    BDVM 0.1.5

The DumpHD Interface
  1. Insert the Blu-ray title into your Blu-ray drive.
  2. Next to the “Source” section at the top-right of the DumpHD window is a “Browse” button. Click on it.
  3. Navigate to the path of your Blu-ray drive (generally “/media/cdrom” will work fine). and hit the OK button.Choosing a source to rip from. Click for full size.
Setting up the ripping source
  1. DumpHD will read the disc and will pass it through AACSKeys to identify the title’s descryption key. If it is successful, it will output some data about the disc in the lower pane. In the case of my Day Watch title, it shows the following:

    Initializing source…
    Disc type found: Blu-Ray BDMV
    Collecting input files…
    Source initialized Identifying disc… OK
    DiscID : 73886D08811073F45AD8C75012689097E17EBD3C
    Searching disc in key database…
    Disc found in key database

Identifying the disc and getting the decryption keys to rip with
  1. This is good. We can decrypt this. If the title is not one you have ripped before, you have the option to click on the “Title” button at the top-left of the DumpHD window to give the movie a name in your Key Database.
  2. In the “Destination” section on the right, click on the “Browse” button.
  3. Choose a place to dump the decrypted disc to. Note that most titles will dump at least 20GB worth of data and in some cases 50GB. Ensure that you have enough hard-drive space in the location you choose to dump to.
  4. We’re ready to rock and/or roll. Click on the “Dump” button and decryption will begin, automatically executing the BD VM and applying the Conversion Table to correct the deliberate corruption in the video stream. Here’s a small extract of what you will see in the lower pane of the DumpHD window:

    AACS data processed
    Initializing the BDVM… OK
    Executing the BDVM… OK
    Parsing the Conversion Table… OK
    Processing: BDMV/BACKUP/CLIPINF/00000.clpi
    Processing: BDMV/BACKUP/CLIPINF/00001.clpi
    Processing: BDMV/BACKUP/CLIPINF/00002.clpi
    etc…

Beginning the ripping process
  1. And after awhile it will finish with something like:
    Processing: BDMV/STREAM/00211.m2ts
    Searching CPS Unit Key… #1
    0x0000000000 Decryption enabled
    Processing: BDMV/STREAM/00212.m2ts
    Searching CPS Unit Key… #1
    0x0000000000 Decryption enabled
    Processing: BDMV/index.bdmv
    Disc set processed

Finished decrypting the Blu-ray title.
  1. That’s it! You’ve successfully decrypted the disc and fixed up the corrupted video track. Identify and playback the actual movie M2TS file using a player like MPlayer or VLC, and you should now find that it contains no corruption whatsoever. In the case of Day Watch, the movie file was under BDMV/STREAM/00012.m2ts identifiable simply because it was the largest file in the directory. Using MPlayer, you can play this file with:

    $ mplayer -fs BDMV/STREAM/00012.mt2s

    Thankfully this title does not have the movie broken up into multiple files (I’ll be writing another article soon showing you how to deal with multi-part movies).