Skip to main content

Our media encoding process


It occurred to me that I started posting to debugfs as if I just left off from my old domain (a story for another time). I had something in the order of 300 articles on the site so naturally posts on this site don't have the context they had on the old site.

When I first started debugfs I talked about the Handbrake command-line script I run on our KODI server to shrink the size of Blu-rays (since they can be huge). I didn't really get into the details of the whole process and I've since changed how I rip and encode media.

When I buy a DVD/Blu-ray the first step I take is to back-up the media using MakeMKV. MakeMKV is great for dumping both Blu-ray and DVD content to a .mkv file. I prefer .mkv over .mp4 because I love subtitles and the .mp4 container only lets you "burn" one subtitle into the file. Files in an .mkv container can contain as many subtitles as the DVD/Blu-ray has. I normally rip these on my desktop workstation which has a late 2013 AMD A8-5600K APU with 16GB RAM, a 500GB SSD, and an NVidia GTX 650 Ti Boost video card. I then use Filezilla to SFTP them to the 8TB drive in our i7-2600 (non-K)-based KODI server. The i7-2600 is a much better processor than the A8-5600K with almost double the compute power despite the fact that both have similar clock speeds and core count (the i7 has more threads).

I added an inexpensive Corsair H60 water cooler not too long ago because I planned on re-encoding a lot of video on our KODI machine, it's been doing a wonderful job at keeping the CPU cool despite all 8 threads ramping up to almost 100%.

I wrote a simple script that uses the handbrake-cli package to re-encode the .mkv files and embed all subtitles in the new .mkv container file. I looked at the resulting files on a 1080p TV and couldn't tell the difference. My original script looked very different.


I've since modified the script to look something like this - hb.sh:

#!/bin/bash
for x in *
do
     HandBrakeCLI -i "$x" -o "${x%.*}-264.mkv" --preset="H.264 MKV 1080p30" --all-subtitles
done

 Note that you have to have the handbrake-cli software installed, this doesn't use the GUI version of Handbrake. I still use the graphical version of Handbrake occasionally, but I've found myself using it less and less. The reason? Speed.

Lately I found myself with a large backlog of un-ripped television shows. Almost two months ago I bought the first season of Happy Days because my son and I were watching Night Shift and I thought it would be nice for him to know Henry Winkler as I remember him. We found the first season on sale at one of the movie specialty stores in Cambrige Mall ($10 new if I remember correctly). But I never got around to ripping the media. Also on the slate were several other shows that I managed to all rip in under 2 hours last night using MakeMKV.

MakeMKV preserves the contents of the disc (minus closed captions which seem to be a problem for both Handbrake and MakeMKV) but this means large file sizes. I used my A8-5600K APU-based workstation to rip the media with MakeMKV, then I used Filezilla to transfer them over to our KODI server in the living room. That server has a much more powerful i7-2600 CPU (which by today's standards is pretty middling). I then ssh into the KODI server and run the hb.sh script above to re-encode the video.

It's important to note that the script takes ALL files in the current directory and tried to encode/compress them to the 1080p30 format. If you have .srt subtitle files you should probably temporarily move them to another folder. The script creates another file with the same filename that has -264.mkv appended to the end.

so:
jolly-frog.mkv

would end up as:
jolly-frog-264.mkv

Really this probably isn't saving any time over just using Handbrake to encode from scratch, but it frees my time up since I don't have to babysit Handbrake. The process is: rip on workstation, transfer to server, encode on the server.

The only downside to this process is that the graphical version of Handbrake has the ability to add tags to files, but this is something I haven't really used much and that KODI doesn't seem to care as much about as it does proper naming.

Update:  Because I ssh into our KODI server to run the script I used to have to leave that SSH session open. So what I've done is put all the files I want to newly encode in a subdirectory called 111/. I then run:

nohup hb.sh &

This lets me detach from the session and shut down my (A8-5600K-based) workstation PC.

Comments

  1. Just a follow up: this morning I set the script to re-encode Blu-ray episodes of a popular television show. It was around 7:00a.m. when I started the script running. It's now 8:24p.m. and the script is still running (it's in the middle of season 2 of 7 seasons). This means the CPU has been running at close to 100% for over 13 hours. I just ran in to check the temperature and it's a cool 63 degrees Celsius -- not bad considering it's been running for 13 hours. On the down side of things that means the script still probably has around 28+ hours left! The CoolerMaster H60 is doing it's job. There IS a case use for a water cooler on a non-K processor Linus!

    ReplyDelete
  2. One more follow-up. I needed to shut of the computer I ssh'd in from. Shutting it down meant the hb.sh script would stop running. Fine, I stopped it running. To get it running again and disconnect I used:

    nohup hb.sh &

    Then I disconnected my SSH session, logged in again from another SSH session and did a:

    ps aux | grep hb.sh

    and voila, the process continued to run in the background.

    ReplyDelete

Post a Comment

Popular posts from this blog

CentOS 4.8 in Virtualbox 5.1.x

Years ago I was part of a project that was developed on and ran on CentOS 4.8. The software we developed was only used in-house so we were able to work around any bugs we found. The development cycle was extremely short, 4 months, considering the end result. While we've backed up the source frequently over the years the software really never got a major facelift. I spent a bit of time over the years making some minor changes (I gave the menu a facelift changing it from rotating gifs to CSS, and made some code changes to deal with a tax shift that happened years ago), but what the project really needs is a major overhaul. One of my goals right from the outset was to open source the project, but this didn't happen because I was simply too swamped with other things to completely audit the code. Also I wanted to simplify the project before exposing it to the world. Initially we based the project on an eCommerce suite (OSCommerce). At the time we were using that suite for an...

That itch to upgrade again - Ryzen 5 2600G?

Lately I've been collecting a lot of movies. Each week Maria and I set a small budget for personal "wants," much of mine has gone to DVDs and the occasional Blu-ray disc. I've been lucky to find some really great deals, but it's got to the point where I'm backed up because ripping all the media is taking more time than I have on evenings and weekends. From what I've read ripping Blu-rays (unless you're also re-encoding them) is entirely dependent on the Blu-ray drive - there's little to be gained by upgrading to a new CPU/motherboard. I use a licensed copy of MakeMKV to rip my Blu-rays. Then I transfer the ripped mkv to our KODI machine and using HandbrakeCLI (the command-line version of Handbrake) to compress the large Blu-ray file to a smaller file. I could compress the file on my desktop, but my AMD A8-5600K APU doesn't have the same power as our KODI machine (which has an i7-2600 and is cooled with a Corsair H60 water cooler). The...