Skip to main content

Our media encoding process


It occurred to me that I started posting to debugfs as if I just left off from my old domain (a story for another time). I had something in the order of 300 articles on the site so naturally posts on this site don't have the context they had on the old site.

When I first started debugfs I talked about the Handbrake command-line script I run on our KODI server to shrink the size of Blu-rays (since they can be huge). I didn't really get into the details of the whole process and I've since changed how I rip and encode media.

When I buy a DVD/Blu-ray the first step I take is to back-up the media using MakeMKV. MakeMKV is great for dumping both Blu-ray and DVD content to a .mkv file. I prefer .mkv over .mp4 because I love subtitles and the .mp4 container only lets you "burn" one subtitle into the file. Files in an .mkv container can contain as many subtitles as the DVD/Blu-ray has. I normally rip these on my desktop workstation which has a late 2013 AMD A8-5600K APU with 16GB RAM, a 500GB SSD, and an NVidia GTX 650 Ti Boost video card. I then use Filezilla to SFTP them to the 8TB drive in our i7-2600 (non-K)-based KODI server. The i7-2600 is a much better processor than the A8-5600K with almost double the compute power despite the fact that both have similar clock speeds and core count (the i7 has more threads).

I added an inexpensive Corsair H60 water cooler not too long ago because I planned on re-encoding a lot of video on our KODI machine, it's been doing a wonderful job at keeping the CPU cool despite all 8 threads ramping up to almost 100%.

I wrote a simple script that uses the handbrake-cli package to re-encode the .mkv files and embed all subtitles in the new .mkv container file. I looked at the resulting files on a 1080p TV and couldn't tell the difference. My original script looked very different.


I've since modified the script to look something like this - hb.sh:

#!/bin/bash
for x in *
do
     HandBrakeCLI -i "$x" -o "${x%.*}-264.mkv" --preset="H.264 MKV 1080p30" --all-subtitles
done

 Note that you have to have the handbrake-cli software installed, this doesn't use the GUI version of Handbrake. I still use the graphical version of Handbrake occasionally, but I've found myself using it less and less. The reason? Speed.

Lately I found myself with a large backlog of un-ripped television shows. Almost two months ago I bought the first season of Happy Days because my son and I were watching Night Shift and I thought it would be nice for him to know Henry Winkler as I remember him. We found the first season on sale at one of the movie specialty stores in Cambrige Mall ($10 new if I remember correctly). But I never got around to ripping the media. Also on the slate were several other shows that I managed to all rip in under 2 hours last night using MakeMKV.

MakeMKV preserves the contents of the disc (minus closed captions which seem to be a problem for both Handbrake and MakeMKV) but this means large file sizes. I used my A8-5600K APU-based workstation to rip the media with MakeMKV, then I used Filezilla to transfer them over to our KODI server in the living room. That server has a much more powerful i7-2600 CPU (which by today's standards is pretty middling). I then ssh into the KODI server and run the hb.sh script above to re-encode the video.

It's important to note that the script takes ALL files in the current directory and tried to encode/compress them to the 1080p30 format. If you have .srt subtitle files you should probably temporarily move them to another folder. The script creates another file with the same filename that has -264.mkv appended to the end.

so:
jolly-frog.mkv

would end up as:
jolly-frog-264.mkv

Really this probably isn't saving any time over just using Handbrake to encode from scratch, but it frees my time up since I don't have to babysit Handbrake. The process is: rip on workstation, transfer to server, encode on the server.

The only downside to this process is that the graphical version of Handbrake has the ability to add tags to files, but this is something I haven't really used much and that KODI doesn't seem to care as much about as it does proper naming.

Update:  Because I ssh into our KODI server to run the script I used to have to leave that SSH session open. So what I've done is put all the files I want to newly encode in a subdirectory called 111/. I then run:

nohup hb.sh &

This lets me detach from the session and shut down my (A8-5600K-based) workstation PC.

Comments

  1. Just a follow up: this morning I set the script to re-encode Blu-ray episodes of a popular television show. It was around 7:00a.m. when I started the script running. It's now 8:24p.m. and the script is still running (it's in the middle of season 2 of 7 seasons). This means the CPU has been running at close to 100% for over 13 hours. I just ran in to check the temperature and it's a cool 63 degrees Celsius -- not bad considering it's been running for 13 hours. On the down side of things that means the script still probably has around 28+ hours left! The CoolerMaster H60 is doing it's job. There IS a case use for a water cooler on a non-K processor Linus!

    ReplyDelete
  2. One more follow-up. I needed to shut of the computer I ssh'd in from. Shutting it down meant the hb.sh script would stop running. Fine, I stopped it running. To get it running again and disconnect I used:

    nohup hb.sh &

    Then I disconnected my SSH session, logged in again from another SSH session and did a:

    ps aux | grep hb.sh

    and voila, the process continued to run in the background.

    ReplyDelete

Post a Comment

Popular posts from this blog

I might be upgrading sooner than expected

It looks like I might be upgrading my home desktop workstation sooner than I wanted to. That system was put together December of 2014 and has: AMD A8-5600K APU Gigabyte  F2A85XM-D3H 16GB GSkill DDR3 1866MHz RAM 2GB NVidia GeForce GTX 650 TI Boost video card (currently a 1GB Radeon HD 6670 for testing) 500GB Samsung 860 SSD (Xubuntu Linux) 1TB WD Blue Drive (Windows 10) Corsair 430 Watt PSU The system has been relatively stable over the years but has recently started to develop a few issues: When recording audio in Xubuntu the audio is stuttering. This may be due to a Pulseaudio update since I don't have the issue in Windows 10 (and I do have the issue with multiple audio sources - web cam, microphone, and in several recording programs: audacity, obs studio). More concerning the system suddenly reboots in the middle of playing Diablo III. I suspect the power supply might be to fault, but if it's not this might mean more expensive purchases. I wanted to hold off to

Media centre management

Early Saturday morning I decided to start cleaning up files on our media centre. Awhile back I took inventory of all the movies on our media centre and entered them into a simple 3 field online database that included: the movie title, format of the movie (DVD/Blu-ray/VHS/Digita/Other), and the date of the movie. Over the past few months I've picked up a lot of movies that didn't make it into the online database. I use that database to make sure I don't already have a movie when I'm out looking for new movies. The first step was to take inventory of all the filenames of the movies on the system. All movies are stored under the directory /mnt/media/Movies. That directory is further sub-divided into 2 other directories /Movies/DVDs and /Movies/Blu-rays. Those folders contain both files and folders, but all the files and folders are archives of our DVDs and Blu-ray discs. In both folders I have a directory called 111. When I rip movies with MakeMKV they go into the appr