Subscribe to entries Subscribe to comments

Ah, what better way to spend a lazy Sunday afternoon than compiling ffmpeg and the half dozen dependencies required to encode H.264 for the iPhone / iPod Touch. (I am supposed to be painting and decorating. Waiting for video to encode is only slightly more interesting than watching paint dry…)

This is not about how to compile open source mpeg encoder ffmpeg. There are loads of good guides how to do that, and it’s mostly ./configure, make, make install anyway. If you’re not comfortable with compiling apps from source then the scripts below might not be your cup of tea either. There are loads of good apps for less than $50 to do the same thing, but if you know what ffmeg is, and are familiar with bash or DOS batch files, you might find these scripts useful. Please leave a comment if you see an error, or improvement, or just want to add something.

Here are the settings I use for encoding H.264 videos that are compatible with the iPhone, iPod Touch and Apple TV. The movies need to be high enough resolution to look good on TV, but low enough data rate to play-back on the iPhone/iPod Touch. Another important requirement is that the file and can easily be dragged from iTunes (on Mac or Windows) to any of those devices. (iTunes is really fussy about that). My source videos are not DVD rips (I believe Handbrake is a popular and free open source application for encoding DVD rips to H.264. Handbrake also uses ffmpeg and works on Mac OS X, Linux and Windows). My movies are PVR recordings and home movies I want to watch on the hand-held device. (In a few years DVB-T might be transmitted already encoded as H.264 and this transcoding not be required).

The script will convert a directory full of movies, in any format your OS is capable of playing, to H.264/AAC mp4. I’ve made versions for bash and DOS which work pretty much in the same way on Linux, Mac OS X and Windows. Pre-compiled versions of ffmpeg for Windows with all the required libraries are easily available online. Compiling your own version is recommended so the binary will make best use of your CPU capabilities, like MMX2, SSE2Fast, SSSE3, Cache64, etc.

These setting are high quality 2 pass encoding. Using a 2.33Ghz Intel Core 2 Duo Macbook Pro with 3 GB RAM I get 100 fps on 1st pass, 50 fps on second. That’s 3/4 real-time encoding, or 22.5 mins for a 30 min movie. Interestingly my old dual 2.8Ghz P4 Xeon server with 4GB ram encodes at the same speed as the Core 2 duo laptop. I was surprised by that, I was hoping the dual P4 Xeons would be faster. (Anyone want to buy a dual P4 Xeon Dell workstation? It’s got 4 Gig of RAM, U320 ultra-wide SCSI, Linux and ffmpeg already installed!)

transcode2ipod.sh


#!/bin/sh
# Mac OS X, path will be different on Linux
FFMPEG_EXE=/Volumes/HD/opt/local/bin/ffmpeg
WORKING_DIR=/Volumes/Media/Movies
QUEUE_DIR=${WORKING_DIR}/transcode-queue
FILE_EXTENSION=avi
DONE_DIR=${WORKING_DIR}/transcode-done
OUT_DIR=${WORKING_DIR}/iPod

#BBC TV widescreen to iPod 640x352, cropping 12 pixels all round
#to remove extraneous black outside the TV safe area
WIDTH_BEFORE_CROP=664
HEIGHT_BEFORE_CROP=376
CROP_TOP=12
CROP_BOTTOM=12
CROP_LEFT=12
CROP_RIGHT=12
WIDTH_OUT=640
HEIGHT_OUT=352

#TESTING="-ss 00:01:55 -t 00:01:01"
TESTING=""
VIDEO_BIT_RATE=896k
MAX_VIDEO_BIT_RATE=1024k
VIDEO_BIT_RATE_TOLERANCE=128k
AUDIO_BIT_RATE=128k
AUDIO_FREQ=48000
GOPS=250
KEY_INT_MIN=25

# default
MOTION_ESTIMATION_METHOD=epzs
#slightly faster
#MOTION_ESTIMATION_METHOD=umh
#slightly faster again
#MOTION_ESTIMATION_METHOD=hex

for FULL_FILENAME in ${QUEUE_DIR}/*.${FILE_EXTENSION}
do
    #gets filename without dir path
    FILENAME=${FULL_FILENAME##*/}
    #gets filename without extension, depends on first path manipulation ##*/
    echo "Transcoding ${FILENAME%%.*}.${FILE_EXTENSION}"

    ${FFMPEG_EXE} \
-i ${QUEUE_DIR}/${FILENAME%%.*}.${FILE_EXTENSION} \
-y ${TESTING} \
-s ${WIDTH_BEFORE_CROP}x${HEIGHT_BEFORE_CROP} \
-vcodec libx264 \
-b ${VIDEO_BIT_RATE} \
-bt ${VIDEO_BIT_RATE_TOLERANCE} \
-threads 2 \
-croptop ${CROP_TOP} \
-cropbottom ${CROP_BOTTOM} \
-cropleft ${CROP_LEFT} \
-cropright ${CROP_RIGHT} \
-f mp4 \
-flags +loop \
-cmp +chroma \
-partitions 0 \
-subq 1 \
-trellis 0 \
-refs 1 \
-coder 0 \
-me_range 16 \
-me ${MOTION_ESTIMATION_METHOD} \
-g ${GOPS} \
-keyint_min ${KEY_INT_MIN}  \
-sc_threshold 40  \
-i_qfactor 0.71 \
-maxrate ${MAX_VIDEO_BIT_RATE} \
-bufsize 10M \
-rc_eq 'blurCplx^(1-qComp)' \
-qcomp 0.6 \
-qmin 10 \
-qmax 51 \
-qdiff 4 \
-level 30 \
-aspect ${WIDTH_OUT}:${HEIGHT_OUT} \
-acodec libfaac \
-an \
-pass 1 /dev/null

    ${FFMPEG_EXE} \
-i "${QUEUE_DIR}/${FILENAME%%.*}.${FILE_EXTENSION}" \
-y ${TESTING} \
-s ${WIDTH_BEFORE_CROP}x${HEIGHT_BEFORE_CROP} \
-vcodec libx264 \
-b ${VIDEO_BIT_RATE} \
-bt ${VIDEO_BIT_RATE_TOLERANCE} \
-threads 4 \
-croptop ${CROP_TOP} \
-cropbottom ${CROP_BOTTOM} \
-cropleft ${CROP_LEFT} \
-cropright ${CROP_RIGHT} \
-f mp4 \
-flags +loop \
-cmp +chroma \
-partitions +parti4x4+partp8x8+partb8x8 \
-subq 5 \
-trellis 1 \
-refs 1 \
-coder 0 \
-me_range 16 \
-me ${MOTION_ESTIMATION_METHOD} \
-g ${GOPS} \
-keyint_min ${KEY_INT_MIN} \
-sc_threshold 40 \
-i_qfactor 0.71 \
-maxrate ${MAX_VIDEO_BIT_RATE} \
-bufsize 10M \
-rc_eq 'blurCplx^(1-qComp)' \
-qcomp 0.6 \
-qmin 10 \
-qmax 51 \
-qdiff 4 \
-level 30 \
-aspect ${WIDTH_OUT}:${HEIGHT_OUT} \
-acodec libfaac \
-ab ${AUDIO_BIT_RATE} \
-ar ${AUDIO_FREQ} \
-ac 2 \
-pass 2 ${OUT_DIR}/${FILENAME%%.*}.mp4

    ERROR=$?
    if [ $ERROR -ne 0 ]; then
        echo "Transcoding failed for $ with error $ERROR"
        exit 3
    fi

    rm ${WORKING_DIR}/*.log
    rm ${WORKING_DIR}/*.temp
    mv ${QUEUE_DIR}/${FILENAME%%.*}.${FILE_EXTENSION} ${DONE_DIR}

done;

transcode2ipod.dos.bat


@echo off
set FFMPEG_EXE="C:\Program Files\ffmpeg\bin\ffmpeg.exe"
set BLAT_EXE="C:\Program Files\Blat\Blat.exe"
set UNIX_UTILS=D:\UnixUtils\usr\local\wbin
SET WORKING_DIR=D:\Movies
set QUEUE_DIR=%WORKING_DIR%\xvid-queue
rem set QUEUE_DIR=%WORKING_DIR%\Home
set FILE_EXTENSION=avi
set DONE_DIR=%WORKING_DIR%\xvid-done
set OUT_DIR=%WORKING_DIR%\iPod
set TEMP_DIR=%WORKING_DIR%\tmp
set TEMP_FILENAME="%TEMP_DIR%\filename.txt"
set LOG_FILE="%TEMP_DIR%\ffmpeg.log"

rem email settings
rem whether or not to send an email after each transcoding
set EMAIL=1
set SMTP_HOST=your.mailhost.com
set TO=email@address.com
set FROM=sender@address.com
set MAILBODY_FILE="%TEMP_DIR%\mail.txt"

rem ffmpeg settings for iPod touch / iPhone
rem
rem small PAL
rem DIMENSIONS=480x256
rem pal
rem DIMENSIONS=480x352
rem rem max
rem DIMENSIONS=720x480
rem
rem rem home movies from Sony Camcorder do 356x376 with crop 12 top bottom left right. Output size is Width -cropleft -cropright, & Height -croptop -cropbottom 536 -12 -12 = 512, 376 -12 -12 = 352
rem set WIDTH=536
rem set HEIGHT=376

rem BBC TV widescreen to iPod 640x352, cropping 12 pixels all round to remove extraneous black outside the TV safe area
set WIDTH=664
set HEIGHT=376
set CROP_TOP=12
set CROP_BOTTOM=12
set CROP_LEFT=12
set CROP_RIGHT=12

rem set TEST=-ss 00:03:00 -t 00:00:10
set TEST=
set VIDEO_BIT_RATE=896k
set MAX_VIDEO_BIT_RATE=1024k
set VIDEO_BIT_RATE_TOLERANCE=128k
set AUDIO_BIT_RATE=128k
set AUDIO_FREQ=48000
set GOPS=250
set KEY_INT_MIN=25

rem default
set MOTION_ESTIMATION_METHOD=epzs
rem slightly faster
rem MOTION_ESTIMATION_METHOD=umh
rem slightly faster again
rem MOTION_ESTIMATION_METHOD=hex

rem create tmp dir if not exists
IF NOT EXIST %TEMP_DIR% MKDIR %TEMP_DIR%

rem create out dir if not exists
IF NOT EXIST %OUT_DIR% MKDIR %OUT_DIR%
rem FOR %%f in ("%QUEUE_DIR%\*.%FILE_EXTENSION%") DO (

SETLOCAL ENABLEDELAYEDEXPANSION
set FILES=
for /f %%a IN ('DIR /b "%QUEUE_DIR%\*.%FILE_EXTENSION%"') do (
set FILES=!FILES! %%~na
echo "Transcoding %%~na.%FILE_EXTENSION%"

%FFMPEG_EXE%
-i "%QUEUE_DIR%\%%~na.%FILE_EXTENSION%"
-y %TEST%
-s %WIDTH%x%HEIGHT%
-vcodec libx264
-b %VIDEO_BIT_RATE%
-bt %VIDEO_BIT_RATE_TOLERANCE%
-threads 4
-croptop %CROP_TOP%
-cropbottom %CROP_BOTTOM%
-cropleft %CROP_LEFT%
-cropright %CROP_RIGHT%
-f mp4
-flags +loop
-cmp +chroma
-partitions 0
-subq 1
-trellis 0
-refs 1
-coder 0
-me_range 16
-me %MOTION_ESTIMATION_METHOD%
-g %GOPS%
-keyint_min %KEY_INT_MIN%
-sc_threshold 40
-i_qfactor 0.71
-maxrate %MAX_VIDEO_BIT_RATE%
-bufsize 10M
-rc_eq "blurCplx^(1-qComp)"
-qcomp 0.6
-qmin 10
-qmax 51
-qdiff 4
-level 30
-aspect %WIDTH%:%HEIGHT%
-acodec libfaac
-an
-pass 1 "%TEMP_DIR%\%%~na.pass1"

%FFMPEG_EXE%
-i "%QUEUE_DIR%\%%~na.%FILE_EXTENSION%"
-y %TEST%
-s %WIDTH%x%HEIGHT%
-vcodec libx264
-b %VIDEO_BIT_RATE%
-bt %VIDEO_BIT_RATE_TOLERANCE%
-threads 4
-croptop %CROP_TOP%
-cropbottom %CROP_BOTTOM%
-cropleft %CROP_LEFT%
-cropright %CROP_RIGHT%
-f mp4
-flags +loop
-cmp +chroma
-partitions +parti4x4+partp8x8+partb8x8
-subq 5
-trellis 1
-refs 1
-coder 0
-me_range 16
-me %MOTION_ESTIMATION_METHOD%
-g %GOPS%
-keyint_min %KEY_INT_MIN%
-sc_threshold 40
-i_qfactor 0.71
-maxrate %MAX_VIDEO_BIT_RATE%
-bufsize 10M
-rc_eq "blurCplx^(1-qComp)"
-qcomp 0.6
-qmin 10
-qmax 51
-qdiff 4
-level 30
-aspect %WIDTH%:%HEIGHT%
-acodec libfaac
-ab %AUDIO_BIT_RATE%
-ar %AUDIO_FREQ%
-ac 2
-pass 2 "%OUT_DIR%\%%~na.mp4" > "%LOG_FILE%"

IF EXIST "%TEMP_DIR%\%%~na.pass1" DEL "%TEMP_DIR%\%%~na.pass1"

IF %EMAIL% == 1 (
    echo Finished transcoding %%~na.mp4" > %MAILBODY_FILE%
    echo "" >> %MAILBODY_FILE%
    IF EXIST %LOG_FILE% copy %MAILBODY_FILE%+%LOG_FILE% %MAILBODY_FILE%
    %BLAT_EXE% %MAILBODY_FILE% -to %TO% -subject "%%~na.mp4 is ready" -server %SMTP_HOST%
)

MOVE "%QUEUE_DIR%\%%~na.%FILE_EXTENSION%" %DONE_DIR%

)

GOTO CLEANUP

:CLEANUP
IF EXIST %TEMP_DIR% RD %TEMP_DIR% /S /Q
@echo complete

I’ve left a lot of commented lines in, so you can see what I’ve been tweaking. I could have set the dimensions lower than 640×352 if I was only every going to view the movies on the iPod screen (420 wide), but I want to play the movies on my laptop and TV too. 640 was a good compromise. 720 would be larger than the source, so I would gain nothing in quality, just bigger files.

A few observations:

  • The way the cropping settings interact with the final width and height is a bit weird.
  • level 30 is essential for getting files from iTunes to the iPod/iPhone. iTunes is strict about what H.264 profiles and levels it ‘thinks’ the iPod can and can not play. The device can do all kinds of data rates, it’s iTunes that is fussy. -level 30 sets the level to 3 in the list of H.264 profiles on Wikipedia
  • I chose 25 fps because my source movies are mostly PAL (25fps)
  • I chose 250 for the GOP size because the fps is 25. (GOP is group of picture, or key frame. 250 means a full picture frame sample (key frame with no in-betweening info) will be created once every 10 seconds at 25 fps. This is quite high but keeps the file size low and the movie is still perfectly watchable when you skip ffwd or rewind - the picture may be blocky for a few seconds - until the next keyframe.

How does it look?

Here is a one minute clip from a documentary about musician/composer/arranger Quincy Jones. I chose this section because it contains lots of colours, music, rostrum pans, zooms, talking heads, and some grainy old black and white footage - the kind of thing that can sometimes be blocky or stutter at lower data rates. Click the poster frame to download the movie. (See the TESTING variable for how to encode just a section of a movie). Sorry, the movie is not optimized for streaming, you’ll have to down load all 7MB.

Quincy Jones

Comments 6 Comments »

Nobody tagged me :-( I just volunteered …

WTF am I talking about? - http://michaeleatonconsulting.com/blog/archive/2008/06/04/how-did-you-get-started-in-software-development.aspx

How old were you when you started programming?

Thatcher, Thatcher the milk snatcher had just won her second term as Prime Minister of the United Kingdom. I had just started High School so it was 1983 and I was 11 (yes, High School begins at age 11 in the U.K.) I never liked Thatcher. The milk snatcher nickname came about because she wanted to abolish the policy of free milk to all primary school children. I hated that horrible creamy cows milk, but I thought it was a good idea nevertheless.

How did you get started in programming?

We had a Tandy (Radio Shack in USA) TRS80 at primary school in 1982. I didn’t get close enough to it to start programming it but I was hooked on the green flashing cursor. The closest I got was programming my Casio calculator to say BOOBLESS by entering 55378008 and turning it up-side down. Seriously, I was fortunate enough to be given a Sinclair ZX81 as a gift from my late Uncle Trefor Sawford in 1982 when he upgraded to an Apple II. My cousin, Christopher Sawford, gave me his old copies of Sinclair User magazine. I learned BASIC by keying in the program listings in the back of the magazine. I wanted to learn how to make games because I could not afford to buy them. Two years later Santa delivered a ZX Spectrum with a whopping 48K of RAM, yes kids, Kilobytes. Daley Thompson’s Decathlon, Bombjack and Commando were my favourite games.

What was your first language?

Sinclair BASIC, closely followed by BBC Basic and then Pascal. In the early 80s the British Broadcasting Corporation, a publicly funded company, thought it wise to be involved in the personal computer revolution. They manufactured their own personal computer based on the Motorola 68000. It was very popular with schools and even had its own television show - ‘Making Most of the Micro’. At age 14 this was amazing. Each week a different application of computers was featured - computer aided manufacturing, music, graphics, printing, and even a demonstration of booking an airline ticket and using bulleting boards using a modem. This was 1984 and I was learning this for free from the television! Can anyone believe this? The finale of the show was a transmission of a program - as audio - for the viewers at home to record to cassette and load into their own ‘Micro’. Amazing.

What was the first real program you wrote?

I didn’t write real programs until I went to Leeds University in 1990. I wrote a real program in Fortran to calculate the area of some metal object for a CAD exercise. I passed the module without having to do much work so I retired from real engineering and looked for a degree in software instead. I transferred to the University of Hertfordshire in 1992 and learned Coad and Yourdon’s OOA and OOD. We had great lecturers at Hertfordshire. We studied problem solving skills, principles and paradigms more than specific languages and syntax. This was great training for the real world.

What languages have you used since you started programming?

In no particular order, BASIC, Pascal, Fortran, Miranda, ADA, Prolog, Smalltalk, C, C++, C#, PHP, Lingo, csh, awk, bash, (are they languages?), Java, JavaScript, others I’ve forgotten.

What was your first professional programming gig?

Developing an interactive Philips CDi for Coopers & Lybrand (now Pricewaterhouse-Coopers) in London. It was a video training disk, like one of those adventure books where you choose which page to jump to next by answering multiple choice questions. They hired the narrator from the BBC Television version of The Hitchhiker’s Guide to the Galaxy. It was pretty cool.

If you knew then what you know now, would you have started programming?

Yes. I would have created a thing called the World Wide Web and a search engine with a funky name.

If there is one thing you learned along the way that you would tell new developers, what would it be?

Practice, practice, practice. Creating software is like creating music. You need sharp tools and skills. The tools are easy to acquire. The skills are not. If you are out of work, write stuff anyway. You will be asked about the last thing you wrote at interview, make sure it was not six months ago at your last job. Play in groups. Learn from other people. Show other people what you do. It’s more fun that way and you might learn something about yourself. Get involved in Open Source Projects. Peer review will make you a better developer. There is too much to know to know it all. Be proud of what you create. If you are not, find another career.

What’s the most fun you’ve ever had … programming?

Without doubt the most fun has been connecting with people around the World, traveling and always learning.

Comments 2 Comments »

The background colour of my windows desktop is not the default white. Too many website assume it is and don’t bother explicitly setting the body background colour. The designer quite obviously intended the background to be white. I can’t believe this is done on purpose to save bandwidth either. I think it’s laziness or ignorance. Google, Yahoo, Microsoft and Apple are all guilty. Here’s a quick sample of what I’m talking about.

Google analytics default page

microsoft asp.net 3.5 extensions homepage

The apple site doesn\'t look so great on gray

Gray is the perfect OS window background colour. White, black, orange and blue text work great on it. If Netscape ever did anything right it was to make the default web browser window gray (or was that Mosaic). I don’t want white. White backgrounds are from the 80s when desktop publishing and printing was all the rage, people printed, faxed and shredded. Before that it was black screens with orange or green text. Great! I’m so happy vibrant colours on black are making a comeback. I can’t remember the last time I printed anything, or pretended that my screen was a model of the printed page. The future is dull, the future is gray.

Comments 2 Comments »

I can’t decide whether to use US English or British English in my blog posts. My first language is British but I’m often persuaded to write emails and project documentation using US spellings because the target readers are from the United States of America. Context matters. When visiting a foreign country it is courteous to address your host nation in their own language, if you are able. I see from my web logs that most people reading my blog come from the USA, so should I use US spelling and grammar? I can’t decide already [sic].

True story: “You have really good English for someone who is not from America”, said an American lady passenger on a cruise ship in the Mediterranean. The well meaning World traveler said this to a member of the Pursers’ desk - a polite young lady from Surrey, England (you know, where English is spoken quite a lot and very properly in Surrey). The purser actually spoke many languages … but after that remark she was speechless.

Comments 2 Comments »

About two years ago a very experience developer told me, “you should never need to write ‘else’ conditions if your code is well designed”. Thank you, my friend, for making me think about that every time I start typing } e..l..

I have written about 3 in the last month. Consider this:


public string SomeFunction(string arg)
{
    if (!String.IsNullOrEmpty(arg))
    {
        // do something with the string argument
    }
    else
    {
        // do something else
    }
}

and…


public string SomeFunction(string arg)
{
    if (!String.IsNullOrEmpty(arg))
    {
        // do something with the string argument
        return;
    }

    // do something else
}

The difference is subtle in this contrived example. It becomes more apparent when your if .. else conditions are in deeply nested curly braces.

Consider …


foreach (KeyValuePair<int, Inventory> entry in inventoryDictionary)
{
    if (_inventory.ContainsKey(entry.Value.UniqueReference))
    {
        _inventory[entry.Value.UniqueReference].Add(entry.Value);
    }
    else
    {
        IList<Inventory> inventory = new List<Inventory>();
        inventory.Add(entry.Value);
        _inventory.Add(entry.Value.UniqueReference, inventory);
    }
}

and …


foreach (KeyValuePair<int, Inventory> entry in inventoryDictionary)
{
    if (_inventory.ContainsKey(entry.Value.UniqueReference))
    {
        _inventory[entry.Value.UniqueReference].Add(entry.Value);
        continue;
    }

    IList<Inventory> inventory = new List<Inventory>();
    inventory.Add(entry.Value);
    _inventory.Add(entry.Value.UniqueReference, inventory);
}

OK. That’s not exactly deep nesting, but you can see where this is train of thought is going. I prefer the latter every time because I find fewer braces easier to read (except for single line ifs or ifs with no braces at all, I hate those). Some people may find the early return; and continue; statements hard to spot, but I really like it.

Here is another example of extraneous bracing I see quite often. There are two small refactorings available here: the else is not needed, and the if logic can be inverted to reduce the levels of indentation and bracing. Resharper 4.0 has this suggestion now which is a really nice feature.

Original …


public static string StripQuotes(this String str)
{
    if(! String.IsNullOrEmpty(str))
    {
        var reg = new Regex("[\"\']");
        return reg.Replace(str, String.Empty);
    }
    else
    {
        return null;
    }
}

Refactored to reduce levels of indentation and bracing …


public static string StripQuotes(this String str)
{
    if(String.IsNullOrEmpty(str))
    {
        return null;
    }
    var reg = new Regex("[\"\']");
    return reg.Replace(str, String.Empty);
}

Comments 2 Comments »