Steve's personal blog

In case you haven’t already heard, NetFlix has announced a competition with the grand price of $1 M USD (Convert) for a system that will improve it’s prediction of movies it’s customers would like to rent. (or as NetFlix put it “promote progress in recommendation systems”).

To help contestants NexFlix is providing a huge 100 million anonymous movie ratings (it’s a 665M download) training data set from its archives and they have very carefully updated the data to remove privacy issues (e.g. you can’t find out who actually borrowed what).

If you want a little help understanding why NetFlix would put up so much cash for a small (10%) improvement in their prediction have a read of Why better DVD recommendations are worth a million bucks from Chris Anderson over on the Long Tail (and don’t forget to have a look at his book The Long Tail: Why the Future of Business Is Selling Less of More).

Now if you want to enter the competition or learn some more about machine learning then you may want to have a check out of a recent Channel 9 video MSR Cambridge Tour: Machine Learning Group, Computer Vision and F# from Microsoft Research down the road here in Cambridge. If you are interested in what the future holds for games, video conferences, machine vision and learning then it’s well worth a watch. Also you may want to check out Pattern Recognition and Machine Learning by Christopher Bishop for a read to help you when your wading through all that NexFlix data.


[rant]
BTW: Anybody having issues with dumprep.exe, I’ve had a few crashes recently and this thing takes forever to work and steals all my CPU. IE 6 just crashed when I was hunting round Channel 9 for a link to the video and now dumprep.exe has 133M of my memory and 92% of my CPU, and it’s been that way for a while.

Whilst I’m on a Channel 9 note could they make it any more horrible to navigate, they have some fantastic video’s on there, just you have to go hunting – and you should as there are some really interesting ones. Mind you theirs not even a nice link from MSDN to Channel 9 like their used to be so I guess we just aren’t meant to be watching them.

OK, rant over!
[/rant]

Anyway, the NetFlix competition looks like it’s going to be interesting, good luck to all those that enter.