Watson Wins Jeopardy

The New York Times has a pretty good article On ‘Jeopardy!’ Computer Win Is All but Trivial. I watched all three nights of the competition and was very impressed but not blown away by Watson.

As I understand it, Watson is 2880 processor cores, 16TB of RAM and is populated with 4TB of info from the web (dictionaries, wikipedia, imdb, world fact book, etc.) I think it works by using each core to solve the problem using a different algorithm and then it ranks the most popular results. If you get the same answer a lot of different ways it's probably right.

The hard part from a programming point of view isn't necessarily finding an answer, it's understanding the question. And Watson doesn't really understand anything. In a final Jeopardy question with the category "US Cities" Watson answered Toronto. Programmers explained that it doesn't put a lot of weight to the categories because they aren't always precise. I think it's more because they're harder to parse because they're short.

i was really impressed with Watson after the first day but the second day I noticed something. A large majority of the questions have the word "this" right before describing what they are looking for. "This author", "This prisoner", "This city". While Jeopardy questions had previously seemed pretty complicated with puns and idioms, I now thought of them more like crossword puzzle clues which have their own common formats that you have learn to get good at them. I started thinking that if you searched for the nouns and found terms that were near them in different articles, it wasn't that hard to figure out which one the clue was asking for based on some parsing rules. That seemed to make the feat a little less impressive to me. In the third game I thought Watson came up with a lot of wrong answers (even though he won) and that somehow reinforced the idea that it wasn't that "smart" (though it also means parsing Jeopardy questions isn't that easy).

I was really impressed with the speed of Watson. That's still a lot of data to go through and to coordinate so many possible answers. 6-8 seconds is a lot of computing time, particularly with 2880 cores and a huge amount of RAM. With the data being only about a quarter of the memory, that means more than half was devoted to indexing and organizing it. There's a lot of programming going into making that fast.

Still though it did seem to have an advantage on pushing the button, even if it had to do it mechanically and as I understand that's key to winning the game.

Ken Jennings had this to say about the experience.

I was disappointed that so many of the articles written about this had to mention HAL or other sci-fi AIs that take over. This is AI the same way Deep Blue was with chess. It's doing a lot of brute force and using probabilities to come up with the answer. That's not how I think of intelligence but it does seem to be a great tool and a great accomplishment.

Contests like this are good to push people to build something new but I'm not sure how soon it will affect everyday life. They say they are looking into medical applications for Watson tech, but if it's talent is parsing the question (which I'm not so sure of) then isn't that much less of an issue when it's being used by doctors? Isn't it easier to teach the human how to phrase the query? Given some of the answers it gave, I really don't think it's anywhere near the point where a layman could query it for medical info.

Then again I wonder what Watson thinks about it?