Thursday, February 17, 2011

What Is, An Infomercial?

On Jeopardy, Computer Win Is All But Trivial:

In the end, the humans on “Jeopardy!” surrendered meekly.

From now on, if the answer is “the computer champion on “Jeopardy!,” the question will be, “What is Watson?”

For I.B.M., the showdown was not merely a well-publicized stunt and a $1 million prize, but proof that the company has taken a big step toward a world in which intelligent machines will understand and respond to humans, and perhaps inevitably, replace some of them.

Er, not so fast. I watched the entire three-night run of episodes because my kids were absolutely riveted by Watson and, distressingly, pulling for the machine to win.

And while it did win, my own conclusion is that the computer was merely faster than the humans at buzzing in, not necessarily "smarter."

Both Mr. Jennings and Mr. Rutter are accomplished at anticipating the light that signals it is possible to “buzz in,” and can sometimes get in with virtually zero lag time. The danger is to buzz too early, in which case the contestant is penalized and “locked out” for roughly a quarter of a second.

Watson, on the other hand, does not anticipate the light, but has a weighted scheme that allows it, when it is highly confident, to buzz in as quickly as 10 milliseconds, making it very hard for humans to beat. When it was less confident, it buzzed more slowly. In the second round, Watson beat the others to the buzzer in 24 out of 30 Double Jeopardy questions.

“It sort of wants to get beaten when it doesn’t have high confidence,” Dr. [David] Ferrucci said. “It doesn’t want to look stupid.”

Um, no. The computer doesn't "want" anything, let alone to not "look stupid." That implies human characteristics which simply don't exist.

The fact remains, the computer beat the guys to the buzzer, which in any game of Jeopardy guarantees virtual success. It wasn't like Watson stumped the two humans, or answered questions they had no clue about (listen to me: referring to his opponents last night as "the two humans").

In fact, some of Watson's answers were hilariously off the mark.

Despite its intellectual prowess, Watson was by no means omniscient. On Tuesday evening during Final Jeopardy, the category was U.S. Cities and the clue was: “Its largest airport is named for a World War II hero; its second largest for a World War II battle.”

Watson drew guffaws from many in the television audience when it responded “What is Toronto?????”

My kids fell out on another question; something about an article of clothing one would wear at Wimbledon, and his top answer was "panties."

Anyway, I admit it was entertaining to watch, but I don't know it means anything beyond IBM's ability to still design cutting-edge technology (and market itself in what was basically a 3 night infomercial for nerds). If Alex Trebek had asked it "so Watson, tell us a little about yourself" after the first commercial break, and the computer had gone off with its biography and history and quirks, yes, I would have conceded the future to machines right then and there.

But we're definitely not there yet. As I told my kids, the most brilliant machine is only as brilliant as what we dumb old humans put into it.


MRMacrum said...

I saw a brief news piece on this. Makes me wish I had watched.

Toronto? BTW - what was the answer?

Todd Krohn said...

I think it was Chicago (O'Hare and Midway).