My eyes started darting when I realized what Watson was doing—preying on our cognitive blind spot and vacuuming the highest-value clues off the board before we had a chance to gain our sea legs. But my response surprised me—by my second match, I was doing the same thing.
What do you do when confronted by an inhuman competitor who ignores convention and overruns what was once your sanctuary? How, in other words, do you defeat alien intelligence?
I’ve spent a good amount of time over the past few years discussing how we, as upstarts, defeat the giants we face. Killing Giants: 10 Strategies to Topple the Goliath In Your Industry (Portfolio) discusses how the smart and nimble defeat the behemoths in the war of business.
But how do we treat Watson? Friend or foe, Giant or Giant Killer? Possibly a bit of both.
But the lessons that early “beta competitor” Greg Lindsay learned in Watson’s pre-prime time days give us a glimpse into how to turn the tables on conventional thinking.
Most players start with the easiest, lowest-value ones and work their way down the board, learning from patterns buried in the clues. Not realizing this, or caring, Watson tended to start each new category with the highest-value clue and work his way up.
Lindsay, brought in as an early test competitor to IBM’s Thomas J. Watson Research Center, discovered the enemy’s plans after first contact. Watson didn’t behave like any other Jeopardy player ever had. Watson approached the game with a blank slate, unencumbered by convention. It did more than just question the givens – it approached the game with a different set of “givens,” borne of optimization rather than human convention. But what began as a shock to convention became a strategy to be emulated, as Lindsay went on to explain.
Trailing again late in my third match against Watson, I abandoned all pretense of trying to beat him outright and began scouring the board for the remaining Daily Double, knowing I had to find it and nail it to have any chance of winning. I had never seen anyone adopt such tactics in all my years of watching Jeopardy!. Against humans, I might have been content to grind it out, but I was on the run, and scared. Once again, the plan worked, as I took the lead just in time for Final Jeopardy.
Lindsay went on to defeat Watson in three straight matches.
Many of the pivotal interviews in Killing Giants take on this same tone.
Marco Nussbaum, CEO and co-founder of Germany’s Prizeotel, left a career of hotel management to re-imagine what a hotel stay could be, throwing out conventions that others thought untouchable.
Geoff Ross, CEO and founder of New Zealand’s 42Below vodka, brushed off conventional thinking and launched a brand that many thought a contradiction, an embodiment of both pristine New Zealand purity and Monty Python-esque irreverence.
Searle Canada’s former president, Richard Hinson, described how his team offered a $1 million grant to anyone who could create a diagnostic procedure or device that would positively ID the early warning signs of traditional pain reliever-induced ulcers, essentially “declaring the debate to be over,” throwing both the “givens” and conventional thinking out the window.
There’s a secret for giants to take from all this. Giants are only vulnerable when they become trapped by their own conventions. They grow blind spots over time, borne of hubris and, ironically, success. It’s often been said that managers fail because they focus on winning the last war – and that last war’s rules, conventions, and requirements aren’t often the same as the one they’re in.
Greg Lindsay gives us a glimpse into the real time nature of engaging the enemy, observing and then reacting – killing the machine that blew up both the longest running Jeopardy winner, Ken Jennings, as well as the biggest prizewinner in the game’s history, Brad Rutter.
Against this backdrop, it was Watson that was the Giant Killer, exploiting the entangled hubris and success of these two former champions to win, and that Lindsay had a significant edge over the computer, being less trapped by the conventions of his former success.
Regards.
[…] This post was mentioned on Twitter by StephenDenny, Jeff Gibbard. Jeff Gibbard said: Killing Watson: Defeating Alien Intelligence by Questioning the Givens and Changing the Game http://bit.ly/fPwdlK […]
Interesting article Stephen! I’d be curious at what point did in the process of creating Watson did Lindsay beat it? Because as I understand it, in early days of testing, Watson got answers wrong as much as it did correct. However, after some algorithmic tweaks and closer to the actual contest, it improved drastically. And then, there’s the actual issue of whether Watson had a unfair buzzer advantage by virtue that while it received the clue at the same time other contestants did, it pounced faster on the buzzer due to its electronic nature. See http://www.thenewstribune.com/2011/02/15/1546078/ibms-watson-has-buzzer-advantage.html
Now as to your question of how to beat the next generations of Watson, one answer may be to change the parameters of the game. Watson, like all narrow AI systems, uses a rules based decision engine. It can only do what it’s programmed to do with slight adaptations. That is, Watson operated in a set of constraints, and within those boundaries did very well. However, by changing the rules of game (when possible), the parameters change and suddenly these “rules based” engines have a much harder time. Humans adapt to change quite well. Computers–not so much.
Great lesson Stephen.
The moment I sense an opponent is working from or trying to refine their algorithm, it’s time to change the rules.
Zig…when their algorithm tells them to zag.
@ Paul: not sure – the link points back to the original article for more and I know that plenty has been written on this strange story already. Worth a look.
@Wags: probably a great segue to a new post! I’ll have to put thought into that today. I’ve been in that situation a number of times where an opponent – giant or otherwise – begins to change its stance, which is the ideal time to strike, even if you don’t necessarily have your plans fully hatched. Deadly, confusing and very effective.