Where No Search Engine Has Gone Before
By Farhad Manjoo
I first came across Google’s interest in Star Trek back in the summer of 2010. A company spokesman wanted to show me the firm’s rapidly improving visual search and speech-recognition technology. At the time, those features were available only on Android phones, and, back then, Android was getting shellacked by the iPhone. So when the spokesman told me that he regarded the latest Android devices as something like a combination of Star Trek’s tricorder and the USS Enterprise’s computer, I dismissed it as a gimmick to attract media attention for a struggling brand. Not that he was totally wrong—in 2010, asking your phone to search for something, rather than typing in your query, was pretty cool. It just wasn’t Star Trek-cool.
Since then, though, Star Trek has popped up again and again in my interviews with Googlers. Last year, for a piece in which I praised Google’s Voice Search for trouncing Apple’s Siri, I spoke to Scott Huffman, one of the engineering directors on Google’s search team. When I asked him how he pictured voice search evolving, he too invoked the Star Trek computer. “You would ask, ‘Hey Google, where should I have dinner?’ ” he told me. “And it might say, ‘Well, you seem to like Italian restaurants, so how about this one?’ ” A few weeks ago, I was chatting with Tamar Yehoshua, director of product management on Google’s search. “Is there a roadmap for how search will look a few years from now?” I asked her. “Our vision is the Star Trek computer,” she shot back with a smile. “You can talk to it—it understands you, and it can have a conversation with you.”
Still, I didn’t really put much stock in these references. After all, Google is very likely the nerdiest large company on earth; of course its employees like Star Trek. Then, in March, Amit Singhal, who heads Google’s search rankings team, gave a talk at the South by Southwest Interactive Festival—and Star Trek took center stage. Singhal told the crowd that the original series was one of his favorite all-time shows, and he longed to one day meet William Shatner, “as long as he doesn’t sell me a hotel room.” Then Singhal added: “The destiny of [Google’s search engine] is to become that Star Trek computer, and that’s what we are building.”
OK, really? Despite all the interviews in which Star Trek had come up, I’d long assumed that Google’s Star Trek chatter was meant as marketing—that Googlers kept talking about the Star Trek computer because it was an easy reference point, but that the company wasn’t really trying to build a machine as encyclopedic and humanistic as the all-knowing ship’s computer. But Singhal’s talk got me wondering. What if Google’s vision really is to build the Star Trek computer? What if the picture most of us have of search—type in a few keywords, get back links to other sites—is not the way Google thinks about search? What if, when Google’s search engineers go about building the next iteration of the company’s primary product, they really do look to Captain Kirk for inspiration?
So I went to Google to interview some of the people who are working on its search engine. And what I heard floored me. “The Star Trek computer is not just a metaphor that we use to explain to others what we’re building,” Singhal told me. “It is the ideal that we’re aiming to build—the ideal version done realistically.” He added that the search team does refer to Star Trek internally when they’re discussing how to improve the search engine. “It comes up often,” Singhal said. “For instance, we might say, ‘Captain Kirk never pulled out a keyboard to ask a question.’ So in that way it becomes one of the design principles—we see that because the Star Trek computer actively relies on speech, if we want to do that we need to work to push the barrier of speech recognition and machine understanding.”
What does it mean that Google really is trying to build the Star Trek computer? I take it as a cue to stop thinking about Google as a “search engine.” That term conjures a staid image: a small box on a page in which you type keywords. A search engine has several key problems. First, most of the time it doesn’t give you an answer—it gives you links to an answer. Second, it doesn’t understand natural language; when you search, you’ve got to adopt the search engine’s curious, keyword-laden patois. Third, and perhaps most importantly, a search engine needs for you to ask it questions—it doesn’t pipe in with information when you need it, without your having to ask.
The Star Trek computer worked completely differently. It understood language and was conversational, it gave you answers instead of references to answers, and it anticipated your needs. “It was the perfect search engine,” Singhal said. “You could ask it a question and it would tell you exactly the right answer, one right answer—and sometimes it would tell you things you needed to know in advance, before you could ask it.”
Google’s transformation into the Star Trek computer will take years. But it has already made huge leaps toward building such a machine. For lots of searches today, you’ll notice Google giving you more and more direct answers. Type in “tom cruise height,” for instance, and you’ll see the diminutive star’s digits at the top of the page (5 feet, 7 inches).
Google can do this, in part, because it “understands” concepts. During the last few years, the company has been compiling what it calls the Knowledge Graph, a database of hundreds of millions of real-world “objects.” Before the Knowledge Graph, Google would simply have searched for “tom cruise height” (and related keywords, like “tall”) on pages around the Web. Now, the engine can accumulate direct knowledge about Tom Cruise and other notable objects. After analyzing all of the information on the Web, Google “knows” that Tom Cruise is a specific person, that he’s male, that he’s an actor, that he’s had three spouses, that he starred in Risky Business, and that he was born in Syracuse, N.Y., on July 3, 1962. When you ask Google about Tom Cruise, its brain pictures the same thing yours does—not a string of letters, but a close to fully formed picture of a quite short man.
It’s not just celebrities that Google is getting better at understanding. It is also amassing more and more knowledge about you. Consider the Gmail “field trial,” Google’s name for an opt-in experimental feature that the company unveiled a couple months ago. It connects the search engine to your email, so you can now search for questions about your personal data. Type in “when’s my flight?” and—because it has seen your itinerary in your inbox—Google can tell you when you need to leave for the airport.
At the same time, Google has spent years perfecting its speech recognition and natural-language systems, and (as I’ve written before) can now understand spoken queries shockingly well. If you pull out your phone and ask Google, “how tall is Tom Cruise?” a friendly voice will spit back the answer. Because it understands concepts, it can also hold a very rudimentary conversation with you. If you ask it a question about Barack Obama and then ask it a question about “his wife,” Google will sometimes—but not always—understand that you’re talking about Michelle Obama, even though you referred to her using a pronoun.
Finally, Google has been inching its way toward predicting your needs. The clearest example of this is Google Now, an app for Android phones that offers up little nuggets of useful information as you need them. Not only will Google Now remind you of an upcoming flight, it will also warn you about traffic conditions along the way, and get your boarding pass ready for the TSA agent. When I interviewed Singhal about Google’s search plans back in 2011, he told me, “I can imagine a world where I don’t even need to search. I am just somewhere outside at noon, and my search engine immediately recommends to me the nearby restaurants that I’d like because they serve spicy food.” Two years later, Google is on the way to building this predictive machine.
Still, as much progress as Google has made toward building the Star Trek computer, everyone at the firm told me it still had many problems to solve. Late last year, I participated in a study Google’s user-research lab was conducting to figure out what information people want that they can’t get from Google. The study asked me and more than 100 other participants to list our “information needs” throughout the day. We’d get a ping on our phones, and then we’d have to recall something that we’d just been looking for, and explain whether we’d found an answer, and how. This is one of Google’s main hurdles in trying to become the Star Trek computer: It doesn’t know enough. As much knowledge as Google has amassed, the depth and diversity of people’s questions far surpasses what it can do.
For instance: “Why are men jerks?” According to John Boyd, Google’s head of user experience, that was one of the questions raised in the study in which I participated. Others included, “Why does my dog behave differently around me than he does around my girlfriend?” and “Why does my tooth hurt?” If you ask Google these questions now it can give you lists of links to sites that might give you some clues (e.g., “The Truth Behind Why Men Are Assholes.”) But these questions defy easy answers—the premises may not make sense, or they may require the search company to know a lot more about you, or they may be fundamentally unanswerable. But responding to questions that are hard or verging on impossible to answer is a task that the Star Trek computer will need to master. Unusual-sounding questions aren’t unusual for Google: On any given day, about 16 percent of the questions that people ask Google are totally new—they’re queries that Google has never seen before. Thus, to become the Star Trek computer, Google will somehow have to figure out how to serve up correct answers to questions it has never come across.
Addressing these problems isn’t just a matter of scale—of Google’s machines getting faster and its data getting more capacious. “We’re going to have to come up with new ways of doing things,” says Tamar Yehoshua.
At the same time, the Google search team is very confident that it can realize its Star Trek dreams. “You already see hints of the Star Trek computer in your phone,” Singhal said. “Now we’re trying to get it to a point where it passes the ‘toothbrush test’ of you using it twice a day.” Singhal predicted that will happen in three years’ time—by then, he says, Google’s Star Trek machine will be so good that you’ll ask it a question and expect a correct answer at least twice a day. “And in five years you won’t believe you ever lived without it. You’ll look back at today’s search engine and you’ll say, ‘Is that really how we searched?’” Singhal says. He adds: “These are the best times we’ve ever had in search. I have done this for 22 years, and I’ve been at Google for 12 years, so I should know. This is the most exciting time—every morning I come into work more excited than ever. Strap in. It’s all happening in our lifetimes.”