When Apple unveiled the iPhone 4S last year, the new phone looked just like the previous one. It had a better camera and a faster chip, but it could do only one new thing: Siri.

Siri, as everyone knows by now, is a software assistant that takes spoken orders. No training necessary: just hold down the “Home” button and speak casually.

Siri lit the cultural world on fire. There were YouTube parodies, how-to guides and copycat apps for Android phones. Pundits have proposed new rules of etiquette for using phones in public now that people are speaking to them even when they're not on a call. Speech recognition became all the rage; suddenly, it popped up in television sets and, of course, rival phones. At the crest of the hype, it looked like the way we interact with our gadgets had changed forever.

And then—the backlash.

“Siri Is Apple's Broken Promise” was the headline at gadget site Gizmodo. People griped that sometimes you'd dictate a whole paragraph, the phone would think and then type—nothing at all. Now there has been a class-action lawsuit asserting that Apple made false claims. (According to Apple, Siri is still in beta testing.)

What happened? How could Siri, the savior of electronics, turn out to be such a bust?

What everybody's missing is the difference between Siri, the virtual assistant, and Siri, the speech-recognition engine. As it turns out, these two different functions have wildly different track records for success.

The assistant half of Siri comes from a company called Siri, which Apple bought. (It was a spin-off from a military artificial intelligence project that wound up at the research firm SRI. Get it?)

But the dictation feature—the text-to-speech part—is provided by Nuance, the company that brought us software such as Dragon NaturallySpeaking.

When you dictate, you generate an audio file that is transmitted to Nuance's servers; they analyze your speech and send the text back to your phone. That is why, when your Internet signal isn't great or when the cell network is congested, Siri may come up short. (When you're on Wi-Fi, dictation works far better.)

That requirement to shuttle data to and from remote servers is at the heart of Siri's frustratingly inaccurate dictation talents.

There are other challenges to the dictation feature, too. Irregular background noise, wind and variable distance from mouth to microphone all make transcription perfection on a cell phone a towering task—and the results are much less accurate than what you would get using PC dictation software, which faces none of those difficulties. Using Siri (and the even less polished dictation feature on Android phones), you might have to correct two or three errors per paragraph.

Desktop dictation software fares much better—close to 100 percent accuracy—because it doesn't have any of those particular challenges. And on your PC, you train the software to recognize only one voice: yours. There's no training on the phone. The computational task is ridiculously hard.

The backlashers have a point. We're used to consumer technology that works every time: e-mail, GPS, digital cameras. Dictation technology that relies on cellular Internet, though, only sort of works. And that can be jarring to encounter in this day and age.

But let's not throw the Siri out with the bathwater. The “virtual assistant” portion of Siri—all those commands to set an alarm, call someone, text someone, record an appointment—works solidly. Even if all you use are basic commands such as “Wake me at,” “Call,” “Text” and “Remind me,” you save time and fumbling.

Free-form cellular dictation is a not-there-yet technology. But as an interface for controlling our electronics, it makes the future of speech every bit as bright as Siri promised a year ago.

Just wait till she comes out of beta.

SCIENTIFIC AMERICAN ONLINE
Eight ways to boost Siri's voice recognition: ScientificAmerican.com/aug2012/pogue

This article was originally published with the title Siri, Why Aren't You Smarter?.

10 Comments

1. 1. lamorpa 09:11 AM 7/31/12

   Siri? If you are talking about voice recognition/response, why didn't you report on the more mature and accurate system built into Android phones that has been around for years. Just because it was not touted in nonsensical, totally unrealistic, cutsey commercials making believe famous people (and suggesting the general public) can express themselves colloquially and be understood (proven false), does not mean the successful Android-based solution does not exist. The impossible assertion of "copycat apps for Android phones" could only be true in a xenophobic, blinders-but-for-Apple fantasy world. I would expect a more scientific investigation (or any at all) in an article in a science magazine.

   Reply | Report Abuse | Link to this

2. 2. phylum_sinter in reply to lamorpa 09:26 AM 7/31/12

   I think that's a very good question.
   
   Though some may argue that Android's function that mirrors Siri is the me-too app, if you look at the timeline Android has had voice command before Siri was built in to iOS. It has matured and probably become more of a focus as a result of Siri.
   
   Maybe the headline should be modified to "Why Siri Is Still the Future: Speech-recognition software is great—unless you're trying to use it on an iPhone"
   
   ...If the article text is going to speak only of them. Generally though yes, i think a discussion of this stuff would be more scientific and interesting if it took not just the recent history of one product into account.

   Reply | Report Abuse | Link to this

3. 3. Bionate 12:13 PM 7/31/12

   as a disabled person I have been using speech recognition software for years. It's actually amazing how far it's come and nuance does make a great product. However, I think the failure with Apple was that they over touted what it was capable of. The simple truth is talking to a computer is not like talking to a person computers don't like inflection or inconsistency. However, I think once the iPhone has the computing power and networking power (4G anyone) the reintroduction of speech recognition technology will be something to see.

   Reply | Report Abuse | Link to this

4. 4. KevinCBrown 11:20 PM 7/31/12

   I completely disagree with all points. I am an owner of a competitor to Nuance, on top of being a speech recognition deployment expert complete with customer experience knowledge.
   
   I tried to contact David, but his email has my secure sending SSMTP server blacklisted which makes me wonder if he ever wants to hear from experts.
   
   The fact is David is five years younger than me biologically, while I lump him into the "clueless old fart" category.
   
   No clue what will copy over to this old fart, clear text interface, but will give it a shot and hope that the demons of the twentieth century don't bite my skinny arse!
   
   
   David,
   Have been reading your columns for years.
   
   Though I am five years older than you, this column mentally caused me place you in the "old fart" column for technology writers. It doesn't help that I've had several meetings today with people trying to "extend their knowledge" much further than reality, hoping that I would cut them some slack. So maybe I am the "old fart" with physical and mental "be nice" pains making me a little less generous today.
   
   To save you time, I will include the text here. This is a conversation in the Voice and Speech Specialists Group on LinkedIn. I am owner of the nearly 8,000 member group of primarily speech recognition experts.
   
   I am also Principal of Verbyx, a premier Automatic Speech Recognizer provider in direct competition with Nuance.
   
   Again, not looking to go hunting for your head, but the post that generated my response was yet another - YOU DON'T GET IT article about speech recognition from someone outside of our realm.
   
   Here's a quick link to my Inside Outsourcing column in Speech Technologies Magazine: http://www.speechtechmag.com/Authors/AuthorDetails.aspx?AuthorID=3657
   
   You're a smart guy, I invite you to join my group and respond in public. I don't offer this as an opportunity to bitch slap you, but rather to offer a partial retraction from me if you clarify what you were thinking that didn't come across in the original article, and keep a great dialogue going with some of the key people in the industry. We've had several of these open discussions in the past year about Siri and everyone comes off looking very sharp - we all are open minded on approaches, intent, language fluency and intent. In short, you can't go wrong by replying to my post. I promise that you'll love our people!
   
   To join: http://www.linkedin.com/groupRegistration?gid=78206
   
   Best regards,
   Kevin
   
   Kevin Brown
   Managing Director
   VoxPeritus

   Reply | Report Abuse | Link to this

5. 5. uconnron in reply to lamorpa 09:52 AM 8/1/12

   Well said, Lamorpa. I agree completely. I have often suspected that Apple has the media in their back pocket.

   Reply | Report Abuse | Link to this

6. 6. mypetclone 09:53 PM 8/1/12

   "But the dictation feature—the text-to-speech part—is provided by Nuance, the company that brought us software such as Dragon NaturallySpeaking."
   
   I'm pretty amazed that what I thought was a pretty good publication, in an article linked from ACM TechNews, has no real content AND a failed definition of its central topic.

   Reply | Report Abuse | Link to this

7. 7. samirsshah 09:34 PM 8/2/12

   Yes. Siri is still the future, the interacting vehicle will change from phone to glasses.
   

   Reply | Report Abuse | Link to this

8. 8. directaccess 06:38 AM 9/11/12

   I've recently discovered this NEW Smartphone app called TALKLER — EMAIL FOR YOUR EARS. They describe themselves as the FREE Smartphone app for heads-up, hands-off, reads-aloud-to-you, voice-controlled email. I recommend checking it out: Facebook.com/Talker and www.Talkler.com

   Reply | Report Abuse | Link to this

9. 9. cocoy_rey 02:05 AM 10/15/12

   RE Siri and email... Even with iOS6, and the new iPhone 5, Siri still won’t do email. We need true email inbox management that’s eyes-free and hands-free. There’s a smartphone app called Talkler —billed as “email for your ears.” It's a free smartphone app that's voice-controlled, and reads your emails aloud to you. There's more info at Talkler.com.

   Reply | Report Abuse | Link to this

10. 10. PhatPhred 12:36 AM 3/20/13

    As far as voice assistant goes Google Now is quite handy. As far as voice dictation goes Google Voice Input beats the pants off Nuance (as included with the Swype keyboard 1.4) not only in terms of recognition accuracy but also because Google Voice Input works with Airplane Mode turned ON (Android 4.1.1). Can Siri do that?

    Reply | Report Abuse | Link to this