When Voice Applications Fail

October 10, 2005


It’s always frustrating to hear people complain about poorly designed telephone applications, particularly when these complaints relate to government-sponsored applications. (Encouraging good design principles for developing voice applications is kind of the point of this here blog ““ “˜ya dig?)

You can imagine how disheartened I was to read a story cataloging a litany of complaints about one such system in my hometown newspaper. This article focuses on some extremely negative experiences by callers to the Medicare help line (1-800-MEDICARE). In an analysis of the Medicare help line, the General Accounting Office concluded late last year that less than two thirds of callers to this “help” line received accurate answers to their questions.

To be perfectly fair, there are few things in this world ““ with the possible exception of the U.S. Tax Code ““ as complex as the Medicare program. Not a surprise considering it’s the largest health insurance program in the nation, covering some 40 million Americans (and counting!). When eligibility and benefit rules are complex, the job of providing clear concise answers to recipient questions is difficult, so some degree of frustration is probably inevitable.

It’s also more likely that a phone system set up to answer benefit questions will have to utilize a mixture of automated dialogs and customer service representatives. Given the nature of the program, developing a fully automated system would seem to be highly impractical.

However, when I read about some of the negative caller experiences, I can’t do anything but shake my head:

After two minutes of following instructions from an automated female voice, Bingham arrived at a list of choices.

“If you’re calling on the Medicare discount drug cards say, ‘Drug card,’ ” the voice said. “For Medicare health plans, say ‘Plan choices.’ ”

“Plan choices,” Bingham said.

“I’m sorry I didn’t understand you,” the voice said.

“Plan choices!” Bingham said again.

“I’m sorry, I still didn’t understand you.”

Bingham brought the telephone two inches from her lips.

“PLAN CHOICES!”

“Please hold a moment while I transfer you to a customer service representative who can help you.”

This is just a fundamental lack of proper menu construction and grammar tuning ““ the fact that these things do not appear to have been done for an application as heavily used as this one is almost criminal. At a minimum, the menu should accept both spoken and DTMF input so that a caller can use their key pad to enter a numeric choice if the application is having trouble recognizing what they are saying.

`
POORLY DESIGNED MENU STRUCTURE:</p>


I'm sorry I didn't understand you.
I'm sorry I didn't understand you.
Welcome to the Medicare help line. If you're calling on the Medicare discount drug cards say, Drug card. For Medicare health plans, say Plan Choices.

drug cards
plan choices

` This type of menu structure has several flaws. First, the prompts are too dissimilar to the menu choices "“ if the desired input for the second item is "plan choices," why doesn't the prompt direct the user to this input? "For Medicare health plans, say Plan Choices." should be "For Medicare health plans, say Health Plans." This increases the likelihood that the caller will provide the right input. Second, the menu does not allow for approximate input "“ if a caller says simply "cards," or "discount cards" the application will not recognize the input. Under the VoiceXML 2.0 specification, the [default setting](http://www.w3.org/TR/voicexml20/#dml2.2.1) for menu input is "exact" — in other words, the VoiceXML interpreter will look for an exact match on the menu items, and will not recognize input that includes some (but not all) of the words in the menu item. Third, the menu does not allow for DTMF entry, which would allow a caller to fall back to their key pad for entry if the application was having trouble recognizing their input. Properly constructed voice applications will check for the type of input being provided, and direct callers to modify it accordingly (demonstrated below). Finally, the menu does not use tapered prompts to assist the caller in refining their input, or selecting the appropriate input type. `
PROPERLY DESIGNED MENU STRUCTURE:


</p>



I'm sorry. I didn't hear what you said.


I'm having trouble hearing you. If you are using a speaker phone, you may want to pick up your telephone handset or make sure your phone is not on mute


I'm sorry I'm still having trouble hearing you. Please wait while I transfer you to a customer service representative.


I'm sorry I didn't understand what you said.



I'm still having trouble hearing you. Please try entering your selection using your touch tone key pad.

  
I didn't get that. Please try entering your selection again.



I'm sorry. I'm still having trouble understanding your selection. Please wait while I transfer you to a customer service representative.

Welcome to the Medicare help line.

For the Medicare service, say , or press .


discount drug cards
health plans
</menu>` Additionally, a good system will verify user input if the confidence level on recognition is lower than a pre-specified threshold. There is a good overview of this technique in an [earlier posting](http://www.voiceingov.org/blog/?page_id=33) on this site. This is pretty basic stuff "“ even if the application developers didn't know enough to take this approach when they built the system, it is most certainly something that should have been uncovered during testing. This is the kind of second rate development that gives voice applications a bad rep.