On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
People crossing the Mexican border into Nogales, Ariz., this week will have a chance to meet U.S. Customs and Border Protection's newest officer—a polite yet no-nonsense bilingual gatekeeper with a thick shock of black hair and a striped gray tie. He may not have a name or join his fellow officers for coffee or lunch breaks, but his presence will likely be welcomed both by them and the commuters who regularly pass through this southern Arizona outpost on their way to and from Mexico.
That is because the new recruit is an avatar, a virtual border patrol officer residing in a kiosk developed by researchers at the University of Arizona to facilitate border crossings.
CBP is actually installing an updated version of the University of Arizona's kiosk—the original was tested at the station from December to March—to determine its ability to help enroll applicants in its Trusted Traveler programs at the Mexican border. The programs, also available for airline passengers, were created after 9/11 at various ports of entry into the U.S. to expedite preapproved, low-risk travelers through dedicated lanes and kiosks. All Trusted Traveler applicants must voluntarily undergo a background check against criminal, law-enforcement, customs, immigration, agriculture and terrorist databases. The process also includes biometric fingerprint checks and an interview with a CBP officer.
In Nogales, human CBP officers monitor the avatar-administered pilot-test interviews, which provide them with automated feedback uploaded wirelessly to an iPad tablet that these officers can use to conduct follow-up interviews. Exchanges that the avatar flags as questionable and worthy of follow-up interrogation—using its speech recognition and voice anomaly–detection software—are color coded green, yellow or red to highlight the potential severity of questionable responses. Everyone who applies for Trusted Traveler status at Nogales ends up speaking with an officer after her or his avatar interview. One of CBP's goals is to implement several kiosks that can administer preliminary interviews that save time by making the follow-up, face-to-face interviews more efficient.
The kiosk is not designed to indicate that an interviewee is lying or to diagnose that person's intent, says Aaron Elkins, a University of Arizona postdoctoral researcher in the Management Information Systems department who helped develop the kiosk. Instead the kiosk analyzes an interviewee's voice for anomalies that may prompt a border officer to probe deeper into a particular response.
Anomaly detection is based on vocal characteristics—changes in factors such as rate, volume, pitch and intonation—that may be related to different emotional, arousal and cognitive states. An inflection in one's voice may indicate uncertainty, or a pause might imply that an interviewee may have been devising a deceptive answer, Elkins says. The kiosk's speech recognition software monitors the content of an interviewee's answers and can flag a response indicating when, for example, a person acknowledges having a criminal record.
Unlike its predecessor, which conversed only in English, the new Nogales kiosk speaks and understands both English and Spanish. The researchers also enhanced the speech recognition software. One of the problems with the first kiosk was that interviewees sometimes began answering a question before the avatar was finished asking it, causing the kiosk to miss the initial portion of the answer. The newer version is designed to more quickly detect when an interviewee is speaking and can prompt interviewees to repeat an answer if it does not understand a response.
The idea that physical cues can be used to determine intent often enough for a system to be both fair and effective remains unproved. This is why the U.S. Department of Homeland Security's Future Attribute Screening Technology (FAST) program has caught a lot of flack from privacy watchdogs, including the Electronic Privacy Information Center (EPIC). FAST proposes that travelers be subjected to an array of sensors measuring pulse rate, skin temperature, breathing, facial expression, body movement, pupil dilation and other physiological and behavioral factors to determine whether they are a security risk. The DHS has tested the technology, in the works since 2007, but its future remains unclear.
CBP contacted Elkins and his colleagues last year because it needed help processing a backlog of applications for its Trusted Traveler Programs. Although CBP gave them only about a month to build their first working prototype, the researchers jumped at the chance to take their "embodied conversational agent," as Elkins calls it, out of the lab and into a real-world setting.
Nogales is as real as it gets. In March Nogales Station officers on patrol seized more than 80 kilograms of hard narcotics from two Mexican men, including more than 77 kilograms of cocaine worth almost $2 million and five containers filled with three kilograms of methamphetamine worth nearly $86,000. In May CBP arrested a 43-year-old Mexican national in a wheelchair at Nogales Station for attempting to smuggle more than three kilograms of cocaine worth more than $65,500 into the U.S. in the chair's seat.
Elkins and his colleagues have spent the past few years developing software for analyzing vocal and written responses to questions. The researchers found that an avatar makes a significant difference in how interviewees answer questions. In lab tests prior to the avatar, "it was like they were talking to Moviefone," he says. When an avatar was added, people treated it almost like a person, at times even referring to it as "sir."
For now, the kiosk uses the same avatar regardless of whether it speaks Spanish or English, although the voice is different depending upon the language. "We are still researching the effects of changing how it looks right now, and the cultural implications, so we were reticent to make any major changes," Elkins says.
The researchers are hoping this second phase of the Nogales pilot test will include more than 1,000 interviews. After the kiosk's performance is analyzed, Elkins would like to add a feature that lets interviewees scan their passports and other documents into the device. Another option might be to equip the kiosk with some other sensors—such as video analyzers, eye trackers and thermal or infrared cameras—that could offer additional means of analyzing interviewees.