NASA Tech Tuesday: Seeing Is Communicating

Communicating when a traumatic brain injury, stroke, or disease has made speech impossible can be daunting. Specialized eye-tracking technology uses eye movement to enable people living with disabilities to connect one-on-one over the phone or via the internet.

Eye-tracking systems for computers pinpoint a person’s gaze – where the eye is looking at a screen – by reflecting infrared light off the cornea and capturing it with a camera, using image-processing software to determine the eye’s orientation. The technology isn’t new, but it has become much more widely accessible, thanks in part to a collaboration between NASA’s Jet Propulsion Laboratory in Southern California and a Fairfax, Virginia-based company called Eyegaze Inc.

When the company built the first model in 1988, its computers were bulky, requiring three shipping boxes for equipment and a company staff member to set up the system. That cost limited access, and the learning process could be intimidating.

In 1998, NASA and Eyegaze entered a public-private partnership via Congressional funding to make the hardware smaller, more portable, and affordable without compromising efficiency. It also reduced the weight of the original system by six times and its volume by nearly the same factor. Other advancements served as a springboard for two more decades of development. By collaborating with JPL, the two entities were able to miniaturize and improve the company’s Eyegaze Edge system and lower costs, eliminating barriers to ownership of this communications technology.

“Working with NASA, we were able to make the device less bulky,” said Preethi Vaidyanathan, an engineer with Eyegaze. “Since then, we integrated the external components into a small camera.” It mounts above or below a standard computer screen and requires less than 15 seconds to calibrate to an individual’s gaze.

Visual Surfing

eyegaze-4

Minesweeper and other video games adapted to use eye-tracking technology are just one form of entertainment made possible by EyeGaze for people living with disabilities.

Credit: EyeGaze Inc.

As personal electronic devices and internet access became commonplace, Eyegaze customers wanted to do more than type. The company’s work with NASA and other government partners put it in a position to meet that demand.

Eyeworld integrates with computers, mobile phones, and tablets, allowing the Eyegaze camera to function as an external mouse and keyboard. By enabling almost any computer function, it lets users chat online, post to social media, send emails, text, or make phone calls. It’s also possible to change room lighting, adjust thermostat settings, and operate other environmental controls using Amazon Echo and Google Home via pages of specialized buttons with one-glance button controls.

During the COVID-19 pandemic, Eyegaze integrated communications programs such as Zoom and WhatsApp to allow people to connect with their family and friends. To help combat a sense of isolation, the company added meditation apps and Window Swap, which stream the views from different windows worldwide. Virtual museums take the viewer on guided tours. A music streaming service syncs to favorite music apps and offers the International Radio Garden app to sample music from other cultures. Access to online gaming communities provides another way to connect.

“We are constantly thinking about what the customers want. One thing is freedom of movement, so controlling a wheelchair gives them that to some extent,” said Vaidyanathan. Eyegaze Edge integrates with the Ability Drive application used with motorized mobility devices. Looking at a specific button enables hands-free control of a wheelchair’s motion.

The Right to Speak

eyegaze-3

Eye-tracking hardware and software make it possible for a person to communicate using the movement of their eyes. This keyboard screen is one of many Eyegaze developed for people with a full range of eye motion. A vertical keyboard is available for those who can only look up and down.

Credit: EyeGaze Inc.

Numerous medical conditions can lead to the kind of physical paralysis making it impossible to speak. Well-publicized brain and spinal cord injuries in athletes have led to broader awareness. But there is a host of other causes, such as cerebral palsy, ALS, multiple sclerosis, muscular dystrophy, and spinal muscular atrophy. These conditions can result in locked-in syndrome, which paralyzes all voluntary muscles except those that control eye movement. Eye-tracking software is a lifeline for patients, using what may be their only remaining voluntary movement, but even that can be impeded.

Eyegaze Edge measures several eye features, making it possible for the tracker to work even when less of the pupil is visible. The company employs clinical specialists who can troubleshoot issues that may prevent customers from using the system, eye conditions like cataracts, dry eyes, and eyelids that droop, partly covering the pupil.

The company now serves individuals in 44 countries, including nations in Southeast Asia and Africa. Adults, children as young as 18 months old, military veterans, and others can communicate using only their eyes. If only vertical eye movement is possible, Eyegaze Edge offers a special vertical keyboard screen. Pages of commonly used phrases communicate a statement with a single glance. Large, easy-to-use buttons allow patients to communicate directly with healthcare providers and caregivers for everyday needs.

Clinicians use Eyegaze Edge and teach their patients how to communicate using the technology. In some cases, Vaidyanathan said, this technology even allows people to share their final thoughts with family and friends.

“NASA helped us get our technology to the size of a laptop – small and sleek. But we continue to investigate and design our solutions to meet ongoing needs. Communication is a key right, so we must accommodate these different challenges,” she said.

NASA has a long history of transferring technology to the private sector. The agency’s Spinoff publication profiles NASA technologies that have transformed into commercial products and services, demonstrating the broader benefits of America’s investment in its space program. Spinoff is a publication of the Technology Transfer program in NASA’s Space Technology Mission Directorate (STMD).

For more information on how NASA brings space technology down to Earth, visit:

spinoff.nasa.gov

Written by Margo Pierce
NASA’s Spinoff Publication

Recommend0 recommendationsPublished in Marstronaut

Related Articles

5 Essential Frameworks for Preventing Violent Child Death

The U.S. has a violent child death problem. Developing strategies to prevent violent child deaths death from firearms and traffic crashes is a demanding task that requires consideration of numerous upstream, interrelated, and tangential issues. To help safety advocates develop strategies to prevent violent child death, we compiled five frameworks to help: Understand and explain …
The post 5 Essential Frameworks for Preventing Violent Child Death appeared first on Salud America.

What Are the Risk and Protective Factors for Violent Child Death?

Gun violence and traffic crashes may seem like unpredictable events. But they are not random. They are systematic. Data reveal trends and patterns in gun violence and traffic crashes that can help us identify risk factors and protective factors. This is especially important for addressing violent child deaths. So what does the data show? Join …
The post What Are the Risk and Protective Factors for Violent Child Death? appeared first on Salud America.

As Social Need Screening Advances, Transportation Remains an Afterthought

Some big changes in 2022 and 2023 have set up the healthcare sector to advance screening for non-medical social needs in 2024 and beyond. This is great news as we work to address social determinants of health (SDoH), improve health outcomes, and reduce health disparities. But one key social need – transportation – isn’t getting …
The post As Social Need Screening Advances, Transportation Remains an Afterthought appeared first on Salud America.

We Need to Recognize Toxic Stress as a Health Condition with Clinical Implications

There is a common health condition with serious medical consequences that has not been nationally recognized by the medical or public health community—toxic stress response. Toxic stress is the body’s response to prolonged trauma─like abuse or discrimination─with no support. It can harm lifelong mental, physical, and behavioral health, especially for Latinos and others of color. …
The post We Need to Recognize Toxic Stress as a Health Condition with Clinical Implications appeared first on Salud America.