Leveraging AI in AAC to Improve Conversations

Much of the discourse around AI today centers on concerns that it will replace humans, destroy our cognitive abilities, or invade our privacy. In the world of augmentative and alternative communication (AAC), similar concerns arise around AI-generated content replacing authentic communication from device users. Although similar fears around AAC replacing “authentic communication” have been widely debunked, the question remains: How can AAC device developers integrate AI in ways that help (literally, augment) communication for people with communication challenges while mitigating concerns around privacy, authenticity, and usability?

Lingraphica has been on the forefront of leveraging new technologies to assist people with communication challenges since 1990. Our goal has always been to create research-backed, science-based solutions to help people with communication challenges, their care partners, and their speech-language pathologists communicate more clearly, achieve therapy goals, and connect with others. With the rise of AI in so many aspects of our lives, we set about finding a way to integrate it usefully and ethically for our users.

The Problem Conversations Solves: Spontaneous Speech

One of the most common difficulties for people with communication challenges is spontaneous conversation – the kind of back-and-forth that happens naturally in life. While traditional AAC tools like pre-recorded icon cards, text-to-speech, and whiteboard are extremely useful in conveying information, these take advanced planning and setup. Also, it can take time to create a response using these tools, and conversations can move on quickly. Even experienced AAC device users can feel left behind in these situations.

These spontaneous conversations can arise in almost any situation: unexpected questions at a doctor’s office, chit chat about current events, even making plans for the day. But they’re also more than just moments to answer simple questions. They are opportunities to help a device user express their identity, build rapport with others, advocate for themselves, and build confidence in themselves and their ability to communicate.

We heard this clearly from users, SLPs, and care partners: people wanted a way to participate in conversations in the moment, not just pull from something they’d pre-programmed. Our team at Lingraphica decided to create Conversations, an app on Lingraphica’s AAC devices, powered by Lingraphica’s Hub platform, to address the gap for our users and their care partners.

Developing Conversations

Conversations is built into Lingraphica’s Hub platform and powered by a custom AI backend developed in-house, called the Conversations API. The backend is hosted with data separation in mind, so that Lingraphica can ensure HIPAA compliance and privacy for our users.

At its core, Conversations transcribes what’s being said and generates AI-powered response suggestions the user can choose from. The tools shows the conversation transcript and offers multiple suggested responses based on the context.

Users can also add keywords to help steer the suggestions toward what they want to say, so if they’re talking about a doctor’s appointment, they can flag that to get more relevant responses. If a suggestion doesn’t land, users can click “regenerate” for new options.

At the request level, the program processes each statement in the context of prior statements and the type of statement being replied to. Optional keyword input and intent selection (agree, ask a question, etc.) get folded in as additional context and instruction. When a user regenerates, that choice further informs the kind of answer they’re looking for, helping the system hone in on what’s actually useful.

We’ve built dedicated infrastructure for intelligently routing requests based on complexity and speed requirements, caching inputs, and incorporating reported feedback. That same infrastructure also powers the keyword suggestions themselves, so the system is doing more than generating replies; it’s actively helping users find the right words to guide those replies. Everything is extensively tested through our benchmarking tools before rollout, validating both response quality and response time.

Initial Results

During our initial tests of Conversations, we had a group of 13 people with communication impairments try the app. We vetted the group based on strong auditory comprehension, spoken language, cognition, and reading abilities.

While the initial test group was small and not fully representative of the spectrum of people with communication challenges, we found some stunning results:

• 56% of users completed a full conversation using the traditional layout of Lingraphica Hub apps (Talk icon-based cards, Type text-to-talk, Draw whiteboard, Media, or Scenes), while 100% of users completed the conversation using Conversations.
• It took users 1 minute 14 seconds on average to complete a conversation using the traditional layout, while users on the Conversations app took 15 seconds.

Additionally, when asked what they thought about the Conversations app, the users rated it an average of 4.8 out of 5 in terms of experience and an average of 4.9 out of 5 for usefulness.

Beta testing has also provided promising results. SLPs and users report integrating Conversations with other tools to support existing communication infrastructure; using Conversations respond more quickly in the moment at the doctor’s office; and even using Conversations to practice everyday moments before they happen (like your coffee order at Starbucks). Because the app still provides text-based responses, users with higher reading levels can use the script to prompt their speech, reading the responses aloud instead of having the tool speak aloud for them.

Key Differentiators

Most AAC tools are built around what you plan to say. Conversations is built around what’s actually happening in front of you. The AI is reading the live transcript and generating contextually relevant response options, so the user isn’t hunting through folders or pages.

Compared to other AI developments, Conversations isn’t a wrapper around ChatGPT or Claude. It’s a purpose-built system designed from the ground up to handle the specific constraints and goals of AAC communication, where response time, relevance, and user agency all matter equally.

Additionally, Conversations API does not feed info to a cloud-based, public LLM, like publicly available AI chatbots. All information is stored securely in a compliant backend. The data we collect is anonymized and opt-in only, to allow us to improve the application from real user scenarios.

Finally, we developed Conversations with input from Lingraphica’s in-house licensed speech-language pathologists. Our internal product and development teams have rare access to the clinicians who actually understand what our users need and how they use their devices. This input is invaluable to our design process, and integral to how Lingraphica works as a company.

The Future of Conversations

As with all Lingraphica technology, our team is constantly finding ways to improve Conversations. We’re using anonymized data to improve the tool based on how people actually use it. We want users to have more control over the conversation itself, such as being able to signal when they want to change the subject, who they’re talking to, or what kind of response they want to give. We’re also invested in improving the quality and relevance of the suggestions Conversations provides.

In the longer term, we want Conversations to have even deeper personalization, so the tool gets smarter the more someone uses it, as most generative AI tools do. We are working on that while keeping an eye on privacy and user safety, which is always paramount.

There’s also a lot of potential in expanding Conversations to more languages. We have started exploring Spanish support. And while Conversations has not been tested on pediatric users, it’s possible to find applications for the tool in that environment.

The north star throughout all of this is empowerment. The goal isn’t to speak for the user; it’s to surface the best possible options so they can make the choice. That distinction shapes every design and engineering decision we make.