Ice Blocks to AC
Imagine massive blocks of ice, harvested and transported by thousands of people across the Illinois River, from the river itself and from surrounding lakes. Imagine waiting anxiously for your weekly delivery of ice, not knowing whether or not it would come intact, and getting through the summer months with little to no supply. This is not a tale from the bronze age, but the way people lived a little over a hundred years ago, in the United States.
Pretty hard to conceptualize when sipping a cool drink with cubes made by your own refrigerator, in the comfort of a nice AC keeping the room a comfortable 72 degrees. This innovation was born not out of a desire for comfort, but out of critical necessity. Dr. John Gorrie first began working on a cooling system when looking for a way to keep his feverish patients at a comfortable temperature, and to slow their decline. Gorrie took several previous inventions and made America’s first mechanical refrigerator, a machine that made ice. We are at just such a junction when it comes to the world of caring for our aging population, a time when innovation will be born out of necessity, and that is precisely what Sensi’s AI based Virtual Care Agent is.
While AI, or artificial intelligence may sound like something out of an Isaac Asimov novel to some, AI is becoming a prevalent part of our lives with self-driving cars, voice assistants such as Siri and Alexa, personalized marketing and more. It is now time to apply this same technology to a global crisis – that of population aging. With less people to care for our older adults, and more and more older adults choosing to age at home, good old human intelligence is simply not enough to care for our elderly at the standards that we would want for ourselves and our family. This is exactly what the founding team at Sensi.AI set out to do when building the world’s first virtual care agent. So what exactly is a virtual care agent and how does it work, and what unique challenges does building AI for seniors present?
Creating Tech Tools for Seniors
When setting out to build an AI tool that serves older adults, or any technology for seniors for that matter, it is important to understand that a lot of what is out there in the tech world was not developed with older adults in mind. This is an incredibly important point to understand, and needs to be kept in mind throughout the development process. This has been increasingly prevalent in phones, gadgets, tablets and other gadgets where companies have woken up to the fact that in order to serve this exponentially growing consumer class, you have to cater to them. Calibrating a product to the desired audience is especially important when it comes to AI. The data which is input into AI systems is ultimately what the algorithm will be able to determine, so if the input data excludes a particular group, the results for this group will not be accurate. This was demonstrated when facial recognition tools failed to accurately identify people of color. How does this come into play when building an AI tool for older adults?
Building a Virtual Care Agent
Sensi’s algorithms are trained on over 15 million interactions, and over 700,000 hours of audio. This data set is unique as it is trained on a one-of-a-kind data set from various senior care environments. Today’s common audio analysis solutions such audio classifiers, sentiment analysis, and various NLP models produce low accuracy results when it comes to home environments of older adults and care interactions. The acoustic parameters in care environments behave differently than in any other environment and the interpretation is different. For example, when a caregiver is talking out loud it can indicate hearing impairment of the older adult. We realized not only that we would have to build tools based solely on our data set, but also the unique potential of such a data set. In order to make a tool that accurately addresses the clinical needs of older adults, not only was it important to have the data sample used for training to actually come from older adults, but the right people needed to be involved in building the algorithm.
Prior to building Sensi’s algorithms, a clinical team which included people from the fields of gerontology, occupational therapy as well as care managers conducted research on audio collected and built the initial parameters for which anomalies and trends can be identified through audio. Sensi continues this research and integrates client feedback into research and development plans.
How it Works
So now that you know the challenges of building an AI tool for seniors, how does it actually work? Sensi’s pods are installed in homes of older adults and are distributed to cover the areas inhabited by the senior throughout the day, and where the care interactions occur (bedroom, bathroom, living room). The pods upload the audio to a secure cloud, where a fusion of several algorithms combine to understand the physical and mental state of the senior. These algorithms identify anomalies and trends pertaining to any changes in the mental and physical state of the older adult, as well as in the care interactions that occur in the home. After removing white noise and irrelevant sounds, the first level of sorting occurs with the environmental classifier. The environmental classifier is able to decipher hundreds of types of audio files, and can determine whether human speech or activity is involved. The classifier also determines whether speech is coming from an electronic source (television, phone speaker) or from a human source. When building an environmental classifier, the home environment of an older adult adds some challenges. Noises such as medical machinery (whose cessation can indicate an emergency situation), the sounds that accompany daily tasks such as lifting from the bed and toilet as well as showering are key indicators for the daily routine. Each step is accompanied by a clinical team who contributed their understanding of the unique audio DNA of a home-care environment. This input was key in determining which data was important for the environmental classifier.
The audio that the environmental classifier determines to be of human origin (groans, moans, crying, speech) are then sent to additional classification. Non-speech audio is used to identify occurrences such as mental or physical distress, or difficulty performing a daily task. Defining what exactly constitutes such an event was also done in close tandem with the clinical team, whose understanding of geriatric home care environments helped overcome the challenge that accompanies attempting to understand complex human interactions. Everything determined to be speech is processed by Sensi’s unique NLP (or natural language processing) model that uses speech to text technology paired with Sensi’s data set to detect anomalies – or what’s being said. This model is bolstered and complimented by the Tonal classifier which determines sentiment, or how something is being said. The reality of a home-care environment presents several major challenges when analyzing speech. The first two challenges are sound based – older adults often have difficulty speaking as mental and physical ailments affect their motor speech abilities. That is why a unique data set was necessary in order to fine-tune speech to text and NLP models. Additionally, many of the caregivers who work with older adults are foreign, and the model had to take into account a variety of accents and cadences of speech.
The tonal, or sentiment classifier is used together with the NLP model to ensure the system does not put out false positives. If an older adult says something that triggers the NLP, but is determined to be in a playful tone by the sentiment classifier, it will be flagged and a certain percentage of certainty is assigned to the occurrence. Events that are graded above a 90% of certainty are sent out automatically, while events with a lower grade are reviewed by a human in the loop.
Sensi uses its blend of AI models to provide information to agencies about the successful completion of the daily care protocols of a certain environment as well as anomalies that represent a deviation from the norm. Sensi analyzes client/caregiver matching through identifying positive and negative interactions, as well as resistance to cate, tracks changes in mental and cognitive state and sends push notifications to the agency when an emergency situation has been detected. While Sensi provides daily updates through notifications that appear on the dashboard of each client, Sensi’s real strength is using models to identify trends and changes that can point to cognitive or physical decline, as well as changes in the caregiver/client relationship. This trend analysis can help to lead to earlier diagnosis of dementia and Alzheimer’s, guide agencies to perform physical reassessments, find risk factors in the home environment, and identify caregiver burnout early on, preventing maltreatment and abuse.
The Future is Knocking
In some ways in the home care industry, we are still harvesting ice and transporting it down the Illinois. We are continuing to rely solely on high-touch human interactions in our care for our elderly, without preparing for the necessity of innovative solutions, when we simply won’t have enough human care to meet demands. While we will never be able to replace human interaction, nor should we strive to, we must look towards the future and build tools that will supplement and perfect our ability to give care. We are not replacing human caregivers, rather empowering them with the data necessary to optimize their time, and do a stellar job. The future is already knocking, and Sensi has answered the door.
