Empowering Lives with AI Assistive Technology

Can artificial intelligence (AI) be integrated into assistive technology to empower people with communication disabilities? Explore the journey of Team Lulu as they set out to do just that in the bi-annual INSEAD Venture Competition (IVC) where they emerged winners of the Hoffmann Institute’s Social Impact Prize.

According to the World Health Organization, an estimated one billion people, or 15% of the world's population, experiences some form of disability, which limits the activities and participation they can have in society, often leading to discrimination and social exclusion.

Our research and analysis indicate that an estimated 78 million people worldwide have disorders that affect speech and motor skill disabilities, placing them at a need for assistive technology for their everyday lives. Even with current expensive assistive technology, people with communication disabilities find it difficult and tiresome to express their thoughts, while their interlocutors often lose patience due to the delay in response. Currently, there are no affordable and accessible solutions to solve these challenges, resulting in only 1/10 people with disabilities having access to the assistive technologies they need.

For our friend Gustavo Salas, Cerebral Palsy is a part of his reality. Meeting him and his family in Santiago, Chile in January 2020 through Amigos de Jesus, his mother and the organisation was kind enough to share more about their situation, becoming involved in the development of our project. Cerebral palsy makes it difficult for an individual to sometimes talk, and walking can require tremendous effort or be difficult altogether. Due to these challenges, blinking was the only way Gustavo was able to communicate with his family. His resilience and cheerful smile continued to inspire our team to create an affordable assistive app that will allow anyone in need like Gustavo, to communicate.

Wanting to empower millions of people like Gustavo, we developed our solution. Using the latest machine learning algorithm and deep learning we developed an intelligent Augmentative and Alternative Communication (AAC) software – Lulu. Our innovation is bringing the true capabilities of AI to assistive technology, which has massive potential in the efficiency of communication for speech-impaired individuals. By collecting information from our users' environment using the camera and microphone of a smartphone, tablet, or AR/VR headset, added to location and temporal awareness, and features such as face recognition and object detection, we can process several contextual cues to predict and suggests sentences for our users to communicate, becoming their voices. Powered by eye-tracking and compatible with any ability switch, our app can be downloaded in any smartphone to enhance our users’ communication.

Since the start of this project in October 2020, we have conducted surveys and user-testing with over 100 patients (and their caregivers) with different conditions, starting with amyotrophic lateral sclerosis (ALS), Multiple Sclerosis, and Cerebral Palsy. The data we gathered from every user and caregiver interaction enabled us to define Lulu's functionalities. We discovered, for example, that using object detection and speech recognition, with a trained model with up to 10 people and 50 objects, we can predict around 80% of all the communication requirements for our patients.

In January 2021, we finished developing our first prototype, a Windows app controlled by head movements that shows the power of our predictions using Object Detection and Speech Recognition. It allows the user to speak using a Text to Speech functionality and can also use a Symbol-based library for easy access to commonly used sentences. This prototype has allowed us to show a tangible product to different stakeholders and start the training process of our algorithm. It is also responsible for our proudest achievement to date - Gustavo's first words ever.

Our final validation came during April and June 2021, when we participated in the famous INSEAD Venture Competition (IVC). It was an intense but exciting experience where we got the opportunity to compete against exceptional ideas and teams. Thanks to a great team effort, including the addition of Mohamed Diabi (our teammate Raimundo’s classmate from INSEAD’s GEMBA program to the team as CTO), our hard work placed us third in the competition, and as the winners of the Social Impact prize sponsored by the Hoffmann Global Institute for Business and Society - the perfect validation that our solution can be a real contribution to fostering inclusivity of people that are unfortunately commonly overlooked.

This experience enabled us to share the intentions and journey of Lulu; from pitching in front of judges, explaining our mission and charting the path to get there. But the best feeling in the world is actually showing our product to users, having them test it and getting their feedback. Being there to witness what could be a life-changing product for them is the most rewarding experience anyone of us can have. That’s what really drives us: to be able to deliver a solution that will empower lives of so many people around the world with words they want to speak.

The INSEAD Venture Competition (IVC) is the flagship entrepreneurship experience for all current degree programme participants. This twice a year event represents a natural and exciting step in the development of INSEAD entrepreneurial students’ ventures, allowing them to pitch their start-up concept and business model. Throughout the multi-stage competition, students are immersed in a supportive yet rigorous environment to develop and showcase their venture ideas. Through extensive coaching and feedback from seasoned entrepreneurs and investors, participants gain valuable knowledge and experience on how to launch a start-up.