Skip to main content
Submitted by PatientsEngage on 18 February 2017

Jellow Communicator, a free app compatible with Android tablets and mobiles has been developed by an interdisciplinary team of therapists, graphic designers, software programmers and animators at the IDC School of Design in IIT Bombay. This app aids communication and learning in children on the autism spectrum and speech impairments. An interview with Sudha Srinivasan, post-doctoral fellow, IDC, and paediatric speech therapist, who provided clinical content for the app.

You and your team recently created this app for kids with Autism Spectrum Disorder (ASD) and development difficulties called Jellow Communicator? What does the app do?

Jellow Communicator is a freely downloadable Android app that can aid communication in children with speech impairments. The app uses child-friendly icons to help children communicate with their caregivers. A novel Visual Emotional Language Protocol (VELP) was conceived for Jellow. The app is composed of 9 central category buttons and 6 expressive side buttons. The content of the category buttons has been developed to include the common activities and objects that people encounter in their daily lives. The expressive side buttons of Jellow comprise like, don’t like, yes/want, no/don’t want, more, and less. To speak using Jellow, the user needs to simply press a category button followed by any of the expressive side-buttons. For example, by pressing the “Eating” category button and then the “like” button (happy face), the app will say, “I like to eat.”

Jellow is not only a communication aid for children but can also be used as a learning tool. For instance, the app can be used to teach children the sequential steps of daily activities such as brushing, going to the toilet, and bathing. Since the content is hierarchically arranged, it can also be used to teach children the concept of categories and sub-categories pertaining to body parts, animals and birds, colors, shapes, etc.

How is it going to help ASD kids in meaningful interaction, express emotions and feelings? Can you give examples.

We have envisioned that Jellow will be a communication aid for non-verbal and minimally verbal children. Since it is compatible with Android tablets and mobiles, we hope that children will be able to use the app throughout the day, as and when needed, to communicate their needs, feelings, likes and dislikes with caregivers and other people in different contexts. For example, with a couple of button-clicks, the child can indicate to their caregivers that they want to eat, or they want to play, or they want to go to the toilet etc. In addition to allowing them to request for their basic needs, the app also allows them to engage in meaningful interactions with others using the “Greet and Feel” category options. For example, this category has a sub-category of “Feelings” that allows children to express their emotions easily with their caregivers. Similarly, the “Greetings” sub-category includes common greetings that people generally use to start a conversations including, “Good morning”, “how are you?” etc.

As mentioned above, the content of Jellow is arranged hierarchically into categories and sub-categories. The user can access the vast library of icons by double-clicking on any of the category and sub-category buttons. The app has around 8200 lines of pre-programmed sentences each in English and Hindi. We have tried to encompass the child’s world within the central categories included in the app. But in addition, by using the keyboard feature, the user can make the app speak out aloud custom-typed sentences. This functionality would be of use to more advanced users who need to communicate more than the available content of the basic categories.

Do you think these apps and smart devices will help build up learning and sensory skills of ASD kids significantly?

As a clinician, I strongly believe that technology is an extremely powerful tool and it is upto us to use it wisely and judiciously to benefit our kids with ASD. Technology never can replace the expertise of a clinician. But it is a strong motivator that when used cleverly can aid the clinician in advancing the child’s functional skills and in increasing their participation in the society. So yes, I definitely think that these apps and devices have therapeutic value provided they are used under expert guidance with clear goals.

What was the biggest challenge you faced while designing this app?

Initially, we had conceived the app for Android tablets. But then we thought that in order to increase its accessibility, we should also make it available on Android mobiles as well. We faced several challenges in trying to make the app compatible with varying screen sizes and screen resolutions. In addition, another ongoing challenge we are facing is to see how we can make the app available in different Indian languages (other than Hindi). We are using the Google Text-to-Speech engine for this app, which is currently only available in English, Hindi, and Bengali. We are waiting for Google to develop Text-to-Speech engines in other Indian languages so that we can make the content available in other regional languages as well.

There are already several apps in the market for kids with ASD like Rufus, iPrompts, E-Mintza that are being used for learning and social development. Will Jellow Communicator offer something unique?

As mentioned before, Jellow is meant to aid communication in children with speech impairments. It falls under the category of Augmentative and Alternative Communication (AAC) which are aids that help to either augment or provide an alternative to communication in individuals with severe speech problems. One of the things that makes Jellow unique is that it has been developed by an interdisciplinary team of clinicians, software developers, graphic designers, and animators. We have tried to ensure that the user interface and experience is as intuitive as possible while also ensuring that the content is developed keeping in mind the unique needs of children with special needs. Moreover, some of the other strong points of the app are that it is available in English and Hindi, the content, icons, and accent have been Indianized to suit the needs of the users, and that it could serve as a multipurpose tool to aid communication and learning.

How does Jellow compare with Avaz, the award winning AAC app that too provides a voice for children with autism and has been listed on MIT's Top 35 Innovations?

The following are the points that contrast Jellow and Avaz -

  1. Functionality - Avaz is a comprehensive tool that primarily caters to more advanced users. Jellow will be more useful to users starting out with AAC since the interface design (especially the expressive buttons) allow the user to speak out sentences by just a couple of button clicks without needing to assemble component parts of the sentence.
  2. Languages - Avaz is available in multiple Indian languages but the vocabulary in these languages is limited since they are essentially using recorded voices to speak a limited number of sentences which are replayed as per the user's choices. In contrast, Jellow uses the Google text-to-speech (TTS) functionality which means that the app converts written text into synthetic speech.
  3. The user interface as well as the visual design of the layout and the icons are entirely different. Avaz includes a mix of stick figures and actual photos. Given that Jellow has been developed in a design school, the interface and icons have been carefully designed to make them as consistent, visually appealing, uncluttered, yet intuitive as possible.
  4. The content of Avaz and Jellow is also different in some respects - the activity sequences (within "Daily Activities") such as "Brushing", "Toilet", "bathing" etc. as well as the "Feelings" category are categories that are unique in Jellow.
  5. Cost - Most importantly, especially for a country like India, Avaz is a paid app whereas Jellow is free of cost. Jellow also works on Android mobiles and tablets since it is only 18 MB in size. Avaz has been developed for iOS and Android systems but it works only on iPADs or tablets since it is around 1GB in size.

Overall, I think that there are significant differences in the mode of operation as well as the overall design of both apps. While Avaz has a lot more functions in it which make it a versatile and customizable tool for advanced users, Jellow is a more beginner-level, easy-to-learn tool that helps users to request and communicate their basic and essential needs and feelings with caregivers.

If Jellow is offered free, how are you planning to sustain and continue to develop the app?

The funding for this project has come from - E-kalpa project sponsored under NMEICT by MHRD (Govt. of India) and Microsoft Design Expo 2004.

The Jellow Communicator was launched in January 2017. What has been the feedback and response so far?

So far there have been around 1200 downloads of the app. The preliminary feedback for the app has been very encouraging. Users have found the app to be simple and easy-to-use compared to other apps. They have also liked the fact that the app is available in English and Hindi and that it is compatible with Android tablets and mobiles. The plus point is that the content of the app has been customized to the Indian audience; for example, the food options in the app include items like “roti”, “dal”, “sabzi” etc. that kids will be able to relate to. Moreover, therapists have commented that the sub-categories of daily activity sequences and emotions are likely to be very useful for children with special needs.

Jellow can be downloaded for free on Google Play or at www.dsource.in/tools/jellow

You can contact us for more information or to leave your comments and suggestions at dsource.in@gmail.com

Condition