Augmented Listening and AI

By Drew McCormack

Lately, I’ve been trying to decide on the best terminology to use with Glisten, to best capture its purpose. Until now, I’ve been using terms like “Podcast  Player for Language Learners” and “Listening Workouts”, but these terms are incomplete, and not immediately understandable. I’ve struggled with it, but I think I have now stumbled on some terms that are keepers.

The plan is to start referring to Glisten as a “Language Player”. Terms like “Audio Player” or “Podcast Player” don’t fully express what this app is about, which is playing back audio/video that contains language, and to do this in creative ways conducive to language learning. For now, that includes automatically looping sentences, inserting pauses, and mixing in slow downs, but this is just the beginning…

Let me explain: If you are like me, you are probably sick of hearing about AI, but I think language learning is one place where it can help a lot. In fact, AI is already used in the text extraction for Glisten, which is the basis of everything the app does. But AI also opens up much more exciting possibilities. 

To this point, Glisten has just been using AI to present text, and determine where to loop sentences, but AI can do much more. It can translate, it can interpret and summarize, and it can generate. These possibilities open up a whole realm of new functionality for Glisten’s listening workouts.

This brings me to the second new term I plan to start using: Augmented Listening (AL). We’ve all heard of Augmented Reality, which is mostly about inserting virtual objects into a real 3D scene. The objects typically help you with real world activities in some way. They might present some information about the real world; you could use a virtual screen for your work or to watch a movie, or add a 3D virtual chair from Ikea to see how it looks in the room. 

I believe this same opportunity exists with audio, and nobody is taking full advantage of it. If you can augment your vision, why not your hearing? And language learners stand to gain most: imagine you can listen to the same spoken content that native speakers listen to, but with an app like Glisten augmenting it to help you understand. It would automatically repeat sections, translate and read aloud sentences in your own native language, and insert explanations, hints or observations about grammar.

Augmented Listening offers enormous opportunities, and we are just getting started. My goal is to make Glisten the Augmented Listening app for language learners.