Bionic Voice

Demystifying Human Voice Generation

Bionic Voice is an electronic prosthesis for patients who lose their larynx due to cancer. It functions as an artificial larynx for these patients to restore their missing voice. Bionics is the science of replacing an amputated limb of the body with an electronic prosthesis. Despite the advances in many fields of Bionics (such as Bionic arm or Bionic ear), there has been always a gap in designing a modern Bionic Voice prosthesis to generate voice for larynx amputees (people who lose their larynx due to cancer).

Laryngectomy is a surgical procedure to remove the larynx in advanced stages of larynx cancer. The existing gold standard to provide voice after laryngectomy [1] is a robust solution but it is quite primitive and inherently incapable of generating a natural voice. The other alternative has been the manually controlled Electrolarynx with a robotic sound quality, which has not been much improved despite more than 50 years of research [2].

The Bionic Voice research team at the MARCS Institute at Western Sydney University are pleased to be the pioneering team globally who have invented a novel Bionic Voice solution (Figure 1) with a clear proof of feasibility which is expected to be a paradigm shift in designing future electronic voice prostheses.

We have published our latest achievements in PLOS-ONE:
A Pneumatic Bionic Voice Prosthesis, Pre-clinical Trials of Controlling the Voice Onset and Offset: http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0192257 (opens in a new window)

Authors: Farzaneh Ahmadi*, Farzad Noorian**, Daniel Novakovic**, André van Schaik*

* The MARCS Institute for Brain Behaviour and Development, Western Sydney University, ** The University of Sydney

Bionic voice f1
Figure 1: The Bionic Voice Prosthesis

Bionic Voice – What Bionic voice is and “is not”

We get this question often, so we think we should clarify that Bionic Voice is NOT a technology to generate “speech” for patients who are paralysed or people who suffer e.g. from ALS. Laryngectomy patients (larynx amputees) who will use Bionic Voice are not paralysed. They simply have lost their voice-box (vocal folds) and are unable to generate any voice. When you (as a healthy person) speak, your vocal folds generate a sound for you. We call this sound “the voice”. You then shape your voice into speech by moving your face, tongue, and lip muscles. Laryngectomy patients are also capable of moving their face, tongue and lip muscles, but they lack a voice source to speak with.

Bionic Voice functions as an artificial larynx which is actively involved in voice generation for these patient as they speak (in real-time). Ideally the Bionic Voice generates a voice similar to what the missing vocal folds used to generate.

The rationale – Why do we believe our Bionic Voice is a game changer?

The complicated structure of human larynx had made the task of mimicking its voice generation function, a hard problem to solve. Yet the team at the MARCS Institute have managed to simplify the human larynx in terms of voice generation. We have now established that using respiration without any input from the nerves or muscles of the larynx is sufficient to generate a high-quality and intelligible voice for the patients as good as the existing gold standard. This respiratory driven Bionic Voice source is what we call the “Pneumatic Bionic Voice”.

Why do we claim that is reparation enough to drive a Bionic Voice source? The Pneumatic Bionic Voice we have invented is an electronic adaptation of a device called the “Pneumatic Artificial Larynx” (PAL). The PAL is an old school whistle like voice prosthesis that laryngectomy patients used to talk with. The PAL is exclusively driven by respiration and (figure 2), there is significant proof that the PAL’s quality of voice is even better than the existing gold standard [3-7].

Pneumatic devicePneumatic device

Figure 2. The Pneumatic Artificial Larynx (PAL)

The PAL is less popular due to its cumbersome design (Figure 2). Yet it provides an excellent reference for designing Pneumatic Bionic Voice prostheses. In the Pneumatic Bionic Voice prosthesis, voice generation is performed by a software model of the PAL. The software monitors respiration variations and similarly generates voice. The generated voice is then played inside the mouth for the patient to speak with (Figure 1). The clear advantage of Pneumatic Bionic Voice is that, similar to the PAL and unlike the existing gold standard, the prosthesis will remain non-invasive and non-surgical. This and the amazingly high quality of the PAL voice is why we believe the Pneumatic Bionic Voice will be the future of voice prostheses.

Bionic Voice – What have we done so far?

To design a Pneumatic Bionic Voice source, we needed to answer two questions. 1) how can we enable the patient to control his/her Bionic Voice source naturally and 2) how can we generate a more natural “human” voice. The two questions are equally important. Using the PAL as a reference, our research goal in essence is to train our algorithms to reach a similar functionality as the PAL both in terms of 1) controlling the voice and 2) generating a similar waveform.

So far, we have successfully answered the first question: How does the PAL enable a laryngectomy patient to “control” his voice using respiration? In our pre-clinical trials, our patient can now control his Bionic Voice source in slow and fast speech rates in real-time. The performance of the Pneumatic Bionic Voice source matches the mechanical PAL with 98.4% accuracy. We have published our findings in PLOS-ONE. To the best of our knowledge this is the first time a functional Bionic Voice prosthesis has been implemented that is naturally controlled by a laryngectomy patient at slow and fast speech rates.

A short comparison with other research in the field: The term Bionic Voice has been coined in 2007 at Nanyang Technological University of Singapore as an umbrella term for exploring alternative ways of generating voice for the voice loss. Since then the Bionic Voice research has become more focused on designing naturally controlled voice prostheses to replace the missing larynx [8-10]. The main focus of Bionic Voice research before our current research program had been to use the neural/muscle activity that remained after the laryngectomy to restore voice. This is what is called the “Myoelectric Bionic Voice” [8-10]. Myoelectric Bionic Voice prostheses are based on solid scientific foundations [11] and they seem to be effective in controlling the intonation of the voice [8] but they face several challenges to turning them into a functional voice prosthesis after laryngectomy. Please refer to our latest paper [12] to see why we believe the Pneumatic Bionic Voice is the more feasible option for the future of electronic voice prostheses.

Bionic Voice – What’s happening now?

In case you are asking, yes! We have built the Pneumatic Bionic Voice source and our laryngectomy participant can now control it naturally and speak with it. We are now partnering with another great team at University of Nagoya to answer the second question: To match the performance of the PAL in terms of generating a similar degree of naturalness!

Stay tuned!! We will provide updates of the progress.

Who we are

The Bionic Voice Research Team:

Bionic Voice is a proud collaboration of:

  • The MARCS Institute for Brain Behavior and Development
  • The University of Nagoya

Funding

The Bionic Voice research has been supported by the Garnett Pass and Rodney Williams Memorial Foundation, the largest funding body of ENT field of research in Australia.

References

[1] Hilgers, Frans JM, and Paul F. Schouwenburg. "A new low‐resistance, self‐retaining prosthesis (Provox™) for voice rehabilitation after total laryngectomy." The Laryngoscope100.11 (1990): 1202-1207.

[2] Meltzner, Geoffrey S., and Robert E. Hillman. "Impact of aberrant acoustic properties on the perception of sound quality in electrolarynx speech." Journal of Speech, Language, and Hearing Research 48.4 (2005): 766-779.

[3] T. Y. Ching, R. Williams, and A. V. Hasselt, "Communication of lexical tones in Cantonese alaryngeal speech," Journal of Speech, Language and Hearing Research, vol. 37, p. 557, 1994.

[4] M. Ng, "Speech performance of adult Cantonese-speaking laryngectomees using different types of alaryngeal phonation," Journal of Voice, vol. 11, pp. 338-344, 1997.

[5] I. Law, "Speech intelligibility, acceptability, and communication-related quality of life in Chinese alaryngeal speakers," Archives of Otolaryngology—Head & Neck Surgery, vol. 135, p. 704, 2009.

[6] S. Bennett and B. Weinberg, "Acceptability Ratings of Normal, Esophageal, and Artificial Larynx Speech," Journal of Speech, Language, and Hearing Research, vol. 16, pp. 608-615, 1973.

[7] B. Weinberg and A. Riekena, "Speech produced with the Tokyo artificial larynx," Journal of Speech and Hearing Disorders, vol. 38, p. 383, 1973.

[8] Ahmadi, Farzaneh, Matheus Araújo Ribeiro, and Mark Halaki. "Surface electromyography of neck strap muscles for estimating the intended pitch of a bionic voice source." Biomedical Circuits and Systems Conference (BioCAS), 2014 IEEE. IEEE, 2014.

[9] Fuchs, Anna Katharina, Clemens Amon, and Martin Hagmüller. "Speech/Non-Speech Detection for Electro-Larynx Speech Using EMG." BIOSIGNALS. 2015.

[10] Fuchs, Anna Katharina, Martin Hagmüller, and Gernot Kubin. "The New Bionic Electro-Larynx Speech System." IEEE Journal of Selected Topics in Signal Processing 10.5 (2016): 952-961.

[11] Goldstein, Ehab A., et al. "Design and implementation of a hands-free electrolarynx device controlled by neck strap muscle electromyographic activity." IEEE Transactions on Biomedical Engineering 51.2 (2004): 325-332.

[12] Ahmadi  Farzaneh, Farzad Noorian, Daniel Novakovic, André van Schaik, A Pneumatic Bionic Voice Prosthesis, Pre-clinical Trials of Controlling the Voice Onset and Offset , PLOS-One, 2018.