Text-to-Speech for Mobile Apps

Altova MobileTogether supports text-to-speech functionality for cross-platform mobile apps, empowering developers to add speech and build a rich multi-media user experience.

Speech can be fully integrated with text anywhere it appears in an app. For instance, an app like the Parcel Delivery MobileTogether example can be enhanced by speaking the next delivery address aloud – especially if the driver is in traffic! Or, text in message boxes of other information blocks can also be spoken by the mobile device.

We created a simple demo to experiment with the functionality of text-to-speech for mobile apps then added text-to-speech features to the Parcel Delivery app.

The very simple text-to-speech demo app has four features:

• An input field control lets the user type any text or speak some preloaded default text
• A Speak button vocalizes text from the input
• The Silence button stops if speaking is in progress
• The Exit button offers a message box with a spoken message and an option to quit the app

Here is the app running on an Android device with text in the input field:

Text in an app can come from many different sources, including the mobile device copy/paste clipboard. In the screenshot below, an iPhone user deleted the default text, copied from the Altova Web site, and pasted into the input field:

The input field property for multiple lines is selected, and the box resizes automatically to fit the pasted text.

Text collected in the input field is stored in an element in the app data tree. The action assigned to the Speak button causes the built-in mobile device text-to-speech engine to speak the text.

This single action defined in the MobileTogether Actions dialog works regardless whether the app runs on an Android, iOS, or Windows phone, on a Windows 10 computer, or even in a Web browser on another device.

We assigned Text to Speech action directly to the actions for the Speak button, but it can also be used in an action group. We used an action group for the Exit button. When the user clicks Exit, we want to both display message box and speak the message, as seen here on an Android phone:

Here are the actions defined for the Exit button:

The phrase “Are you sure you want to quit this app” is stored in the data tree and used in XPath expressions for the Text to Speech highlighted action and the MessageBox below it.

When the user clicks OK, a Loop action waits for Text to Speech to complete before the app is cancelled.

Updating the Parcel Delivery App with Text-to-Speech for Mobile Apps

The Parcel Delivery example app was originally created to demonstrate how to work with geo-location data in MobileTogether. In this simulation, the user is a driver for a delivery company assigned to deliver parcels from a warehouse in New Jersey to various destinations in New York City.

When each new destination is assigned, the app displays a Delivery page with the address and current distance to destination, as seen in this view from the MobileTogether Simulator for an Android phone:

We want to add text-to-speech for each new destination and distance as the driver travels.

First, we can add a new user setting on the Configuration page with a switch control to enable or disable text-to-speech. Here is the revised Configuration page running on an iPhone:

Looking at the Delivery page in the Designer, we can see the destination, current location, and distance to destination are all derived from XPath expressions:

These same expressions can be assigned to text-to-speech actions. We first created an action group to speak each new destination. The area highlighted in blue below shows the text-to-speech action. We used the string function concat to add a few additional words to build a conversational phrase that includes the address and starting distance:

As the driver proceeds toward the destination, geolocation is periodically updated and the distance to destination is refreshed. We created a second action group to speak the new distance:

The loop at the top is similar to the action defined for the Exit button of our demo app described above. In this case it prevents the new text-to-speech action from interrupting if the previous text-to-speech action has not completed. This could otherwise happen if a geolocation update occurs before the previous vocalization completed.

Our text-to-speech action groups must be called during execution of the app. First, we can place the Vocalize Next Destination group in the sequence for OnPageLoad actions for the Delivery page:

The Delivery page is loaded once for the first delivery. For the second through last deliveries, we can add the action group to the existing Proceed with Next Destination action group:

All we have left to do is place the Vocalize New Distance action group in OnPageRefresh actions for the Delivery Page:

Now, whether the app executes in the MobileTogether Simulator, or on any mobile device, the driver receives spoken updates and does not have to look away from the street!

You can try creating text-to-speech for mobile apps for yourself by downloading the free-to-use MobileTogether Designer, which comes with integrated help, tutorials and many sample apps including Parcel Delivery.

The solutions described in this post can be downloaded at https://github.com/altova/MobileTogether-TTS

Open the either .mtd file in the MobileTogether Designer to examine how it was built and execute it in the MobileTogether Simulator.

Connect to the app store for your device and download the MobileTogether Mobile App.  Then connect to the MobileTogether Demo Server to run more demo apps. With the MobileTogether Mobile App installed, you can also try the text-to-speech for mobile apps on your own device by using the MobileTogether Designer as a server.