Interacting with James

The user can give James instructions or ask questions using the following methods:

  1. Voice Commands

  2. Tablet

  3. ZBOS Control App

Voice Commands

James can be controlled by using voice-commands.

James will listen to pre-programmed voice commands after the wakeword "OK James" has been understood. Stand in front of the robot for the best results.

Supported languages

The currently supported languages on James are the following:

Language ASR / Listening TTS / Speaking

Japanese

No

Yes

Thai (th-TH)

Yes

Yes

Hungarian

Yes

Yes

Arabic (ar-WW)

Yes

Yes

French (fr-FR)

Yes

Yes

English (en-US)

Yes

Yes

English (en-GB)

Yes

No

Dutch (nl-NL)

Yes

Yes

German (de-DE)

Yes

Yes

Russian

Yes

Yes

Chinese (zh-CN)

Yes

Yes

French (Canadian)

Yes

Yes
The language the robot will listen to can be set in the option speech-language of the kiosk. When connecting ZBOS Control and stating messages, only the TTS languages that the robot can speak will be shown.

Settings

The following speech-related settings can be found in the advanced-section of the settings in the Kiosk.

  1. Listen to voice-commands

  2. Rotate the robot when the wake-word is recognized

These options can be toggled on or off to suit the user’s needs.

Indicator

The microphone at the bottom of the home-screen on the tablet can also be used as an indicator to see if James is listening.

Not listening

Listening
NotRadiating
Radiating

Prerequisites for the voice commands

Not all voice commands can be used all of the time. Some voice-commands will only be available once the following conditions have been met:

  1. Settings of the robot

The setting 'Spoken commands' in the general settings needs to be turned ON before the robot will respond to the wake-word "Okay James" and start listening.

If the option 'Spoken commands' is off, the option 'Rotate the robot when wake-word is recognized' will be greyed out and unable to be changed.

  1. Internet

Certain features require an active internet-connection to retrieve the latest data and to keep up-to-date.

For example:

  • Weather

  • Verklizan or VoIP/SIP

  • Installed apps

  1. VoIP

In addition to having a connection to the internet, a stable connection is needed to make calls using VoIP-services (the minimum bandwidth needed for a VoIP-call is between 85 and 100 kbps).

The section VoIP / SIP in the settings has been created to enter the account-information needed to call the Verklizan help center (or any other VoIP-service). A SIP-account needs to be registered at Verklizan before any VoIP-calls to their help-center can be made.

Voice Commands

The following is (an extract of) the list of voice-commands.

Command Actions taken by the robot

Call help

Triggers an alarm at Verklizan. (1) (2)

I need/want help

What is the date of today
What is the date (today)

The robot states the date according to location specifications.

What year is it?

The robot states the current year.

What’s the day?
Which day are we today?

The robot states the day of the week.
(e.g. Monday)

What time is it?
How late is it?

The robot states the time as hour + minute.

Hello

The robot states a self-introduction.

Goodbye

The robot says goodbye.

Introduce yourself
Who are you

The robot introduces itself

What will the weather be like this morning/afternoon/evening?

The robot states the weather forecast for the specified time of day.

What is the weather for today
What is the weather forecast for today

The robot states the weather forecast for the day.

What will the weather be like (the day after) tomorrow

The robot states the weather forecast for the specified date.

Go to <Point-Of-Interest>

The robot will move to the registered POI (3).

Enable/disable offer more help feature

The robot will (no longer) state the message ‘Can I help you with something else’.

(1): This option is only available for robots where this option has been installed and configured.

(2): Confirmation is required before the alarm will be triggered and is only available if a valid SIP- and VoIP is configured in the settings of James.

(3): The correct map needs to be loaded on the robot.

Advanced features

James can be further customized so the following options can be used:

  1. Turn to the source of the voice

James will turn to the direction of the voice if the wake-word has been understood and the option ‘Rotate the robot when wake-word is recognized’ is enabled.

  1. Display the command in the kiosk

If the kiosk-app (home screen) is active, the recognized command will be displayed on the tablet.

  1. Start an activity when no input has been detected (inactivity)

James can start an activity (composition, moving to charging station, …​) if no input has been detected by the user for some time. This action can then be stopped by any user-interaction (speech, tapping the tablet, …​)

Tablet

On the tablet, the Kiosk app is the default launcher for ZBOS-powered Android devices. This app is used as:

  • Main display

  • User-interface to configure features

  • Platform to launch other apps

For more information, see the Kiosk-manual

ZBOS Control App

Please refer to the ZBOS Control manual for more information on using ZBOS Control to interact with James.