Speech

API for communication with ZBOS by Zora Robotics.

Channels

zbos/dialog/languages/available/get

PUB Get available languages

Legacy topic for getting all installed languages (both tts and asr combined). Better not to use this.see zbos/dialog/languages/available/response/\{key} for response

Channel tags

  • Speech

KeyResult

Random key

Payload

Name

Type

Description

Accepted values

key

string

Required random key

Any
Examples of payload
{
  "key": "ABCxyz"
}

zbos/dialog/languages/available/response/\{key}

SUB response: Get available languages

response: Legacy topic for getting all installed languages (both tts and asr combined)

Channel tags

  • Speech

Array<String>
Payload

Name

Type

Description

Accepted values

Message Payload

array (string)

Any
Examples of payload
"en-US"
"nl-BE"
"fr-FR"

zbos/dialog/languages/available/event

SUB event: Available languages changed

Published when the available languages have changed

Channel tags

  • Speech

Array<LanguageInfo>
Payload

Name

Type

Description

Accepted values

Message Payload

array (object)

Any

Message Payload. language_code

string

Any
Examples of payload
{
  "language_code": "en-US"
}
{
  "language_code": "nl-NL"
}

zbos/dialog/languages/current/get

PUB Get current language

see zbos/dialog/languages/current/response/\{key} for response

Channel tags

  • Speech

KeyResult

Random key

Payload

Name

Type

Description

Accepted values

key

string

Required random key

Any
Examples of payload
{
  "key": "ABCxyz"
}

zbos/dialog/languages/current/response/\{key}

SUB response: Get current language

Channel tags

  • Speech

String
Payload

Name

Type

Description

Accepted values

Message Payload

string

Any
Examples of payload
"en-US"

zbos/dialog/languages/current/set

PUB Set current language

Set the active language, example: 'en-US'. Note: this will be probably be changed to zbos/dialog/set/language in a future release.

Channel tags

  • Speech

String
Payload

Name

Type

Description

Accepted values

Message Payload

string

Any
Examples of payload
"en-US"

zbos/dialog/set/message

PUB Speak a message

Use this to make the robot say something.

Channel tags

  • Speech

String
Payload

Name

Type

Description

Accepted values

Message Payload

string

Any
Examples of payload
"Hello world"

zbos/dialog/set

PUB Speak a message with parameters

Use this to make the robot say something with parameters. Only message is required, other params are optional.

Channel tags

  • Speech

DialogOptions
Payload

Name

Type

Description

Accepted values

requestId

string

Any

message

string

Any

speed

integer

Any

language

string

Language string like "en-US"

Any

volume

integer

Any

gesticulation

boolean

Any

voice

string

Any

pitch

integer

Any

blocking

boolean

Any
Examples of payload
{
  "requestId": "1",
  "message": "Hello world",
  "speed": 50,
  "language": "en-US",
  "volume": 50,
  "gesticulation": true,
  "voice": "Ava",
  "pitch": 120
}

zbos/dialog/languages/current/event

SUB event: Current language

Event when the system is done with changing language to a new language.

Channel tags

  • Speech

String
Payload

Name

Type

Description

Accepted values

Message Payload

string

Any
Examples of payload
"en-US"

zbos/dialog/animatedspeech/enable

PUB Enable animated speech

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/dialog/animatedspeech/disable

PUB Disable animated speech

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/dialog/animatedspeech/get

PUB Get status of animated speech

see zbos/dialog/animatedspeech/response/\{key} for response

Channel tags

  • Speech

KeyResult

Random key

Payload

Name

Type

Description

Accepted values

key

string

Required random key

Any
Examples of payload
{
  "key": "ABCxyz"
}

zbos/dialog/animatedspeech/response/\{key}

SUB response: Get status of animated speech

Channel tags

  • Speech

Boolean
Payload

Name

Type

Description

Accepted values

Message Payload

boolean

Any
Example of payload (generated)
true

zbos/dialog/tts/start

PUB event: TTS started

Event when the robot starts speaking and what it is saying

Channel tags

  • Speech

String
Payload

Name

Type

Description

Accepted values

Message Payload

string

Any
Examples of payload
"Hi, my name is James"

zbos/dialog/tts/stop

PUB Stop robot speech

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/dialog/tts/end

SUB event: TTS ended

Event when the robot has finished speaking

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/dialog/hotword/recognised

SUB event: robot has recognised a hotword

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/event/dialog/listen/started

SUB event: robot starts listening

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/event/dialog/listen/stopped

SUB event: robot stops listening

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/dialog/listen/start

PUB Start listening

Start listening for hotword and commands

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/dialog/listen/stop

PUB Stop listening

Cancel hotword recognition

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/dialog/service/start

PUB Start dialog service

Start the dialog service so the robot can listen to the mic.

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/dialog/service/stop

PUB Stop dialog service

Stop the dialog service so the mic is free to use by other applications.

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/dialog/service/stopped

SUB Event: dialog service stopped

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/dialog/service/started

SUB Event: dialog service started

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/iflytek/rotation/started

SUB Iflytek rotation started

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/iflytek/rotation/stopped

SUB Iflytek rotation stopped

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/tts/phonemereached

SUB event. Phoneme reached

Channel tags

  • Speech

PhonemeInfo
Payload

Name

Type

Description

Accepted values

jawOpen

integer

Range between 0 (jaw closed) and 255 (jaw open)

Any

lipTension

integer

Range between 0 (relaxed) and 255 (tense)

Any

mouthHeight

integer

Range between 0 (mouth closed) and 255 (mouth open)

Any

mouthUpturn

integer

Range between 0 (mouth corner turned down) and 255 (mouth corner turned up)

Any

mouthWidth

integer

Range between 0 (mouth puckered) and 255 (mouth wide)

Any

teethLowerVisible

integer

Range between 0 (hidden), 128 (teeth visible) and 255 (gums visible)

Any

teethUpperVisible

integer

Range between 0 (hidden), 128 (teeth visible) and 255 (gums visible)

Any

tonguePosition

integer

Range between 0 (tongue flat) and 255 (tongue against upper teeth)

Any
Examples of payload
{
  "jawOpen": 128,
  "lipTension": 0,
  "mouthHeight": 128,
  "mouthUpturn": 64,
  "mouthWidth": 255,
  "teethLowerVisible": 128,
  "teethUpperVisible": 128,
  "tonguePosition": 255
}

zbos/dialog/grammars/add/multiple

PUB Grammars: add multiple

This is an extension for the grammar add topic. It removes all grammars first and then triggers a single platform reload. Hence grammars are added a lot faster.see zbos/dialog/grammars/add/response for response

Channel tags

  • Speech

Array<CustomGrammar>
Payload

Name

Type

Description

Accepted values

Message Payload

array (object)

Any

Message Payload. id

string

a unique id of the grammar. This must be unique among the application (next field).

Any

Message Payload. app

string

The application name. This will be used to determined which MQTT topic will be used to post the json output to.

Any

Message Payload. continuous

boolean

Any

Message Payload. input

object

Contains an array for each language with all the possible sentences. The sentences need to be in a format that the speech software understands.

Any

Message Payload. variables

object

Any
Examples of payload
{
  "id": "question_1_variable",
  "app": "App",
  "continuous": true,
  "input": {
    "nl-BE": [
      "Ken jij $name"
    ],
    "en-US": [
      "Do you know $name"
    ]
  },
  "variables": {
    "name": {
      "nl-BE": [
        {
          "input": "Arno",
          "data": "name_in_dutch"
        },
        {
          "input": "Gilles",
          "data": "name_in_dutch"
        }
      ],
      "en-US": [
        {
          "input": "Arno",
          "data": "name_in_english"
        },
        {
          "input": "Gilles",
          "data": "name_in_english"
        }
      ]
    }
  }
}

zbos/dialog/grammars/add/response

SUB response: grammar add

Channel tags

  • Speech

SuccessMessage

Success message

Payload

Name

Type

Description

Accepted values

success

boolean

Any

message

string

Optional error message

Any
Examples of payload
{
  "success": true
}

zbos/dialog/grammars/remove/multiple

PUB Grammars: remove multiple

This is an extension for the grammar remove topic. It removes all grammars first and then triggers a single platform reload. Hence grammars are removed a lot faster.see zbos/dialog/grammars/remove/response for response

Channel tags

  • Speech

Array<CustomGrammar>
Payload

Name

Type

Description

Accepted values

Message Payload

array (object)

Any

Message Payload. id

string

a unique id of the grammar. This must be unique among the application (next field).

Any

Message Payload. app

string

The application name. This will be used to determined which MQTT topic will be used to post the json output to.

Any

Message Payload. continuous

boolean

Any

Message Payload. input

object

Contains an array for each language with all the possible sentences. The sentences need to be in a format that the speech software understands.

Any

Message Payload. variables

object

Any
Examples of payload
{
  "id": "id send in add",
  "app": "app from add"
}

zbos/dialog/grammars/remove/response

SUB response: grammar add

Channel tags

  • Speech

SuccessMessage

Success message

Payload

Name

Type

Description

Accepted values

success

boolean

Any

message

string

Optional error message

Any
Examples of payload
{
  "success": true
}

zbos/dialog/grammars/match/\{appId}

PUB Event: grammar match

Event when a custom grammar match is found, the output data is posted here.

Channel tags

  • Speech

GrammarMatch
Payload

Name

Type

Description

Accepted values

app

string

Any

triggered-by

string

The id of the grammer that has matched with the user input. This is the id from the input json.

Any

language

string

Language that speech software was configured in when the sentence is recognised.

Any

variables

object

Any
Examples of payload
{
  "language": "en-US",
  "variables": {
    "beverage": {
      "input": "Wine",
      "data": "this is wine"
    },
    "dish": {
      "input": "hamburger",
      "data": "this is a hamburger"
    }
  },
  "triggered-by": "input ID"
}

zbos/dialog/custom/grammar/rules/get

PUB Gets all custom grammar rules

see zbos/dialog/custom/grammar/rules/get/response for response

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/dialog/custom/grammar/rules/get/response

SUB response: Get all custom grammar rules

Channel tags

  • Speech

Array<CustomGrammar>
Payload

Name

Type

Description

Accepted values

Message Payload

array (object)

Any

Message Payload. id

string

a unique id of the grammar. This must be unique among the application (next field).

Any

Message Payload. app

string

The application name. This will be used to determined which MQTT topic will be used to post the json output to.

Any

Message Payload. continuous

boolean

Any

Message Payload. input

object

Contains an array for each language with all the possible sentences. The sentences need to be in a format that the speech software understands.

Any

Message Payload. variables

object

Any
Example of payload (generated)
[
  {
    "id": "string",
    "app": "string",
    "continuous": true,
    "input": {},
    "variables": {}
  }
]

zbos/dialog/tts/languages/get

PUB Get TTS languages

Get a list of languages that the TTS engine can use to talk.see zbos/dialog/tts/languages/response/\{key} for response

Channel tags

  • Speech

KeyResult

Random key

Payload

Name

Type

Description

Accepted values

key

string

Required random key

Any
Examples of payload
{
  "key": "ABCxyz"
}

zbos/dialog/tts/languages/response/\{key}

SUB response: TTS languages list

Channel tags

  • Speech

Array<TtsVoiceInfo>
Payload

Name

Type

Description

Accepted values

Message Payload

array (object)

Any

Message Payload. service

string

Any

Message Payload. name

string

Any

Message Payload. age

string

Any

Message Payload. language

string

Any

Message Payload. languageCode

string

Any

Message Payload. gender

string

Any

Message Payload. languageVersion

string

Any

Message Payload. frequency

integer

Any
Examples of payload
"BE"

zbos/dialog/asr/languages/get

PUB Get ASR languages

Get a list of languages recognized by the ASR engine. note: For now this is limited to the language code defined in the config. No Vocon load check is done yet.see zbos/dialog/asr/languages/response/\{key} for response

Channel tags

  • Speech

KeyResult

Random key

Payload

Name

Type

Description

Accepted values

key

string

Required random key

Any
Examples of payload
{
  "key": "ABCxyz"
}

zbos/dialog/asr/languages/response/\{key}

SUB response: ASR languages list

Channel tags

  • Speech

Array<OdpAsrLanguagesResponse>
Payload

Name

Type

Description

Accepted values

Message Payload

array (object)

Any

Message Payload. languageCode

string

Any
Examples of payload
"nl-BE"

zbos/asr/recognition/result

SUB Speech recognised event

Event sent by the dialog service when user said something. Mainly for subtitles.

Channel tags

  • Speech

String
Payload

Name

Type

Description

Accepted values

Message Payload

string

Any
Examples of payload
"Okay James"

zbos/dialog/asr/pause

PUB Pause ASR

Channel tags

  • Speech

KeyResult

Random key

Payload

Name

Type

Description

Accepted values

key

string

Required random key

Any
Examples of payload
{
  "key": "ABCxyz"
}

zbos/dialog/asr/pause/response/\{key}

SUB response: Pause ASR

Channel tags

  • Speech

SuccessMessage

Success message

Payload

Name

Type

Description

Accepted values

success

boolean

Any

message

string

Optional error message

Any
Examples of payload
{
  "success": true
}

zbos/dialog/asr/pause/event

SUB event: Pause ASR

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/dialog/asr/resume

PUB Resume ASR

Channel tags

  • Speech

KeyResult

Random key

Payload

Name

Type

Description

Accepted values

key

string

Required random key

Any
Examples of payload
{
  "key": "ABCxyz"
}

zbos/dialog/asr/resume/response/\{key}

SUB response: Resume ASR

Channel tags

  • Speech

SuccessMessage

Success message

Payload

Name

Type

Description

Accepted values

success

boolean

Any

message

string

Optional error message

Any
Examples of payload
{
  "success": true
}

zbos/dialog/asr/resume/event

SUB event: Resume ASR

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/dialog/asr/state/get

PUB Get ASR state

Channel tags

  • Speech

KeyResult

Random key

Payload

Name

Type

Description

Accepted values

key

string

Required random key

Any
Examples of payload
{
  "key": "ABCxyz"
}

zbos/dialog/asr/state/get/response/\{key}

SUB response: Get ASR state

Channel tags

  • Speech

AsrState
Payload

Name

Type

Description

Accepted values

paused

boolean

Any

enabled

boolean

Any
Examples of payload
{
  "paused": true,
  "enabled": true
}

zbos/dialog/text/command

PUB Send a text message to dialog system

Channel tags

  • Speech

String
Payload

Name

Type

Description

Accepted values

Message Payload

string

Any
Example of payload (generated)
"string"

zbos/dialog/language/setup/start

PUB Start language setup

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/dialog/language/setup/stop

PUB Stop language setup

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/dialog/set/config

PUB Set the current speech config

see zbos/dialog/set/config/response/\{key} for response

Channel tags

  • Speech

SetDialogConfigRequest
Payload

Name

Type

Description

Accepted values

config_items

object

One or more config items.

Any

key

string

Required random key

Any
Examples of payload
{
  "key": "ABCxyz",
  "config_items": {}
}

zbos/dialog/set/config/response/\{key}

SUB response: Speech config was set

Channel tags

  • Speech

EmptyMessage

Empty message

zbos/dialog/get/config

PUB Get the current speech config

see zbos/dialog/get/config/response for response

Channel tags

  • Speech

GetDialogConfigRequest
Payload

Name

Type

Description

Accepted values

config_items

array (string)

One or more config keys.

Any

key

string

Required random key

Any
Examples of payload
{
  "key": "ABCxyz",
  "config_items": []
}

zbos/dialog/get/config/response

SUB response: Current dialog config

Channel tags

  • Speech

Map
Payload

Name

Type

Description

Accepted values

Message Payload

object

Any
Examples of payload
{
  "tts.phonemes": true,
  "asr.language": "en-US"
}