Interactive demo

We have programmed an interactive demonstration that can be used for showcasing visitors the interactive capabilities of Pepper as well as Robot House functions and the interplay between them.

Starting the demo

To start the demonstration, you have to start the demo script on one of the pre-configured robot house computers. Please follow Starting up to prepare the robot.

Note

You can safely omit starting Choregraphe but it will not interfere in case you want to have it ready for other demonstrations, e.g. Applications.

Setup your terminal’s Robot environment to Pepper (and common robot house infrastructure) and set a ROS IP:
```
$ robot_env common pepper
$ ros_ip
```
Start the script:
```
$ rosrun uh_pepper demo_interactive.py
```
If not awake, Pepper will wake up and calibrate. Blue LEDs at the ears signal an active dialogue system.
Use verbal commands to trigger behaviours, see below.

Interacting with Pepper

You can verbally interact with Pepper in an interactive demo that currently has the following functionalities:

Hello and goodbye
Play a Promotional video
Execute a Dancing routine
Display Photo albums
Face learning and recognition
Set a reminder
Change its Degree of interactivity

When Pepper is deployed in Robot House, it additionally has the following functions:

Display the Sensor map
Activity detection reports
Device manipulation

The below sections explain how to trigger and use these functions in detail. First, they will present a general command structure that let’s you start or stop a certain routine. These commands usually contain some keywords, like command or reference which should be replaced by a list of words or phrases as indicated in a table below the command. Commands also may contain optional words or phrases that might be left out when speaking, indicated like this.

Often, the command structure looks like this, where the words Pepper and please are optional, followed by some command:

Pepper please do something.

Functions available everywhere

Hello and goodbye

This routine lets you start and end an interaction with Pepper where the robot will reply with a randomly selected greeting or farewell phrase.

Command structure:

greeting Pepper.

Example Utterancess

Hello.

Good bye Pepper.

greeting

Effect

hello

how are you?

good morning

good afternoon

are you all right?

what’s up?

hi

hiya

hey

Pepper greets back

good bye

have a lovely evening

see you later

take care

Pepper says goodbye

Promotional video

Pepper can display a promotional video, by default a video of Robot House, when asked to do so. The video can be played, paused, restarted and stopped.

Command structure:

Pepper please command the reference.

Example Utterances

Pepper can I see the video.

Please show me Robot House.

`command`	Effect
show show me open play I want to see I would like to see can I see	(Re-)starts the video
pause	Pauses the video (stays visible)
resume	Resumes a paused video
stop hide close	Stops the video and hides it

reference

video

promotion video

project video

Robot House

Note

The video can also be started or stopped by triggering the robot’s back bumper with your foot and paused using its left bumper. Moreover, all sound output can be muted with the robot’s right bumper.

Dancing routine

With the dancing routine, Pepper can display some sequence where it plays the song Greased Lightning and performs some dance moves. A still image of the movie is shown on its screen.

Command structure:

Pepper please command get active together.

Example Utterances

Let’s get active.

command

Effect

I want to

let’s

Starts dancing routine (Grease)

Note

The behaviour can be stopped by triggering the robot’s back bumper with your foot. Moreover, all sound output can be muted with the robot’s right bumper.

Photo albums

Using this routine, Pepper can display a curated list of photo albums from unsplash. Currently available are albums about travel, animals, nature, and arts & culture.

Command structure:

Pepper please command modifier * reference.

*modifier is optional and valid with display command only

Example Utterances

Show some photo albums.

I want to see other photos.

command

Effect

show

show me

open

display

I want to see

I would like to see

can I see

Displays a randomly selected photo album

hide

close

stop showing

Hide photo album

modifier

some

more

other

reference

photos

images

pictures

photo albums

Face learning and recognition

This routine makes use of Pepper’s people perception apabilities. The robot can attempt to learn a face and associate it with a name that is typed in on its virtual keyboard. It can also be asked to recognise a face and recall the associated name.

Command structure:

Pepper please command reference.

Example Utterances

Please remember my name.

Recognise me.

command

Effect

remember

learn

Start face learning routine

forget

Forget face currently seen

recognise

List people in field of view

reference

my name

my face

me

who I am

Set a reminder

This routine allows you to set a reminder to be triggered after a certain time (currently limited to 1, 2, 5, 10, 20 minutes). The topic or reason for the reminder can be entered via a virtual keyboard on the robot’s tablet.

Command structure:

Pepper please command.

Example Utterances

Pepper set a reminder.

`command`	Effect
set a reminder	Start reminder routine asking for duration and topic then triggering reminder after timeout

Degree of Interactivity

It is sometimes desirable to disable the interactive dialogue or Pepper’s head movements, for exmaple when explaining to an audience or taking photos. This routine can temporarily disable and re-enable such functions.

Command structure:

Pepper please command.

Example Utterances

Pepper stop listening.

Stop moving.

`command`	Effect
start moving resume moving	(Re-)start pepper attention towards person
stop moving stand still	Stop pepper attention towards person and look straight
stop listening stop talking be quiet ignore us	Stop pepper dialogue
go to sleep	Stop dialogue and attention towards person, go to resting posture
wake up	(Re-)start dialogue and attention towards person, go to resting posture

Note

The dialogue can also be started or stopped by touching the robot’s left hand. Attention can be started or stopped by touching the robot’s right hand. Moreover, all sound output can be muted when triggering the robot’s right bumper with your foot.

Functions only in Robot House

Sensor map

When deployed in Robot House, Pepper can display the floor plan of the house with a live view of the smart home sensors overlayed.

Command structure:

Pepper please command the reference.

Example Utterances

Hide the sensor map.

Please show the sensors.

command

Effect

show

show me

open

display

I want to see

I would like to see

can I see

Displays a website that contains the Robot House sensor map

hide

close

stop showing

Hide website with sensor map

reference

map

sensors

sensor map

floor plan

Activity detection

Pepper can report on various activities in Robot House using the smart sensor infrastructure, potentially devising some recommendations for actions like increasing the room temperature or taking a drink.

Command structure:

Pepper please inform my report.

Example Utterances

Pepper summarise my activities today.

Tell me about my hydration.

inform

Effect

summarise

tell me about

report

Generates a summary report based on sensor readings

report

Type

activities today

bed, TV, room usage, etc.

hydration

hydration level

tap, fridge, kettle usage, etc.

Device manipulation

You can use the following commands to manipulate certain devices (or device groups) in the house.

Command structure for enabling a device:

Pepper please switch/turn on the device.

Command structure for disabling a device:

Pepper please switch/turn the device off.

Example Utterances

Pepper switch on the kettle.

Please turn the coffee machine off.

`device`	Description
light(s)	Living room lights
kettle	Kettle plug (kitchen)
coffee machine	Coffee machine plug (kitchen)

Note

Individual commands are more flexible, see below.

Individual commands	Description
Prepare/make a/some tea Boil some water	Kettle plug (kitchen) on
Stop the kettle/making tea	Kettle plug (kitchen) off
Prepare/make a/some coffee	Coffee machine plug (kitchen) on
Stop the kettle/making tea	Coffee machine plug (kitchen) off
Charge your friends/the/all robots	All robot charging plugs on
Stop charging your friends/the/all robots	All robot charging plugs off
Increase the brightness/illumination.	Increase brightness (living) 30%
Decrease the brightness/illumination.	Decrease brightness (living) 30%