User Tools

Site Tools


public:bmlr:smartbody

BML Realizer - SmartBody

About SmartBody

SmartBody is an open sourced, modular framework for animating embodied conversational agents in real time, based on the notion of hierarchically connected animation controllers. Using the Behavior Markup Language, a growing standard of human behavior description for real time animated characters, SmartBody is capable of carefully synchronized skeletal animation and speech behaviors, independent of game engine or rendering environment. It is the product of years of work at the University of Southern California, and built on the Motion Engine animation library by Marcelo Kallmann.

http://www.smartbody-anim.org/

SmartBody can be run in a distributed environment and uses ActiveMQ as a message broker. It allows external programs to send BML commands to ActiveMQ that are later parsed by SmartBody. SmartBody in the BML Realizer toolkit is not configured for distributed environment by default. In the BML Realizer toolkit, the renderer is able to send commands back to SmartBody instead of external programs.

SmartBody resources

SmartBody requirements

SmartBody is written in C++ for the Windows platform. All dependencies are included, see Libraries below. To build SmartBody, Visual Studio 2008 or Visual C++ 2008 (free) is required.

SmartBody should run on any fairly recent computer.

Known issues

  • Speech BML element must be the first BML element in a block.
  • Does not support all BML elements, see BML support below.
  • Sync points only work within the same BML request. Synchronizing between multiple BML requests or between characters is not supported.
  • Synchpoints are not fully working, see Synchpoint hack below.
  • Non-ASCII characters in Speech BML elements may not work.
  • SmartBody will output a lot of error messages regarding missing “audio synch points” when using the Speech BML element. Don't worry about it.

Architecture


SmartBody takes BML input and converts it to joint rotations on characters. The renderer listens to SmartBody animates the characters with the given joint rotations.

Libraries

The following libraries are used by SmartBody and are stored in the SmartBody\lib folder. Some are only needed for the distributed version of SmartBody.

External open-source libraries:

Name Req. Description
activemq No Apache ActiveMQ is a messaging broker
boost Yes Boost is a collection of C++ Libraries extend the functionality of C++. Only the used libraries are included
pthreads No Library that defines an API for creating and manipulating threads
sdl Yes Window manager and multimedia library
xerces-c Yes XML parser

Internal libraries, created at ISI:

Name Req. Description
commapi Yes CommAPI is a library that encapsulates the communication to the renderer
playsound Yes Small lightweight library that lets you play sounds
vhmsg No Wrapper around ActiveMQ
wsp No Wrapper around tt_utils

If a library is not labeled required, it means that it is only needed if you want to build the distributed version of SmartBody.

Compiling

After downloading the source from the BMLR website, open the SmartBody-CADIA.sln file in Visual studio 2008, pick the correct build target and build.

There are 4 build targets, two without ActiveMQ (named “Debug-NOAMQ” and “Release-NOAMQ”) and two with ActiveMQ (“Debug” and “Release”). You might need to do a “rebuild all” when changing to or from “NOAMQ” build targets.

To build the distributed versions you need to download the additional SmartBody_Amq_libs file from the BMLR download site and extract to your SmartBody\libs directory.

Running

SmartBody Standalone (Without ActiveMQ message broker)

  • Debug: sbm-fltkd_noamq.exe
  • Release: sbm-fltk_noamq.exe

SmartBody Distributed

  • Debug: sbm-fltkd.exe
  • Release: sbm-fltk.exe

If no arguments are provided, SmartBody will load the default.seq file from the same directory as the exe file.

Command line arguments

These are the command line arguments sbm.exe accepts. All of them are optional.

Argument Effect Default value
-host= Hostname to send render data to None
-seq Path to .seq file to run on startup.
Takes full file path or only file name.
sbm-dfl-init
-procid= ? ?
-audio If specified then internal audio will be played False
-lockdt If specified then lock_dt = True False
-fps= Max desired frames per second 100
-perf= ? ?
-facebone If specified then net_face_bones = True False
-mepath Path to the motion engine data None
-seqpath Path to directory that contains seq files None
Example: sbm.exe -host=localhost -seq sbm-dfl-init -seq ..\myinit.seq

You can have many -seq arguments to load many files.

ActiveMQ

To use SmartBody with the ActiveMQ message broker, you need to set up a couple of system variables and the ActiveMQ service, as described below.

USING ActiveMQ IS OPTIONAL!
It is in fact not used by default and you should probably only use it if you know what it is and the purpose of it.

System variables

Variable Example value Description
ELVISH_SESSION_HOST “localhost” Name/IP of computer running ActiveMQ service
ELVISH_SCOPE “CADIA” Name of ActiveMQ session. New one is created if it doesn't exist.

ActiveMQ setup

You need to set up the ActiveMQ service. For Win32, run:

SmartBody\lib\activemq\apache-activemq-5.0.0\bin\win32\InstallService.bat

The service should now be installed, but you need to start it with:

net start ActiveMQ

Using

Commands

SmartBody command reference
List of commands accepted in the smartbody console and how to use them.

Coordinate system

SmartBody and PandaBMLR use a different coordinate system. PandaBMLR takes care of conversions automatically as long as raw commands are not input to SmartBody, but if raw commands are used, care must be taken to convert coordinates for positions, quaternions and rotations. PandaBMLR's Converter class has built in functionality for this.

BML support

SmartBody does not yet (as of May 2008) have support for all BML elements defined in the BML specification. To see elements currently implemented go to the SmartBody BML page.

Bml 2 animations

SmartBody has a lot of pre-recorded animations that for the most part need to be called by the name of the animation file, with a special BML extension, instead of a simple, compliant BML block that requests a similar behavior. Because of this, we have created a scheme where animations can be mapped to certain BML tags. This allows the BML code to be more general instead of referring to specific files that might be completely different from system to system. The mapping configuration is stored in an XML file.

The XML file (bml2anim.xml)

<bml2anim>
  <gesture type="beat">HandsOnHip_RArm_MidBeat</gesture>
</bml2anim>

BML input

<bml>
  <gesture type="beat" />
</bml>

maps to

<bml>
  <sbm:animation name="HandsOnHip_RArm_MidBeat" />
</bml>

Matching is performed from top of the bml2anim file and down. The first map element that matches the element type and ALL its attributes will be selected. This is why more generic mappings (fewer arguments) should be placed at the bottom. See the default bml2anim file for a full listing and examples.

Sequences

Sequence file contains a list of commands. Each line in the file starts with the time of execution and then has a SmartBody command. You can comment lines out by placing # in front of the line. Sequence can be executed with the seq command.

Each line has this structure.

<time> <command>

Example

This example seq file creates a pawn and moves it in a (rough) circle around location (0, 0, 0) over 5 seconds:

 0 pawn CirclePawn init
 1 set pawn CirclePawn world_offset x 100 y 0 z 0
 2 set pawn CirclePawn world_offset x 0 z 100
 3 set pawn CirclePawn world_offset x -100 z 0
 4 set pawn CirclePawn world_offset x 0 z -100
 5 set pawn CirclePawn world_offset x 100 z 0

Example BML

Character nods his head.

<bml>
  <head type="nod"/>
</bml>

Character says “Hello world!”.

<bml>
  <speech>Hello World!</speech>
</bml>

This example uses synchpoints to schedule the behavior. The character will say “Hello, my name is Sam”, and will do a beat gesture when he says “my”.

<bml>
  <speech id="sp">Hello, <mark name="m1" time="1" />my name is Sam</speech>
  <gesture type="beat" stroke="sp:m1" />
</bml>

Synchpoint hack

In the current version of SmartBody (May 2008), the synchpoints are not fully working. To make them work you need to define the event you are listening to (set to zero).

Without hack

<bml>
  <head type="nod" id="n1" />
  <gaze target="Sam" start="n1:relax" />
</bml>

With hack

<bml>
  <head type="nod" id="n1" relax="0" />
  <gaze target="Sam" start="n1:relax" />
</bml>
/var/www/ailab/WWW/wiki/data/pages/public/bmlr/smartbody.txt · Last modified: 2008/09/10 21:32 by hannes