Table of Contents
BML Realizer - SmartBody
About SmartBody
SmartBody is an open sourced, modular framework for animating embodied conversational agents in real time, based on the notion of hierarchically connected animation controllers. Using the Behavior Markup Language, a growing standard of human behavior description for real time animated characters, SmartBody is capable of carefully synchronized skeletal animation and speech behaviors, independent of game engine or rendering environment. It is the product of years of work at the University of Southern California, and built on the Motion Engine animation library by Marcelo Kallmann.
http://www.smartbody-anim.org/
SmartBody can be run in a distributed environment and uses ActiveMQ as a message broker. It allows external programs to send BML commands to ActiveMQ that are later parsed by SmartBody. SmartBody in the BML Realizer toolkit is not configured for distributed environment by default. In the BML Realizer toolkit, the renderer is able to send commands back to SmartBody instead of external programs.
SmartBody resources
- Official SmartBody wiki - This page will try not to reproduce information from the official wiki, so check that out as well
- Official SmartBody wiki - Commands
SmartBody requirements
SmartBody is written in C++ for the Windows platform. All dependencies are included, see Libraries below. To build SmartBody, Visual Studio 2008 or Visual C++ 2008 (free) is required.
SmartBody should run on any fairly recent computer.
Known issues
- Speech BML element must be the first BML element in a block.
- Does not support all BML elements, see BML support below.
- Sync points only work within the same BML request. Synchronizing between multiple BML requests or between characters is not supported.
- Synchpoints are not fully working, see Synchpoint hack below.
- Non-ASCII characters in Speech BML elements may not work.
- SmartBody will output a lot of error messages regarding missing “audio synch points” when using the Speech BML element. Don't worry about it.
Architecture
SmartBody takes BML input and converts it to joint rotations on characters. The renderer listens to SmartBody animates the characters with the given joint rotations.
Libraries
The following libraries are used by SmartBody and are stored in the SmartBody\lib folder. Some are only needed for the distributed version of SmartBody.
External open-source libraries:
| Name | Req. | Description |
|---|---|---|
| activemq | No | Apache ActiveMQ is a messaging broker |
| boost | Yes | Boost is a collection of C++ Libraries extend the functionality of C++. Only the used libraries are included |
| pthreads | No | Library that defines an API for creating and manipulating threads |
| sdl | Yes | Window manager and multimedia library |
| xerces-c | Yes | XML parser |
Internal libraries, created at ISI:
| Name | Req. | Description |
|---|---|---|
| commapi | Yes | CommAPI is a library that encapsulates the communication to the renderer |
| playsound | Yes | Small lightweight library that lets you play sounds |
| vhmsg | No | Wrapper around ActiveMQ |
| wsp | No | Wrapper around tt_utils |
If a library is not labeled required, it means that it is only needed if you want to build the distributed version of SmartBody.
Compiling
After downloading the source from the BMLR website, open the SmartBody-CADIA.sln file in Visual studio 2008, pick the correct build target and build.
There are 4 build targets, two without ActiveMQ (named “Debug-NOAMQ” and “Release-NOAMQ”) and two with ActiveMQ (“Debug” and “Release”). You might need to do a “rebuild all” when changing to or from “NOAMQ” build targets.
To build the distributed versions you need to download the additional SmartBody_Amq_libs file from the BMLR download site and extract to your SmartBody\libs directory.
Running
SmartBody Standalone (Without ActiveMQ message broker)
- Debug: sbm-fltkd_noamq.exe
- Release: sbm-fltk_noamq.exe
SmartBody Distributed
- Debug: sbm-fltkd.exe
- Release: sbm-fltk.exe
If no arguments are provided, SmartBody will load the default.seq file from the same directory as the exe file.
Command line arguments
These are the command line arguments sbm.exe accepts. All of them are optional.
| Argument | Effect | Default value |
|---|---|---|
| -host= | Hostname to send render data to | None |
| -seq | Path to .seq file to run on startup. Takes full file path or only file name. | sbm-dfl-init |
| -procid= | ? | ? |
| -audio | If specified then internal audio will be played | False |
| -lockdt | If specified then lock_dt = True | False |
| -fps= | Max desired frames per second | 100 |
| -perf= | ? | ? |
| -facebone | If specified then net_face_bones = True | False |
| -mepath | Path to the motion engine data | None |
| -seqpath | Path to directory that contains seq files | None |
Example: sbm.exe -host=localhost -seq sbm-dfl-init -seq ..\myinit.seq
You can have many -seq arguments to load many files.
ActiveMQ
To use SmartBody with the ActiveMQ message broker, you need to set up a couple of system variables and the ActiveMQ service, as described below.
USING ActiveMQ IS OPTIONAL!
It is in fact not used by default and you should probably only use it if you know what it is and the purpose of it.
System variables
| Variable | Example value | Description |
|---|---|---|
| ELVISH_SESSION_HOST | “localhost” | Name/IP of computer running ActiveMQ service |
| ELVISH_SCOPE | “CADIA” | Name of ActiveMQ session. New one is created if it doesn't exist. |
ActiveMQ setup
You need to set up the ActiveMQ service. For Win32, run:
SmartBody\lib\activemq\apache-activemq-5.0.0\bin\win32\InstallService.bat
The service should now be installed, but you need to start it with:
net start ActiveMQ
Using
Commands
SmartBody command reference
List of commands accepted in the smartbody console and how to use them.
Coordinate system
SmartBody and PandaBMLR use a different coordinate system. PandaBMLR takes care of conversions automatically as long as raw commands are not input to SmartBody, but if raw commands are used, care must be taken to convert coordinates for positions, quaternions and rotations. PandaBMLR's Converter class has built in functionality for this.
BML support
SmartBody does not yet (as of May 2008) have support for all BML elements defined in the BML specification. To see elements currently implemented go to the SmartBody BML page.
Bml 2 animations
SmartBody has a lot of pre-recorded animations that for the most part need to be called by the name of the animation file, with a special BML extension, instead of a simple, compliant BML block that requests a similar behavior. Because of this, we have created a scheme where animations can be mapped to certain BML tags. This allows the BML code to be more general instead of referring to specific files that might be completely different from system to system. The mapping configuration is stored in an XML file.
The XML file (bml2anim.xml)
<bml2anim> <gesture type="beat">HandsOnHip_RArm_MidBeat</gesture> </bml2anim>
BML input
<bml> <gesture type="beat" /> </bml>
maps to
<bml> <sbm:animation name="HandsOnHip_RArm_MidBeat" /> </bml>
Matching is performed from top of the bml2anim file and down. The first map element that matches the element type and ALL its attributes will be selected. This is why more generic mappings (fewer arguments) should be placed at the bottom. See the default bml2anim file for a full listing and examples.
Sequences
Sequence file contains a list of commands. Each line in the file starts with the time of execution and then has a SmartBody command. You can comment lines out by placing # in front of the line. Sequence can be executed with the seq command.
Each line has this structure.
<time> <command>
Example
This example seq file creates a pawn and moves it in a (rough) circle around location (0, 0, 0) over 5 seconds:
0 pawn CirclePawn init 1 set pawn CirclePawn world_offset x 100 y 0 z 0 2 set pawn CirclePawn world_offset x 0 z 100 3 set pawn CirclePawn world_offset x -100 z 0 4 set pawn CirclePawn world_offset x 0 z -100 5 set pawn CirclePawn world_offset x 100 z 0
Example BML
Character nods his head.
<bml> <head type="nod"/> </bml>
Character says “Hello world!”.
<bml> <speech>Hello World!</speech> </bml>
This example uses synchpoints to schedule the behavior. The character will say “Hello, my name is Sam”, and will do a beat gesture when he says “my”.
<bml> <speech id="sp">Hello, <mark name="m1" time="1" />my name is Sam</speech> <gesture type="beat" stroke="sp:m1" /> </bml>
Synchpoint hack
In the current version of SmartBody (May 2008), the synchpoints are not fully working. To make them work you need to define the event you are listening to (set to zero).
Without hack
<bml> <head type="nod" id="n1" /> <gaze target="Sam" start="n1:relax" /> </bml>
With hack
<bml> <head type="nod" id="n1" relax="0" /> <gaze target="Sam" start="n1:relax" /> </bml>