====== BML Realizer - PandaBMLR ======
{{ bml:bmlr:rob_read.jpg}}
===== About PandaBMLR =====
PandaBMLR is the [[http://www.panda3d.org|Panda3D]] rendering engine in the [[bml::bmlr:main|BML Realizer (BMLR) toolkit]]. It is basically a rendering engine and an object oriented wrapper for [[bml:bmlr:smartbody|SmartBody]].
Panda3D is a free and open source game engine developed by Disney and Carnegie Mellon University. The **Python programming language** is used to interface with the engine (and PandaBMLR).
PandaBMLR is simply an API, or a module, for normal Panda3D applications. Therefore, applications are built by Panda3D's rules and guidelines, in **Python**. You can construct entire scenes completely independent of PandaBMLR and then add characters through PandaBMLR afterwards.
==== The Three Rules ====
These are the three main rules that need to be followed to make use of PandaBMLR:
- The configuration file must be parsed before anything else
- Instance of PandaBMLR must be created at startup
- The PandaBMLR instance must be passed into any CharactersPawns or Pawns created (see below)
Each step is explained in detail in this document.
==== PandaBMLR vs BMLR? ====
To clear up any confusion about the difference between BMLR and PandaBMRL;
* BMLR is the name of the project/toolkit as a whole. This includes SmartBody (our modified version), PandaBMLR and anything else (such as modeling scripts).
* PandaBMLR is only the Panda3D part of the project. It acts as a front end to SmartBody.
* Other rendering engines could be added to the BMLR toolkit, and they would be named for example; OgreBMLR, if we added support for the Ogre 3D engine. Both PandaBMLR and OgreBMLR would then use the same SmartBody core.
==== Panda3D resources ====
* [[http://panda3d.org/wiki/index.php/Main_Page|Panda3D Manual]]
* [[http://panda3d.org/apiref.php?page=classes|Panda3D Class Reference]]
* [[http://cadia.ru.is/wiki/public:t-vien-08-1:main|Class at RU on Virtual Environments, taught with Panda3D]]
* [[http://www.ru.is/kennarar/karih/20082_T203_TOLV/pdf-a4-2.5/Python-Docs-2.5/tut.pdf|Python tutorial]]
* [[http://www.ru.is/kennarar/karih/20082_T203_TOLV/diveintopython.pdf|Dive into Python]]
==== PandaBMLR requirements ====
We are not going to nail down exact system requirements, but you will generally be safe with a fairly recent computer (3 years old max) that has a 3D accelerator. Panda3D works fine with even the worst on-board 3D cards (like Intel), but do not expect high performance or good visual quality.
Panda3D is multi-platform, but our implementation has only been tested on Windows XP and Windows Vista. We however do not expect problems with other operating systems as long as they are supported by Panda3D.
==== Known issues ====
* SbmAutoStart will not work on operating systems other than Windows.
* Characters in "Motex" postures will appear to "float" above the ground like ghosts. Cause unknown, but SmartBody related.
* Facial animation is not supported at this time.
===== Configuration =====
It is possible to configure options such as screen resolution and network ports in a config file. By default the config file is named PandaBMLR.config and is located in the PandaBMLR directory. The file is on plain text format and uses the Panda3D configuration file parsing mechanism. It is very easy to add your own configuration variables, see [[http://panda3d.org/wiki/index.php/Accessing_Config_Vars_in_a_Program|Panda3D Manual]]
The table below lists PandaBMLR specific configuration values, all of whom are set to reasonable values by default. It is also possible to put in any [[http://panda3d.org/wiki/index.php/Configuring_Panda|default Panda3D configuration variable]] in the file, such as "win-size" for screen resolution and "fullscreen" to specify if the application should be launched fullscreen or not.
^ Variable ^ Example value ^ Accepted values ^ Description ^
| TcpPort | 15102 | 1 - 65536 | Port number that listens to SmartBody's TCP packets. |
| UdpPort | 15100 | 1 - 65536 | Port number that listens to SmartBody's UDP packets. |
| SbmAutoStart | 1 | 1 or 0 | Whether SmartBody is started automatically on BMLR startup or not. |
| SbmPath | ..\Sbm\bin\ | dir path | The path to the directory that contains the SmartBody binaries. Relative to the directory that Panda3D starts in. |
| SbmBin | sbm.exe | filename | The name of the SmartBody binary to run. It is assumed it is located in the SbmPath dir. |
| SbmArgs | -host=localhost | string | Arguments that SmartBody is started with. See [[bml:bmlr:smartbody|SmartBody]] for further information. |
| SbmInitSeq | sbm-init-empty | string | The name of the initial sequence (.seq) file to run SmartBody with. Do not include .seq in the filename! |
| DefaultPawnModel | Pawn.egg | filename | The default pawn model to load if no model is specified. Relative to the directory that Panda3D starts in. |
| ClassDir | Classes/ | dir path | The directory that contains the definition of the [[#character_classes|character classes]] |
| ChatBubbleScale | 3 | float numbers | How much to scale the size of chat bubbles. |
| ChatBubbleOffsetY | 20 | float numbers | How much to offset the chat bubble position, from left to right. |
| ChatBubbleOffsetZ | 0 | float numbers | How much to offset the chat bubble height. |
| NamePlateScale | 3 | float numbers | How much to scale the size of characters' name plates. |
| SpeechOn | 0 | 1 or 0 | Whether to attempt to use a command line speech synthesizer. |
| SpeechPath | ..\Speech\ | dir path | The path to the directory that contains the speech synthesizer binaries. Relative to the directory that Panda3D starts in. |
| SpeechBin | msspeech.exe | filename | The name of the speech synthesizer binary to run. It is assumed it is located in the SpeechPath dir. |
| SpeechArgs | -type=say | string | Arguments that the speech synthesizer is always started with. Additional arguments are character specific. |
===== Key bindings =====
PandaBMLR will bind a few keys to key functions. This is in addition to the optional [[#camera|camera key bindings]] listed below.
^ Key ^ Effect ^
| F1 | Hide name plates |
| F2 | Hide HUD |
| F11 | Toggle wireframe rendering |
| F12 | Take a screenshot |
| ESC | Exit |
===== Running =====
PandaBMLR applications are run the same way as normal Panda3D applications. Assuming you have created a scene with an entry point in main.py, this is how you start it (from command line):
**Panda3D installed (NOPANDA version)**
ppython main.py
**Panda3D redistributable**
# This assumes you are in the PandaBMLR directory, otherwise adjust the path to ppython.exe accordingly
Panda3D\python\ppython -E main.py
===== Usage =====
This section explains how to integrate PandaBMLR (and therefore BMLR) into an existing Panda3D program.
==== Parsing the config file ====
PandaBMLR assumes that a few [[#configuration|configuration]] variables are present in the Panda3D configuration file. You can place them in a special configuration file, as we have done so in the demo programs. To parse a special Panda3D configuration file, the following lines must be placed BEFORE Panda3D's DirectStart is imported (basically the first lines in your entire program). The filename can of course be changed to whatever, assuming the file exists.
from pandac.PandaModules import loadPrcFile
loadPrcFile("PandaBMLR.config")
==== Creating a PandaBMLR instance ====
The main PandaBMLR class is located in the PandaBMLR package. When you initialize your program, you need to instanciate it and store a reference to it. It will register itself to the Panda3D task manager and run automatically.
This is how an instance is created:
from PandaBMLR.PandaBMLR import BMLR
...
self.BMLR = PandaBMLR()
Once created, PandaBMLR will set up TCP and UDP sockets and listen for connections on the ports specified in the config file. There should only ever be **one instance** of the PandaBMLR class in your Panda3D program. This instance will maintain a list of characters and handle incoming net traffic.
==== About Characters and Pawns ====
CharacterPawn is a character animated and controlled by SmartBody. The CharacterPawn name (as opposed to Character) is because by design, the Characters inherit the Pawn. Furthermore, Panda3D already has a class named Character. CharactersPawns have a skeleton and can move their joints. They can animate in accordance to BML.
Pawn is an entity controlled by SmartBody. It can be visible or invisible, and can in fact use any static model imaginable. The purpose of Pawns is basically to create objects in the scene that SmartBody knows about, other than characters. For example, one could create a Pawn (placed in an interesting position) that characters are supposed to gaze at. This could be extended with Panda3D functionality by parenting the Pawn to the camera, and thus making the characters gaze at the camera wherever it goes.
CharacterPawns and Pawns inherit Panda3D's [[http://panda3d.org/apiref.php?page=NodePath|NodePath]] class, which mean that they behave in the exact same way as every other Panda3D node/object. The position can be changed with the method setPos(x,y,z), rotation with setHpr(h,p,r) and so forth. The one exception is that they are automatically parented to render, but they can easily be re-parented to other NodePaths if desired. Changes of the position or rotation will automatically be sent to SmartBody.
==== CharacterPawns ====
=== Creation ===
CharacterPawns can be created whenever. At startup or while the program is running. The CharacterPawn constructor will automatically place itself in the scene graph to be rendered. If a SmartBody client is connected when the character is created, it will register itself to SmartBody. If not, it will register itself automatically any time that SmartBody connects to the program in the session.
This is how a CharacterPawn is created:
from PandaBMLR.CharacterPawn import CharacterPawn
...
char = CharacterPawn(bmlr, name, class)
The CharacterPawn constructor has 3 required attributes:
^ Attribute ^ Description ^
| bmlr | Reference to the PandaBMLR instance |
| name | The name of the character |
| [[#character_classes|class]] | The [[#character_classes|class]] of the character |
=== Usage ===
Now that you've created a character, you can make it come to life in various ways. The first thing to do is to find the reference to the class. If you did not store it somewhere, you can search for characters (and pawns) with the PandaBMLR instance (that you should always keep a reference to) with the GetPawn(name) method:
char = bmlrInstance.GetPawn(nameOfChar)
A more interesting way of bringing the character to life is the **BML(bmlString)** method of CharacterPawn. When called, it will send a BML block to SmartBody, that will in turn respond by a set of joint rotations or other behavior. Here is an example:
char.BML('')
The above code will automatically be wrapped in tags, but it is also okay to include them. It will change the posture of the character to the posture "CrossedArms", _Motex means that it is an animated posture, not a static one.
Another example:
char.BML('I am Sam, and these are my friends Rob and Kelly.')
char.BML('')
char.BML('')
This above code will make Sam introduce you to his friends, and gaze at Rob and Kelly when he says their name. Refer to the BML specification for a better explanation of this behavior.
** Timing behavior to outside synch points is not working 100% at the moment **
==== Pawns ====
=== Creation ===
This is how a pawn is created:
from PandaBMLR.Pawn import Pawn
...
self.Pawn = Pawn(bmlr, name, geom, geomType)
The Pawn constructor has 2 required attributes, but also 2 optional ones.
^ Attribute ^ Description ^
| bmlr | Reference to the BMLR instance, ( self.BMLR in the context of the above code bits) |
| name | The name of the pawn |
| geom | **(Optional, default = None)** The location of the model that represents the Pawn. By sending in "DEFAULT" or "", the default pawn model is loaded. By sending in None, you make the pawn not represented by a model at all. |
| geomType | **(Optional, default = 1)** Specifies how the geom attribute should be handled. The values accepted are: 0 (a static model), 1 (a static model that is forcefully animated) and 2 (a character class(should never be used directly)). |
The Pawn constructor will automatically place itself in the scene graph to be rendered. If a SmartBody client is connected when the character is created, it will register itself to SmartBody. If not, it will register itself automatically any time that SmartBody connects to the program in the running session.
=== Usage ===
There aren't many things you can do with Pawns aside from moving them around, but that behavior is crucial as your characters often rely on the position of your pawns. As with CharacterPawn, you can look up pawns with the with the GetPawn(name) method of the BMLR class.
pawn = self.BMRL.GetPawn(nameOfPawn)
==== Intervals ====
You can move CharacterPawns and Pawns around with the setPos(x,y,z) command. Panda3D however has a very powerful mechanism to move entities from place to place over time, known as intervals. It is outside the scope of this article to cover Panda3D's intervals but an a detailed guide can be found [[http://panda3d.org/wiki/index.php/Intervals|here]] (note the **Next** links on the page).
Here is an quick example to give the rough idea:
char = bmlrInstance.GetPawn("Sam")
char.posInterval(10, Vec3(100, 200, 300))
The above example will move the character Sam (assuming he exists) from the position he is in, to the position (100, 200, 300) over the course of 10 seconds.
==== More control ====
PandaBMLR's method SbmCommand(cmd) will send a command directly to the SmartBody module and have it evaluated there. This includes all SmartBody commands.
Here is an example that shows how to send commands to SmartBody:
bmlrInstance.SbmCommand("test bml char * head nod")
===== Character classes =====
All characters created need to have a character class defined. A character class defines the model that represents the characters, the texture to apply to the model (optional) and so on. Each of these character class is saved in the ClassDir folder, as specified in the configuration file. The file needs to have the .class file extension to be parsed. At startup (when PandaBMLR instance is created) all of these character classes are cached into memory. The name of the file will become the name of the class, minus the .class extension. For example, the "CADIA.George.class" file will be parsed into the class "CADIA.George".
Here is an example of a character class file:
model=Classes/CADIA.George/George.bam
flip=1
scalex=0.74
scaley=0.74
scalez=0.74
offsetx=0
offsety=0
offsetz=0
skeletontype=1
bonefile=Classes/CADIA.DoctorBones.bones
These are the variables supported:
^ Variable ^ Type ^ Description ^
| model | string | Path to the character model, relative to the application startup directory. |
| scale | float | Scale of the model. |
| skeletontype| integers| The type of skeleton the model has. Currently only type 1 is supported. |
| bonefile | string | Path to the file that defines the bone structure for the model, see below. |
| offset | float | (Optional) Offset of the model. (Not tested) |
| texture | string |( Optional) Path to a texture to apply to the model. |
| envmap | 1 or 0 | (Optional) Whether to use the texture as environment map. Requires texture to be specified. |
A number of character classes are included, take a look at them if anything is unclear.
==== Bone files ====
Character classes rely on bone files to map SmartBody's bone ID's to their joints. This is because SmartBody only sends over the ID's of the bones but we usually only know the name of the bone (joint). Therefore we need to create a mapping between the two. It can also allow us to disable or enable bones at will.
Here is an example of a bones file:
#0=unused
#1=skeleton
#2=base
3=l_hip
4=l_knee
5=l_ankle
6=l_forefoot
7=l_toe
...
111=r_thumb2
112=r_thumb3
113=r_thumb4
In that example, the first 3 bones are disabled (rotatational and posititional changes from SmartBody are ignored).
==== Coordinate system ====
[[bml:bmlr:smartbody|SmartBody]] and PandaBMLR use a different coordinate system. PandaBMLR takes care of conversions automatically as long as raw commands are not input to SmartBody, but if raw commands are used, care must be taken to convert coordinates for positions, quaternions and rotations. PandaBMLR's Converter class has built in functionality for this.
{{ bml:bmlr:coordsystem.png }}
===== Camera =====
A default camera handler is included in PandaBMLR, but its usage is optional. The camera handler allows you to navigate and rotate the camera around the scene with your keyboard and mouse. It also allows you to take control over your characters and pawns and move them around the scene.
This is how the camera is created:
from PandaBMLR.Camera import BMLR_Camera
...
self.Cam = BMLR_Camera(bmlrInstance)
It should be placed AFTER the BMLR instance is created, as you need to pass a valid instance of BMLR as an argument.
The camera will bind the following keys for navigation:
^ Key ^ Effect ^
| W | Move forwards |
| S | Move backwards |
| A | Move left |
| D | Move right |
| Z | Move down |
| Space | Move up |
| Q | Rotate left |
| E | Rotate right |
| I | Roll up |
| K | Roll down |
| U | Pithch up |
| J | Pitch down |
| B | Reset camera target |
| N | Next camera target |
| M | Previous camera target |
| Shift | Hold to increase camera speed |
| Right MouseButton | Hold and drag mouse to look around |
===== HUD and Console =====
{{ bml:bmlr:hud.jpg|This is the HUD}}
A default HUD (Heads Up Display) is included in PandaBMLR, but its usage is optional. It displays information such as the camera position and heading, SmartBody connection status and incoming data rate. Furthermore, it sets up a "console" window and input, that lets you send commands directly to SmartBody and view debug and error messages from within the Panda3D window instead of the command line window. The HUD's box can outlined in three colors; red (SmartBody not connected), yellow (SmartBody connected but not ready to accept commands) and green (SmartBody connected and ready to receive commands (but not necessarily done loading motions and data).
This is how the HUD is created:
from PandaBMLR.Hud import BMLR_Hud
...
hud = BMLR_Hud(self.BMLR)
The HUD instance should be created AFTER the PandaBMLR instance is created, as you need to pass a valid instance of BMLR as an argument. It should also be AFTER the camera is initialized, if it is used.
To bring up the console, press TAB. Press TAB to close it again. When the console is active, you can use the mouse-wheel or page-up/down to scroll up and down the console. A command is executed with the press of the ENTER button. Furthermore, you can use the up and down arrows to bring up previously entered commands.
Note that when the console is active the camera control ceases to function.
==== Console Commands ====
Commands can be entered into the console that have various effects.
Commands abbreviated with a **>** will be sent directly to SmartBody and executed there.
Commands abbreviated with a **%** will be executed by Python's exec() method **(EXPERIMENTAL)**.
Other commands will be executed by PandaBMLR. Two commands are reserved, **q**, which exits PandaBMLR, and **sbm**, which starts SmartBody. A console command that ends with **%%'%%%%'%%%%'%%** (three single quotes) will remove the trailing ''', execute the command, and close the console. This can be useful if you are testing commands and want to see what happens right away without having to close the console manually.
=== Custom console commands ===
Commands other than the ones mentioned above can be bound to custom functions.
This example will bind the "Test"(case-sensitive) console command to the MyFunction method:
self.accept("CMD_Test", self.MyFunction)
...
def MyFunction(self):
print "The test was successful!"
Note that the class needs to inherit Panda3D's DirectObject (as we have done in the demos).
So basically what happens is that whenever you write "" in the console, a global event named "CMD_" will be sent out, and any class can listen to that event with the accept(eventName, method) method.
===== Speech synth =====
**Usage of speech synthesizers is a purely experimental function of PandaBMLR.**
But if you want to test it anyway, you need a speech synthesizer that works by accepting command line arguments. For example:
c:\myspeech.exe -name Kate My name is Kate!
or
c:\myspeech.exe Kate 2 My name is Kate!
The above commands would make the speech synthesizer say the text "My name is Kate!" with the "Kate" voice. In the latter example, the voice Kate is called with a speed of 2. If your speech synthesizer works in a different way (for example, accepts commands over network, or outputs .wav files instead of playing voice) you will need to modify PandaBMLR's code.
To use a speech synthesizer in such a way you need to take a look at the Speech* configuration variables. Basically, after you have set the path to the binary and the name of the binary itself, there are 2 ways arguments can be passed to the program. Arguments specified in the SpeechArgs configuration variable are arguments that will always be sent, no matter what character is speaking. Additionally, character specific speech arguments can be set with the SetSpeechArgs(string) method of the CharacterPawn class. This allows you to specify a voice for each character, for example.
kate.SetSpeechArgs("-name Kate") # For example 1
kate.SetSpeechArgs("Kate 2") # For example 2
===== Editing PandaBMLR's code =====
PandaBMLR is written in Python, with heavy dependency on Panda3D. Editing the code is very straightforward. No compile or update is needed, simply go into the PandaBMLR\PandaBMLR folder, pick a file, and hack away.
Here is a basic description of the files included:
^ File ^ Description ^
| Camera.py | Reference implementation of the camera navigator. |
| CharacterPawn.py | The CharacterPawn class and chat bubble related classes. CharacterPawn inherits Pawn. |
| ClassCacher.py | Handles the parsing and caching of character classes. |
| Converter.py | Static methods for conversion between coordinate systems. |
| Hud.py | Reference implementation of a HUD and Console. |
| NetHandler.py | Handles setting up and listening to incoming network traffic. |
| PandaBMLR.py | Does not do much by itself except glue everything together. |
| Pawn.py | The Pawn and NamePlate classes. |
| Scene.py | The Scene handles incoming commands (after they have been received by the NetHandler). Also maintains a list of all characters and pawns. |