1. Overview

This tutorial will walk you through the creation of a simple test sequence. We want to create a monocular sequence in which a camera is moving on a straight line in front of a textured plane. This information is stored in three repository items:

The last step towards rendering requires specifying a rendering options repository item and tying everything together in a so-called setup.

When the setup is prepared, rendering is done by typing make sequence_<setupname>. This generates a set of images and a sequence description. This sequence description is the input to the VSLAM interface which is covered in the API Documentation.

Note All the files used in this tutorial can be found here.

2. Designing the Sequence

The repository contains scenes, camera descriptions, and trajectories which can be recombined in several setups. All repository items are stored in subdirectories of repository/.

2.1 Building the scene

The scene is described in the POV-Ray scene description language (SDL). The purpose of this tutorial is not to give an introduction to POV-Ray (the POV-Ray website provides good documentation and tutorials). A scene description is usually split across multiple files.

Note Every scene is contained in a subdirectory of repository/scenes. The name of the subdirectory is the name of the scene, by which it is referenced in the framework.

Let's create a subdirectory for our new textured_plane scene.

$ cd repository/scenes
$ mkdir textured_plane
$ cd textured_plane
Note The main file for every scene has to be named scene.pov.

Basically, this file defines all the objects that make up the scene (in our case the textured plane) and also light source definitions, etc. scene.pov will later be used for determining ground truth feature positions, too. A ground truth feature position is found by sending a ray from a certain camera pose and computing the nearest intersection with a scene object. Your scene might contain objects that should not be used for tracing because they will not generate valid ground truth positions. Examples are transparent window panes, smoke, fog, etc. You can exclude such objects from the tracing process by querying the tracing parameter in your scene.pov. Usually, you will also exclude non-object parts of the scene (like light sources) from tracing.

Here is our POV-Ray description of the textured plane scene:

#if (!tracing)
  light_source { < 0, 100, -100 > color 1}
#end

polygon {
  // define a textured square of side-length 6 meters
  4,
  <0, 0, 0>,<0, 1, 0>,<1, 1, 0>,<1, 0, 0>
  texture {
    finish { ambient 0 diffuse 1 }
    pigment { image_map { jpeg "texture.jpg"  } }
  }
  translate <-0.5,-0.5,0>
  scale 6

  // shift it 5 meters in Z direction
  translate <0,0,5>
}

The statement light_source { < 0, 100, -100 > color 1} defines a point light source. The surrounding #if block will exclude the light source from tracing. The rest of the file defines a square which is textured with the image "texture.jpg". The square is 6 by 6 meters in size, parallel to the XY-plane and shifted 5 meters in Z direction.

Povray uses a left-handed coordinate system. If no camera definition is given, povray places a default camera at the origin, looking in the positive Z direction. We can use this to render a simple preview of our scene by calling povray from the command line.

$ povray Declare=tracing=0 scene.pov

Note, that we have to define the tracing variable because it's referenced in our scene file. This should produce a scene.png image like this:

example scene .

We are satisfied with the result, and can move on to defining a camera configuration.

2.2 Building a camera configuration

Note Camera configurations are stored as XML files in the subdirectory repository/cameras/. The name of the XML file (without the .xml extension) is the name of the camera configuration, by which it is referenced in the framework.

We model our camera after a Unibrain Fire-i camera with a 2.1 mm lens (specifications taken from unibrain website).

We create a file repository/cameras/fire-i.xml which contains the camera specification:

<?xml version="1.0"?>
<!DOCTYPE camera SYSTEM "http://vslam.inf.tu-dresden.de/dtds/v0.1/camera.dtd">

<camera>
  <type>mono</type>
  <res_x>640</res_x>
  <res_y>480</res_y>
  <focal_length>375</focal_length>
  <princ_point_x>319.5</princ_point_x>
  <princ_point_y>239.5</princ_point_y>
  <shutter_speed>0.033</shutter_speed>
  <frame_rate>30</frame_rate>
</camera>

The shutter_speed is used as the exposure time for motion blur simulation. The other parameters should be self-explanatory.

Note The princ_point parameters are currently not used by the renderer (the principal point is always the center of the image.) However, the princ_point parameters are read and interpreted by the VSLAM API, so it is important that they are set to the image center as in the example.

2.3 Creating a camera trajectory

The next repository item we need to specify is a trajectory.

Note Trajectories are stored as XML files in the subdirectory repository/trajectories/. The name of the XML file (without the .xml extension) is the name of the trajectory, by which it is referenced in the framework.

A trajectory describes a camera motion as a list of keyframes. Every keyframe comprises a timestamp and a camera pose (translation and rotation).

Note Except in the POV-Ray scene descriptions, the framework uses right-handed coordinate systems throughout. This is in accord with the usual convention in the literature. POV-Rays deviation from this convention is unfortunate but can not easily be fixed.

The following picture illustrates the relationship between the coordinate systems:

The framework uses a right-handed coordinate system     with X pointing left

We will let the camera move in a straight line from left (-1,0,0) to right (1,0,0). The camera is always looking in Z direction (forward).

Let's create the file straight.xml with the following content:

<?xml version="1.0"?>
<!DOCTYPE trajectory SYSTEM "http://vslam.inf.tu-dresden.de/dtds/v0.1/trajectory.dtd">
<trajectory>
  <keyframes>
    <camera_pose>
      <timestamp>0</timestamp>
      <camera_rotation>
        <w>1</w>
        <x>0</x>
        <y>0</y>
        <z>0</z>
      </camera_rotation>
      <camera_position>
        <x>-1</x>
        <y>0</y>
        <z>0</z>
      </camera_position>
    </camera_pose>
    <camera_pose>
      <timestamp>3</timestamp>
      <camera_rotation>
        <w>1</w>
        <x>0</x>
        <y>0</y>
        <z>0</z>
      </camera_rotation>
      <camera_position>
        <x>1</x>
        <y>0</y>
        <z>0</z>
      </camera_position>
    </camera_pose>
  </keyframes>
</trajectory>

The trajectory describes the movement of the camera using a list of keyframe poses. We let the camera move in a straight line, so we only need to specify two keyframes at the begin and end of the trajectory. Every keyframe is specified by a camera_pose. Every camera pose has a timestamp, which specifies the time in seconds from the start of the sequence. The first and last keyframe determine the duration of the sequence. In our example, the sequence is 3 seconds long. Together with the framerate of the fire-i camera (30Hz) this means, that 90 frames will be rendered. Camera poses in between keyframes are linearly interpolated.

A camera pose is specified by a camera_position and a camera_rotation (given as a quaternion).

Our first keyframe has timestamp 0, is located at -1 meter on the X axis, and has zero rotation. The second keyframe has timestamp 3, is located at 1 meter on the X axis, and has zero rotation.

2.4 Rendering options

Before we can render the sequence we need one final repository item, the rendering options. The rendering options control rendering quality settings such as anti-aliasing and motion blur accuracy.

Note Rendering options are stored as XML files in the subdirectory repository/render_options/ The name of the XML file (without the .xml extension) is the name of the rendering options, by which it is referenced in the framework.

For simplicity we will use one of the predefined rendering option files in repository/render_options/, namely medium_quality.xml:

<?xml version="1.0"?>
<!DOCTYPE render_options SYSTEM "http://vslam.inf.tu-dresden.de/dtds/v0.1/render_options.dtd">

<render_options>
  <quality>9</quality>
  <antialiasing>true</antialiasing>
  <aa_sampling_method>1</aa_sampling_method>
  <aa_threshold>0.1</aa_threshold>
  <aa_depth>3</aa_depth>
  <num_blur_samples>5</num_blur_samples>
</render_options>

Most of these options map directly to corresponding POV-Ray settings. For a detailed explanation we refer to the POV-Ray manual. quality is the POV-Ray Quality= setting, antialiasing is Antialias=, aa_sampling_method is Sampling_Method=, aa_threshold is Antialias_Threshold=, and aa_depth is Antialias_Depth=. The values in the example correspond to medium quality settings.

num_blur_samples is a framework specific setting. Motion blur effects are simulated by rendering and averaging multiple images from nearby camera poses. The amount of blur is controlled by the shutter speed, which defines the range of poses around the timestamp of the current frame. num_blur_samples controls the number of such nearby images that are used for averaging. In the example, num_blur_samples is 5, that means that 5 images have to be rendered to create one output frame.

3. Creating Setup and Rendering

The main directory for rendering is rendering/. Cd to that directory now.

3.1 Creating a setup

Now we have every item we need to render the sequence. We tell the rendering engine which repository items to use by listing them in a setup file.

Note Setups are stored as XML files in the rendering/ directory. The name of the XML file (without the .xml extension) is the name of the setup, by which it is referenced in the framework.

We create a setup file named textured_plane.xml

<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE setup SYSTEM "http://vslam.inf.tu-dresden.de/dtds/v0.1/setup.dtd">

<setup>
  <scene_name>textured_plane</scene_name>
  <camera_name>fire-i</camera_name>
  <render_options_name>medium_quality</render_options_name>
  <trajectory_name>straight</trajectory_name>
</setup>

The content of the file is straightforward. We combine the repository items described above into a setup.

3.2 Rendering the sequence

Now we can render the sequence. In the rendering/ directory type

$ make
These are the targets you may wish to make:

sequence_textured_plane
$

This yields a list of make targets, each corresponding to a setup. Now we can start rendering by

$ make sequence_textured_plane

You should see something like this:

Creating Makefile.deps, this may take some time...
Creating footage/textured_plane/images/frame_000.000000 ...

---- Rendering motion blur image  1  of  5  at  0.0165 ...
---- Camera position:  [-0.989  0.     0.   ] , rotation:  (1.0, 0.0, 0.0, 0.0)
...

Rendering should be finished after a few minutes. Depending on your hardware, now is the time for one to five nice cups of coffee.

To speed up rendering on multicore machines you may use the make -jN switch to start N parallel processes.

After rendering is finished, the results are placed in rendering/footage/textured_plane

$ cd footage/textured_plane
$ ls
images/  sequence.avi  sequence.xml  setup.xml

The subdirectory images contains the rendered .png images. The first frame should look like this: images/frame0.png

The images are also put together into the sequence.avi video using the camera frame rate (for visual inspection of the result). setup.xml is a copy of the setup file used to create the sequence, that is textured_plane.xml. sequence.xml contains a timestamped list of the output image filenames and will be used as the input to the VSLAM interface.