Prev

Prev-tail

Tail

Up

Chapter 20
Building Urbi SDK

This section is meant for people who want to build the Urbi SDK. If you just want to install a pre-built Urbi SDK, see paragraph 16.

 20.1 Requirements
  20.1.1 Bootstrap
  20.1.2 Build
  20.1.3 Check
 20.2 Check out
 20.3 Bootstrap
 20.4 Configure
  20.4.1 configuration options
  20.4.2 Windows: Cygwin
  20.4.3 Building For Windows
  20.4.4 Building for Windows using Cygwin
 20.5 Compile
 20.6 Install
 20.7 Relocatable
 20.8 Run
 20.9 Check
  20.9.1 Lazy test suites
  20.9.2 Partial test suite runs

20.1 Requirements

This section details the dependencies of this package. Some of them are required to bootstrap the package (Section 20.1.1), i.e., to build it from the repository. Others are required to build the package (Section 20.1.2), i.e., to compile it from a tarball, or after a bootstrap. Finally, some are needed to run the test suite (Section 20.1.3).

The reader in a hurry can simply run one of the following commands, depending on her environment.

Debian or Ubuntu  

 
sudo apt-get install  \ 
   aspell aspell-en autoconf automake bc ccache colordiff  \ 
   coreutils cvs doxygen flex g++ gettext git-core gnuplot  \ 
   graphviz help2man imagemagick libboost-all-dev  \ 
   liblinphone-dev make pkg-config python-docutils  \ 
   python-yaml python2.6 socat sun-java6-jdk swig tex4ht  \ 
   texinfo texlive-base texlive-binaries  \ 
   texlive-latex-extra transfig valgrind  

MacPorts  

 
sudo port install  \ 
   ImageMagick aspell aspell-dict-en autoconf automake bc  \ 
   boost ccache colordiff coreutils cvs doxygen flex gcc42  \ 
   gettext git-core gmake gnuplot graphviz help2man libxslt  \ 
   linphone pkgconfig py26-docutils py26-yaml python26  \ 
   python_select socat swig swig-java texinfo texlive  \ 
   texlive-bin-extra texlive-htmlxml texlive-latex-extra  \ 
   transfig valgrind-devel  

20.1.1 Bootstrap

To bootstrap this package from its repository, and then to compile it, the following tools are needed.

Autoconf 2.64 or later
 
 
package: autoconf  
Automake 1.11.1 or later
Note that if you have to install Automake by hand (as opposed to “with your distribution’s system”), you have to tell its aclocal that it should also look at the files from the system’s aclocal. If ‘/usr/local/bin/aclocal’ is the one you just installed, and ‘/usr/bin/aclocal’ is the system’s one, then run something like this:

 
$ dirlist=$(/usr/local/bin/aclocal --print-ac-dir)/dirlist 
$ sudo mkdir -p $(dirname $dirlist) 
$ sudo /usr/bin/aclocal --print-ac-dir >>$dirlist  
 
package: automake  
Cvs
This surprising requirement comes from the system Bison uses to fetch the current version of the message translations.
 
package: cvs  
colordiff
Not a requirement, but a useful addition. Used if exists.
 
package: colordiff  
Git 1.6 or later
Beware that older versions behave poorly with submodules.
 
package: git-core  
Gettext 1.17
Required to bootstrap Bison. In particular it provides autopoint.
 
package: gettext  
GNU sha1sum
We need the GNU version of sha1sum.
 
package: coreutils  
Help2man
Needed by Bison.
 
package: help2man  
Python
You need ‘xml/sax’, which seems to be part only of Python 2.6. Using python_select can be useful to state that you want to use Python 2.6 by default (‘sudo python_select python26’).
 
deb: python2.6 
MacPorts: python26 python_select  
Texinfo
Needed to compile Bison.
 
package: texinfo  
yaml for Python
The AST is generated from a description written in YAML. Our (Python) tools need to read these files to generate the AST classes. See http://pyyaml.org/wiki/PyYAML. The installation procedure on Cygwin is:

 
$ cd /tmp 
$ wget http://pyyaml.org/download/pyyaml/PyYAML-3.09.zip 
$ unzip PyYAML-3.09.zip 
$ cd pyYAML-3.09 
$ python setup.py install  
 
MacPorts: py26-yaml 
deb:   python-yaml  

On OS X you may also need to specify the PYTHONPATH:

 
export PYTHONPATH="\ 
/opt/local/Library/Frameworks/Python.framework/Versions/2.6\ 
/lib/python2.6/site-packages:$PYTHONPATH"
 

20.1.2 Build

Boost 1.40 or later
Don’t try to build it yourself, ask your distribution’s management to install it.
 
deb: libboost-all-dev 
MacPorts: boost 
windows: http://www.boostpro.com/download  
Ccache
Not a requirement, but it’s better to have it.
 
package: ccache  
Doxygen
Needed if you enable the Doxygen documentation (via configure’s option ‘--enable-documentation=doxygen,...’).
 
package: doxygen  
Flex 2.5.35 or later
Beware that 2.5.33 has bugs than prevents this package from building correctly.
 
package: flex  
G++ 4.0 or later
GCC 4.2 or later is a better option. You may have a problem with GCC 4.4 which rejects some Bison generated code. See Section 17.1.2.
 
deb: g++ 
MacPorts: gcc42  
Gnuplot
Required to generate charts of trajectories for the documentation.
 
package: gnuplot  
GraphViz
Used to generate some of the figures in the documentation. There is no GraphViz package for Cygwin, so download the MSI file from GraphViz’ site, and install it. Then change your path to go into its bin directory.

 
wget http://www.graphviz.org/pub/graphviz/stable/windows/graphviz-2.26.msi 
PATH=/cygdrive/c/Program\ Files/Graphviz2.26/bin:$PATH  
 
package: graphviz  
ImageMagick
Used to convert some of the figures in the documentation.
 
MacPorts: ImageMagick 
deb: imagemagick  
JDK
In order to compile support for Java UObjects, you need Sun’s JDK. In particular the ‘jni.h’ must be available. On OS X Snow Leopard, you need the “Java Developer” package from Apple. See http://connect.apple.com/.
 
deb: sun-java6-jdk  
GNU Make
Although we use Automake that does provide portability across flavors of Make, we do use GNU Make extensions. Actually, the most common version, 3.81, behaves improperly on our Makefiles, so be sure to use GNU Make 3.82.
 
deb: make 
MacPorts: gmake  
oRTP
The UObject middleware can use the RTP protocol to provide better throughput for streams. The implementation relies on the oRTP library which is now part of the linphone package. oRTP is not needed, but it is strongly recommended.
 
MacPorts: linphone 
deb: liblinphone-dev  
PDFLaTeX
TeX Live is the most common TEX distribution nowadays. We use pdfcrop from ‘texlive-bin-extra’.
 
deb: texlive-base texlive-latex-extra texlive-binaries 
MacPorts: texlive texlive-latex-extra texlive-bin-extra  
pkg-config
To find some libraries (such as oRTP), we use pkg-config.
 
deb: pkg-config 
MacPorts: pkgconfig  
py-docutils
This is not a requirement, but it’s better to have it. Used by the test suite. Unfortunately the name of the package varies between distributions. It provides rst2html.
 
deb: python-docutils 
MacPorts: py26-docutils  
ROS CTurtle or later
Needed to enable our ROS bridge. Versions CTurtle, Diamondback, and Electric Emys (1.2.X, 1.4.X, 1.6.X) are known to work. The packages and installation processes are detailed at http://www.ros.org/wiki/ROS.
SWIG 1.3.36 or later
Used to generate the Java support for UObjects. Versions 1.3.36, 1.3.40, and 2.0.0 are known to work.
 
package: swig 
MacPorts: swig-java  
TeX4HT
Used to generate the HTML documentation.
 
deb: tex4ht 
MacPorts: texlive-htmlxml  
Transfig
Needed to convert some figures for documentation (using fig2dev).
 
package: transfig  
xsltproc
This is not a requirement, but it’s better to have it. Used to generate reports about the urbiscript grammar.
 
MacPorts: libxslt  

20.1.3 Check

aspell
Needed by the test suite.
 
deb: aspell aspell-en 
MacPorts: aspell aspell-dict-en  
bc
Needed by the test suite.
 
package: bc  
socat
Needed by the test suite to send messages to an Urbi server.
 
package: socat  
Valgrind
Not needed, but if present, used by the test suite.
 
deb: valgrind 
MacPorts: valgrind-devel  

20.2 Check out

Get the open source version tarball from http://www.gostai.com/downloads/urbi/2.x/ and uncompress it. This version is bootstrapped, you can directly proceed to the “Configure” step.

20.3 Bootstrap

Run

 
$ ./bootstrap  

20.4 Configure

Do not compile where the source files are (the test suite of a sub-component of ours, Libport, is known to fail in that situation). So, compile in another directory than the one containing the sources, for instance as follows.

 
$ mkdir _build 
$ cd _build 
$ ../configure options...  

Also, do not compile in a directory whose name, or the name of some of its ancestor, use “special characters”: stick to plain ASCII. This is a limitation of the Java VM which seems to be unable to traverse properly directories with UTF-8 names when the locale is set to C (Section 17.1.4).

20.4.1 configuration options

See ‘../configure --help’ for detailed information. Unless you want to do funky stuff, you probably need no option.

To use ccache, pass ‘CC=’ccache gcc’ CXX=’ccache g++’’ as arguments to configure:

 
$ ../configure CC=ccache gcc CXX=ccache g++ ...  

For the records, here are some of the ‘--enable-’/‘--disable-’ specific options.

--enable-bindings
 
Whether to generate the bindings to other languages, such as UObject Java support. Disabling is fine:‘--disable-bindings’.
--enable-compilation-mode=mode
 
Specify the flavor of the build. You should use ‘--enable-compilation-mode=speed’.
--enable-doc=formats
 
Specify what formats of the documentation must be built. Disabling is fine: ‘--disable-doc’.
--enable-doc-sections=sections
 
If the PDF or HTML documentation is to be generated, specify what optional sections of the documentation must be built.
--enable-examples
 
Whether to build the sample programs (urbi-ball-tracking and so forth). Disabling is fine: ‘--disable-examples’.
--enable-library-suffix=suffix
 
Specify that libraries should be named ‘libfoosuffix.*’ instead of ‘libfoo.*’. If suffix is auto (or yes, or if no argument is passed), then use the same suffix as Boost (e.g., ‘-vc80-d’ for Visual 2005, debug libraries). If suffix is autodebug, then just append ‘-d’ for debug builds.
--enable-sdk-remote
 
Whether to build the Urbi SDK Remote libraries. Disabling is fine: ‘--disable-sdk-remote’. Some of the components that are shared between Urbi and SDK Remote will be built anyway, so there will be compilations to perform in ‘sdk-remote’.
--enable-ufloat=kind
 
Do not use this option, it is experimental and unreliable.

20.4.2 Windows: Cygwin

The builds for Windows use our wrappers. These wrappers use a database to store the dependencies (actually, to speed up their computation). We use Perl, and its DBI module. So be sort to have sqlite, and DBI.

 
$ perl -MCPAN -e install Bundle::DBI  

It might fail. The most important part is

 
$ perl -MCPAN -e install DBD::SQLite  

It might suffice…

20.4.3 Building For Windows

We support two builds: using Wine on top of Unix, and using Cygwin on top of Windows.

Both builds use our wrappers around MSVC’s tools (cl.exe, dumpbin.exe, link.exe). It is still unclear whether it was a good or a bad idea, but the wrappers use the same names as the tools themselves. To set up Libtool properly, you will need to pass the following as argument to configure:

 
AR=lib.exe 
CC=ccache cl.exe 
CC_FOR_BUILD=gcc 
CXX=ccache cl.exe 
LD=link.exe 
DUMPBIN=dumpbin.exe 
RANLIB=: 
host_alias=mingw32 
--host=mingw32  

where: ‘cl.exe’, ‘dumpbin.exe’, ‘lib.exe’, and ‘link.exe’ are the wrappers.

Since we are cross-compiling, we also need to specify CC_FOR_BUILD so that config.guess can properly guess the type of the build machine.

20.4.4 Building for Windows using Cygwin

We use our wrappers, which is something that Libtool cannot know. So we need to tell it that we are on Windows with Cygwin, and pretend we use GCC, so we pretend we run mingw.

The following options have been used with success to compile Urbi SDK with Visual C++ 2005. Adjust to your own case (in particular the location of Boost and the ‘vcxx8’ part).

 
$ ../configure                                          \ 
  -C                                                    \ 
  --prefix=/usr/local/gostai                            \ 
  --enable-compilation-mode=speed                       \ 
  --enable-shared                                       \ 
  --disable-static                                      \ 
  --enable-dependency-tracking                          \ 
  --with-boost=/cygdrive/c/gt-win32-2/d/boost_1_40      \ 
  AR=lib.exe                                            \ 
  CC=ccache cl.exe                                    \ 
  CC_FOR_BUILD=gcc                                      \ 
  CXX=ccache cl.exe                                   \ 
  LD=link.exe                                           \ 
  DUMPBIN=dumpbin.exe                                   \ 
  RANLIB=:                                              \ 
  host_alias=mingw32                                    \ 
  --host=mingw32  

20.5 Compile

Should be straightforward.

 
$ make -j3  

Using distcc and ccache is recommended.

20.6 Install

Running ‘make install’ works as expected. It is a good idea to check that your installation works properly: run ‘make installcheck’ (see Section 20.9).

20.7 Relocatable

If you intend to make a relocatable tree of Urbi SDK (i.e., a self-contained tree that can be moved around), then run ‘make relocatable’ after ‘make install’.

This step requires that you use a DESTDIR (see the Automake documentation for more information). Basically, the sequence of commands should look like:

 
$ DESTDIR=/tmp/install 
$ rm -rf $DESTDIR 
$ make install DESTDIR=$DESTDIR 
$ make relocatable DESTDIR=$DESTDIR 
$ make installcheck DESTDIR=$DESTDIR  

You may now move this tree around and expect the executable to work properly.

20.8 Run

There are some special variables in addition to the environment variables documented in the manual.

URBI_ACCEPT_BINARY_MISMATCH

As a safety net, Urbi checks that loaded modules were compiled with exactly the same version of Urbi SDK. Define this variable to skip this check, at your own risks.

URBI_CHECK_MODE

Skip lines in input that look like Urbi output. A way to accept test files (‘*.chk’) as input.

URBI_DESUGAR

Display the desugared ASTs instead of the original one.

URBI_IGNORE_URBI_U

Ignore failures (such as differences between kernel revision and ‘urbi.u’ revision) during the initialization.

URBI_INTERACTIVE

Force the interactive mode, as if ‘--interactive’ was passed.

URBI_LAUNCH

The path to urbi-launch that urbi.exe will exec.

URBI_NO_ICE_CATCHER

Don’t try to catch SEGVs.

URBI_PARSER

Enable Bison parser traces.

URBI_REPORT

Display statistics about execution rounds performed by the kernel.

URBI_ROOT_LIBname

The location of the libraries to load, without the extension. The LIBname are: LIBJPEG4URBI, LIBPLUGIN (libuobject plugin), LIBPORT, LIBREMOTE (libuobject remote), LIBSCHED, LIBSERIALIZE, LIBURBI.

URBI_SCANNER

Enable Flex scanner traces.

URBI_TEXT_MODE

Forbid binary communications with UObjects.

URBI_TOPLEVEL

Force the display the result of the top-level evaluation into the lobby.

20.9 Check

There are several test suites that will be run if you run ‘make check’ (‘-j4’ works on most machines).

To rerun only the tests that failed, use ‘make recheck’. Some tests have explicit dependencies, and they are not rerun if nothing was changed (unless they had failed the previous time). It is therefore expected that after one (long) run of ‘make check’, the second one will be “instantaneous”: the previous log files are reused.

Note that running ‘make’ in the top-level is needed when you change something deep in the package. If you forget this ‘make’ run, the timestamps on which the test suite depends will not be updated, and therefore the results of the previous state of the package will be used, instead of your fresh changes.

Some tests are extremely “touchy”. Because the test suite exercises Urbi under extreme conditions, some tests may fail not because of a problem in Urbi, but because of non-determinism in the test itself. In this case, another run of ‘make check’ will give an opportunity for the test to pass (remind that the tests that passed will not be run again). Also, using ‘make check -j16’ is a sure means to have the Urbi scheduler behave insufficiently well for the test to pass. Do not send bug reports for such failures.. Before reporting bugs, make sure that the failures remain after a few ‘make check -j1’ invocations.

20.9.1 Lazy test suites

The test suites are declared as “lazy”, i.e., unless its dependencies changed, a successful test will be run only once — failing tests do not “cache” their results. Because spelling out the dependencies is painful, we rely on a few timestamps:

libraries.stamp

Updated when a library is updated (libport, libuobject, etc.).

executables.stamp

Updated when an executable is updated (urbi-launch, etc.). Depends on libraries.stamp.

urbi.stamp

When Urbi sources (‘share/urbi/*.u’) are updated.

all.stamp

Updated when any of the three aforementioned timestamps is.

These timestamps are updated only when make is run in the top-level. Therefore, the following sequence does not work as expected:

 
$ make check -C tests     # All passes. 
$ emacs share/urbi/foo.u 
$ make check -C tests     # All passes again.  

because the timestamps were not given a chance to notice that some Urbi source changed, so Make did not notice the tests really needed to be rerun and the tests were not run.

You may either just update the timestamps:

 
$ make check -C tests     # All passes. 
$ emacs share/urbi/foo.u 
$ make stamps             # Update the timestamps. 
$ make check -C tests     # All passes again.  

or completely disable the test suite laziness:

 
$ make check -C tests LAZY_TEST_SUITE=  

20.9.2 Partial test suite runs

You can run each test suite individually by hand as follows:

sdk-remote/libport/test-suite.log

Tests libport.

 
$ make check -C sdk-remote/libport  
sdk-remote/src/tests/test-suite.log

Checks liburbi, and some of the executables we ship. Requires the kernel to be compiled in order to be able to test some of the uobjects.

 
$ make check -C sdk-remote/src/tests  
tests/test-suite.log

Tests the kernel and uobjects.

 
$ make check -C tests  

Partial runs can be invoked:

 
$ make check -C tests TESTS=2.x/echo.chk  

wildcards are supported:

 
$ make check -C tests TESTS=2.x/*  

To check remote uobjects tests:

 
$ make check -C tests TESTS=uob/remote/*  
doc/tests/test-suite.log

The snippets of code displayed in the documentation are transformed into test files.

 
$ make check -C doc  

Partial runs for the doc tests:

 
$ make -C doc check-TESTS TESTS=tests/specs/float-00.chk