Prev

Prev-tail

Tail

Up

Chapter 18
Building Urbi SDK

This section is meant for people who want to build the Urbi SDK. If you just want to install a pre-built Urbi SDK, see Listing 15.

 18.1 Requirements
  18.1.1 Bootstrap
  18.1.2 Build
  18.1.3 Check
 18.2 Check out
 18.3 Bootstrap
 18.4 Configure
  18.4.1 configuration options
  18.4.2 Windows: Cygwin
  18.4.3 Building For Windows
  18.4.4 Building for Windows using Cygwin
 18.5 Compile
 18.6 Install
 18.7 Relocatable
 18.8 Run
 18.9 Check
  18.9.1 Lazy test suites
  18.9.2 Partial test suite runs

18.1 Requirements

This section lists all the dependencies of this package. Some of them are required to bootstrap the package, i.e., to build it from the repository. Others are required to build the package, i.e., to compile it from a tarball, or after a bootstrap.

18.1.1 Bootstrap

To bootstrap this package from its repository, and then to compile it, the following tools are needed.

Autoconf 2.64 or later
 

 
package: autoconf  
Automake 1.11.1 or later
Note that if you have to install Automake by hand (as opposed to “with your distribution’s system”), you have to tell its aclocal that it should also look at the files from the system’s aclocal. If ‘/usr/local/bin/aclocal’ is the one you just installed, and ‘/usr/bin/aclocal’ is the system’s one, then run something like this:

 
$ dirlist=$(/usr/local/bin/aclocal --print-ac-dir)/dirlist 
$ sudo mkdir -p $(dirname $dirlist) 
$ sudo /usr/bin/aclocal --print-ac-dir >>$dirlist  

 
package: automake  
Cvs
This surprising requirement comes from the system Bison uses to fetch the current version of the message translations.
 
package: cvs  
colordiff
Not a requirement, but a useful addition. Used if exists.
Git 1.6 or later
Beware that older versions behave poorly with submodules.
 
package: git-core  
Gettext 1.17
Required to bootstrap Bison. In particular it provides autopoint.
 
package: gettext  
GNU sha1sum
We need the GNU version of sha1sum.
 
package: coreutils  
Help2man
Needed by Bison.
 
package: help2man  
Python
You need ‘xml/sax’, which seems to be part only of Python 2.6. Using python_select can be useful to state that you want to use Python 2.6 by default (‘sudo python_select python26’).
 
deb: python2.6 
MacPorts: python26 
MacPorts: python_select  
Texinfo
Needed to compile Bison.
 
package: texinfo  
yaml for Python
The AST is generated from a description written in YAML. Our (Python) tools need to read these files to generate the AST classes. See http://pyyaml.org/wiki/PyYAML. The installation procedure on Cygwin is:

 
$ cd /tmp 
$ wget http://pyyaml.org/download/pyyaml/PyYAML-3.09.zip 
$ unzip PyYAML-3.09.zip 
$ cd pyYAML-3.09 
$ python setup.py install  

 
MacPorts: py26-yaml 
deb:   python-yaml  

On OS X you may also need to specify the PYTHONPATH:

 
export PYTHONPATH="\ 
/opt/local/Library/Frameworks/Python.framework/Versions/2.6\ 
/lib/python2.6/site-packages:$PYTHONPATH"
 

18.1.2 Build

Boost 1.38 or later
Don’t try to build it yourself, ask your distribution’s management to install it.
 
deb: libboost-dev 
MacPorts: boost 
windows: http://www.boostpro.com/download  
Ccache
Not a requirement, but it’s better to have it.
 
package: ccache  
Doxygen
Needed if you enable the Doxygen documentation (via configure’s option ‘--enable-documentation=doxygen,...’).
Flex 2.5.35 or later
Beware that 2.5.33 has bugs than prevents this package from building correctly.
 
package: flex  
G++ 4.0 or later
GCC 4.2 or later is a better option. Beware that you may find a problem with GCC-4.4 which rejects some Bison generated code. See Section 16.1.2.
 
deb: g++-4.2 
MacPorts: gcc42  
Gnuplot
Required to generate charts of trajectories for the documentation.
 
package: gnuplot  
GraphViz
Used to generate some of the figures in the documentation. There is no GraphViz package for Cygwin, so download the MSI file from GraphViz’ site, and install it. Then change your path to go into its bin directory.

 
wget http://www.graphviz.org/pub/graphviz/stable/windows/graphviz-2.26.msi 
PATH=/cygdrive/c/Program\ Files/Graphviz2.26/bin:$PATH  

 
package: graphviz  
ImageMagick
Used to convert some of the figures in the documentation.
 
MacPorts: ImageMagick 
deb: imagemagick  
JDK
In order to compile support for Java UObjects, you need Sun’s JDK. In particular the ‘jni.h’ must be available. On OS X Snow Leopard, you need the “Java Developer” package from Apple. See http://connect.apple.com/.
 
deb: sun-java6-jdk  
oRTP
The UObject middleware can use the RTP protocol to provide better throughput for streams. The implementation relies on the oRTP library which is now part of the linphone package. oRTP is not needed, but it is strongly recommended.
 
MacPorts: linphone 
deb: liblinphone-dev  
PDFLaTeX
TeX Live is the most common TEX distribution nowadays. We use pdfcrop from ‘texlive-bin-extra’. We also use TeX4HT, which is part of ‘texlive-htmlxml’ on MacPorts.
 
deb: texlive-base texlive-latex-extra texlive-binaries 
MacPorts: texlive texlive-latex-extra texlive-bin-extra texlive-htmlxml  
pkg-config
To find some libraries (such as oRTP), we use pkg-config.
 
deb: pkg-config 
MacPorts: pkgconfig  
py-docutils
This is not a requirement, but it’s better to have it. Used by the test suite. Unfortunately the name of the package varies between distributions. It provides rst2html.
 
deb: python-docutils 
MacPorts: py26-docutils  
SWIG 1.3.36 or later
Used to generate the Java support for UObjects. Versions 1.3.36, 1.3.40, and 2.0.0 are known to work.
 
package: swig 
MacPorts: swig-java  
TeX4HT
Used to generate the HTML documentation.
 
deb: tex4ht 
MacPorts: texlive_texmf-full  
Transfig
Needed to convert some figures for documentation (using fig2dev).
 
package: transfig  
xsltproc
This is not a requirement, but it’s better to have it. Used to generate reports about the urbiscript grammar.

18.1.3 Check

aspell
Needed by the test suite.
 
deb: aspell aspell-en 
MacPorts: aspell aspell-dict-en  
bc
Needed by the test suite.
 
package: bc  
socat
Needed by the test suite to send messages to an Urbi server.
 
package: socat  
Valgrind
Not needed, but if present, used by the test suite.
 
deb: valgrind 
MacPorts: valgrind-devel  

18.2 Check out

Get the open source version tarball from http://www.urbiforge.com/index.php/Main/Downloads and uncompress it. With this version, the bootstrap step can be skipped.

18.3 Bootstrap

Run

 
$ ./bootstrap  

18.4 Configure

As of today, building where the source files are is a bad idea (the test suite of a sub-component of ours, Libport, is known to fail in that situation). So, compile in another directory than the one containing the sources, for instance as follows.

 
$ mkdir _build 
$ cd _build 
$ ../configure options...  

18.4.1 configuration options

See ‘../configure --help’ for detailed information. Unless you want to do funky stuff, you probably need no option.

To use ccache, pass ‘CC=’ccache gcc’ CXX=’ccache g++’’ as arguments to configure:

 
$ ../configure CC=ccache gcc CXX=ccache g++ ...  

18.4.2 Windows: Cygwin

The builds for windows use our wrappers. These wrappers use a database to store the dependencies (actually, to speed up their computation). We use Perl, and its DBI module. So be sort to have sqlite, and DBI.

 
$ perl -MCPAN -e install Bundle::DBI  

It might fail. The most important part is

 
$ perl -MCPAN -e install DBD::SQLite  

It might suffice, I don’t know...

18.4.3 Building For Windows

We support two builds: using Wine on top of Unix, and using Cygwin on top of Windows.

Both builds use our wrappers around MSVC’s cl.exe and link.exe. It is still unclear whether it was a good or a bad idea, but the wrappers use the same names as the tools themselves. Yet libtool will also need to use the genuine link.exe. So to set up Libtool properly, you will need to pass the following as argument to configure:

 
AR=lib.exe 
CC=ccache cl.exe 
CC_FOR_BUILD=gcc 
CXX=ccache cl.exe 
LD=link.exe 
DUMPBIN=/cygdrive/c/vcxx8/VC/bin/link.exe -dump -symbols 
RANLIB=: 
host_alias=mingw32 
--host=mingw32  

where:

Since we are cross-compiling, we also need to specify CC_FOR_BUILD so that config.guess can properly guess the type of the build machine.

18.4.4 Building for Windows using Cygwin

We use our cl.exe wrappers, which is something that Libtool cannot know. So we need to tell it that we are on Windows with Cygwin, and pretend we use GCC, so we pretend we run mingw.

The following options have been used with success to compile Urbi SDK with Visual C++ 2005. Adjust to your own case (in particular the location of Boost and the ‘vcxx8’ part).

 
$ ../configure                                                  \ 
  -C                                                            \ 
  --prefix=/usr/local/gostai                                    \ 
  --enable-compilation-mode=debug                               \ 
  --enable-shared                                               \ 
  --disable-static                                              \ 
  --enable-dependency-tracking                                  \ 
  --with-boost=/cygdrive/c/gt-win32-2/d/boost_1_39              \ 
  AR=lib.exe                                                    \ 
  CC=ccache cl.exe                                            \ 
  CC_FOR_BUILD=gcc                                              \ 
  CXX=ccache cl.exe                                           \ 
  LD=link.exe                                                   \ 
  DUMPBIN=/cygdrive/c/vcxx8/VC/bin/link.exe -dump -symbols    \ 
  RANLIB=:                                                      \ 
  host_alias=mingw32                                            \ 
  --host=mingw32  

18.5 Compile

Should be straightforward.

 
$ make -j3  

Using distcc and ccache is strongly recommended.

18.6 Install

Running ‘make install’ works as expected. It is a good idea to check that your installation works properly: run ‘make installcheck’ (see Section 18.9).

18.7 Relocatable

If you intend to make a relocatable tree of Urbi SDK (i.e., a self-contained tree that can be moved around), then run ‘make relocatable’ after ‘make install’.

This step requires that you use a DESTDIR (see the Automake documentation for more information). Basically, the sequence of commands should look like:

 
$ DESTDIR=/tmp/install 
$ rm -rf $DESTDIR 
$ make install DESTDIR=$DESTDIR 
$ make relocatable DESTDIR=$DESTDIR 
$ make installcheck DESTDIR=$DESTDIR  

You may now move this tree around and expect the executable to work properly.

18.8 Run

There are some special variables in addition to the environment variables documented in the manual.

URBI_CHECK_MODE

Skip lines in input that look like Urbi output. A way to accept test files (‘*.chk’) as input.

URBI_DESUGAR

Display the desugared ASTs instead of the original one.

URBI_IGNORE_URBI_U

Ignore failures (such as differences between kernel revision and ‘urbi.u’ revision) during the initialization.

URBI_INTERACTIVE

Force the interactive mode, as if ‘--interactive’ was passed.

URBI_LAUNCH

The path to urbi-launch that urbi.exe will exec.

URBI_NO_ICE_CATCHER

Don’t try to catch SEGVs.

URBI_PARSER

Enable Bison parser traces.

URBI_REPORT

Display stats about execution rounds performed by the kernel.

URBI_ROOT_LIBname

The location of the libraries to load, without the extension. The LIBname are: LIBJPEG4URBI, LIBPLUGIN (libuobject plugin), LIBPORT, LIBREMOTE (libuobject remote), LIBSCHED, LIBSERIALIZE, LIBURBI.

URBI_SCANNER

Enable Flex scanner traces.

URBI_TOPLEVEL

Force the display the result of the top-level evaluation into the lobby.

18.9 Check

There are several test suites that will be run if you run ‘make check’ (‘-j4’ works on most machines).

To rerun only the tests that failed, use ‘make recheck’. Some tests have explicit dependencies, and they are not rerun if nothing was changed (unless they had failed the previous time). It is therefore expected that after one (long) run of ‘make check’, the second one will be “instantaneous”: the previous log files are reused.

Note that running ‘make’ in the top-level is needed when you change something deep in the package. If you forget this ‘make’ run, the timestamps on which the test suite depends will not be updated, and therefore the results of the previous state of the package will be used, instead of your fresh changes.

Some tests are extremely “touchy”. Because the test suite exercises Urbi under extreme conditions, some tests may fail not because of a problem in Urbi, but because of non-determinism in the test itself. In this case, another run of ‘make check’ will give an opportunity for the test to pass (remind that the tests that passed will not be run again). Also, using ‘make check -j16’ is a sure means to have the Urbi scheduler behave insufficiently well for the test to pass. Do not send bug reports for such failures.. Before reporting bugs, make sure that the failures remain after a few ‘make check -j1’ invocations.

18.9.1 Lazy test suites

The test suites are declared as “lazy”, i.e., unless its dependencies changed, a successful test will be run only once — failing tests do not “cache” their results. Because spelling out the dependencies is painful, we rely on a few timestamps:

libraries.stamp

Updated when a library is updated (libport, libuobject, etc.).

executables.stamp

Updated when an executable is updated (urbi-launch, etc.). Depends on libraries.stamp.

urbi.stamp

When Urbi sources (‘share/urbi/*.u’) are updated.

all.stamp

Updated when any of the three aforementioned timestamps is.

These timestamps are updated only when make is run in the top-level. Therefore, the following sequence does not work as expected:

 
$ make check -C tests     # All passes. 
$ emacs share/urbi/foo.u 
$ make check -C tests     # All passes again.  

because the timestamps were not given a chance to notice that some Urbi source changed, so Make did not notice the tests really needed to be rerun and the tests were not run.

You may either just update the timestamps:

 
$ make check -C tests     # All passes. 
$ emacs share/urbi/foo.u 
$ make stamps             # Update the timestamps. 
$ make check -C tests     # All passes again.  

or completely disable the test suite laziness:

 
$ make check -C tests LAZY_TEST_SUITE=  

18.9.2 Partial test suite runs

You can run each test suite individually by hand as follows:

sdk-remote/libport/test-suite.log

Tests libport.

 
$ make check -C sdk-remote/libport  
sdk-remote/src/tests/test-suite.log

Checks liburbi, and some of the executables we ship. Requires the kernel to be compiled in order to be able to test some of the uobjects.

 
$ make check -C sdk-remote/src/tests  
tests/test-suite.log

Tests the kernel and uobjects.

 
$ make check -C tests  

Partial runs can be invoked:

 
$ make check -C tests TESTS=2.x/echo.chk  

wildcards are supported:

 
$ make check -C tests TESTS=2.x/*  

To check remote uobjects tests:

 
$ make check -C tests TESTS=uob/remote/*  
doc/tests/test-suite.log

The snippets of code displayed in the documentation are transformed into test files.

 
$ make check -C doc