Prev

Prev-tail

Tail

Up

Chapter 16
Building Urbi SDK

This section is meant for people who want to build the Urbi SDK. If you just want to install a pre-built Urbi SDK, see Section 13.

 16.1 Requirements
 16.2 Check out
 16.3 Bootstrap
 16.4 Configure
  16.4.1 configuration options
  16.4.2 Windows: Cygwin
  16.4.3 building For Windows
  16.4.4 Building for Windows using Cygwin
 16.5 Compile
 16.6 Run
 16.7 Check
  16.7.1 Lazy test suites
  16.7.2 Partial test suite runs

16.1 Requirements

To bootstrap this package from its repository, and then to compile it, the following tools are needed.

Autoconf 2.63 or later
 

 
package: autoconf  
Automake 1.11.1 or later
Note that if you have to install Automake by hand (as opposed to “with your distribution’s system”), you have to tell its aclocal that it should also look at the files from the system’s aclocal. If ‘/usr/local/bin/aclocal’ is the one you just installed, and ‘/usr/bin/aclocal’ is the system’s one, then run something like this:

 
$ dirlist=$(/usr/local/bin/aclocal --print-ac-dir)/dirlist 
$ sudo mkdir -p $(dirname $dirlist) 
$ sudo /usr/bin/aclocal --print-ac-dir >>$dirlist  

 
package: automake  
bc
Needed by the test suite.
 
package: bc  
Boost 1.38 or later
Don’t try to build it yourself, ask your distribution’s management to install it.
 
deb: libboost-dev 
MacPorts: boost 
windows: http://www.boostpro.com/download  

At Gostai, you can use the script ‘common/install/install-boost’ to set up Boost as we use it.

Ccache
Not a requirement, but it’s better to have it.
 
package: ccache  
Cvs
This surprising requirement comes from the system Bison uses to fetch the current version of the message translations.
 
package: cvs  
Flex 2.5.35 or later
Beware that 2.5.33 has bugs than prevents this package from building correctly.
 
package: flex  
G++ 4.0 or later
GCC 4.2 or later is a better option. Beware that we have a problem with GCC-4.4 which rejects some Bison generated code. See Section 14.1.2.
 
deb: g++-4.2 
MacPorts: gcc42  
Git 1.6 or later
Beware that older versions behave poorly with submodules.
 
package: git-core  
Gettext 1.17
Required to bootstrap Bison. In particular it provides autopoint.
 
package: gettext  
GNU sha1sum
We need the GNU version of sha1sum.
 
package: coreutils  
Gnuplot
Required to generate charts of trajectories for the documentation.
 
package: gnuplot  
GraphViz
Used to generate some of the figures in the documentation. There is no GraphViz package for Cygwin, so download the MSI file from GraphViz’ site, and install it. Then change your path to go into its bin directory.

 
wget http://www.graphviz.org/pub/graphviz/stable/windows/graphviz-2.26.msi 
PATH=/cygdrive/c/Program\ Files/Graphviz2.26/bin:$PATH  

 
package: graphviz  
Help2man
Needed by Bison.
 
package: help2man  
ImageMagick
Used to convert some of the figures in the documentation.
 
MacPorts: ImageMagick 
deb: imagemagick  
Libtool 2.2 or later
Actually it should no longer be needed, as we embed all the file we need in libport.
PDFLaTeX
TeX Live is the most common TEX distribution nowadays.
 
deb: texlive-base texlive-latex-extra 
MacPorts: texlive texlive_texmf-full  
Python
You need ‘xml/sax’, which seems to be part only of Python 2.6. Using python_select can be useful to state that you want to use Python 2.6 by default (‘sudo python_select python26’).
 
deb: python2.6 
MacPorts: python26 
MacPorts: python_select  
py-docutils
This is not a requirement, but it’s better to have it. Used by the test suite. Unfortunately the name of the package varies between distributions. It provides rst2html.

 
MacPorts: py26-docutils  
socat
Needed by the test suite to send messages to an Urbi server.
 
package: socat  
TeX4HT
Used to generate the HTML documentation.
 
deb: tex4ht 
MacPorts: texlive_texmf-full  
Texinfo
Needed to compile Bison.
 
package: texinfo  
Transfig
Needed to convert some figures for documentation (using fig2dev).
 
package: transfig  
Valgrind
Not needed, but if present, used by the test suite.
 
package: valgrind  
xsltproc
This is not a requirement, but it’s better to have it.
yaml for Python
The AST is generated from a description written in YAML. Our (Python) tools need to read these files to generate the AST classes. See http://pyyaml.org/wiki/PyYAML. The installation procedure on Cygwin is:

 
$ cd /tmp 
$ wget http://pyyaml.org/download/pyyaml/PyYAML-3.09.zip 
$ unzip PyYAML-3.09.zip 
$ cd pyYAML-3.09 
$ python setup.py install  

 
MacPorts: py26-yaml 
deb:   python-yaml  

At least on OS X you also need to specify the PYTHONPATH:

 
export PYTHONPATH="\ 
/opt/local/Library/Frameworks/Python.framework/Versions/2.6\ 
/lib/python2.6/site-packages:$PYTHONPATH"
 

16.2 Check out

We heavily use Git submodules, i.e., we subscribe to other Git repositories, which themselves may depend on other submodules. Unfortunately Git’s support for recursive submodules is not nice. We have scripts in common (named ‘git-my-foo ’, also available as ‘git my-foo ’) that make this easier. Since they call each other, set your PATH to go into common/git.

Git 1.6.6 or higher
There is direct support for submodules from clone.

 
$ git clone --branch master --recursive git@git.gostai.com:kernel  
Old versions of Git
The following sequence expects that you have common/git (common being the Gostai repository) in your PATH.

 
$ git my-clone git@git.gostai.com:kernel 
$ cd kernel 
$ git checkout -b master --track origin/master 
$ git my-update  

16.3 Bootstrap

Run

 
$ ./bootstrap  

16.4 Configure

You should really compile in another directory than the one containing the sources.

 
$ mkdir _build 
$ cd _build 
$ ../configure OPTIONS...  

16.4.1 configuration options

See ‘../configure --help’ for detailed information. Unless you want to do funky stuff, you probably need no option.

To use ccache, pass ‘CC=’ccache gcc’ CXX=’ccache g++’’ as arguments to configure:

 
$ ../configure CC=ccache gcc CXX=ccache g++ ...  

16.4.2 Windows: Cygwin

The builds for windows use our wrappers. These wrappers use a database to store the dependencies (actually, to speed up their computation). We use Perl, and its DBI module. So be sort to have sqlite, and DBI.

 
$ perl -MCPAN -e install Bundle::DBI  

It might fail. The most important part is

 
$ perl -MCPAN -e install DBD::SQLite  

It might suffice, I don’t know...

16.4.3 building For Windows

We support two builds: using Wine on top of Unix, and using Cygwin on top of Windows.

Both builds use our wrappers around MSVC’s cl.exe and link.exe. It is still unclear whether it was a good or a bad idea, but the wrappers use the same names. Yet libtool will also need to use the genuine link.exe. So to set up Libtool properly, you will need to pass the following as argument to configure:

 
$ AR=lib.exe                                                    \ 
  CC=ccache cl.exe                                            \ 
  CC_FOR_BUILD=gcc                                              \ 
  CXX=ccache cl.exe                                           \ 
  LD=link.exe                                                   \ 
  DUMPBIN=/cygdrive/c/vcxx8/VC/bin/link.exe -dump -symbols    \ 
  RANLIB=:                                                      \ 
  host_alias=mingw32                                            \ 
  --host=mingw32  

where:

Since we are cross-compiling, we also need to specify CC_FOR_BUILD so that config.guess can properly guess the type of the build machine.

16.4.4 Building for Windows using Cygwin

We use our cl.exe wrappers, which is something that Libtool cannot know. So we need to tell it that we are on Windows with Cygwin, and pretend we use GCC, so we pretend we run mingw.

The following options have been used with success to compile Urbi SDK with Visual C++ 2005. Adjust to your own case (in particular the location of Boost).

 
$ ../configure                                                  \ 
  -C                                                            \ 
  --prefix=/usr/local/gostai                                    \ 
  --enable-compilation-mode=debug                               \ 
  --enable-shared                                               \ 
  --disable-static                                              \ 
  --enable-dependency-tracking                                  \ 
  --with-boost=/cygdrive/c/gt-win32-2/d/boost_1_39              \ 
  AR=lib.exe                                                    \ 
  CC=ccache cl.exe                                            \ 
  CC_FOR_BUILD=gcc                                              \ 
  CXX=ccache cl.exe                                           \ 
  LD=link.exe                                                   \ 
  DUMPBIN=/cygdrive/c/vcxx8/VC/bin/link.exe -dump -symbols    \ 
  RANLIB=:                                                      \ 
  host_alias=mingw32                                            \ 
  --host=mingw32  

16.5 Compile

Should be straightforward.

 
$ make -j3  

Using distcc and ccache is recommended.

16.6 Run

There are some special variables in addition to the environment variables documented in the manual.

URBI_CONSOLE_MODE

Skip lines in input that look like shell output. A way to accept *.chk as input.

URBI_DESUGAR

Display the desugared ASTs instead of the original one.

URBI_IGNORE_URBI_U

Ignore failures (such as differences between kernel revision and ‘urbi.u’ revision) during the initialization.

URBI_LAUNCH

The path to urbi-launch that urbi.exe will exec.

URBI_NO_ICE_CATCHER

Don’t try to catch SEGVs.

URBI_PARSER

Enable Bison parser traces.

URBI_REPORT

Display stats about execution rounds performed by the kernel.

URBI_ROOT_LIBname

The location of the libraries to load, without the extension. The LIBname are: LIBJPEG, LIBPLUGIN (libuobject plugin), LIBPORT, LIBREMOTE (libuobject remote), LIBSCHED, LIBSERIALIZE, LIBURBI.

URBI_SCANNER

Enable Flex scanner traces.

URBI_TOPLEVEL

Force the display the result of the top-level evaluation into the lobby.

16.7 Check

There are several test suites that will be run if you run ‘make check’ (‘-j4’ works on most machines). To rerun only the tests that failed, use ‘make recheck’.

16.7.1 Lazy test suites

The test suites are declared as “lazy”, i.e., unless its dependencies changed, a successful test will be run only once — failing tests do not “cache” their results. Because spelling out the dependencies is painful, we rely on a few timestamps:

libraries.stamp

Updated when a library is updated (libport, libuobject, etc.).

executables.stamp

Updated when an executable is updated (urbi-launch, etc.). Depends on libraries.stamp.

urbi.stamp

When Urbi sources (‘share/urbi/*.u’) are updated.

all.stamp

Updated when any of the three aforementioned timestamps is.

These timestamps are updated only when make is run in the top-level. Therefore, the following sequence does not work as expected:

 
$ make check -C tests     # All passes. 
$ emacs share/urbi/foo.u 
$ make check -C tests     # All passes again.  

because the timestamps were not given a chance to notice that some Urbi source changed, so Make did not notice the tests really needed to be rerun and the tests were not run.

You may either just update the timestamps:

 
$ make check -C tests     # All passes. 
$ emacs share/urbi/foo.u 
$ make stamps             # Update the timestamps. 
$ make check -C tests     # All passes again.  

or completely disable the test suite laziness:

 
$ make check -C tests LAZY_TEST_SUITE=  

16.7.2 Partial test suite runs

You can run each test suite individually by hand as follows:

sdk-remote/libport/test-suite.log

Tests libport.

 
$ make check -C sdk-remote/libport  
sdk-remote/src/tests/test-suite.log

Checks liburbi, and some of the executables we ship. Requires the kernel to be compiled in order to be able to test some of the uobjects.

 
$ make check -C sdk-remote/src/tests  
tests/test-suite.log

Tests the kernel and uobjects.

 
$ make check -C tests  

Partial runs can be invoked:

 
$ make check -C tests TESTS=2.x/echo.chk  

wildcards are supported:

 
$ make check -C tests TESTS=2.x/*  

To check remote uobjects tests:

 
$ make check -C tests TESTS=uob/remote/*  
doc/tests/test-suite.log

The snippets of code displayed in the documentation are transformed into test files.

 
$ make check -C doc