Running compiled code on OSv

This document explains how to run compiled code (C code, C++ code, etc.) on OSv

Compiling Linux applications for OSv

OSv in ABI-compatible with Linux, meaning that it should be able to run (minus any bugs still in OSv...) executable code compiled for Linux, as long as this code does not use a few unsupported features such as fork(). This means that you can compile your existing Linux application with its normal build process, and run the resulting Linux executable on OSv.

There's one snag, though: OSv cannot currently run "normal" (fixed-position) executables, and can only run a relocatable shared-object (a file normally given a ".so" extension). It looks for the main() function in that shared object, and runs it.

Converting a compilation process to produce a shared-object instead of an executable is fairly trivial: All you need to do is to add "-fPIC" option to the compilation of each source file (to produce an object file with position-independent code) and to add "-shared" to the linking stage, to produce a shared-object instead of an executable. That's it, and you can run the result on OSv (the next section explains how to do that).

By the way, some projects already create ".so"s that can be run in OSv unmodified. For example, the Java runtime (the JVM) is mostly a collection of shared objects, with the "java" executable being a simple main() that calls the right functions from libjvm.so. So to run Java in OSv, all we needed to do is to take unmodified ".so"s (we can take precompiled versions, e.g., from some Linux distribution), and add to it our own simple main which we compiled as explained above (see java/java.cc in OSv's source repository).

Putting the application into OSv's image and running it

There are many ways of getting your application into OSv. You can upload it with scp, or with a Web application. But here we'll describe a more direct way (that is probably not recommended to anyone but OSv developers) - add the executable to OSv's image during OSv's compilation.

During the build of OSv ("make"), it consults a file bootfs.manifest on which files to copy into the image. For example, we compiled the "Memcached" application to a shared-object memcached.so as described above, and then added it to the built image by adding the following lines to bootfs.manifest:

/memcached.so: /home/nyh/memcached/memcached.so
/libevent-2.0.so.5: /usr/lib64/libevent-2.0.so.5.1.6

Note we needed to copy not only memcached.so, but also libevent, a system shared-library that memcached was compiled with and is needed for running it. As shown, we can simply copy an unmodified shared library (64-bit version, of course) straight out of any Linux distribution. We put these files in the root directory of the image; If you want to put them in /usr, you'll need to add these lines to usr.manifest instead of bootfs.manifest.

We needed to copy libevent.so, but not the standard C and C++ libraries (libc, libm, libpthread, librt, libdl, libstdc++), as those are already included in the OSv kernel.

Now, to run this new memcached.so, all you need to do is

scripts/run.py -e "memcached.so ..."

For example, to run memcached on a single vCPU, run scripts/run.py -e "memcached.so -u root -t 1"

Compiling applications which uses OSv APIs

While initially most applications running on OSv will be Linux applications, OSv is not just a Linux clones, and it has additional non-Linux APIs that applications can use to further improve their performance on OSv.

Currently, our build process does not copy OSv's header files anywhere, so currently the easiest way to compile a program which uses both Linux and OSv APIs is to to just add this program to OSv's makefile ("build.mk") and have it compile together with OSv with all the right include paths. As an example, consider a version of Memcached modified to use some OSv APIs. We put the source code into a subdirectory "tools/memcached" of OSv's source tree, and add to build.mk:

memcachedobjs := assoc.o cache.o daemon.o hash.o items.o memcached.o \
    slabs.o stats.o thread.o util.o
memcachedobjs := $(addprefix tools/memcached/, $(memcachedobjs))
tools/memcached/memcached.so: CFLAGS+=-levent
tools/memcached/memcached.so: $(memcachedobjs)
        $(makedir)
        $(q-build-so)
$(memcachedobjs): CFLAGS += -fPIC -DHAVE_CONFIG_H -isystem /usr/include
tools += tools/memcached/memcached.so

Now "make", which also builds the "tools" target, will build tools/memcached/memcached.so, and it can be installed on the image as explained above.

The "-isystem /usr/include" is needed above to allow the memcached to also include system header files - namely those of libevent. Without this option, only OSv header files are available, and those provide the standard libc header files, but not additional header files which might be installed on the system (in this example, /usr/include/event.h).