Add a trivial sleep() test, which sleep()s for 2 seconds, and verifies
that this finishes and has slept for roughly 2 seconds.

I used this for debugging issue #26 - the attempts there ruined timers,
and in particular this trivial test hangs, as sleep() never returns.

(A note to our future automatic testing implementor: We need to allow
for the possibility that a test doesn't cleanly fail, but rather hangs,
and consider this a failure too).

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>

This regression test trigger issue #8
"Make Java InetAddress.getHostName() work" by exercising the DNS
resolver on localhost and a dns root server.

The test takes care of specifying NI_NOFQDN to resolve only the
hostname part of localhost ip.

It appears that the DNS ip is not communicated to the libc by
core/dhcp.cci: /etc/resolv.conf is not filled.

Test contributed while waiting for an fix idea to implement.

Signed-off-by: Benoit Canet <benoit@irqsave.net>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This is a test for the effectiveness of our scheduler's load balancing while
running several threads on several cpus.

A full description of the test and its expected results is included in
comments in the beginning of the code. but briefly, the test runs multiple
concurrent busy-loop threads, and an additional "intermittent" thread (one that
busy-loops for a short duration, and then sleeps), and expects that all busy
threads will get their fair share of the CPU, and the intermittent thread
won't bother them too much.

Testing the current code, this tests demonstrates the following problems
we have:

1. Two busy-loop threads on 2 cpus are 5%-10% slower than just one.
   This is not kernel overhead (profiling show 100% of the time in the
   test's inner loop), and I see exactly the slowdown when running this
   test on the Linux host, so it might be related to the host's multitasking?
   For now, let's not worry about that.

2. Much more worrying is that the intermittent thread sometimes (in about half
   the tests) causes us to only fully use one CPU, and of course get bad
   performance.

3. In many of the tests involving more than 2 threads (2 threads +
   intermittent, or 4 threads) load balancing wasn't fair and some
   threads got more CPU than the others.

Later I'll send patches to fix issues 2 and 3, which appear to happen because
the load balancer thread doesn't run as often as it should, because of vruntime
problems.

Added a new function, osv::reboot() (declared in <osv/power.hh>)
for rebooting the VM.

Also added a Java interface - com.cloudius.util.Power.reboot().

NOTE: Power.java and/or jni/power.cc also need to be copied into
the mgmt submodule.

a test where the guest connects to the host and sends a small packet of data.
used to verify that retransmits is working in Van Jacobson and the TCP
stack in general.

The shell call to stat(1) is evaluted when the rule to build the image is
triggered, at which point loader-stripped.elf does not exist yet.  This
causes stat to fail and the build to break.

Fix by moving the creation of loader-stripped.elf to its own target, so
that by the time the recipe is evaluated, the file is known to exist.

Changed "$[]" to "$(())" when calculating zfs start/size

This is the size that goes in our bootloader count32. But since we will be
copying over the stripped binary anyway, we are probably reading too much data,
for no reason. That should increase boot time a bit.

This patch implement the HPET clock driver, that should work as a fallback for
both Xen and KVM, in case the paravirtual clock is not present. This is
unfortunately the situation for all HVM guests running on EC2, so support for
this is paramount. I have tested on KVM forcing the kvmclock to disappear, and
it seems to work all right.

Add simple tests for munmap() for file-backed memory maps. This exposes
a limitation in munmap() not writing out MAP_SHARED mappings.

gcc 4.7 doesn't define a few builtins, which we don't use, but the 4.8
headers require.

Declare them so compilation passes.

These aren't used in the build and will bitrot badly once
changing the VFS significantly.

The xen block driver needs some extra state not needed for the network drivers.
Namely, the same way virtio-blk does, we need to tell the block layer which is
our strategy, read and write functions. For that, we need some extra code that
I am implementing in xenfront-blk.cc.

Simple implementation of BSD's bus_dma interface. Since we are constrained by virtual
environments, we are able to cut out most of the things.

We need to add our headers first, but the rest should be ready to go.

With all the infrastructure in place, we can compile all our xenbus code in,
and enble the netfront driver.

We need bsd/ directory listed in includes not only for bsd/ files, but
for our cpp code as well. So it has to be in the main includes.

Second, I was overly optmistic about the interface version. The drivers
expect an inteface version slightly older than the maximum present in headers.

Changes needed for xenbus operation. They are, as usual:
* delete function tables and make previously static functions on it public
* comment out sysctl code
* change order of includes between sbuf.h and malloc.h. sbuf calls into our
  functions, and those have a single malloc instead of a 3-argument one. This
  is by far the easiest way to handle this
* Modify calling convention for device_add_child. It is just way easier if we
  now the path at creation time. BSD does not need it because it creates all
  devices equal (they are the same device_t structure), but for us is way more
  convenient if we can create the appropriate classes.

In particular, I am not implementing the struct filling in the end of
the file. Just comment it out, and make the relevant static functions
public. We will call them from our code.

Mostly trivial changes needed to compile the pv event channel.  We need some
type adjustments, but the most complex ones are assembly fixes.  Because BSD
seems to only do this for 32-bit guests, we need to adjust the inline asm
instructions to take quad words for longs, and force int types for double
words.

After this, the evtchn can be compiled.

The BSD pv event channel will expect the underlying OS to be able to register a
PIC.  For now we will just allow that for xen, and provide the expected
translation functions to allow xen to work.

The design I have chosen is to let the xen event handler run in interrupt
context. We can do threaded if it really becomes a problem, but right now it
should do. The handlers themselves, though, will be threaded. So the
intr_execute_handlers() function will do nothing more than to wake the
respective threads.

BSD will provide us functions, not threads. So we have a common thread that
executes the function that we were given. One exception for this is the xenstore.
The xenstore is already threaded, and its interrupt handler will also just wake
up a thread. So for that we could do better in the future.

With this patch, the grant table code is compiled into osv.
The edits in the file reflect the fact that we don't need to go through PCI
memory for the Xen special device even for HVM. We have mappings that are
way simpler, so we can just use them. All the rest is kept as unchanged as I
could.

Already specified elsewhere.  Noted by Nadav.

Currently we statically link to libstdc++ and libgcc_s, and also dynamically
link to the same libraries (since the payload requires them).  This causes
some symbols to be available from both the static and dynamic version.

With the resolution order change introduced by 82513d41, we can
resolve the same symbol to different addresses at different times.  This
violates the One Definition Rule, and in fact breaks std::string's
destructor.

Fix by only linking in the libraries statically.  We use ld's --whole-archive
flag to bring in all symbols, including those that may be used by the payload
but not by the kernel.

Some symbols now become duplicates; we drop our version.

This adds fairly basic support for rcu.

Declaring:

   mutex mtx;
   rcu_ptr<my_object> my_ptr;

Read-side:

   WITH_LOCK(rcu_read_lock) {
      const my_object* p = my_ptr.read();
      // do things with *p
      // but don't block!
   }

Write-side:

  WITH_LOCK(mtx) {
    my_object* old = my_ptr.read_by_owner();
    my_object* p = new my_object;
    // ...
    my_ptr.assign(p);
    rcu_dispose(old);  // or rcu_defer(some_func, old);
  }

The previous commit (fix symbol resolution order) caused a regression -
tst-pipe.so stopped working, aborting on segfault while handling an
expected exception (one of the only places in OSV where we use an
exception to signal an error - running out of file descriptors).

However, it turns that commit just exposed an already existing bug
in our exception unwinding support. The following trivial test of
exceptions, throwing an integer and catching it, crashes both with
the previous commit, and without it.

Christoph discovered a bug in our dynamic linker, where symbols which
exist in the kernel cannot be used in a shared object, which can
cause nasty bugs when trying to run existing programs.

This test demonstrates this bug, and verifies its fix (in the
previous commit).

Add fsx from xfstests to our test cases.

Currently fails on the first OP due to our lacking truncate implemenation.

BSD register a structure with its per cpu data. We can do the same, using
just the fields we need.

Here are some compile-time definitions that will make importing of BSD code
easier. They should be straightforward and trivially true for our current
setup: x86_64 is our current arch (and when we support others we can just replace
the static -D__x86_64__ with a test that goes precisely here, it makes no sense
to support code that is not SMP, and the id string is included by pretty much
every BSD file - and I believe we have no interest in that.

__attribute__((cold)) on labels isn't supported by gcc 4.7.  Detect support
for the feature and enable it conditionally.

Use DHCP to discover an IP address per each interface, DHCP packets are being hooked
in the networking stack in ip_input and queued for deffered processing using a
dhcp worker thread. Sending dhcp packet is done directly over ethrernet (building
IP and UDP).

There's still alot to be done: setting up lease time, timeouts, more error handling
but it's possible to implement these later on.

  sprintf(fmt, ...) - returns a std::stream
  fprintf(os, fmt, ...) - prints to a std::ostream&

Easier than the usualy method of constructing a temporary ostringstream
and formatting into that.