Add a "java" command to the CLI, using the same syntax of java.so and
attempting to emulate as closely as possible the "java" command on Linux.
So for example one can run

        java Hello

to run /java/Hello.class (/java is on the classpath by default), or

        java -jar /java/bench.jar

to run the main class of this jar, or a more sophisticated
command lines, such as the following which runs Jetty (if the
appropriate files are in your image):

        java -classpath /jetty/* org.eclipse.jetty.xml.XmlConfiguration /jetty/jetty.xml

Note that like java.so, the new "java" command basically runs the RunJava
class (/java/RunJava.class). Remember that java.so adds /java to the parent
class loader, so we can always find the RunJava class even though it's not
in cli.jar or cloudius.jar).

these tests are a bit outdated, they change the system configuration and are
not useful anymore, they were basically written to understand how stuff works.

tst-bsd-netdriver.c - was made just to figure out the network driver model of
                      freebsd.
tst-bsd-netisr.c    - same for isr layer, this tests runs over the ARP isr and
                      the system is badly wounded after it runs, it is useless today
                      and was written to figure out how netisr works.
tst-virtionet.c     - testing network interface creation using virtio,
                      today the interface is created anyway.

This patch adds pipe(). The pipes are built using the same FIFO implementation,
"af_local_buffer", as used by the existing unix-domain socketpair
implementation - while the socket-pair used two of these buffers, a pipe
uses one.

This implementation deviates from traditional POSIX pipe behavior in two
ways that we should fix in followup-patches:

1. SIGPIPE is not supported: A write to a pipe whose read end is closed
   will always return EPIPE, and not generate a SIGPIPE signal.
   Programs that rely on SIGPIPE will break, but SIGPIPE is completely out
   of fashion, and normally ignored.

2. Unix-style "atomic writes" are not obeyed. A write(), even if smaller
   than PIPE_BUF (=4096 on Linux, whose ABI we're emulating), may partially
   succeed if the pipe's buffer is nearly full. Only a write() of a single
   byte is guaranteed to be atomic.

   We hope that Java doesn't rely on multi-byte write() atomicity
   (single-byte writes are enough for waking poll, for example), and users
   of Java's "Pipe" class definitely can't (as Java is not Posix-only),
   so we hope this will not cause problems. Fixing this issue (which is easy)
   is left as a TODO in the code.

Additionally, this patch marks with a FIXME (but doesn't fix) a serious
bug in the code's iovec handling, so writev() and readv() are expected
not to work in this version of pipe() - and also on the existing socketpair.

Added a simple readdir() and readdir_r() test.
The test is successful - it turns out readdir() had no bug, and the bug
was in mkbootfs.py, but since I already wrote the test I guess might as
well add it.

Java.so used to correctly support the "-jar" option, but did not fully
allow the other "mode" of running Java: specifying a class name which is
supposed to be searched in the class path. The biggest problem was that
it only know to find class files, but not a class inside a jar in the class
path - even if the classpath was correctly set.

Unfortunately, fixing this C code was impossible, as JNI's FindClass()
simply doesn't know to look in Jars.

So this patch overhauls java.so: Java.so now only runs a fixed class,
/java/RunJava.class. This class, in turn, is the one that parses the
command line arguments, sets the class path, finds the jar or class to
run, etc.. The code is now much easier to understand, and actually works
as expected :-) It also fixes the bug we had with SpecJVM2008's "compiler.*"
benchmarks, which forced us to tweak the class path manually.

The new code supports running a class from the classpath, and also the
"-classpath" option to set the class path. Like the "java" command line
tool in Linux, this one also recognizes wildcard classpaths. For example,
to run Jetty, whose code is in a dozen jars in /jetty, one can do:

        run.py -e "java.so -classpath /jetty/* org.eclipse.jetty.xml.XmlConfiguration jetty.xml"

Builds on osv an Linux.

Tests context switch performance:
  - between threads co-located on the same cpu
  - between threads on different cpus
  - between threads placed by the scheduler policy

The following test currently frequently crashes - with an abort or
assertion failure.

It's a very simple test, where 10 threads do an endless yield() loop.
While yield() itself is not very important - and doesn't even implement
the promise of sched_yield(2) to move the thread to the end of the run
queue - this test failure may be the sign of a scheduler bug that needs to
be fixed.

Until now, OSV's console defaulted to raw mode, to make the CLI
happy. The problem is that on Linux, applications expect to be
run in cooked mode, so if we ever run a simple application that
tries to read user input, it can be confused.

This patch makes OSV console default to cooked mode, and the
CLI switch to raw mode before reading an input line - and reset
to the default mode just before running the user's command.

Unfortunately, we had to resort to adding a JNI class "Stty",
since Java has no builtin support for the ioctls required for
changing the tty settings.

While trying to run the "compiler.compiler" benchmark from SPECjvm2008,
I noticed we seem to have a problem with concurrent use of filesystem
operations - which often hang (waiting on a vn_lock()) or cause
assertion failures.

This trivial test - which does stat() calls in 10 concurrent threads -
reproduces this bug, and usually (but not always) crashes on one of
several assertion failures, or hangs.

APIs. In particular, the "crypto.aes" and "crypto.rsa" benchmarks
fail without them.

At some point, we should consider just adding most of the stuff in
/usr/lib/jvm/java/jre/lib to our distribution... Everything in that
directory has to be useful for something, and it might take a while
before we try all possible APIs in OSV...

split cloudius.jar to cloudius.jar an cli.jar
the following patch will introduce proper code reusing between the two jars
basically, cli.jar will be dependant on cloudius.jar

Clean up the lockfree/mutex.hh code. This introduces no new algorithms or
major bug fixes - what was blocking the lockfree mutex from working were
mainly preemption bugs and a bug in the lockfree queue implementation.

Also introduced a test for the mutex variants (mutex, lockfree::mutex and
spinlock) - several threads do a non-atomic increment to a shared variables,
and unless a mutex is used, the sum ends up being wrong. The test seems to
be working correctly, and also exercising the interesting cases of the code
(the "responsibility hand-off" protocol), but does so very rarely, so we'll
still need more testing before we can fully trust this new code.

The test also begins to do performance testing on the mutex variants.
uncontended lockfree::mutex is currently twice slower than uncontended
mutex, so some more work is needed in this area.

expects to recieve a String[] array as an argument, invokes run_elf()
and communicates the return value by setting a static int of the invoking
class

Add a much better test for the lock-free multi-producer single-consumer
queue implementation, where a 20 threads push numbers onto the queue,
and a single consumer reads them and verifies that they all are received
and in the right order. This test uncovered a bug which will be fixed by
a later commit.

The test added here patch was written using C++11's concurrency APIs
(std::thread, std::condition_variable, etc.) instead of osv's own APIs,
for two reasons:

1. It also helped uncover a bunch of bugs in our pthread and C++11 thread
   support, so this test can double as another pthread test.
2. It allows running the same test in Linux, to tell if a certain bug is a
   bug in lockfree:queue_mpsc, or in underlying osv scheduler mechanisms.

the TLS), which led to preempt_counter not being intialized to zero and
therefore preemption not being enabled for new threads.

Such non-preemptable threads can monopolize their CPU and cause other
threads with the misfortune of being assigned to this CPU to never run.

This patch adds a simple test reproducing this bug. Because we don't have
an implementation of __tls_get_addr(), we can't "extern" a TLS symbol
(here sched::preempt_counter) from the test, and so I had to create a
new sched::get_preempt_counter() function which I can "extern" from the
test.

Added some basic tests for condition variables. I'm sure more tests
can be written but this is a good start. They didn't find any bug,
however, so as Avi says, it may mean the test isn't good enough,
not there isn't a bug ;-)

By the way, I also checked the condition variables in more runs using
pthread (not yet committed), and Java (which uses condition variables),
and they seem to be working fine.

Not sure we want it, but for now, it makes the resolver stuff work.

1. a simple command that invokes the run method of a test class
2. supports autocomplete with test names

Add tst-lsroute a test utility that shows the routing tables in a similar way to "netstat -rn" ro "route" on Linux (I don't show the Metric, Ref and Use columns but they can probably be added if desired). This change required adding sysctl(0 functionality to osv. I have added a new function osv_sysctl(), which uase the same arguments as sysctl(). At the moment we only support the routing related functions (I have only tested NET_RT_DUMP - dump the routing tables and NET_RT_IFLIST - return the network interfaces. Other routing ops may run as well). Last but not least tst-lsroute supports IPv6 once we implement it. And yes it is possible to implement the whole things in Java with a little JNI magic...

Instead of specifying

  /foo/some/common/path/name: /bar/some/common/path/name

say

  /foo/&/some/common/path/name: /bar/&

The '/&/' tells mkbootfs where the common name begins; everything after that
is substituted for '&' in the host filesystem filename.

Needed for libawt.so.