Added missing bench.jar file back to the image.
Fixed an obsolete description in the README file.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>

This patch adds support for the Linux kill(2) function.
The next patch will add alarm(2) support, which uses kill(2).

To be honest, we sort-of implement kill(). This implementation is
compatible with the API, but the semantics are somewhat different:
While in Linux kill() causes the signal handler to run on one of the
existing threads, in this implementation, the signal handler is run in a
*new* thread.

Implementing the exact Linux semantics in OSv would require tracking when
OSv runs kernel code (i.e., code in the main executable, not a shared
object) so we can delay running the signal handler until returning to the
user code. Moreover, we'll need to be able to interrupt sleeping kernel
code. This is complicated and adds overhead even if signals aren't used
(and they aren't used in most modern code).

I expect that this code will be "good enough" in many use cases.
This code will *not* be good in enough in programs that expect one of the
following:

1. A program that by using Posix Thread's "signal masks" tried to ensure
   that the signal is delivered to one specific thread, and not to an
   arbitrary thread.

2. A program that used kill() or alarm() not intending to run a signal
   handler, but rather intending to interrupt a sleeping system call
   like sleep() or read(). Our kill() does not interrupt sleeping OSv
   function calls, which will continue to sleep on the thread they run
   on.

The support in this patch (and see next patch, for alarm()) is good
enough for netperf's use of alarm().

P.S. kill() can be used only to send a signal to the current process, the
only process we have in OSv (you can also use pid=0 and pid=-1 to achieve
the same results).

This patch also adds a test for kill() and alarm(). The alarm() test
will fail until the next patch :-)

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>

Add a basic test case for filesystem hard link support. You can run it as
follows:

  ./scripts/run.py -e "tests/tst-fs-link.so"

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Add a trivial sleep() test, which sleep()s for 2 seconds, and verifies
that this finishes and has slept for roughly 2 seconds.

I used this for debugging issue #26 - the attempts there ruined timers,
and in particular this trivial test hangs, as sleep() never returns.

(A note to our future automatic testing implementor: We need to allow
for the possibility that a test doesn't cleanly fail, but rather hangs,
and consider this a failure too).

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>

Having that implementation in the tree even though it was superceded is
confusing and really unnecessary.

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This regression test trigger issue #8
"Make Java InetAddress.getHostName() work" by exercising the DNS
resolver on localhost and a dns root server.

The test takes care of specifying NI_NOFQDN to resolve only the
hostname part of localhost ip.

It appears that the DNS ip is not communicated to the libc by
core/dhcp.cci: /etc/resolv.conf is not filled.

Test contributed while waiting for an fix idea to implement.

Signed-off-by: Benoit Canet <benoit@irqsave.net>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

clearing cloudius cli and sshd jar (they are replaced by crash jar in usr.manifest) and moving tmp and crash into usr.manifest

This is a test for the effectiveness of our scheduler's load balancing while
running several threads on several cpus.

A full description of the test and its expected results is included in
comments in the beginning of the code. but briefly, the test runs multiple
concurrent busy-loop threads, and an additional "intermittent" thread (one that
busy-loops for a short duration, and then sleeps), and expects that all busy
threads will get their fair share of the CPU, and the intermittent thread
won't bother them too much.

Testing the current code, this tests demonstrates the following problems
we have:

1. Two busy-loop threads on 2 cpus are 5%-10% slower than just one.
   This is not kernel overhead (profiling show 100% of the time in the
   test's inner loop), and I see exactly the slowdown when running this
   test on the Linux host, so it might be related to the host's multitasking?
   For now, let's not worry about that.

2. Much more worrying is that the intermittent thread sometimes (in about half
   the tests) causes us to only fully use one CPU, and of course get bad
   performance.

3. In many of the tests involving more than 2 threads (2 threads +
   intermittent, or 4 threads) load balancing wasn't fair and some
   threads got more CPU than the others.

Later I'll send patches to fix issues 2 and 3, which appear to happen because
the load balancer thread doesn't run as often as it should, because of vruntime
problems.

Parsed by JLine (in CRaSH)
Console should now better understand keys like home/end/arrows

a test where the guest connects to the host and sends a small packet of data.
used to verify that retransmits is working in Van Jacobson and the TCP
stack in general.

Add simple tests for munmap() for file-backed memory maps. This exposes
a limitation in munmap() not writing out MAP_SHARED mappings.

The previous commit (fix symbol resolution order) caused a regression -
tst-pipe.so stopped working, aborting on segfault while handling an
expected exception (one of the only places in OSV where we use an
exception to signal an error - running out of file descriptors).

However, it turns that commit just exposed an already existing bug
in our exception unwinding support. The following trivial test of
exceptions, throwing an integer and catching it, crashes both with
the previous commit, and without it.

Christoph discovered a bug in our dynamic linker, where symbols which
exist in the kernel cannot be used in a shared object, which can
cause nasty bugs when trying to run existing programs.

This test demonstrates this bug, and verifies its fix (in the
previous commit).

2 threads are created on 2 different vcpus, one consumer and one producer.

Both threads are pushing and popping concurrently 1,000,000,000 elements,
the producer is pushing a random number between 0 and 7 and consumer pops
those numbers. Both of the threads keeps track on the values they
pushed/popped. per each value, the number of pushed elements
should be equal to the number of popped elements.

 - ring_spsc: 14.8 Mop/s per core

Add tst-epoll.cc for testing the epoll_*() functions.
This test finds a bug, which will be fixed in a separate patch.

This reverts commit ccaad30c.

Added a test for wake_with(). It tries to ensure that the problematic
case solved by wake_with() actually happens quickly, by:
 1. Spin a long time between the setting of the flag and t->wake()
 2. Do a spurious wake() to ensure that the waiting thread is woken
    up right after setting the flag, before the intended wake.
 3. Use mprotect() to ensure that working with an already join()ed
    thread crashes immediately, instead of just maybe crashing.

This test fails when wake_with() doesn't use ref()/unref(), and succeeds
with the full wake_with().

tst-wake contains a second test, which does the same thing but without
the additional measures we used to show the bug (spinning, spurious
wake and mprotect). Without these additional measures the test iteration
is much faster, which allows us to stress wake/join much more.

Usage:

  perf list (lists all tracepoints)
  perf stat tp... (counts tracepoints)

Example:

[/]$ perf stat mutex_lock ctxsw=sched_switch mutex_unlock wake=sched_wake
  mutex_lock   ctxsw  mutex_unlock    wake
          40       3          1909       2
        2075     147           190      82
         193     138           193      78
         146     139           146      92
         317     179           317      78
         146     139           146      78
         146     139           186      78
         205     139           165      78
         146     139           146      78
         146     139           146      78
         146     139           146      80
         193     143           193      81
         151     147           151      78
         146     139           146      78
         146     139           146      78
         146     139           146      78
         159     139           159      78
         149     139           149      78
         146     139           146      78
         164     139           164      78
         146     139           176      78
         176     139           146      78
         149     139           149      78
         146     139           146      78
         146     139           146      78
  mutex_lock   ctxsw  mutex_unlock    wake
         146     139           146      79
         715     147           715      80
         188     139           204      78

Exposes tracepoints and counters

Add a "java" command to the CLI, using the same syntax of java.so and
attempting to emulate as closely as possible the "java" command on Linux.
So for example one can run

        java Hello

to run /java/Hello.class (/java is on the classpath by default), or

        java -jar /java/bench.jar

to run the main class of this jar, or a more sophisticated
command lines, such as the following which runs Jetty (if the
appropriate files are in your image):

        java -classpath /jetty/* org.eclipse.jetty.xml.XmlConfiguration /jetty/jetty.xml

Note that like java.so, the new "java" command basically runs the RunJava
class (/java/RunJava.class). Remember that java.so adds /java to the parent
class loader, so we can always find the RunJava class even though it's not
in cli.jar or cloudius.jar).

these tests are a bit outdated, they change the system configuration and are
not useful anymore, they were basically written to understand how stuff works.

tst-bsd-netdriver.c - was made just to figure out the network driver model of
                      freebsd.
tst-bsd-netisr.c    - same for isr layer, this tests runs over the ARP isr and
                      the system is badly wounded after it runs, it is useless today
                      and was written to figure out how netisr works.
tst-virtionet.c     - testing network interface creation using virtio,
                      today the interface is created anyway.

This patch adds pipe(). The pipes are built using the same FIFO implementation,
"af_local_buffer", as used by the existing unix-domain socketpair
implementation - while the socket-pair used two of these buffers, a pipe
uses one.

This implementation deviates from traditional POSIX pipe behavior in two
ways that we should fix in followup-patches:

1. SIGPIPE is not supported: A write to a pipe whose read end is closed
   will always return EPIPE, and not generate a SIGPIPE signal.
   Programs that rely on SIGPIPE will break, but SIGPIPE is completely out
   of fashion, and normally ignored.

2. Unix-style "atomic writes" are not obeyed. A write(), even if smaller
   than PIPE_BUF (=4096 on Linux, whose ABI we're emulating), may partially
   succeed if the pipe's buffer is nearly full. Only a write() of a single
   byte is guaranteed to be atomic.

   We hope that Java doesn't rely on multi-byte write() atomicity
   (single-byte writes are enough for waking poll, for example), and users
   of Java's "Pipe" class definitely can't (as Java is not Posix-only),
   so we hope this will not cause problems. Fixing this issue (which is easy)
   is left as a TODO in the code.

Additionally, this patch marks with a FIXME (but doesn't fix) a serious
bug in the code's iovec handling, so writev() and readv() are expected
not to work in this version of pipe() - and also on the existing socketpair.

Added a simple readdir() and readdir_r() test.
The test is successful - it turns out readdir() had no bug, and the bug
was in mkbootfs.py, but since I already wrote the test I guess might as
well add it.

Java.so used to correctly support the "-jar" option, but did not fully
allow the other "mode" of running Java: specifying a class name which is
supposed to be searched in the class path. The biggest problem was that
it only know to find class files, but not a class inside a jar in the class
path - even if the classpath was correctly set.

Unfortunately, fixing this C code was impossible, as JNI's FindClass()
simply doesn't know to look in Jars.

So this patch overhauls java.so: Java.so now only runs a fixed class,
/java/RunJava.class. This class, in turn, is the one that parses the
command line arguments, sets the class path, finds the jar or class to
run, etc.. The code is now much easier to understand, and actually works
as expected :-) It also fixes the bug we had with SpecJVM2008's "compiler.*"
benchmarks, which forced us to tweak the class path manually.

The new code supports running a class from the classpath, and also the
"-classpath" option to set the class path. Like the "java" command line
tool in Linux, this one also recognizes wildcard classpaths. For example,
to run Jetty, whose code is in a dozen jars in /jetty, one can do:

        run.py -e "java.so -classpath /jetty/* org.eclipse.jetty.xml.XmlConfiguration jetty.xml"

Builds on osv an Linux.

Tests context switch performance:
  - between threads co-located on the same cpu
  - between threads on different cpus
  - between threads placed by the scheduler policy

The following test currently frequently crashes - with an abort or
assertion failure.

It's a very simple test, where 10 threads do an endless yield() loop.
While yield() itself is not very important - and doesn't even implement
the promise of sched_yield(2) to move the thread to the end of the run
queue - this test failure may be the sign of a scheduler bug that needs to
be fixed.