Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Rename blacklisted tests, from tst-wake.cc et al. to misc-wake.cc.

The different name will cause these tests not to be automatically run
by "make check" - without needing the separate blacklist in test.py
(which this patch deletes).
After this patch, testrunner.so will also only run tests called tst-*,
so will not run the misc-* tests.

The misc-* tests can still be run manually, e.g.,
  run.py -e tests/misc-mutex.so

In addition to the previously blacklisted tests, this patch "blacklists"
(renames) a few additional tests which fail quickly, but test.py didn't
know because they didn't use the word "fail". An example is tst-schedule.so,
which existed immediately when not run on 1 vcpu. So this patch also
renames it to misc-schedule.so, so "make check" or testrunner.so won't
run this test.

Note that after this patch, testrunner.so is a new way to run all tests,
but it isn't working well yet because it still exposes new bugs that do not
exist in the separate tests (depending on your view point, this might be
considered a feature, not a bug, in testrunner.so...).

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Using hard-coded path names is problematic because other test cases may
use the same path names and forget to clean up after them.

Make tst-fs-link.so more robust by using mktemp() to generate unique
path names.

Reviewed-by: Tomasz Grabiec <tgrabiec@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The last part of the standard thread tests created 4 threads and calls the
detach of one from the body of the other. They live in the same block to
guarantee that they will all be destroyed more or less at the same time (we
expect). Avi, however, demonstrated that a mistake prevents that from being
the actual case:

    t1 starts to run
    t2 starts to run
    t3 starts to run
    t4 starts to run
    t4 is detached
    t4 is destroyed (ok)
    t3 is destroyed. wasn't detached or join, to terminate
    t1, t2, t3 are detached, but too late

This introduces a simple wait mechanism to avoid having the threads
terminated after the block is gone.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

        sched::thread *t2 = nullptr;
        sched::thread *t1 = new sched::thread([&]{
            // wait for the t2 object to exist (not necessarily run)
            sched::thread::wait_until([&] { return t2 != nullptr; });
            if (quick) {
                return;
            }
            sched::thread::sleep_until(nanotime() + 10_ms);
        }, sched::thread::attr(sched::cpus[0]));

        t2 = new sched::thread([&]{
            t1->wake();
        }, sched::thread::attr(sched::cpus[1]));

        t1->start();
        t2->start();
        delete t1
        delete t2;

t1 may start, complete, and be destroyed before t2 gets a chance to run.  In
this case the call to t1->wake() will access deallocated memory.

Fix by making sure t1 is only destroyed after t2 completes.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

This patch fixes the error codes in four error cases:

1. unlink() of a directory used to return EPERM (as in Posix), and now
   returns EISDIR (as in Linux).

2. rmdir() of a non-empty directory used to return EEXIST (as in Posix)
   and now returns ENOTEMPTY (as in Linux).

3. rmdir() of a regular file (non-directory) used to return EBADF
   and now returns ENOTDIR (as in Linux).

4. readdir() of a regular file (non-directory) used to return EBADF
   and now returns ENOTDIR (as in Linux).

This patch also adds a test, tst-remove.cc, for the various unlink() and
rmdir() success and failure modes.

Fixes #123.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Our implementation of operate() will try to fill as much as the address space
as possible with huge pages. If that fails, we should be able to fill the range
with small pages instead of failing. This test should make sure that in such
scenarios, the resulting mapping looks sane.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This reverts commit bdd99c7b.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

tst-pipe.cc read a buffer after freeing it, which could have theoretically
caused segfaults (it didn't in practice, but better fix this oversight).

Also, it forgot to return a return code, so it doesn't play nicely in
a test framework like testrunner.so. I'm surprised the C++ compiler wasn't
bothered by an int main() not returning an int.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The remove() function is part of the ISO C 1989 standard, and used, for
example, to implement Java's File.delete(). It's supposed to remove a
file, regardless of whether unlink() or rmdir() is needed to remove it.

Our implementation (from Musl's) assumed that unlink() on a directory fails
with EISDIR, and only on that case it tried rmdir(). However, returning
EISDIR on unlink() is a Linux extension, which (deliberately) goes against
the Posix standard - which specified EPERM should be returned in that case.
Our ZFS implementation of unlink, following Solaris and FreeBSD (and not
Linux), returns EPERM in that case.

This meant that remove() used to fail deleting empty directories, and
Java code (like the SpecJVM2008 "derby" benchmark) using it to recursively
delete a directory, left behind undeleted empty directories.

So this patch fixes remove() to try rmdir() if unlink() returned either
the Linux-specific EISDIR, or the Posix-standard EPERM. It also adds
to the readdir test another test which verifies that remove() can delete
all files in a directory - both regular files and empty directories.

Fixes #112.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Before this patch, tst-readdir.cc assert()ed on every test, meaning that
a failure will cause a crash. Change this to use a report() function,
which counts failures instead of immediately crashing on the first one.

This patch doesn't change anything in what this test actually tests for.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

That test goes together with thread detach, but I am also calling joins
to make sure we're not breaking them. It is unfortunate that this is quite
non-deterministic and we can't really surely test for failure. But on the
flip side, it did help me catch a couple of bugs in my implementation. So
it will eventually explode somewhere if a bug appears.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Since we are using ZFS now, we don't need workarounds
for tmpfs bugs.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Revive testrunner.so (run with run.py -e testrunner.so), which runs all tests
in the tests/ directory in the image.

Changed it to C++, and for simplicity use the osv::run() API.

The next steps on the TODO list should be:
1. Change the manifest so that non-tests are NOT copied to the tests/ on the
   image.
2. Add an option to run tests in parallel, for maximum testing speed. Will not
   be trivial (will need to redirect the tests' output so they don't mix, and
   also figure out the load so we can run as many tests in parallel as we need
   to fill up the CPUs).

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

A ';' at the end of a parameter mark the end of a program's arguments list.

The goal of this patch is to be able to split mkfs.so in to parts mkfs.so and
cpiod.so.

The patch uses a full spirit parser to escape "" and split commands around ';'.

Signed-off-by: Benoit Canet <benoit@irqsave.net>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The main purpose is to exercise the new parameter checks added to
mmap to handle wrong combination of parameters.
Let's intentionally force errors and catch them with the respective errno codes.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

storage/runtime arguments for tracepoint can be inferred from
assign() function signature instead of specified explicitly
by storage_args/runtime_args. This makes boilerplate code smaller.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Add a simple test case for "/dev/random" which just reads from the file
and prints out the random bytes.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Add missing join() in tst-loadbalance, to avoid rare crashes during the
test.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch adds tst-scheduler.cc, containing a few tests for the fairness
of scheduling of several threads on one CPU (for scheduling issues involving
load-balancing across multiple CPUs, check out the existing tst-loadbalance).

The test is written in standard C++11, so it can be compiled and
run on both Linux and OSv, to compare their scheduler behaviors.
It is actually more a benchmark then a test (it doesn't "succeed" or "fail").

The test begins with several tests of the long-term fairness of the
schduler when threads of different or identical priorities are run for
10 seconds, and we look at how much work each thread got done in those
10 seconds. This test only works on OSv (which supports float priorities).

The second part of the test again tests long-term fairness of the scheduler
when all threads have the default priority (so this test is standard C++11):
We run a loop which takes (when run alone) 10 seconds, on 2 or 3
threads in parallel. We expect to see that all 2 or 3 threads
finish at (more-or-less) exactly the same time - after 20 or 30
seconds. Both OSv and Linux pass this test with flying colors.

The third part of the test runs two different threads concurrently:
 1. One thread wants to use all available CPU to loop for 10 seconds.
 2. The second thread wants to loop in an amount that takes N
    milliseconds, and then sleep for N milliseconds, and so on,
    until completing the same number of loop iterations that (when run
    alone) takes 10 seconds.

The "fair" behavior of the this test is that both threads get equal
CPU time and finish together: Thread 2 runs for N milliseconds, then
while it is sleeping for N more, Thread 1 gets to run.
This measure this for N=1 through 32ms. In OSv's new scheduler, indeed both
threads get an almost fair share (with N=32ms, one thread finishes in 19
seconds, the second in 21.4 seconds; we don't expect total fairness because
of the runtime decay).

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Allows longer tests to be run.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

v2: Let's convert everything to std::chrono::timepoint (Avi Kivity)
v3: Use the to_timeptr approach suggested by Nadav Har'El

This test checks the functionality of the utimes support.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

As suggested by Nadav, add tests for mincore() interraction with demand
paging.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This adds a simple mmap microbenchmark that can be run on both OSv and
Linux.  The benchmark mmaps memory for various sizes and touches the
mmap'd memory in 4K increments to fault in memory.  The benchmark also
repeats the same tests using MAP_POPULATE for reference.

OSv page faults are slightly slower than Linux on first iteration but
faster on subsequent iterations after host operating system has faulted
in memory for the guest.

I've included full numbers on 2-core Sandy Bridge i7 for a OSv guest,
Linux guest, and Linux host below:

  OSv guest
  ---------

  Iteration 1

       time (seconds)
   MiB demand populate
     1 0.004  0.000
     2 0.000  0.000
     4 0.000  0.000
     8 0.001  0.000
    16 0.003  0.000
    32 0.007  0.000
    64 0.013  0.000
   128 0.024  0.000
   256 0.052  0.001
   512 0.229  0.002
  1024 0.587  0.005

  Iteration 2

       time (seconds)
   MiB demand populate
     1 0.001  0.000
     2 0.000  0.000
     4 0.000  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.004  0.000
    64 0.010  0.000
   128 0.019  0.001
   256 0.036  0.001
   512 0.069  0.002
  1024 0.137  0.005

  Iteration 3

       time (seconds)
   MiB demand populate
     1 0.001  0.000
     2 0.000  0.000
     4 0.000  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.005  0.000
    64 0.010  0.000
   128 0.020  0.000
   256 0.039  0.001
   512 0.087  0.002
  1024 0.138  0.005

  Iteration 4

       time (seconds)
   MiB demand populate
     1 0.001  0.000
     2 0.000  0.000
     4 0.000  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.004  0.000
    64 0.012  0.000
   128 0.025  0.001
   256 0.040  0.001
   512 0.082  0.002
  1024 0.138  0.005

  Iteration 5

       time (seconds)
   MiB demand populate
     1 0.001  0.000
     2 0.000  0.000
     4 0.000  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.004  0.000
    64 0.012  0.000
   128 0.028  0.001
   256 0.040  0.001
   512 0.082  0.002
  1024 0.166  0.005

  Linux guest
  -----------

  Iteration 1

       time (seconds)
   MiB demand populate
     1 0.001  0.000
     2 0.001  0.000
     4 0.002  0.000
     8 0.003  0.000
    16 0.005  0.000
    32 0.008  0.000
    64 0.015  0.000
   128 0.151  0.001
   256 0.090  0.001
   512 0.266  0.003
  1024 0.401  0.006

  Iteration 2

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.000  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.005  0.000
    64 0.009  0.000
   128 0.019  0.001
   256 0.037  0.001
   512 0.072  0.003
  1024 0.144  0.006

  Iteration 3

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.001  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.005  0.000
    64 0.010  0.000
   128 0.019  0.001
   256 0.037  0.001
   512 0.072  0.003
  1024 0.143  0.006

  Iteration 4

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.001  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.003  0.000
    32 0.005  0.000
    64 0.010  0.000
   128 0.020  0.001
   256 0.038  0.001
   512 0.073  0.003
  1024 0.143  0.006

  Iteration 5

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.001  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.003  0.000
    32 0.005  0.000
    64 0.010  0.000
   128 0.020  0.001
   256 0.037  0.001
   512 0.072  0.003
  1024 0.144  0.006

  Linux host
  ----------

  Iteration 1

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.001  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.005  0.000
    64 0.009  0.000
   128 0.019  0.001
   256 0.035  0.001
   512 0.152  0.003
  1024 0.286  0.011

  Iteration 2

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.000  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.004  0.000
    64 0.010  0.000
   128 0.018  0.001
   256 0.035  0.001
   512 0.192  0.003
  1024 0.334  0.011

  Iteration 3

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.000  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.004  0.000
    64 0.010  0.000
   128 0.018  0.001
   256 0.035  0.001
   512 0.194  0.003
  1024 0.329  0.011

  Iteration 4

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.000  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.004  0.000
    64 0.010  0.000
   128 0.018  0.001
   256 0.036  0.001
   512 0.138  0.003
  1024 0.341  0.011

  Iteration 5

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.000  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.004  0.000
    64 0.010  0.000
   128 0.018  0.001
   256 0.035  0.001
   512 0.135  0.002
  1024 0.324  0.011

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Avoid possible blocking.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Make sure to wait until the running thread count drops to zero
before destroying things.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Previously, _Unwind_Resume wasn't available, so functions that handled an
exception implicitly (by running a few destructors) crashed.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The test creates and destroys threads, each of which creates a random number
of connections, each transferring a random number of bytes to an echo server.

This is used to stress the tcp/ip stack.

The test is portable, and builds on the host with the command

  g++ -O2  -g3 -pthread -std=gnu++11 -lboost_program_options -lboost_system tests/tst-tcp.cc

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Add the actual test case that was forgotten from commit a9f8092a
("Introduce test for libc locking").

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Dead code, and boost::signals2::signal<> does the job better.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

It is needed for systems where python points to python3

Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The tst-zfs-simple.so test case has serverd its purpose for bringup.  As
OSv build relies on working ZFS now, there's no need to run the tests.

Furthermore, we have the full ztest stress test in the tree:

  bsd/cddl/contrib/opensolaris/cmd/ztest/ztest.c

which we can use if needed.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

When system classloader is used as a parent some of its protected
methods are called to lookup resources.

This also adds delegation for all remaining protected and public
methods.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>