Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Start using spaces instead of tabs and surround all single-line
control statements with curly braces.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

It will be useful to take better and safer VFS decisions in the future.
For example, avoiding code that uses the absolute path to determine something.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

newmp->m_covered must be released if not NULL.
Found this problem while dumping dcache content.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Currently, vflush is used in the unmount process to release remaining
dentries. vflush in turn calls vevict that is releasing dentries that
it doesn't own.
This behavior is not correct neither good to the future of VFS.

So Avi suggested switching to a different approach. We could only
release those dentries owned by the mountpoint when unmounting it as
there wouldn't be anything else in the dcache (given its functionality).

The problem was fixed by doing the following steps:
 - Drop vflush calls in sys_umount2, make vevict an empty function,
and remove vevict.

 - Created the function release_mp_dentries to release dentries of a mount
point which will be called by VFS_UNMOUNT. It cannot be called before
VFS_UNMOUNT as failures must be considered, neither after as the mount point
would be considered busy.
Don't respect this "rule", and that previously seen ZFS replay transaction
error would happen.

NOTE: vflush is currently duplicated in zfs unmount cases to address the problem
above. This patch fixes this duplication as well.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

It's a good practice to propagate failures so that make does not
succeed if something goes wrong.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

To make it work self._closed needs to be initialized early because
even though exception is thrown from __init__(), __exit__() still will
be called and that one expects self._closed to be defined.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Found during code inspection.

Qemu process is started with stdout=PIPE but the pipe will not be read
until _process.communicate() is called and it is called when nbd_file
is closed. So if the pipe buffer fills up we may have a deadlock. Qemu
will be blocked on stdout.write() and imgedit.py will be waiting for
it to open a socket.

Solution here is to redirect to /dev/null as this has the same
effect as current code.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This is not a documented behavior but glibc's dlopen behaves this way.

Spotted when debugging assertion failure when java is started with a
debugger, we try to read elf header from "/" on dlopen("", ...):

Assertion failed: data.uio_resid == 0 (osv/fs/fs.cc: read: 27)

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Useful if you want to know who created that large pile of threads.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

gcc -O3 complains about uninitialized use of 'nims'. In fact,
inm_get_source() can return an error without setting nims which will
be later read from RB_FOREACH_REVERSE_FROM macro. It looks like 'nims'
is intended to hold the last value for which inm_get_source() returned
success. The uninitialized access would happen if this function never
succeded. I am not sure if this is possible in practice, but let's
initialize nims to NULL, which will cause no iteration in
RB_FOREACH_REVERSE_FROM macro.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Gcc comaplins about attempt to read 'size' via dereferencing a pointer
in xdr_u_int() in case xdrs->x_op == XDR_ENCODE.  However, in this
case the size will be set from the switch case inside xdr_string()
before xdr_u_int() is invoked. I think it's spurious because the
code clearly assumes that xdrs->x_op cannot change between these two
execution points.

Let's initialize 'size' to 0 to make gcc happy.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Currently map_file() do three passes over vma memory in a worst case.
First it maps memory with write permission while zeroing it, then it
reads a file into memory and, if vma is read only, it does one more
pass to fix memory permissions. Fix it by providing new specialization
of fill_page class which builds iovec of all allocated memory and
reads from a file using the iovec at the end of populate stage.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Commit 2ebfd915 ("build.mk: Generate a manifest of tests automatically")
separated native tests from Java tests but unfortunately broke the build
because it forgot to add dependency to the new java_tests variable.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

In issue #145 I reported a crash during boot in start_early_threads().
I wasn't actually able to replicate this bug on master, but it happens
quite frequently (e.g., on virtually every "make check" run) with some
patches of mine that seem unrelated to this bug.

The problem is that start_early_threads() (added in 63216e85)
iterates on the threads in the thread list, and uses
t->remote_thread_local_var() for each thread. This can only work if
the thread has its TLS initialized, but unfortunately in thread's
constructor we first added the new thread to the list, and only later
called setup_tcb() (which allocates and initializes the TLS). If we're
unlucky, start_early_threads() can find a thread on the list which still
doesn't have its TLS allocated, so remote_thread_local_var() will crash.

The simple fix is to switch the order of the construction: First
set up the new thread's TLS, and only then add it to the list of
threads.

Fixes #145.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Generate a manifest of tests automatically from the makefile
based on the value of $(tests) and $(java_tests) variables.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Currently we just call poweroff() which does not umount
file systems which leads to data loss on next boot.

This happens for example if you start the CLI and type exit. On next
boot you will get:

 TX_CREATE
 TX_CREATE
 Solaris: WARNING: ZFS replay transaction error 95, dataset osv/zfs,
 seq 0x1, txtype 2

Reported-by: Pekka Enberg <penberg@cloudius-systems.com>
Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

In order to reuse the logic it needs to be extracted.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

When compiling with -O3 gcc detects that 'th' may be uninitialized.
This is true, it happens when prio_find_thread is given
which == PRIO_USER or which == PRIO_PGRP.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

MultiJarLoader reads the command line parameters from a file
and uses split to make a list of params out of them before
calling parseArgs.

Currently the regular expression is \\s which means that
consecutive space will be split it to an empty string.
It should be \\s+ so consecutive spaces will be ignored.

Found by code inspection.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>

Instead of the old C-style file-operation function types and fo_*()
functions, since recently we have methods of the "file" class. All our
filesystem code is now C++, and can use these methods directly.

So this patch drops the old types and functions, and uses the class methods
instead.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

The code has bitrotted, and it doesn't support wait morphing.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Instead of free standing functions, use member functions, which are easier
to work with.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Prevent memset() overwriting an initialized mutex.  Instead initialize
members individually.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

the _ prefix helps to distinguish between members and non-members; helps with
the next patch.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Make the code more maintainable by removing the #ifdefs; it doesn't make
sense to disable wait morphing.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Drop the socket uma zone and replace it with calls to new and delete.  This
allows the constructor and destructor to be called, so we can add C++
objects to the socket structure.

Take care to use the default-initializing form of the constructor since
the socket code requires zero initialization.

We lose the ability to limit the socket count, so we'll have to re-add it in
the future if we want it.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Each implementation of "struct file" needs to implement 8 different file
operations. Most special file implementations, such as pipe, socketpair,
epoll and timerfd, don't support many of these operations. We had in
unsupported.h functions that can be reused for the unsupported operation,
but this resulted in a lot of ugly boiler-plate code.

Instead, this patch switches to a cleaner, more C++-like, method:
It defines a new "file" subclass, called "special_file", which implements
all file operations except close(), with a default implementation identical
to the old unsupported.h implementations.

The files of pipe(), socketpair(), timerfd() and epoll_create() now inherit
from special_file, and only override the file operations they really want
to implement.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Right now most of mmap related functions have the same bug related to vma
locking: they validate mapping under vma lock then release the lock
and do actual vma operation, but since between validation and operation
the mapping can go away it is incorrect. This patch fixes this by doing
validation and operation under the same lock.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Currently offset calculation is incorrect. Fix it by tracking base
address of a region and calculating an offset by subtracting base
address from current mapping address.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

map_range() is a entry point into a page mapper, so make it
impossible to instantiate map_level class directly by making all of its
function and constructor private and declaring map_range() as a friend.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

One for initial call, another for recursion.

This gets rid of default parameters, the need for std::integral_constant
and makes code much more readable.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Bring back haproxy.py that I deleted by accident in commit 3e48ad65
("modules: support "include"ing a module list").

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Instead of listing modules from the apps/ repository in OSv's config.json,
this patch adds support for an "include" specification in config.json,
which we use to include apps/modules.json.

This allows adding modules to the apps/ repository without needing to
patch also the main OSv repository.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Example output looks as follows:

(gdb) osv zfs
:: ZFS TUNABLES ::
    zil_replay_disable=0
    zfs_nocacheflush=0
    zfs_prefetch_disable=0
:: ARC SIZES ::
    Actual ARC Size: 16742872
    Target size of ARC: 16777216
    Min Target size of ARC: 16777216
    Max Target size of ARC: 16777216
        Most Recently Used (MRU) size: 1222656 (7.29%)
        Most Frequently Used (MFU) size: 15554560 (92.71%)
:: ARC EFFICIENCY ::
Total ARC accesses: 47477
    ARC hits: 44668 (94.08%)
        ARC MRU hits: 18737 (41.95%)
            Ghost Hits: 183
        ARC MFU hits: 25733 (57.61%)
            Ghost Hits: 1084
    ARC misses: 2809 (5.92%)
Total Hash elements: 341
    Max Hash elements: 449
    Hash collisions: 98
    Hash chains: 13

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Useful for tracing virtio queue utilization issues.

To get plottable data:
 1. Save traces to gdb.txt
 2. grep virtio_add_buf gdb.txt | grep "queue=0" \
     | awk '{print $3 " " $6}' | sed -r 's/avail=(.*)/\1/'

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Clean up virtio-vring.hh and move add_buf_wait() out of line. Benefits:

  - Plays well with tracepoints

  - Less recompilation when code is changed

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Most likely introduced in eb48b150

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>