This patch addresses a corner-case in our zfs_inactive which can potentially
leak a znode object.

*** Some background on znode/zfs_inactive ***
- Used to deallocate fs-specific data.

- Before destroying the znode, a DMU transaction is created to sync the znode
to the backing store *if* its z_atime_dirty is set (Not relevant to this
patch though).

- When removing a link, zfs_remove sets the field zp->z_unlinked of the
underlying znode if the number of links reached 0 (Simply put, not present in
the fs anymore).

*** The problem ***
The actual problem shows up when zfs_inactive is used on znodes with the
unlinked field set.

The code wrapped around by this patch was previously added to speed up the call
to vrecycle, whose name partially explains itself. Its first functionality is
to eliminate all activity associated to the vnode, then put the vnode back into
a list of free vnodes.

OSv VFS layer doesn't support vrecycle, but our zfs_inactive is acting as if it
were supported. Another thing is that vrecycle call was also removed.

*** Solution ***
Let's fix this problem by simply wrapping around the test which prevented
zfs_inactive from working properly on unlinked znodes, thus leaking references
to the underlying mount point afterwards.

The commentary added into zfs_inactive also explains why these changes are
needed. It would also make things easier when people look at it in the future,
and try to understand why things are the way they are.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The zfs_remove() function calls zfs_dirent_lock, which in turn calls
zfs_zget() which bumps up the underlying znode reference count once.

However, neither zfs_remove() or zfs_rmdir() release the reference count
after using it. This prevents zfs_zinactive() which is used to destroy
the znode object from working properly. Another consequence is that each
znode holds a reference to the underlying mount point, keeping it busy
for unmount.

Fix the znode refcnt by calling zfs_zinactive after znode usage.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

In the Http server implementation, some of the code was
based on an example from the boost library.

The boost license itself was not included in the license
directory.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

There were 2 places with ioctl definitions, Xen netfront driver
was compiled with IOCTL definitions from wrong place.

Fixed by changing include and deleting file with improper definitions

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Add mruby on apps, support make all image=mruby.

Reviewed-by: Tomasz Grabiec <tgrabiec@gmail.com>
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Functions which do not modify input parameters are easier to analyze
and more friendly for its users.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

To start the JVM with debugger server just run:

  sudo scripts/run.py -nv --jvm-debug

If you want the JVM to wait for debugger client:

  sudo scripts/run.py -nv --jvm-suspend

Listens on port 5005.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

A previous test changed loader.cc's command line parsing to also support
"&" as a separator of commands, causing the previous command to be executed
in a new thread. To achieve this, the parser ends each command with another
string, containing "&", ";" or "", depending on what appeared on the end
of this command.

This change caused tst-commands.cc, which checks the results of the
command line parsing, to fail. So this patch fixes the test to match
the code.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Our loader's command line (what is given to the "-e" option of run.py)
already allows running multiple commands (each a shared object with
arguments) separated with a semicolon - e.g.,

    run.py -e "program1.so; program2.so; program3.so"

This patch allows, just like in Unix, to use a "&" instead of a ";",
in which case the preceding program is run in the background, in our case
this means in a new thread.

For example,
    run.py -e "httpserver.so& java.so ..."

As before a command line can constitute multiple commands, and whitespaces
around the separators (; or &) are optional.

Take care if you intend to run the *same* object multiple times concurrently,
e.g., "something.so& something.so". For an object to support this use case,
it should support its main() being called in parallel, and in particular
avoid using global variables.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

At least in its current incarnation, it is risky to start the balloon without
the monitor. Tomek found a problem under which a problem with Tomcat's logging
system would cause an exception to be thrown.

While we were mistaken about the effects of that problem, namely, the exception
seems to caught by the logger itself and won't affect the functioning of the
monitor, it drew our attention to the fact that exceptions do can happen. And if
they do, we should be prepared to catch them and abort.

Reviewed-by: Tomasz Grabiec <tgrabiec@gmail.com>
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The second stack trace mentioned in issue #178 happens because of a
bug in tst-queue-mpsc (this is what happens when tests become too
complex, and have bugs of their own...):

The "popper" thread reads an "item" from a lockfree:queue_mpsc, and wakes
the "pusher" thread in that item. But we have a bug when the pusher thread
is done and returns: While the condvar remains valid, the "item" containing
it does not! We cannot continue to use the index item->value.waiter after
we woke this thread, because it can return and item points to invalid
memory... We need to save the index "item->value.waiter" before waking
the thread.

Unfortunately, this does *not* completely solve issue #178 - the timer
bug (similar to the two stack traces on issue #178) is still seen
(rarely) after this patch.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

It lives now in apps.git/netperf.

I'm sending this as RFC because there is a text file which
is not in apps.git, but I'm not sure if it is relevant enymore.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Issue #178, and recent netperf 3 experiments by Vlad, suggests we have a
bug in our timer code which very rarely causes crashes on
timer_list::fired(). It appears we somehow corrupt our per-cpu timer list,
or some armed timer object - but I still haven't been able to figure out
where.

This patch adds an assertion that the timer we find on the list is
actually armed. Unfortunately, it consistantly fails (I can see the same
assertion failure once every 100-200 runs of tst-queue-mpsc.so), and this
failure seems to replace the page-fault crash of issue #178.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Move definitions from <debug.h> to <osv/debug.h> and update includes to
use the latter.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch fixes chdir() on a normal file, which used to succeed (!?),
but now will fail as it should, with ENOTDIR.

The patch also adds an exhaustive test for chdir's success and error cases.
Before the latest chdir() patches, most of these tests would fail, and now
all of them succeed.

This test is standard C++ & Posix code, so it can be run also on Linux.
This is important for verifing that whatever we expect from OSv, Linux
really does the same.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch adds the O_DIRECTORY flag to sys_open(), which causes the open
to fail with ENOTDIR if the given file is any type of file but a directory.

We need this flag as part of a correct implementation of chdir() (which
should fail on a non-directory file), and it is also required for Linux
compatibility (the O_DIRECTORY flag exists since Linux 2.1.126).

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Add cwd (current directory) and env (environment variable) option to the
loader. Can be useful for certain applications that expect to be in a
certain directory, or certain environment variables to exist.

Example usage:

   run.py -e "--cwd=/tmp /usr/bin/something.so"

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Test case for dladdr(), to make sure it returns correct symbol in these cases:
 - addr is less than 'vfprintf'. Should returns another symbol.
 - addr is equals to 'vfprintf'. Should returns 'vfprintf' as the result.
 - addr is bigger than 'vfprintf', and also inside of it. Should returns 'vfprintf' as the result.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Instead of acquiring sockbuf::sb_mtx inside sblock() and sbunlock(), rely
on the caller to take the lock for us.  Expand existing lock hold regions
in callers to make it so.  This reduces acquisitions of sb_mtx.

As a side effect, copies to and from userspace are done under the lock.
This can affect MSG_NOWAIT with demand paging major faults, but these are
screwed anyway.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

sb_rwlock is used to serialize concurrent writers (or readers) to the same
socket buffer, but is quite expensive as it requires 4 atomic operations
per transaction, even if there is no contention.

Replace it with a waitqueue, and use the sockbuf::sb_mtx for serialization.
This still has exactly the same cost, but we can later move sblock() and
sbunlock() into contexts where the sockbuf::sb_mtx is already acquired.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>