No functional changes, just renames to more adequate names.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

No need to create an empty bootfs.manifest anymore.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Currently importing module from a module definition would fail because
we cannot call import module with the same name (module.py)
recursively, __import__ will complain that we removed 'module' from
sys.modules.

There is a simple solution to this problem, we can use runpy.run_path()
which works like a charm.

In addition to this we cache loaded modules so that we don't
have to load the file twice.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

In his review of timerfd.cc, Avi asked that I simplify the implementation
by having a single "timerfd" object (instead of two I had - timerfd_file
and timerfd_object), and by using a single mutex instead of the complex
combination of mutexes and atomic variable.

This new version indeed does this. It should be easier to understand this
code, and it is 30 lines shorter.

The performance of this code is slightly inferior to the previous one -
in particular poll() now locks and unlocks a mutex - but this should be
negligible in practice.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch improves the command by adding useful info for debugging ZFS
in general, and also addresses some stylistic issues.

The new output is as follow:
(gdb) osv zfs
:: ZFS TUNABLES ::
	zil_replay_disable:       0
	zfs_nocacheflush:         0
	zfs_prefetch_disable:     0
	zfs_no_write_throttle:    0
	zfs_txg_timeout:          5
	zfs_write_limit_override: 0
	vdev_min_pending:         4
	vdev_max_pending:         10
:: ARC SIZES ::
	Actual ARC Size:        122905056
	Target size of ARC:     1341923840
	Min Target size of ARC: 167740480
	Max Target size of ARC: 1341923840
		Most Recently Used (MRU) size:   670961920 (50.00%)
		Most Frequently Used (MFU) size: 670961920 (50.00%)
:: ARC EFFICIENCY ::
Total ARC accesses: 42662
	ARC hits: 41615 (97.55%)
		ARC MRU hits: 12550 (30.16%)
			Ghost Hits: 0
		ARC MFU hits: 29045 (69.79%)
			Ghost Hits: 0
	ARC misses: 1047 (2.45%)
Prefetch workload ratio: 0.0097%
Prefetch total:          412
	Prefetch hits:   20
	Prefetch misses: 392
Total Hash elements: 1053
	Max Hash elements: 1053
	Hash collisions:   13
	Hash chains:       11

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

A previous patch renamed mutex.cc to spinlock.cc. This fixes the build.mk
dependency to make the code compile again... Sorry about that.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This document describe OSv coding style.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

In very early OSv history, the spinlock was used in the mutex's
implementation so it made sense to put it in mutex.cc and mutex.h.

But now that the spinlock is all that's left in mutex.cc (the real mutex
is in lfmutex.cc), rename this file spinlock.cc. Also, move the spinlock
definitions from <osv/mutex.h> to a new <osv/spinlock.h>, so if someone
wants to make the grave mistake of using a spinlock - they will at least
need to explicitly include this header file.

Currently, the only remaining user of the spinlock is the console.
Using a spinlock (and not a mutex) in the console allows printing debug
messages while preemption is disabled. Arguably, this use-case is no
longer important (we have tracepoints), so in the future we can consider
dropping the spinlock completely.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch implements a simplified version of pthread_getcpuclockid
that should be enough for our needs.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch reserves some thread ids, that are kept unused. This is so we can
construct values that reuse the thread public id and add it together with other
information and still fit in 32-bits.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This will be used later to determine for how long have a thread been running.
It can easily be updated right before we call ran_for(), reusing its interval
parameter.

Fixes #135

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Start using spaces instead of tabs and surround all single-line
control statements with curly braces.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Previously, scripts/test.py had no option to do that. It launched an OSv
instance for each test case.
Terribly slow PCs like mine took a bunch of time to run all test cases
through 'make check'.

Then let's take advantage of testrunner.so which will use a single OSv instance
to run all test cases, consequently boosting the speed considerably.
Let's also change testunner.so to conform our needs, e.g. blacklist.

To run this fast check, do: scripts/test.py --single;

Results show that this option is about 2.5x faster than the current one.

By now, let's not use this approach as the default version given that its
output has to be better formatted.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Add on the top of README.md a short introduction to what OSv is.
If someone gets to our github page, https://github.com/cloudius-systems/osv

,
and scrolls down, it's strange that we only explain how to build OSv,
without first mentioning what it is.

Fixes #148

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Start using spaces instead of tabs and surround all single-line
control statements with curly braces.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

It will be useful to take better and safer VFS decisions in the future.
For example, avoiding code that uses the absolute path to determine something.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

newmp->m_covered must be released if not NULL.
Found this problem while dumping dcache content.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Currently, vflush is used in the unmount process to release remaining
dentries. vflush in turn calls vevict that is releasing dentries that
it doesn't own.
This behavior is not correct neither good to the future of VFS.

So Avi suggested switching to a different approach. We could only
release those dentries owned by the mountpoint when unmounting it as
there wouldn't be anything else in the dcache (given its functionality).

The problem was fixed by doing the following steps:
 - Drop vflush calls in sys_umount2, make vevict an empty function,
and remove vevict.

 - Created the function release_mp_dentries to release dentries of a mount
point which will be called by VFS_UNMOUNT. It cannot be called before
VFS_UNMOUNT as failures must be considered, neither after as the mount point
would be considered busy.
Don't respect this "rule", and that previously seen ZFS replay transaction
error would happen.

NOTE: vflush is currently duplicated in zfs unmount cases to address the problem
above. This patch fixes this duplication as well.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

It's a good practice to propagate failures so that make does not
succeed if something goes wrong.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

To make it work self._closed needs to be initialized early because
even though exception is thrown from __init__(), __exit__() still will
be called and that one expects self._closed to be defined.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Found during code inspection.

Qemu process is started with stdout=PIPE but the pipe will not be read
until _process.communicate() is called and it is called when nbd_file
is closed. So if the pipe buffer fills up we may have a deadlock. Qemu
will be blocked on stdout.write() and imgedit.py will be waiting for
it to open a socket.

Solution here is to redirect to /dev/null as this has the same
effect as current code.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This is not a documented behavior but glibc's dlopen behaves this way.

Spotted when debugging assertion failure when java is started with a
debugger, we try to read elf header from "/" on dlopen("", ...):

Assertion failed: data.uio_resid == 0 (osv/fs/fs.cc: read: 27)

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Useful if you want to know who created that large pile of threads.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

gcc -O3 complains about uninitialized use of 'nims'. In fact,
inm_get_source() can return an error without setting nims which will
be later read from RB_FOREACH_REVERSE_FROM macro. It looks like 'nims'
is intended to hold the last value for which inm_get_source() returned
success. The uninitialized access would happen if this function never
succeded. I am not sure if this is possible in practice, but let's
initialize nims to NULL, which will cause no iteration in
RB_FOREACH_REVERSE_FROM macro.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Gcc comaplins about attempt to read 'size' via dereferencing a pointer
in xdr_u_int() in case xdrs->x_op == XDR_ENCODE.  However, in this
case the size will be set from the switch case inside xdr_string()
before xdr_u_int() is invoked. I think it's spurious because the
code clearly assumes that xdrs->x_op cannot change between these two
execution points.

Let's initialize 'size' to 0 to make gcc happy.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Currently map_file() do three passes over vma memory in a worst case.
First it maps memory with write permission while zeroing it, then it
reads a file into memory and, if vma is read only, it does one more
pass to fix memory permissions. Fix it by providing new specialization
of fill_page class which builds iovec of all allocated memory and
reads from a file using the iovec at the end of populate stage.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Commit 2ebfd915 ("build.mk: Generate a manifest of tests automatically")
separated native tests from Java tests but unfortunately broke the build
because it forgot to add dependency to the new java_tests variable.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

In issue #145 I reported a crash during boot in start_early_threads().
I wasn't actually able to replicate this bug on master, but it happens
quite frequently (e.g., on virtually every "make check" run) with some
patches of mine that seem unrelated to this bug.

The problem is that start_early_threads() (added in 63216e85)
iterates on the threads in the thread list, and uses
t->remote_thread_local_var() for each thread. This can only work if
the thread has its TLS initialized, but unfortunately in thread's
constructor we first added the new thread to the list, and only later
called setup_tcb() (which allocates and initializes the TLS). If we're
unlucky, start_early_threads() can find a thread on the list which still
doesn't have its TLS allocated, so remote_thread_local_var() will crash.

The simple fix is to switch the order of the construction: First
set up the new thread's TLS, and only then add it to the list of
threads.

Fixes #145.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Generate a manifest of tests automatically from the makefile
based on the value of $(tests) and $(java_tests) variables.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Currently we just call poweroff() which does not umount
file systems which leads to data loss on next boot.

This happens for example if you start the CLI and type exit. On next
boot you will get:

 TX_CREATE
 TX_CREATE
 Solaris: WARNING: ZFS replay transaction error 95, dataset osv/zfs,
 seq 0x1, txtype 2

Reported-by: Pekka Enberg <penberg@cloudius-systems.com>
Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

In order to reuse the logic it needs to be extracted.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

When compiling with -O3 gcc detects that 'th' may be uninitialized.
This is true, it happens when prio_find_thread is given
which == PRIO_USER or which == PRIO_PGRP.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

MultiJarLoader reads the command line parameters from a file
and uses split to make a list of params out of them before
calling parseArgs.

Currently the regular expression is \\s which means that
consecutive space will be split it to an empty string.
It should be \\s+ so consecutive spaces will be ignored.

Found by code inspection.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>

Instead of the old C-style file-operation function types and fo_*()
functions, since recently we have methods of the "file" class. All our
filesystem code is now C++, and can use these methods directly.

So this patch drops the old types and functions, and uses the class methods
instead.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

The code has bitrotted, and it doesn't support wait morphing.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>