Signed-off-by: Asias He <asias.hejun@gmail.com>
[ penberg: keep comments vertically aligned ]
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

bio is already signalled in all cases. No need to do it in destructor.

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

It's better not to indent after the namespace specifier.

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Change from blk::make_virtio_request to blk::make_request.

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

We are under the virtio namespace, it makes no sense to repeat the
virtio prefix again in the virito_blk driver.

Change the naming from virtio::virtio_blk to virtio::blk.

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The check is moved to bdev_read and bdev_write.

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The check is moved to bdev_read and bdev_write.

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The check is moved to bdev_read and bdev_write.

Sigaed-off-by: Asias He <asias@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Over the process of creating a zpool, history_str_get() will be eventually
called by zfs_ioc_pool_create(). history_str_get() will not work as expected as
it fully relies on the function copyinstr() which is currently unimplemented.

After that, spa_create() will be called with history_str storing trash, thus
making the spa_history_log filled with completely wrong entries.

Let's fix this problem simply by implementing copyinstr. By the way, copystr
has basically the same semantics as copyinstr, so let's implement it as well.
It's not being used anywhere, but who knows? It might be needed in the future
by some bsd compliant application.

As far as I know, this problem didn't manifest *visibly* anywhere as the spa
history log wasn't intentionally used by us yet. I found the problem while
auditing code.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This reverts commit 97626d36. As pointed
out Tomek, "-u" flag is already passed in mkzfs.py.

mprotect() should fails with ENOMEM if it is called on non mapped
virtual address, but this check is done by mmu::ismapped().

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Just came across this. Trivial to fix.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Since there's no user data on the image, we can safely cache it, improving
build performance.  Helpful duing 'git bisect'.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Commit 649654af ("Fix race between join() and thread completion") fixed
the problem so bring back the test case.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Currently we use implementation from
bsd/sys/cddl/compat/opensolaris/kern/opensolaris_atomic.c
which is using global mutex.

This showes up on wait profile of ZFS writes:

(...)
   \-- 469.96 us (0.00%, #250) arc_do_user_evicts
       \-- 469.96 us (0.00%, #250) dbuf_do_evict
           |-- 355.93 us (0.00%, #154) dbuf_destroy
(...)
           |   \-- 76.60 us (0.00%, #53) zrl_add
           |                             atomic_cas_32
           |                             lockfree_mutex_lock
           |                             sched::thread::wait()
           |
           |-- 66.14 us (0.00%, #56) dbuf_clear
           |                         zrl_add
           |                         atomic_cas_32
           |                         lockfree_mutex_lock
           |                         sched::thread::wait()
           |
           \-- 47.89 us (0.00%, #40) dbuf_rele_and_unlock
                                     atomic_add_64_nv
                                     lockfree_mutex_lock
                                     sched::thread::wait()

This patch switches to amd64 implementation.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

af_local.h declares a couple of functions implemented in af_local.cc.
There is no reason for pipe.cc to include it.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch adds the basic of memory tracking, and exposes an interface to for
that data to be collected.

We basically start with all stats at zero, and as we add memory to the System,
we bump it up and recalculate the watermarks (to avoid recomputing them all the
time). When a page range comes up, it will be added as free memory.

We operate based on what is currently sitting in the page ranges. This means
that we are effectively ignoring memory that sit in pools for memory usage. I
think it is a good assumption because it allow us to focus in the big picture,
and leave the pools to be used as liquid currency.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This adds initial virtio-scsi support. We have no scsi layer in osv, in
this implementation virtio-scsi works directly with the bio layer. It
translates BIO_READ, BIO_WRITE and BIO_FLUSH to SCSI CMD.

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

bio with BIO_SCSI flag contains a SCSI command.

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This can be used by the low-level driver to store private data, e.g.
virtio-scsi driver uses it to store a virtio-scsi request bound to this
bio.

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The lock used to protect _waiting_request_thread can go away.

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

With net channels, poll() needs to wait not only on poll wakeups and the
timeout, but also requests from network interfaces to flush net channels
for polled sockets.

In preparation for that, switch from bsd msleep() to native wait_until().

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

While booting with many cpus (e.g., run.py -c 20, but I sometimes saw this
with as few as 7), we crashed while trying to sleep with preemption disabled:

Assertion failed: preemptable() (/home/nyh/osv/include/sched.hh: do_wait_until: 605)
Aborted

Turns out that since commit 223b2252 (half a year ago), main_cont()
ran smp_launch() with preemption disabled. But smp_launch creates threads,
and thread's constructor may sleep (e.g., on the thread_list_mutex), and
we cannot do this with preemption disabled...

So dropped the preempt_disable()/enable() from main_cont(). The reasoning for
it as expressed in 223b2252 appears to be no longer relevant.

Fixes #130.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Printing asynchronously is not affected by write() stalls and hence the period
for partial amount can be more or less constant.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Reviewed-by: Dor Laor <dor@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

We can skip to construct a vring::sg_node.

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

bsd defines some m_ macros, for example m_flags, to save some typing.  However
if you have a variable of the same name in another header, for example
m_flags, have fun trying to compile your code.

Expand the code in place and eliminate the macros.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This code seems obviously broken to me:

  tls_data().size

there is no tls_data function, it is a struct. So this is creating
temporary uninitialized struct and reads size field from it.

What it meant instead is probably the TLS size, which is calculated by
tls() function and returned in a tls_data structure.

I am not able to actually test this change because I don't have any
DSO which has R_X86_64_TPOFF64 relocations. Any idea how to test it?

tls() is also broken, because it initializes file_size field instead
of the size field. The file_size field was added at some point but
this place wasn't updated. As it appears that tls() is not actually
used anywhere, this patch gets rid of it.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Dynamically loaded modules use __tls_get_addr() to locate thread local
symbols. Symbol is identified by module index and offset in module's
TLS area. Module index and offset are filled in by dynamic linker when
DSO is loaded.

TLS area for given DSO is allocated dynamically, on-demand. OSv keeps
TLS areas in a vector indexed by module index, inside per-thread
vector.

TLS area of core module should be handled differently than that of
dynamically loaded modules. The TLS offsets for thread local symbols
defined in core module are known at link time and code inside core
module can use these offsets directly. The offsets are relative to TCB
pointer (fs register on x86).

The problem was that __tls_get_addr() was treating core module as a
dynamically loaded module and returned pointer inside dynamically
allocated TLS area instead of a pointer inside core module's TLS. As a
result code inside core module was reading value from different
location than code inside DSO has written value to.

The offending thread local varaible was __once_call. It was set by
call_once() defined in DSO (inlined from a definition inside header)
and read by __once_proxy() defined in core module.

Fixes #125.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

In order to have unform naming, ulong is used in several places.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>