First step towards moving x64-specific stuff into arch
subdirectories:

move the submodules in external/ to external/x64/,

rename amd64->x64 in opensolaris,

rename x86_64->x64 in include/api,

adapt build.mk to reference the updated paths,
and improve detection of include dirs and libs in the
external submodules.

Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Hopefully this will reduce porting efforts.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Automatically:
 s/\t/    /g
 s/ +$//

Manually: align constant definitions in block.h

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Commit 622d8bba re-enabled code to register vm_lowmem event,
so the respective deregister should also be re-enabled.

'For every OSv specific ifdef we remove in ZFS, God ressurects a kitten.'
by Glommer.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

I have previously seen situations in which we had a lot more requests than we
can handle, leading to an OOM. I've initially thought that this was because
since there are only max_requests (usually 32) requests, new requests would
overwrite the old bio queue, leading to some of the bio_done functions never
being called.

A closer look at the code was enough to show me wrong, since at the end of the
ring consumption, we do iterate over old requests, processing them.

The real problem then, is that we can generate memory a lot faster than we can
process disk IO. To give an idea about order of magnitude, when a spike would
happen in Xen block (the very problem I am tracking), causing processing to be
interrupted by ~1s, 18k requests would accumulate in the queue. Since after
draining the ring we can only requeue 32 new requests, the end result is that
the 18k requests are never really processed. And at the new spike, 18k more
sit, and so on.

The solution is to adopt a strategy similar to virtio. We'll add a condition
variable to the queue and block new requests while we are processing new ones.

With this patch applied, misc-bdev-write.so runs to completion, but
unfortunately still quite slow.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
[ penberg: Fix mixed formatting ]
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

They are going to work exactly the same. But with std::list instead

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch make some of the blkfront structures a bit more C++ friendly. I am
creating a C++ version of the device private structure, and the goal here is to
convert it slowly until all or most fields are inside it. At that point, I
expect it to be easy to turn the functions into methods, etc. I am sorry, I
would really like to do a mass translation here, but there are urgent bugs
requiring my attention.

Some notes:
 * I am converting blkfront only, and leaving netfront with the old code. It
   shouldn't be hard, I just deferred the risk of mistakes to a future
   oportunity.
 * you will notice that lines that are added by me are not following the
   original file's identation. This is deliberate. As much as we have in our
   guidelines to preserve the original ident, since we are slowly converting
   those files, our identation should eventually win. Think of it as a form
   of cultural domination. (and shouldn't bother viewers if tabs=4 spaces)

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Xen boot is not totally silent. The devices usually print through
device_printf, which right now is hooked at printf. Changing that to debugf
already gets rid of most messages. However, netfront by itself manually prints
some messages with printf. They are changed as well.

Please note that the messages in netfront are changed directly to debug,
instead to device_printfs as well. This is because although we just hook that
to a printer, device_printf is supposed to prepend the messages with device
information.  We don't do it now, but could do it in the future. That is why
the original code used printf directly, and I am keeping that intent.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Remove unused #include of <drivers/clock.hh>.
Except the clock drivers and <osv/clock.hh>, no source file now now
include this header. Rather, <osv/clock.hh> should be used. Code including
<sched.hh> will also get <osv/clock.hh> automatically.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Replace the old function condvar::wait(mutex*, uint64_t) with one taking
a timepoint. This timepoint can use any clock which the timer supports,
namely osv::clock::uptime or osv::clock::wall (as usual, wall-clock timers
are not recommended, and are converted to an uptime timer at the point
of instantiation).

Leave a C-only function condvar_wait(convar*, mutex*, s64) but comment on
what it takes.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Fix sbwait implementation to use the new <osv/clock.hh> APIs and the
monotonic clock.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Reimplement the BSD functions getmicrotime(9), getmicrouptime(9)
and variable "ticks", using the new clock APIs.

getmicrotime() returns the system time ("wall clock"), while getmicrouptime
and ticks return the time since boot.

I believe this is the correct implementation according to the FreeBSD
documentation, but our previous implementation didn't quite do this and
it also worked ;-) The previous implementation pretended, according to
getmicrouptime() and get_ticks(), that the system is up since 1970,
and yet the variable "time_uptime" (which FreeBSD has) is never updated,
and is fixed at 1 second :-)

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Fix msleep implementation to use the new <osv/clock.hh> APIs and the
monotonic clock.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Change callout implementation to use the new <osv/clock.hh> APIs and the
monotonic clock.

Since _callout.h now uses the C++ type osv::clock::uptime::time_point,
it can only be used from C++ code. All the relevant code is already C++.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

BUILD SUCCESSFUL

Total time: 35.396 secs
make -r -C build/release/ all
make[1]: Entering directory `/home/zwu/osv/build/release'
  CXX loader.o
  CXX runtime.o
  CXX drivers/vga.o
  CXX bsd/net.o
  CXX bsd/porting/networking.o
/home/zwu/osv/bsd/porting/networking.cc: In function ‘int osv::if_set_mtu(std::string, u16)’:
/home/zwu/osv/bsd/porting/networking.cc:43:32: error: missing braces around initializer for ‘char [16]’ [-Werror=missing-braces]
cc1plus: all warnings being treated as errors
make[1]: *** [bsd/porting/networking.o] Error 1
make[1]: Leaving directory `/home/zwu/osv/build/release'
make: *** [all] Error 2

Signed-off-by: Zhi Yong Wu <zwu.kernel@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch addresses a corner-case in our zfs_inactive which can potentially
leak a znode object.

*** Some background on znode/zfs_inactive ***
- Used to deallocate fs-specific data.

- Before destroying the znode, a DMU transaction is created to sync the znode
to the backing store *if* its z_atime_dirty is set (Not relevant to this
patch though).

- When removing a link, zfs_remove sets the field zp->z_unlinked of the
underlying znode if the number of links reached 0 (Simply put, not present in
the fs anymore).

*** The problem ***
The actual problem shows up when zfs_inactive is used on znodes with the
unlinked field set.

The code wrapped around by this patch was previously added to speed up the call
to vrecycle, whose name partially explains itself. Its first functionality is
to eliminate all activity associated to the vnode, then put the vnode back into
a list of free vnodes.

OSv VFS layer doesn't support vrecycle, but our zfs_inactive is acting as if it
were supported. Another thing is that vrecycle call was also removed.

*** Solution ***
Let's fix this problem by simply wrapping around the test which prevented
zfs_inactive from working properly on unlinked znodes, thus leaking references
to the underlying mount point afterwards.

The commentary added into zfs_inactive also explains why these changes are
needed. It would also make things easier when people look at it in the future,
and try to understand why things are the way they are.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The zfs_remove() function calls zfs_dirent_lock, which in turn calls
zfs_zget() which bumps up the underlying znode reference count once.

However, neither zfs_remove() or zfs_rmdir() release the reference count
after using it. This prevents zfs_zinactive() which is used to destroy
the znode object from working properly. Another consequence is that each
znode holds a reference to the underlying mount point, keeping it busy
for unmount.

Fix the znode refcnt by calling zfs_zinactive after znode usage.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

There were 2 places with ioctl definitions, Xen netfront driver
was compiled with IOCTL definitions from wrong place.

Fixed by changing include and deleting file with improper definitions

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Instead of acquiring sockbuf::sb_mtx inside sblock() and sbunlock(), rely
on the caller to take the lock for us.  Expand existing lock hold regions
in callers to make it so.  This reduces acquisitions of sb_mtx.

As a side effect, copies to and from userspace are done under the lock.
This can affect MSG_NOWAIT with demand paging major faults, but these are
screwed anyway.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

sb_rwlock is used to serialize concurrent writers (or readers) to the same
socket buffer, but is quite expensive as it requires 4 atomic operations
per transaction, even if there is no contention.

Replace it with a waitqueue, and use the sockbuf::sb_mtx for serialization.
This still has exactly the same cost, but we can later move sblock() and
sbunlock() into contexts where the sockbuf::sb_mtx is already acquired.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

sblock() takes sb_rwlock for reading, and is the only such locker, so it
clearly has no effect.

Change it to acquire the lock for writing, so it serializes access to the
socket buffer as intended.

Bug introduced in 6296cbab.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This allows placing C++ objects in ifnet.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Include the system header and remove duplicate definitions.

Change some solaris imports to use _GNU_SOURCE to make rlim64_t defined
consistently.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Allows making ifnet a C++ class.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Add sockbuf::sb_cc_wq for using instead of msleep(&sb->sb_cc).  This paves
the way for lockless wakeups for net channels, as the wait primitive is now
thread::wait_for() instead of msleep().  In addition this reduces lock
acquisition and improves netperf bandwidth by about 10%.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

This was used to ensure all socket-using code was converted to C++, but not
cleaned up later.

Clean it up now.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

This patch introduces support for MTU option as described in
RFC2132, chapter 5.1. Interface MTU Option

Amazon EC2 networking uses this option in some cases and it gives
throughput improvement of about 250% on big instances with 10G networking.

Netperf results for hi1.4xlarge instances, TCP_MAERTS test, OSv runs netserver:

Send buffer size     Throughput w/ patch (Mbps)     Throughput w/o patch (Mbps)     Improvement (%)

32                   4912.29                        1386.28                         254
64                   4832.01                        1385.99                         249
128                  4835.09                        1401.46                         245
256                  4746.41                        1382.28                         243
512                  4849.04                        1375.23                         253
1024                 4631.8                         1356.69                         241
2048                 4859.59                        1371.92                         254
4096                 4864.99                        1383.67                         252
8192                 4627.07                        1364.05                         239
16384                4868.73                        1366.48                         256
32768                4822.69                        1366.63                         253
65536                4837.67                        1353.87                         257

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Total time: 30.77 secs
make -r -C build/release/ all
make[1]: Entering directory `/home/zwu/osv/build/release'
  CXX bsd/sys/xen/gnttab.o
In file included from /home/zwu/osv/bsd/sys/xen/hypervisor.h:40:0,
                 from /home/zwu/osv/bsd/sys/xen/gnttab.cc:29:
/home/zwu/osv/bsd/machine/xen/hypercall.h: In function ‘int HYPERVISOR_set_trap_table(const trap_info_t*)’:
/home/zwu/osv/bsd/machine/xen/hypercall.h:146:9: error: unable to find string literal operator ‘operator"" STR’
/home/zwu/osv/bsd/machine/xen/hypercall.h: In function ‘int HYPERVISOR_mmu_update(mmu_update_t*, unsigned int, unsigned int*, domid_t)’:
/home/zwu/osv/bsd/machine/xen/hypercall.h:154:9: error: unable to find string literal operator ‘operator"" STR’
/home/zwu/osv/bsd/machine/xen/hypercall.h: In function ‘int HYPERVISOR_mmuext_op(mmuext_op*, unsigned int, unsigned int*, domid_t)’:
......
make[1]: *** [bsd/sys/xen/gnttab.o] Error 1
make[1]: Leaving directory `/home/zwu/osv/build/release'
make: *** [all] Error 2

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Zhi Yong Wu <zwu.kernel@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Add 'struct file' to VOP_READ API. This is needed for procfs which
generates file contents at open() time and read() must operate on it,
not the vnode.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>