Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The current tracepoint coverage does not handle all situations
well. In particular:

 * it does not cover link layer devices other than virtio-net. This
   change fixes that by tracing in more abstract layers.

 * it records incoming packets at enqueue time, whereas sometimes it's
   better to trace at handling time. This can be very useful when
   correlating TCP state changes with incoming packets. New tracepoint
   was introduced for that: net_packet_handling.

 * it does not record protocol of the buffer. For non-ethernet
   protocols we should set appropriate protocol type when
   reconstructing ethernet frame when dumping to PCAP.

We now have the following tracepoints:

 * net_packet_in - for incoming packets, enqueued or handled directly.

 * net_packet_out - for outgoing packets hitting link layer (not
   loopback).

 * net_packet_handling - for packets which have been queued and are
   now being handled.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This tracepoint is invoked whenever TCP state is changed.

Example output:

  tcp_state     tp=0xffffc0003dbb6c00, 6 -> 9

There is no pretty-printer yet for state numbers, they can be decoded
using defines from tcp_fsm.h.

It's useful when tracing reaction of the stack on particular network
packets, when used together with net_packet* tracepoints and 'trace
list --tcpdump'. We can also get information from which code location
given state was changed, by enabling backtrace collection and using
-b[-L][-F] flags to 'trace list'.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This will allow to insert tracepoint invocation in the setter which
will cover all occurrences of TCP control block state change.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

(u16)_avail_added_since_kick wrap-around should be avoided since we may loose
an _avail_event update and thus miss the point where the HV should have
been woken (kicked).

We choose half a u16 range as a threshold since kick() is usually called for a
bulk of buffers and not for every separate buffer. Number of buffers in such a bulk is
limited by a size of a virtio ring. The virtio ring size is unlikely to have 32K
entries in the nearest future (it has 256 entries now) thus kicking() at least
every 32K buffers will ensure that we prevent the mentioned above wrap-around
as a result of such a bulking.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch enables to run OSv on VMware using run.py.
Both serial console and VGA are supported.

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Exploits issue #288.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Currently tracepoint's signature string is encoded into u64 which
gives 8 character limit to the signature. When signature does not fit
into that limit, only the first 8 characters are preserved.

This patch fixes the problem by storing the signature as a C string of
arbitrary length.

Fixes #288.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Attempt to get read ARC buffer for a hole in a file results in temporary
ARC buffer which is destroyed immediately after use. It means that
mapping such buffer is impossible, it is unmapped before page fault
handler return to application. The patch solves this by detecting that
hole in a file is accessed and mapping special zero page instead. It is
mapped as COW, so on write attempt new page is allocated.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

It will be used by random_kthread from Yarrow as a hardware source of entropy
whenever the thread wakes up, which happens around 10 times a second.

Systems that don't provide a hardware for rng will lack the property, then
should gather environmental noise from specific interrupts.
Systems that do provide will still gather environmental noise from interrupts
if available, and do 'one round' of gathering from the rng hardware.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch changes the device behind /dev/random to start using FreeBSD Yarrow
CPRNG algorithm. Similar to FreeBSD, our /dev/urandom was created as an alias
to /dev/random.
The device operations are linked to Yarrow through the random adaptor API.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Simply put, FreeBSD-specific code such as SYSCTL API, etc, was disabled.
Besides, I had gone throughout the code, and made sure that the overall
semantics remained the same on OSv.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

rijndael encryption algorithm is a dependency for porting the
random infrastructure from FreeBSD, so let's add it into our
BSD tree.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Glauber says:
I would advise commiting the original files from BSD in their unmodified state
(don't include them in the build), and in a different patch put on the changes
that OSv needs.
It has proven useful more than once for me in conjunction with git blame to
determine whether a snippet was already in the BSD code or was ours.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

They are needed by trace.py, which is now run as part of 'make check'
harness.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Current OSv implementation suppress most of output when it's not verbose mode.

It may speed up bootup speed, but supress printing out IP address is
inconvenient for most of users.  So I added infof(), which acts like debugf()
but print out string even not on verbose mode, and use it in dhcp.cc to print
out IP address.

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Porting CRuby now, and this is call execve and etc.  I won't support exec from
Ruby, so this function just need to fail.

Signed-off-by: KANATSU Minoru <icc.pot.tyew272@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This test passes on Linux but failed on OSv before the patch which
changed solisten_proto() to always use SOMAXCONN.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

The TCP stack in current form is causing a lot of connection resets
when server running inside OSv is swarmed with connection attempts.
This does not happen on Linux for the same kind of workload.

The scenario is as follows. Client sends SYN to the server which runs
inside OSv. A SYN cache entry is created and SYN+ACK is sent back. The
client sends ACK in response and connect() call exits on its
side. Form now on the client may already start to write() into the
socket. When server receives client's ACK, it looks up the entry in
SYN cache, finds it, removes it, and tries to create a full-blown
connection socket. When it attempts to append it to the listening
socket's backlog it fails to do so because the backlog is full. As a
result the ACK is dropped and there is no longer SYN cache entry for
that connection. The client doesn't know about that. Then the next
client's packet arrives with data. The server cannot find a SYN cache
entry for that and tries to do SYN-cookie check, which fails because
client sequence does not match. As a result the server sends RST to
the client.

We could get more Linux-alike behavior in such situation by applying
the following modifications to our stack:

  * Drop SYN requests when the backlog is full _and_ SYN cache has
    more than one entry. Client will try to resend SYN packet.

  * When backlog is full upon receiving ACK, keep the entry in SYN
    cache. The client will eventually retransmit unacknowledged data
    and the connection state promotion will be retried at later time.

As an interim workaround let's set the socket backlog to SOMAXCONN
(128) regardless of the desired backlog size. Apparently some
benchmarking tools (iperf) set it to a really low value (5). Since OSv
is designated to run only one process, which will typically listen on
only a handful of ports, this should not be a big threat to memory
footprint.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch implements the sigsetjmp()/siglongjmp() functions. Fixes #241.

sigsetjmp() and siglongjmp() are similar to setjmp() and longjmp(), except
that they also save and restore the signals mask. Signals are hardly useful
in OSv, so we don't necessarily need this signal mask feature, but we still
want to implement these functions, if only so that applications which use
them by default could run (see issue #241).

Most of the code in this patch is from Musl 1.0.0, with a few small
modifications - namely, call our sigprocmask() function instead a Linux
syscall. Note I copied the x64 version of sigsetjmp.s only. Musl also
has this file for ARM and other architectures. Interestingly we already
had in our source tree, but didn't use, block.c, and this patch starts
to use it.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

MongoDB will call sigwait and expect the process to actually stop, so we can't
stub them. Also, some of its features may depend on the fact that the program
will resume after a wait if a signal actually reaches it, (for cleanup and
whatever).

So let's come up with an actual implementation.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Functions to be run when a thread finishes.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

While working with blocked signals and notifications, it would be good to test
what's the current state of other thread's pending signal mask.

That machinery exists in sched.cc but isn't exposed. This patch exposes that,
together with a more convenient helper for when we are interested in the
pointer itself, without dereferencing it.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The test verifies basic tracing functionallity:
 * that samples are collected by OSv
 * extraction via `trace extract` succeeds
 * `trace summary` shows expected tracepoint names
 * listing via `trace list` shows tracepoints with arguments
 * network packet parsing works

This test is run as part of standard 'make check'.

Note: this single test can be executed as easily as:

 $ scripts/test.py --test tracing_smoke_test

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Since we adveritse use of a system package in README, we should also
recommend the same here.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

It's needed by TCP-related tracing commands, as of now:

 trace.py list --tcpdump
 trace.py tcpdump

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Also, move platform dependent fast dispatch to platform arch code tree(s)

The patching code is a bit more complex than would seem immediately (or
even factually) neccesary. However, depending on cpu, there might be
issues with trying to code patch across cache lines (unaligned).
To be safe, we do it with the old 16-bit jmp + write + finish dance.

[avi: fix up build.mk]

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

tcpdump port number conversions vary from system to system. Turn of the
conversions to have consistent output which is also simpler to assert
for in tests.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>