Current OSv implementation suppress most of output when it's not verbose mode.

It may speed up bootup speed, but supress printing out IP address is
inconvenient for most of users.  So I added infof(), which acts like debugf()
but print out string even not on verbose mode, and use it in dhcp.cc to print
out IP address.

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Porting CRuby now, and this is call execve and etc.  I won't support exec from
Ruby, so this function just need to fail.

Signed-off-by: KANATSU Minoru <icc.pot.tyew272@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This test passes on Linux but failed on OSv before the patch which
changed solisten_proto() to always use SOMAXCONN.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

The TCP stack in current form is causing a lot of connection resets
when server running inside OSv is swarmed with connection attempts.
This does not happen on Linux for the same kind of workload.

The scenario is as follows. Client sends SYN to the server which runs
inside OSv. A SYN cache entry is created and SYN+ACK is sent back. The
client sends ACK in response and connect() call exits on its
side. Form now on the client may already start to write() into the
socket. When server receives client's ACK, it looks up the entry in
SYN cache, finds it, removes it, and tries to create a full-blown
connection socket. When it attempts to append it to the listening
socket's backlog it fails to do so because the backlog is full. As a
result the ACK is dropped and there is no longer SYN cache entry for
that connection. The client doesn't know about that. Then the next
client's packet arrives with data. The server cannot find a SYN cache
entry for that and tries to do SYN-cookie check, which fails because
client sequence does not match. As a result the server sends RST to
the client.

We could get more Linux-alike behavior in such situation by applying
the following modifications to our stack:

  * Drop SYN requests when the backlog is full _and_ SYN cache has
    more than one entry. Client will try to resend SYN packet.

  * When backlog is full upon receiving ACK, keep the entry in SYN
    cache. The client will eventually retransmit unacknowledged data
    and the connection state promotion will be retried at later time.

As an interim workaround let's set the socket backlog to SOMAXCONN
(128) regardless of the desired backlog size. Apparently some
benchmarking tools (iperf) set it to a really low value (5). Since OSv
is designated to run only one process, which will typically listen on
only a handful of ports, this should not be a big threat to memory
footprint.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch implements the sigsetjmp()/siglongjmp() functions. Fixes #241.

sigsetjmp() and siglongjmp() are similar to setjmp() and longjmp(), except
that they also save and restore the signals mask. Signals are hardly useful
in OSv, so we don't necessarily need this signal mask feature, but we still
want to implement these functions, if only so that applications which use
them by default could run (see issue #241).

Most of the code in this patch is from Musl 1.0.0, with a few small
modifications - namely, call our sigprocmask() function instead a Linux
syscall. Note I copied the x64 version of sigsetjmp.s only. Musl also
has this file for ARM and other architectures. Interestingly we already
had in our source tree, but didn't use, block.c, and this patch starts
to use it.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

MongoDB will call sigwait and expect the process to actually stop, so we can't
stub them. Also, some of its features may depend on the fact that the program
will resume after a wait if a signal actually reaches it, (for cleanup and
whatever).

So let's come up with an actual implementation.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Functions to be run when a thread finishes.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

While working with blocked signals and notifications, it would be good to test
what's the current state of other thread's pending signal mask.

That machinery exists in sched.cc but isn't exposed. This patch exposes that,
together with a more convenient helper for when we are interested in the
pointer itself, without dereferencing it.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The test verifies basic tracing functionallity:
 * that samples are collected by OSv
 * extraction via `trace extract` succeeds
 * `trace summary` shows expected tracepoint names
 * listing via `trace list` shows tracepoints with arguments
 * network packet parsing works

This test is run as part of standard 'make check'.

Note: this single test can be executed as easily as:

 $ scripts/test.py --test tracing_smoke_test

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Since we adveritse use of a system package in README, we should also
recommend the same here.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

It's needed by TCP-related tracing commands, as of now:

 trace.py list --tcpdump
 trace.py tcpdump

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Also, move platform dependent fast dispatch to platform arch code tree(s)

The patching code is a bit more complex than would seem immediately (or
even factually) neccesary. However, depending on cpu, there might be
issues with trying to code patch across cache lines (unaligned).
To be safe, we do it with the old 16-bit jmp + write + finish dance.

[avi: fix up build.mk]

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

tcpdump port number conversions vary from system to system. Turn of the
conversions to have consistent output which is also simpler to assert
for in tests.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

On python 2.7.5 python complains about struct() not accepting unicode
argument. Fix by adding explicit conversion.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

phys_ptr<T>: unique_ptr<> for physical memory
make_phys_ptr(): allocate and initialize a phys_ptr<>
make_phys_array(): allocate a phys_ptr<> referencing an array

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

If dentry has elevated reference count during unlink (a file is opened
for instance) it is not destroyed and remains in the dentry hash.
As a result unlinked file is still accessible.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

If dentry has elevated reference count during rename (a file is opened
for instance) it is not destroyed and remains in the dentry hash. As a
result old file is accessible by destination path instead of new one.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

If dentry has elevated reference count during rename (a file is opened
for instance) it is not destroyed and still hashed with old path. As a
result renamed file can still be accessed by its old name.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Move static variables for alarm and related function to itimer class.
itimer class offers interfaces to implement setitimer/getitimer, so added setitimer/getitimer.
Now alarm rewrite as a just a wrapper function to call setitimer.

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

When a small allocation is requested with large alignment, we ignored
the alignment, and as a consequence posix_memalign() or
alloc_phys_contiguous_aligned() could crash when it failed to achieve
the desired alignment. This is not a common case (usually, size >= alignment,
and the new C11 aligned_alloc() even supports only this case), but still
it might happen, and we saw it in cloudius-systems/capstan#75.

When size < alignment, this patch changes the size so we can achieve the
desired alignment. For small alignments, this means setting size=alignment,
so for example to get an alignment of 1024 bytes we need at least 1024-byte
allocation. This is a waste of memory, but as these allocations are rare,
we expect this to be acceptable. For large alignments, e.g., alignment=8192,
we don't need size=alignment but we do need size to be large enough so we'll
use malloc_large() (malloc_large() already supports arbitrarily large
alignments).

This patch also adds test cases to tst-align.so to test alignments larger
than the desired size.

Fixes #271 and cloudius-systems/capstan#75.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

AHCI spec says cmd_table need to be aligned to 128 bytes, respect it.

AHCI spec does not specify the alignment requirement of _cmd_list and
_recv_fis. To be safe, we align to their size, i.e. 1024 and 256 bytes.

This fixes the issue reported by Roman Shaposhnik.

   $ capstan run -p vbox
   Created instance: i1398297574
   OSv v0.07
   Assertion failed: !(reinterpret_cast<uintptr_t>(ret) & (align - 1))
   (/home/penberg/osv/core/mempool.cc: alloc_phys_contiguous_aligned:
   1378)

   [backtrace]
   0x398316 <memory::alloc_phys_contiguous_aligned(unsigned long,
   unsigned long)+214>
   0x3542bb <ahci::port::setup()+283>
   0x3549ae <ahci::port::port(unsigned int, ahci::hba*)+142>
   0x354a5f <ahci::hba::scan()+111>
   0x354d34 <ahci::hba::hba(pci::device&)+148>
   0x354e07 <ahci::hba::probe(hw::hw_device*)+119>
   0x33e65c
   <hw::driver_manager::register_driver(std::function<hw::hw_driver*
   (hw::hw_device*)>)+188>
   0x33c8c3 <hw::device_manager::for_each_device(std::function<void
   (hw::hw_device*)>)+51>
   0x33e3de <hw::driver_manager::load_all()+78>
   0x20edf5 <do_main_thread(void*)+629>
   0x3ff0a5 <sync+69>
   0x3a3eaa <thread_main_c+26>
   0x361585 <thread_main+7>

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

There are 2 types of HVM AMIs on Amazon:

1. Windows based that support all instances types
2. Linux based that support instances larger than medium only

Linux based AMIs suit OSv better because their default seciurity
policy exactly fits OSv model (SSH access) and doesn't require
reconfiguration.

Windows based AMIs require default security policy reconfiguration
in order to allow SSH access but from other hand allow running OSv
on free-tier eligible micro instances which is good for enthusiasts
and initial evaluations.

In order to support both options this patch extends release scripts
with ability to choose between Windows and Linux based templates
during AMIs creation.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This reverts commit 532abb89 which makes
Mr. Jenkins unhappy.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The test verifies that:
 * samples are collected by OSv
 * extraction via `trace extract` succeeds
 * `trace summary` shows expected tracepoint names
 * listing via `trace list` shows expected tracepoints
 * network packet parsing works

This test is run as part of standard 'make check'.

Note: this single test can be executed as easily as:

 $ scripts/test.py --test tracing_smoke_test

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The command runs GDB which is very chatty. This will spoil our clean
output of 'make check' once the tracing test is added. Let's hide GDB
output unless it fails or the tracefile is not created.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The --tcpdump switch enables in-line decoding of net_packet* samples
so that packet content is displayed in human-readable form next to
other samples. This allows to corelate packets with other samples
easily.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The 'scripts/trace.py pcap-dump' command will process all packet
capturing samples and output a pcap stream. This is the format used by
tcpdump. It can be used together with tcpdump to display packets in
human-readable form like this:

  scripts/trace.py pcap-dump | tcpdump -r - | less

Because pcap stream is often used together with tcpdump, I also
introduce a shorthand for the above:

  scripts/trace.py tcpdump

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

There was no way to sniff packets going through OSv's loopback
interface. I faced a need to debug in-guest TCP traffic. Packets are
logged using tracing infrastructure. Packet data is serialized as
sample data up to a limit, which is currently hardcoded to 128 bytes.

To enable capturing of packets just enable tracepoints named:
  - net_packet_loopback
  - net_packet_eth

Raw data can be seen in `trace list` output. Better presentation
methods will be added in the following patches.

This may also become useful when debugging network problems in the
cloud, as we have no ability to run tcpdump on the host there.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Tracepoint argument which extends 'blob_tag' will be interpreted as a
range of byte-sized values. Storage required to serialize such object
is proportional to its size.

I need it to implement storage-fiendly packet capturing using tracing layer.

It could be also used to capture variable length strings. Current
limit (50 chars) is too short for some paths passed to vfs calls. With
variable-length encoding, we could set a more generous limit.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>