As I have stated previously, what is true for qemu (that we always have
a user-provided network interface) is not true for Xen. It is quite possible
that we boot with no network interface at all. In that case, we will get stuck
asking for an IP that will never come.

This patch takes care to call for dhcp only if our interface is really up. Since
networking is such a core service, we'll print a message if we can't do that.

Some time ago I have moved the console initialization a bit earlier, so
messages could be seen earlier. This has been, however, creating spurious
problems (1 at each 10 - 15 boots) on Xen HVM. The reason is that the isa
serial reset code enables interrupts upon reset, and the isa irq interrupt
will call wake() in the pool thread, which at this point is not yet started.

Since these days we already have simple_write() dealing with the early stuff,
move it back to where it used to be.

P.S: Dima found a way to make this problem 100 % reproduceable, by queueing
data in the input line before the console starts. With this patch, the problem
is gone even if Dima's method is used.

msleep() measure times in units of 1/hz seconds. We had hz = 1,000,000,
which gives excellent resolution (microsecond) but a terible range
(limits msleep()'s timeout to 35 minutes).

We had a program (Cassandra) doing poll() with a timeout of 2 hours,
which caused msleep to think we gave a negative timeout.

This patch reduces hz to 1,000, i.e., have msleep() operate in the same units
as poll(). Looking at the code, I don't believe this change will have any
ill-effects - we don't need higher resolution (freebsd code is used to
hz=1,000, which is the default there), and the code converts time units to
hz's correctly, always using the hz macro. The allowed range for timeouts will
grow to over 24 days - and match poll()'s allowed range.

Needed by JMX.

Fixes the following problem when connecting to the JVM via JMX/RMI:

ERROR 08:22:55,278 Exception in thread Thread[RMI TCP Connection(idle),5,RMI Runtime]
java.lang.UnsatisfiedLinkError: no rmi in java.library.path
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1878)
	at java.lang.Runtime.loadLibrary0(Runtime.java:849)
	at java.lang.System.loadLibrary(System.java:1087)
	at sun.security.action.LoadLibraryAction.run(LoadLibraryAction.java:67)
	at sun.security.action.LoadLibraryAction.run(LoadLibraryAction.java:47)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.server.MarshalInputStream.<clinit>(MarshalInputStream.java:122)
	at sun.rmi.transport.StreamRemoteCall.getInputStream(StreamRemoteCall.java:133)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:142)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:724)

Fixes the following problem when JMX is enabled in the JVM:

Error: Config file not found: /usr/lib/jvm/jre/lib/management/management.properties
program exited with status 1
Aborted

I'm not sure about the target location of libnpt.so, but when it was under JVM
libraries, the debug agent didn't find it.

You should be able to start the debug agent as recommended with these JVM
options:
-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005

Remote debugging through your favorite IDE should be enabled.

Add a "qcow2" target to Makefile that uses "qemu-img convert" to build a
smaller qcow2 image out of the OSv raw image.

the musl implementation immidiately issues a DNS query, instead of looking
first at the /etc/hosts file, this patch fixes it.

As a side effect, the long boot duration is shortened by 3-4 seconds.

Command line option: --trace-backtraces

This patch implements GSI interrupt support for Xen bus.
Needed in Xen environments w/o vector callbacks for HVM.
One example of such an environment is Amazon EC2.

This is a test for the effectiveness of our scheduler's load balancing while
running several threads on several cpus.

A full description of the test and its expected results is included in
comments in the beginning of the code. but briefly, the test runs multiple
concurrent busy-loop threads, and an additional "intermittent" thread (one that
busy-loops for a short duration, and then sleeps), and expects that all busy
threads will get their fair share of the CPU, and the intermittent thread
won't bother them too much.

Testing the current code, this tests demonstrates the following problems
we have:

1. Two busy-loop threads on 2 cpus are 5%-10% slower than just one.
   This is not kernel overhead (profiling show 100% of the time in the
   test's inner loop), and I see exactly the slowdown when running this
   test on the Linux host, so it might be related to the host's multitasking?
   For now, let's not worry about that.

2. Much more worrying is that the intermittent thread sometimes (in about half
   the tests) causes us to only fully use one CPU, and of course get bad
   performance.

3. In many of the tests involving more than 2 threads (2 threads +
   intermittent, or 4 threads) load balancing wasn't fair and some
   threads got more CPU than the others.

Later I'll send patches to fix issues 2 and 3, which appear to happen because
the load balancer thread doesn't run as often as it should, because of vruntime
problems.

Pass the "-D" command line options that are used to configure JMX, for
example, to the JVM.

XAPIC is supported as a fall-back when X2APIC is not available

Nobody cares about vma address and it's easy to confuse it to the vma start address so drop it from "osv mmap" output.

Suggested by Nadav Har'El.

Strangely, C++11's new std::chrono::system_clock::now() (which I wanted
to use in a test case) calls clock_gettime() not through the function,
but with the syscall() interface. So add support for that too.

Added a new function, osv::reboot() (declared in <osv/power.hh>)
for rebooting the VM.

Also added a Java interface - com.cloudius.util.Power.reboot().

NOTE: Power.java and/or jni/power.cc also need to be copied into
the mgmt submodule.

Statically allocated mutexes are very common.  Make the mutex constructor
constexpr to ensure that a statically allocated mutex is initialized before
use, even if that use is from static constructors.

Fix up memory layout of 'class vma' for 'osv mmap' gdb command.

Commit 3510a5ea ("mmu: File-backed VMAs") forgot to fix vma::split() to
take file-backed mappings into account. Fix the problem by making
vma::split() a virtual function and implementing it separately for
file_vma.

Spotted by Avi Kivity.

Parsed by JLine (in CRaSH)
Console should now better understand keys like home/end/arrows

Rarely (about once every 20 runs) we had OSV crash during boot, in the
DHCP code. It turns out that the code first sends out the DCHP requests,
and then creates a thread to handle the replies. When a reply arrives,
the code wake()s the thread, but on rare occasions the thread hasn't yet
been set up (still a null pointer) so we have a crash.

Fix this by reversing the order - first create the reply handling thread,
and only then send the request.

use to dump tracepoints to a file - trace.txt, x100 faster ;)

The load_balance() code checks if another CPU has fewer threads in its
run queue than this thread, and if so, migrates one of this CPU's threads
to the other CPU.

However, when we count this core's runnable threads, we overcount it by
1, because as soon as load_balance() goes back to sleep, one of the
runnable threads will start running. So if this core has just one more
runnable threads than some remote's core runnable threads, they are
actually even, so in that case we should *not* migrate a thread.

Overcounting the number of threads on the core running load_balance
caused bad performance in 2-core and 2-thread SpecJVM: Normally, the
size of the run queue on each core is 1 (each core is running one of
the two threads, and on the run queue we have the idle thread). But
when load_balance runs it sees 2 runnable threads (the idle thread and
the preempted benchmark thread), and the second core has just 1, so
it decides to migrate one of its threads to the second CPU. When this
is over, the second CPU has both benchmark threads, and the first CPU
has nothing, and this will only be fixed some time later when the
second CPU's load_balance thread runs, and later the balance will be
ruined again. All this time that the two threads run on the same CPU
significantly hurt performance, and on the host's "top" we see qemu
taking just 120%-150% instead of 200% as it should (and as it does
after this patch).

Currently, clock::get()->time() jumps (by system_time(), i.e., the host's
uptime) at some point during the initialization. This can be a huge jump
(e.g., a week if the host's uptime is a week). Fixing this jump is hard,
so we'd rather just tolerate it.

reschedule_from_interrupt() handles this clock jump badly. It calculates
current_run, the amount of time the current thread has run, to include this
jump while the thread was running. In the above example, a run time of
a whole week is wrongly attributed to some thread, and added to its vruntime,
causing it not to be scheduled again until all other threads yield the
CPU.

The fix in this patch is to limit the vruntime increase after a long
run to max_slice (10ms). Even if a thread runs for longer (or just thinks
it ran for longer), it won't be "penalized" in its dynamic priority more
than a thread that ran for 10ms. Note that this cap makes sense, as
cpu::enqueue already enforces a similar limit on the vruntime "bonus"
of a woken thread, and this patch works toward a similar goal (avoid
giving one thread a huge bonus because another thread was given a huge
penalty).

This bug is very visible in the CPU-bound SPECjvm2008 benchmarks, when
running two benchmark threads on two virtual cpus. As it happens, the
load_balancer() is the one that gets the huge vruntime increase, so
it doesn't get to run until no other thread wants to run. Because we start
with both CPU-bound threads on the same CPU, and these hardly yield the
CPU (and even more rarely are the two threads sleeping at the same time),
the load balancer thread on this CPU doesn't get to run, and the two threads
remain on the same CPU, giving us halved performance (2-cpu performance
identical to 1-cpu performance) and on the host we see qemu using 100% cpu,
instead of 200% as expected with two vcpus.