Used for implementing support for indirect functions referenced from shared
libraries.

Recently Guy fixed abort() so it will *really* not infinitely recurse trying
to print a message, using a lock, causing a new abort, ad infinitum.

Unfortunately, that didn't fix one remaining case: DUMMY_HANDLER (see
exceptions.cc) used the idiom

        debug(....); abort();

which can again cause infinite recursion - a #GP calls debug() which causes a
new #GP, which again calls debug, etc.

Instead of the above broken idiom, created a new function abort(msg), which is
just like the familiar abort(), just changes the "Aborted" message to some
other message (a constant string). Like abort(), the new variant abort(msg) will
only print the message once even if called recursively - and uses a lockless
version of debug().

Note that the new abort(msg) is a C++-only API. C will only see the abort(void)
which is extern "C". At first I wanted to call the new function panic(msg) and
export it to C, but gave when I saw the name panic() was already in use in a
bunch of BSD code.

Before this patch tracepoints required manual tracepoint numbers:

    tracepoint<17, unsigned int> trace_event1("event1", "%d");
    tracepoint<18> trace_event2("event2", "");

While the numbers only had to be unique in the file, so it wasn't hard to
achieve, this was still tedious and verbose.

This patch adds an additional, shorter, tracepoint syntax, not requiring
those numbers and in general less repetitive and clearer:

    TRACEPOINT(trace_event1, "%d", unsigned int);
    TRACEPOINT(trace_event2, "");

The first parameter is the name of the generated tracepoint function -
it's convenient to see it so that grep can find it, for example.
The name of the tracepoint itself (shown in "osv trace") is this string
without the prefix trace_ (if the name of the tracepoint function, for some
reason, doesn't start with trace_, the full function name is used as the
tracepoint name).

given the scheduler state, wake() sometimes rescheduled the dispatcher thread
immidiately, and then it blocked on the mutex that is still held by the caller
of _callout_stop_safe_locked().

this patch does wake() outside of the lock to eliminated these spurious context
switches.

controlling the logger output in tests is simplistic, it is enough to
change the severity level to logger_error and all logging messages
will appear.

next patch is changing the debug function to tprintf_d, which may be
implemented as do{}while(0) in case conf-logger_debug=0, in this case
compilation breaks complaining about unused variables.

these debug prints are not very useful today, so I remove them. Instead,
they may be implemented as tracepoints.

the conf-logger_debug option control whether tprintf_d (verbose debug
logging) is enabled at all, if it is set to 0 then tprintf_d is
implemented as do{}while(0)

We want to use the clock in tracepoints.

Add some extern "C" versions of the lockfree::mutex methods. They will be
necessary for providing the lockfree::mutex type to C code - as you'll see
in later patches, C code will see an opaque type, a byte array, and will
call these functions to operate on it.

Do not include <sched.hh> in <lockfree/mutex.hh>.

Including <sched.hh> creates annoying dependency loops when we (in a
later patch) replace <osv/mutex.h> by <lockfree/mutex.hh>, and some header
files included by <sched.hh> themselves use mutexes, so they include
<osv/mutex.h>. This last include does nothing (because of the include guard)
but the compiler never finished reading osv/mutex.h (it was only in its
beginning, when it included sched.hh) so the inner-included code lacks the
definitions it assumes after including mutex.h.

Add to lockfree::mutex the simple owned() and getdepth() methods which
existed in ::mutex and were used in a few places - so we need these
methods to switch from ::mutex to lockfree::mutex.

When I developed lockfree mutex, the scheduler, preemption, and related code
still had a bunch of bugs, so I resorted to some workarounds that in hindsite
look unnecessary, and even wrong.

When it seemed that I can only wake() a thread in wait state, I made an
effort to enter the waiting state (with "wait_guard") before adding the
thread to the to-awake queue, and then slept with schedule(). The biggest
problem with this approach was that a spurious wake(), for any reason, of
this thread, would cause us to end the lock() - and fail on an assert that
we're the owners of the lock - instead of repeating the wait. When switching
to lockfree mutex, the sunflow benchmark (for example) would die on this
assertion failure.

So now I replaced this ugliness with our familiar idiom, wait_until().
The thread is in running state for some time after entering queue, so
it might be woken when not yet sleeping and the wake() will be ignored -
but this is fine because part of our protocol is that the wake() before
waking also sets "owner" to the to-be-woken thread, and before sleeping
we check if owner isn't already us.

Also changed the comment on "owner" to say that it is not *just* for
implementing a recursive mutex, but also nessary for the wakeup protocol.

The existing shutdown() code only worked with AF_INET sockets, and returned
ENOTSOCK for AT_LOCAL sockets, because we implemented the latter sockets in
completely different code (in af_local.cc).

So in uipc_syscalls_wrap.c, the same place we call a the special af-local
socketpair(), we also need to call the special af-local shutdown().

The way we do it is a bit ugly, but effective: shutdown() first calls
shutdown_af_local(), and if that returns ENOTSOCK (so it's not an af_local
socket), we continue trying the regular socket shutdown code.

A better way would have been to add shutdown() to the fileops table -
either the generic one (why not?), or invent a new mechanism whereby
certain file types (in this case, "sockets" of all types) can have additional
ops tables including in this case a shutdown() operation. Linux has
something of this sort for implementing shutdown().

we have to disable virio interrupts before msix EOI so disabling
must be done in the ISR handler context. This patch adds an std::function
isr to the bindings.

references to the rx and tx queues are saved as well (_rx_queue and _tx_queue),
so they can be used in the ISR context.

this patch reduces virtio net rx interrupts by a factor of 450.

lazy_indirect<T> which only allocates T, on the heap, on first use. The
lazy_indirect<T> object itself only takes 8 bytes of memory (a single pointer).

This template is useful for implementing pthread_mutex_t and pthread_condvar_t
using the larger lockfree::mutex and condvar (containing a lockfree::mutex)
objects.

Optimize tracepoints fast path to a single nop instruction.

Allows networking to work without reconfiguration or dhcp.

Place the tracepointv type in an anonymous namespace.  This makes every
translation unit have its own unique tracepoint types, so we only need
to ensure uniqueness within a source file.

Use the type's type_info to select the correct patch sites.

Idea from Nadav.

When a tracepoint is disabled, we want it to have no impact on running code.

This patch changes the fast path to be a single 5-byte nop instruction.  When
a tracepoint is enabled, the nop is patched to a jump instruction to the
out-of-line slow path.

In order to optimize the fast path of tracepoints, we need to patch
the call sites to skip calling the slow path code completely.  In turn,
that requires that each call site be unique -- a separate function.

In the current implementations, tracepoints with the same signature map
to the same type.  It would have been great to use the name as a discriminant
(tracepoint<"sched_queue", thread*> trace_sched_queue(...);), but C++ does
not support string literals as template arguments.

We could do

  const char* trace_sched_queue_name = "sched_queue";
  tracepoint<trace_sched_queue_name, thread*> trace_sched_queue(...);

but that doubles the code for declaring a tracepoint.  Add a unique ID instead
(and code to verify it is unique).

Until now, lockfree::mutex functions were entirely inline, which won't
fly if we want to make it our default mutex. Change them to be out-of-line,
implemented in a new source file core/lfmutex.cc.

This has a slight performance impact - uncontended lock/unlock pair used
to be 17ns, and is now 22ns.

In the future we can get this performance back by making these functions
partially inline (the uncontended case inline, the waiting case in a
separate function), although we'll need to consider the speed/code-size
tradeoff.

When abort()ing on failed symbol lookup, if this is a C++ symbol, also
show its demangled form, which in many case can be more helpful.
Here is an example lookup failure now:

failed looking up symbol _ZN8lockfree5mutex4lockEv (lockfree::mutex::lock())
Aborted

Significantly speed up lockfree::mutex's uncontended case, by avoiding
sequential memory ordering in atomic operations. I *think* I did this
correctly, but can't be really sure ;-) Moreover, I didn't change the
memory ordering on the rarer cases, and these should also be reduced in
the future.

Uncontended lock&unlock of lockfree::mutex is now 17ns. This is faster
than the previous mutex (24ns) but slower than spinlock (10ns).

The recent change, to add the program name as argv[0] for C code's
main(), make sense for C code, but less for Java code, where main()
normally expects args[0] to be the first argument, not the program name.

So the change to RunJava.java was un-Java-like; It also broke the "java"
CLI command which didn't put "java" in argv[0] for the arguments to
RunJava.main(), so the "java" command no longer worked after the previous
patch.

Instead, we change java.cc (which compiles to java.so). This is what
calls RunJava.class, and it should remove the new argv[0] before calling its
main() - instead of expecting that RunJava.class to do this.

the convention in linux is that argv[0] holds the program executable.
I had an attempt to run netserver not from the CLI and it didn't work because
its argument parsing got broken.

Added the possibility to pass to cli.jar a command, which it runs instead
of taking commands interactively. Note that the initialization script is
run before the given command.

After this patch,

        scripts/run.py -e "java.so -jar /java/cli.jar"

Continues to run the interactive command line editor loop, as before.
But additionally, one can do:

        scripts/run.py -e "java.so -jar /java/cli.jar ls"

To run just the command "ls" and exit - exactly as if the user would type
this command on the command line and exit the VM.

The given command can be, of course, much longer. For example to run Jetty
after the CLI's normal initialization script, the following monster can
be used:

scripts/run.py -n -e "java.so -jar /java/cli.jar java -classpath /jetty/* org.eclipse.jetty.xml.XmlConfiguration /jetty/jetty.xml"

(Funny how a single command should say "java" 3 times and "jetty" 4 times :-))

Add a "java" command to the CLI, using the same syntax of java.so and
attempting to emulate as closely as possible the "java" command on Linux.
So for example one can run

        java Hello

to run /java/Hello.class (/java is on the classpath by default), or

        java -jar /java/bench.jar

to run the main class of this jar, or a more sophisticated
command lines, such as the following which runs Jetty (if the
appropriate files are in your image):

        java -classpath /jetty/* org.eclipse.jetty.xml.XmlConfiguration /jetty/jetty.xml

Note that like java.so, the new "java" command basically runs the RunJava
class (/java/RunJava.class). Remember that java.so adds /java to the parent
class loader, so we can always find the RunJava class even though it's not
in cli.jar or cloudius.jar).

This patch adds support for O_NONBLOCK on pipes and unix domain sockets.

Java's EPollSelectorImpl uses a pipe to interrupt a sleeping poll, and it,
quite understandably, sets them to non-blocking (if you only write a
single byte to a pipe, you don't expect any blocking anyway).
So we can't croak if this option is used, and better just implement it
correctly.