read_random() is used indirectly by the TCP stack to randomize a local port number,
before this patch it was identical in each execution, and that caused NAT problems.

also add a FIXME note to implement real random one day.

the linux/musl definition is 2 bytes long instead of 1 byte as in
freebsd, this patch is fixing this issue which caused ifconfig to fail

Attempting to put an end to the linux<->freebsd confusion.

"osv leak show" can take a very long time until you start seeing output.
This patch shows the progress of the first phase (getting data from the
debugged program), and then says the second phase (sorting) starts.
By the way, it turns out the sorting phase is slower (complexity-wise,
this is obvious, but I didn't really expect it in this case).

The previous implementation of backtrace() required frame pointers.
This meant it could only be used in the "debug" build, but worse,
it also got confused by libstdc++ (which was built without frame pointers),
leading to incorrect stack traces, and more rarely, crashes.

This changes backtrace() to use libunwind instead, which works even
without frame pointers. To satisfy the link dependencies, libgcc_eh.a
needs to be linked *after* libunwind.a. Because we also need it linked
*before* for other reasons, we end up with libgcc_eh.a twice on the
linker's command line. The horror...

mmu::allocate(), implementing mmap(), used to first evacuate the
region (marking it free), then allocate a tiny vma object (a start,end
pair), and finally populate the region.

But it turns out that the allocation, if it calls backtrace() for the first
time, ends up calling mmap() too :-) These two running mmap()s aren't
protected by the mutex (it's the same thread), and the second mmap could
take the region just freed by the first mmap - before returning to the
first mmap who would reuse this region.

We solve this bug by allocating the vma object before evacuating the
region, so the other mmap picks different memory.

Before this fix, "--leak tests/tst-mmap.so" crashes with assertion
failure. With this fix, it succeeds.

This patch fixes two deadlocks which previously existed in the allocation
tracking code in core/mempool.cc. These deadlock have become more visible
in the new backtrace() implementation which uses libunwind and more often
allocates memory during its operation (especially from dl_iterate_phdr()).

1. alloc_page() wrongly held the free_page_ranges lock for a bit too
   long - including when calling tracker_remember(). tracker_remember()
   then takes the tracker mutex. Unfortunately, the opposite lock order
   also occurs: Consider a tracker_remember() (e.g., from malloc)
   needing to allocate memory and through the memory pool, end up calling
   alloc_page().

   This is a classic deadlock situation. The solution is for alloc_page()
   to drop free_page_ranges_lock before calling tracker_remember().

2. Another deadlock occured between the tracker lock and a pool::_lock -

   thread A: malloc calls remember() taking the TRACKER LOCK, and then
   calling malloc() (often in dl_iterate_phdr()), which calls
   memory::pool::alloc and takes the POOL LOCK.

   thread B: malloc calls memory::pool::alloc which takes the POOL LOCK
   and then if the pool is empty, calls alloc_page() which is also
   tracked so it takes the TRACKER LOCK.

   Here the solution is not to track page allocations and deallocations
   from within the memory pool implementation. We add
   untracked_alloc_page() and untracked_free_page() and use those in
   the pool class. This not only solves the deadlock, it also provides
   better leak detection because pages held by the allocator are now
   no longer considered "leaks" (just the individual small objects
   themselves).

   The fact alloc_page() now calls untracked_alloc_page()
   also made solving problem 1 above more natural (the free_pages
   lock is held during untracked_alloc_page()).

When the allocation tracking code itself does allocations, we do not
track these allocations - and we notice this by the "depth" of the
alloc_tracker mutex. This avoids some messy infinite recursion, and also
improves performance because alloc_tracker does do a bunch of allocations
(most of them not apparent from the code), and tracking them too would
significantly slow it down, with no benefit because we're not debugging
the allocation tracker itself.

But while the existing code ignored nested alloc_tracker::remember(),
we forgot to also ignore nested alloc_tracker::forget()! This meant
that for each allocation inside alloc_tracker, we never tracked it,
but wasted our time trying to delete it from the list of living
allocations. This oversight caused a huge slowdown of alloc_tracker(),
which this patch fixes. alloc_tracker() is now just very slow, not
very very very slow ;-)

libunwind, which the next patches will use to implement a more reliable
backtrace(), needs the msync() function. It doesn't need it to actually
sync anything - just to recognize valid frame addresses (stacks are
always mmap()ed).

Note this implementation does the checking, but is missing the "sync" part
of msync ;-) It doesn't matter because:

1. libunwind doesn't need (or want) this syncing, and neither does anything
   else in the Java stack (until now, msync() was never used).

2. We don't (yet?) have write-back of mmap'ed memory anyway, so there's
   no sense in doing any writing in msync either. We'll need to work on
   a full read-write implementation of file-backed mmap() later.

As Nadav found out, we don't have to call defineClass and register each visible
Java class within the js scope, this makes it much simpler to integrate with Java
from the cli.

Currently, if we're requested to preempt while preemption is disabled, we
just ignore the request and wait until the next interrupt (which is not even
guaranteed to arrive) hits us.  This isn't the best arrangement.

Fix by latching the preemption request into a 'need_reschedule' thread local
variable, and examining it during preempt_enable().

Use includes from musl and external/, avoiding the host development
environment.

Not provided by the musl headers, and also unneeded.

We already get most of this from <sys/param.h>

This way we make sure any headers with the same name in the Solaris porting
glue get used first.  Also make sure the jni header is available to C code
as well.

libc: include/api
boost: external/misc.bin

bsd sources now prefer bsd's sys/ to our sys/, should be reconciled somehow
later.

Probably unneeded with the prototype fix, but can't hurt.

musl provides those, so use their definitions.

Not provided by musl, so mark as extern "C".

add extern "C" where needed, fix up prototypes.

musl doesn't provide these defines, and it isn't really necessary to put them
in some header.