This patch implements GSI interrupt support for Xen bus.
Needed in Xen environments w/o vector callbacks for HVM.
One example of such an environment is Amazon EC2.

I would like to call read_partition_table automatically from device_register,
which would guarantee that every device that comes up have its partitions
scanned. Although totally possible on KVM, it is not possible on Xen, due to
the assynchronous nature of the bringup protocol: the device is exposed and
created in a moment where IO is not yet possible, so reading the partition
table will fail. Just read them both from the drivers when we are sure the
driver is ready.

To support multiple partitions to a disk, I found it easier to add a
post-processing offset calculation to the bio just before calling the strategy.

The reason is, we have many (really many) entry points for bio preparation
(pre-strategy) and only two entry points for the strategy itself (the drivers).
Since multiplex_strategy is a good thing to be used even for virtio (although I
am not converting it now), since it allows for arbitrary sized requests, we
could very well reduce it to just one.

At this moment, the offset is always 0 and everything works as before.

Currently we get it from the private data, but since I plan to use the same
private data for all partitions, we need a unique value, that already exists in
the device. So use it.

This patch implement the HPET clock driver, that should work as a fallback for
both Xen and KVM, in case the paravirtual clock is not present. This is
unfortunately the situation for all HVM guests running on EC2, so support for
this is paramount. I have tested on KVM forcing the kvmclock to disappear, and
it seems to work all right.

Right now we are doing it right before we parse the MADT, but this is by far
not MADT specific. Other users are planned, and the best way to resolve the
disputes is to have it in a separate constructor

As a cleanup, use wait_for_queue() like virtio-net does.

The xen block driver needs some extra state not needed for the network drivers.
Namely, the same way virtio-blk does, we need to tell the block layer which is
our strategy, read and write functions. For that, we need some extra code that
I am implementing in xenfront-blk.cc.

With all the infrastructure in place, we can compile all our xenbus code in,
and enble the netfront driver.

This commit includes the basic translation layer so we can support BSD devices
for Xen as is. I believe there is room for improvement, but that provides a
very good starting point. I would like to provide you guys with a smaller
commit, but it is quite cumbersome to separate the device implementation from
the bus implementation per se. Part of it is due to the way Xen does things.

For instance: when we are talking virtio devices, each of them will have its
own pci device. For xen, only one pci device will be exposed, with its own
interrupt, and that will be the xenbus. The xenbus is then responsible for
querying the xenstore and figuring out which devices are there. Maybe there
is a different way of doing things (I believe for instance that there is no
reason to have shared interrupts between all devices) but that is the way BSD
does things. And about the bus-is-the-device, even Linux as far as I know.

Another thing to note, is that BSD devices registered with a device_method_t
seem to be fully hierarchical. I am not doing it this way here (I am mostly
ignoring the xenbus device_method_t definitions), which means there are a lot
of assumptions that probably won't hold with other kinds of drivers. But it
should not be hard to extend, and that simplifies things.

Without this, the networking stack will ignore TSO.

Improves Java Echo test from ~2Gbps to ~5Gbps.

With Dor Laor.

Our console works well with KVM, that redirects the serial to stdio. In Xen,
this seems not to work very well (some documentation sources - not all - lists
'serial="stdio"' as a valid option, but it doesn't really work). In that case,
LFs are not automatically translated into CRLF and the terminal output is totally
borked.

This is obviously not exclusive to Xen: by changing our qemu command line to run
a pts instead of stdio, and then connecting to that pts, the same thing happens.

This patch modifies our console's write() function to respect the termios ONLCR
flag, which is now set by default. When write() sees a newline character and
this flag is set we output a CF before any LF.

The termios structure is left outside the specific console, and we pass a
pointer down to it. Our pre-console implementation of simple_write outputs CRs
unconditionally.

The normal stdio serial still works. Telnet still works.

Our console write() takes 3 parameters. The last one controls whether or not we
will issue a newline at the end of input. If it is true, we will call the
console's implementation of newline(). It is always passed as false, though.
Remove it and fix the callers.

There is no harm in leaving it mappend, and there is a potential harm in unmapping,
since our mapping code does not keep reference counters. So if we map an already
mapped area, the unmap code will just unmap it for good. Avi spotted this.

We are currently using PCI's brute force enumeration, which consists of scanning
all possible combination of b,d,f. This makes us issue many inb operations, a lot
more than we could do if we were using smarter enumeration. In my system, the inbs
alone account for more than 5M cycles, which takes around 4s to run (remember this
is xen, it is faster in KVM)

We can do a lot better by using recursive discovery, and only scanning the buses
we need to. In the case of virtual systems the difference is even bigger, since it
is very unlikely that we will have any complex setup with more than one bridge.

Boot for me feels instantaneous now (once we get to start OSV)

I have recently ran into an early init bug due that ended up being tracked
down to a changeset in which the initialization priorities of the constructors
were changed. One of the changed ones was kvmclock, but the change did not
update kvmclock.

I propose we use constants for that. To avoid things like this in the future,
wherever priorities are used, I believe they should come from the same place
so that the order is utterly obvious. To handle that, I am creating the prio.hh
file, and sticking all priority definitions in there.

The correct fix is to add setup_features call in the constructor

This reverts commit db65a9fe.

We need to read the various config bits from the device config, not the
guest config.

Without this we get errors reading/writing from larger imageṡ.  Our current
10g usr.img actually is enough to trigger this, but our lack of error handling
papered over it so far.

bio queue is the name used by BSD. Since it is just a name difference,
I would better change our code, since there are few users (only ramdisk),
than to patch all code I am importing from BSD that uses it.

Based on FreeBSD virtio code
Provides a x7 boost for rx netperf

Fix memcached networking issue that was caused by unsecure, parallel device invocation. I wasn't aware that the a device lock has to be held on the tx callback. Will look into it deeper tomorrow. The patch solves the issue

This way it's possible to wake a thread while holding the lock
that protects the thread pointer of going away. The lock itself
won't be held by the waker and thus the wakee will be able to
use it immedietly w/o ctx. Suggested by Nadav.

Make the early allocator available earlier to support the dynamic
per-cpu allocator.

No code change.

Instead of cancelling block requests due to no space on the ring
that lead to corruption of the upper layer, block until there is
space.

Required by the virtio spec.

virtio_blk pre-allocates requests into a cache to avoid re-allocation
(possibly an unneeded optimization with the current allocator).  However,
it doesn't take into account that requests can be completed out-of-order,
and simply reuses requests in a cyclic order. Noted by Avi although
I had it made using a peak into the index ring but its too complex
solution. There is no performance degradation w/ smp due to the good
allocator we have today.

virtio_driver::wait_for_queue() would often hang in a memcached and
mc_benchmark workload, waiting forever for received packets although
these *do* arrive.

As part of the virtio protocol, we need to set the host notification
flag (we call this, somewhat confusingly, queue->enable_interrupts())
and then check if there's anything in the queue, and if not, wait
for the interrupt.

This order is important: If we check the queue and only then set the
notification flag, and data came in between those, the check will be
empty and an interrupt never sent - and we can wait indefinitely for
data that has already arrived.

We did this in the right order, but the host code, running on a
different CPU, might see memory accesses in a different order!
We need a memory fence to ensure that the same order is also seen
on other processors.

This patch adds a memory fence to the end of the enable_interrupts()
function itself, so we can continue to use it as before in
wait_for_queue(). Note that we do *not* add a memory fence to
disable_interrupts() - because no current use (and no expected use)
cares about the ordering of disable_interrupts() vs other memory
accesses.

I'm returning Dor's original virtio_driver::wait_for_queue().

The rewrite just masked, with its slightly different timing and redundant
second check before waiting, the real bug which a missing memory barrier
(see separate patch fixing that).

Dor's original code has the good feature that after waking up from a
sleep - when presumably we already have something in the queue - we
check the queue before pessimisticly enabling the host notifications.
So let's use Dor's original code.