move the arch-specific stuff in premain to
arch/x64/arch-setup.cc.

Introduce arch_init_premain() and arch_setup_tls().

arch_init_premain() is supposed to perform arch-specific
initialization before the common premain code is run.

arch_setup_tls() is run _after_ the common setup_tls code.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The scenario

  run elf file
    demand fault
      allocation
        leak detector tracking
          backtrace_safe()
            page fault

leads to a nested exception.  Add support for it by allocating an extra
stack.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Introduce unregister_level_triggered_handler to handle unregister of
shared vector.

Signed-off-by: Asias He <asias@cloudius-systems.com>

The overloaded version of register_interrupt_handler with a gsi
parameter is only used by register_level_triggered_handler. Fold it into
register_level_triggered_handler.

Signed-off-by: Asias He <asias@cloudius-systems.com>

nullptr check is done in rcu_dispose.

Signed-off-by: Asias He <asias@cloudius-systems.com>

Add set_ack_and_handler helper to setup ack and handler function of gsi
level interrupt.

Signed-off-by: Asias He <asias@cloudius-systems.com>

This patch makes multiple devices which share the same interrupt line
works, e.g. on GCE, it shares virito-net and virtio-scsi interrupts.

Signed-off-by: Asias He <asias@cloudius-systems.com>

pre_eoi return 'true' when the interrupt is for the device, 'false'
otherwise. This is the preparation for interrupt sharing support.

Signed-off-by: Asias He <asias@cloudius-systems.com>

In the past, we have struggled with long delays while reading data from disk in
real mode, leading to big boot times (not that they are totally gone). For that
reason, it is useful to know how much time is being spent in that process.  As
unstable and broken the TSC is, it is pretty much our only ally for that.

What I am proposing in this patch, is that we take timings from key states of
the bootloader, and pass that to main loader. We will do that by adding some
space at the end of the multiboot_info structure, so that we can pass some
fields to it. Right now, we are using 16 bytes so we can pass 2 64-bit tsc
reads.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

I am proposing a mechanism here that will allow us to have a better idea about
how much time do we spend booting, and also how much time each of the pieces
contribute to. For that, we need to be able to get time stamps really early, in
places where tracepoints may not be available, and a clock most definitely
won't.

With my proposal, one should be able to register events. After the system
boots, we will calculate the total time since the first event, as well as the
delta since the last event. If the first event is early enough, that should
produce a very good picture about our boot time.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This following patch implements two of the tracepoints that helped me debug
some of the xen blkfront problems. There are two tracepoint pairs: one of them
for measuring time spent processing an interrupt, and the other time between
interrupts themselves. Huge latencies could be due to either of them.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Adds insl and outsl to perform PIO on IDE driver.

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

If abort() called the result of an exception (ex: segv), my
backtrace-on-abort patch outputs incorrect symbol name on exception
handler.  Because it doesn't have enough information on elf header.

This patch adds the information.

Before the patch applied:
page fault outside application
[backtrace]
0x2f3b31 <mmu::vm_sigsegv(unsigned long, exception_frame*)+37>
0x2f3d4d <mmu::vm_fault(unsigned long, exception_frame*)+215>
0x325b34 <page_fault+157>
0x324797 <void std::__introsort_loop<fault_fixup*, long>(fault_fixup*, fault_fixup*, long)+1129>  <-- This is incorrect !
0x312a05 <vmware::vmxnet3::vmxnet3(pci::device&)+473>

After applied:
page fault outside application
[backtrace]
0x2f3b31 <mmu::vm_sigsegv(unsigned long, exception_frame*)+37>
0x2f3d4d <mmu::vm_fault(unsigned long, exception_frame*)+215>
0x325b34 <page_fault+157>
0x324797 <ex_pf+35> <-- Correct name
0x312a05 <vmware::vmxnet3::vmxnet3(pci::device&)+473>

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

idle, load balancer, init.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

put all of .ctors, .init_array, .ctors.*, init_array.*
into the init_array section.
This fixes the build for mixed ctors / init_array tooling
and dependencies.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

OSv's timer mechanism hinges on the Local APIC's (per-cpu) one-shot timer,
which delivers an interrupt after the requested number of nanoseconds.

The API to set this timer, clock_event::set(), took the absolute time of
the next interrupt. However, what it really needs is the duration in
nanoseconds until the next interrupt.

So this patch we change the basic clock_event::set() API to take a
duration, and implement the original clock_event::set(s64) - taking an s64
absolute wall-clock time - as a simple wrapper.  The next patch will add
more wrappers for set() taking absolute times from different clocks.
Later patches in this series will stop using the old set(s64) version,
until it is dropped in the end of the series.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The "APIC base" message is not very useful to users. Drop it.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Currently, OSv prints out the following at boot:

  acpi 0 apic 0
  acpi 1 apic 1
  acpi 2 apic 2
  acpi 3 apic 3

replace that with a simpler message:

  4 CPUs detected

We do lose the ACPI ID -> CPU ID mapping but it is not terribly
important for users.

Suggested-by: Nadav Har'El <nyh@cloudius-systems.com>
Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

When we start using the JVM balloon, our memcpy could fail for valid reasons
when the JVM is moving memory that is now in an unmapped region. To handle that,
register a fixup that will trigger a JVM call when the fault happens. If all goes
well, we will be able to continue normally.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch provides a backwards version of memcpy. It works all the same, but
will start the copy from dst + n <= src + n, instead of dst <= src. That is
needed for memmove when the source and destination operands overlap.

Being a nonstandard interface, I've just named it "memcpy_backwards"

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Without this, only even megabytes are accessible, and accesses to odd megabytes
hit the even megabytes.  For example address 0x312345 is aliased to address
0x212345.

Enable the A20 gate to prevent this.

Fixes boot on VMware.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Helps making bsd header changes that xen includes.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

We use sched::thread::attr to pass parameters to sched::thread creation,
i.e., create a thread with non-default stack parameters, pinned to a
particular CPU, or a detached thread.

Previously we had constructors taking many combinations of stack size
(integer), pinned cpu (cpu*) and detached (boolean), and doing "the
right thing". However, this makes the code hard to read (what does
attr(4096) specify?) and the constructors hard to expand with new
parameters.

Replace the attr() constructors with the so-called "named parameter"
idiom: attr now only has a null constructor attr(), and one modifies
it with calls to pin(cpu*), detach(), or stack(size).

For example,
    attr()                                  // default attributes
    attr().pin(sched::cpus[0])              // pin to cpu 0
    attr().stack(4096).pin(sched::cpus[0])  // pin and non-default stack
    and so on.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Move the x86-64 PTE definitions to a new arch specific arch-mmu.hh
header file to make core/mmu.cc smaller and more portable.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Context: going to wait with irqs_disabled is a call for disaster.  While it is
true that not every time we call wait we actually end up waiting, that should
be an invalid call, due to the times we may wait. Because of that, it would
be good to express that nonsense in an assertion.

There is however, places we sleep with irqs disabled currently. Although they
are technically safe, because we implicitly enable interrupts, they end up
reaching wait() in a non-safe state. That happens in the page fault handler.
Explicitly enabling interrupts will allow us to test for valid / invalid wait
status.

With this test applied, all tests in our whitelist still passes.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Simplify core/mmu.cc and make it more portable by moving the page fault
handler to arch/x64/mmu.cc.  There's more arch specific code in
core/mmu.cc that should be also moved.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>