Our APIC code is so wrong, but so wrong, that it even produce incorrect
results.  X2APIC is fine, but XAPIC is using xapic::ipi() for all its
interrupts. The problem with that, is that the costumary place for "vector" is
inverted in the case of allbutself delivery mode, and therefore, we're sending
these IPIs to God Knows Where - not to the processors, that is for sure.
As a result, we would spin waiting for IRQ acks that would never arrive.

I could invert and reorganize the parameters and comment this out, but I've
decided it is a lot clearer just to open code it. Also, there is no need at all
to set ICR2 for allbutself, because the destination is already embedded in the
firing mode.

One issue: NMI is copied over because it is also wrong by the same reasons, so
I fixed. But I don't have a test case for this.

Fixes #110

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch fixes the Amazon crash many CPUs. Funny enough, it is not
actually related to the number of CPUs. In that situation, the port
numbers allocated for the event channels are quite high. I don't
really know the reason for that, maybe the Hypervisor reserves the
small bits for CPU related things...

As we have more and more CPUs and the bits shift more and more rightwards,
they eventually reach the second long word of the event channel quadword.

But we have been operating this with brsl, which will only reach a long word.
If bit 32, for instance, is 1, it  will be interpreted as bit 0 == 1. Bit 0
having no registered handler, that will turn into a nullptr access.

Cheers for Dima for doing most of the debugging and heavy lifting here.
The hang issues are still present.

Fixes #109.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

When source code is compiled with -D_FORTIFY_SOURCE on Linux, various
functions are sometimes replaced by __*_chk variants (e.g., __strcpy_chk)
which can help avoid buffer overflows when the compiler knows the buffer's
size during compilation.

If we want to run source compiled on Linux with -D_FORTIFY_SOURCE (either
deliberately or unintentionally - see issue #111), we need to implement
these functions otherwise the program will crash because of a missing
symbol. We already implement a bunch of _chk functions, but we are
definitely missing some more.

This patch implements 6 more _chk functions which are needed to run
the "rogue" program (mentioned in issue #111) when compiled with
-D_FORTIFY_SOURCE=1.

Following the philosophy of our existing *_chk functions, we do not
aim for either ultimate performance or iron-clad security for our
implementation of these functions. If this becomes important, we
should revisit all our *_chk functions.

When compiled with -D_FORTIFY_SOURCE=2, rogue still doesn't work, but
not because of a missing symbol, but because it fails reading the
terminfo file for a yet unknown reason (a patch for that issue will
be sent separately).

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The SMP bringup code uses a linked list of stacks. The APs are all
brought up concurrently and so to acquire a stack, they compare and swap
on smp_stack_free. The current code works as long as the compare and
swap succeeds, if it fails, then smp_stack_free MUST be read again
otherwise the AP will be deadlocked. This patch fixes the current code
by enforcing a re-read on a cmpxchg failure.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Dan Schatzberg <schatzberg.dan@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

There is no need to hold the lock while waiting for the host to refill the
entropy buffer; drop it.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

std::valarray does not guarantee its elements will be allocated contiguously,
so the form &v[0] is only guaranteed to point to the first element, not the
rest.

Switch to std::vector, where contiguity is guaranteed.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Suppose N threads try to acquire a byte of entropy from an empty pool.  They
will all serialize on the mutex, waiting for the pool to refill.  However,
when the pool is eventually refilled, only one consumer will be awakened;
the rest will continue sleeping even though there is entropy available in the
pool.  They will eventually be awakened when the worker refills the pool,
but that's unneeded latency.

Fix by using wake_all() to wake all consumers.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

It's a bad idea to claim to support /dev/urandom but rely on HW RNG
because starting up Cassandra, for example, takes ages.

Drop it until we have a cryptographically secure PRNG in OSv that can be
used to implement /dev/urandom properly.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Convert file::f_ops/fileops to file's vtable.

Reviewed-by: Pekka Enberg <penberg@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Everyone is now overriding file's virtual functions; we can make them
pure virtual and remove fileops completely.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Unused.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Everyone switched to the nifty variadic type.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Useful for C -> C++ conversions.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Not everyone wants it.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Derived file objects will be initialized by the class constructor, no need
for fo_init().

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Once file::~file() is called, virtual functions no longer dispatch to the
derived type (which has since been destroyed) but to the base type, which
is uninteresting.

Move the call to close() from the destructor to fdrop().

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This is a temporary measure before the module definition is moved to
the mgmt repo.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Manifests should contain paths in the same file system in which the
image is built. Currently manifests assumed that the module will be
downloaded/copied into build/*/module/ and that the CWD is inside
build/*. This is not always the case now that we have "direct-dir"
module type, which is is not copied.

Example use:

    /usr/x:${MODULE_DIR}/x

This patch adds MODULE_DIR variable, which is replaced with the path
to the module in local file system. Depending on the module type this
can be either direct path or a path inside build/*/module/.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

    == Description of the problem ==

Currently modules can only specify files which need to be
copied. There is a need for a module to also declare ways it can be
run, so that we can automatically prepare a runnable image. It
should be easy to switch between run configurations.

Currently it is enough for image configuration to happen at the
time of image make process. In future this should be allowed
on the already built image.

We also need to be able to combine multiple modules in one
image. For example in addition to the main application one might want
to start various services like CLI, management API server, etc.

Java apps should be able to specify JVM arguments, which must be
passed to the JVM upon its creation, as well as regular run-java
arguments (classpath, main classes, main arguments, etc.)

    == Solution ==

This is not intended to be a permanent solution. The aim is to solve
immediate need to have a fully modularized build in a scalable way.

Every module has a new kind of file in its root directory which
holds its configuration. The file is named 'module.py' and is
a python script which uses osv's api for declaring run configurations.

Using python as config language has several advantages:
 - more expresiveness, unlike json it allows for expression reuse
 - it's easier to extend the config language
 - we don't need as much parsing, gluing, error checking, error
   reporting code because we have it already

There are currently two kinds of applications which can be declared:

   run(cmdline)  <- basic .so application
   run_java(jvm_args=[], classpath=[], args=[])  <- java applications

Run configurations can be declared as simple module attributes
which can be referenced from the image configuration file.

Image configuration

There is a new configuration file kind, which defines which modules
and which run configurations should be included in the image. Files
are located using path: ${OSV_BASE}/images/$(image-name).py

Syntax:

  require(module) <-- declares that module should be included in the image
  		      and returns an object which allows to access module's
		      attributes.

  run = []  <-- list of run configurations

Example:

  _mgmt = require('mgmt')
  run = [ _mgmt.shell ]

To use a particular image configuration run make like this:

  make image=fancy-tomcat

The default configuration is named 'default'.

This patch extracts mgmt into a module, which is embedded under
${OSV_BASE}/modules/mgmt

The purpose of ${OSV_BASE}/config.json has been changed. It does not
list modules which should be included anymore, image config file does
that. It's a module look-up configuration which tells the build where
to look for modules.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

They are temporary files for vim and should be ignored.

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

We have an intermediate bare.img now. Rename usr.raw to bare.raw to make the
naming more consistent: bare.raw -> bare.img -> usr.img.

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Besides simplifying mmu::map_file interface, let's make it more similar
to mmu::map_anon.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This will allow the users to populate /usr with simples binaries like haproxy
and their config at runtime.
It's intented to be combined with the loader ability to run multiple binaries
in a sequential way.
For example -e '/tools/cpiod.so; /usr/haproxy.so -f /usr/haproxy.conf'

Signed-off-by: Benoit Canet <benoit@irqsave.net>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This pave the way for the creation of bare images that the user can populate
using cpiod when he need it.

Signed-off-by: Benoit Canet <benoit@irqsave.net>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

A ';' at the end of a parameter mark the end of a program's arguments list.

The goal of this patch is to be able to split mkfs.so in to parts mkfs.so and
cpiod.so.

The patch uses a full spirit parser to escape "" and split commands around ';'.

Signed-off-by: Benoit Canet <benoit@irqsave.net>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

We had at least one report for a system in which the __mm_setcsr function
we were using is not found. Since this is heavily arch specific anyway,
we should just use inline assembly to do so.

Generated code verified to be the same:

  33c6e7:       c7 45 e0 80 1f 00 00    movl   $0x1f80,-0x20(%rbp)
  33c6ee:       0f ae 55 e0             ldmxcsr -0x20(%rbp)

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The main purpose is to exercise the new parameter checks added to
mmap to handle wrong combination of parameters.
Let's intentionally force errors and catch them with the respective errno codes.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

(flags & MAP_ANONYMOUS) must be instead of (fd == -1) to determine
the mapping type as the latter one is a valid argument to file mappings.
Tests related to files were added into mmap_validate_file.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
[ penberg: cleanups ]
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Rename mmap_validate_flags to mmap_validate as it's not only related to
flags now.  Add new tests to check bad paramater values.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
[ penberg: cleanups ]
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

storage/runtime arguments for tracepoint can be inferred from
assign() function signature instead of specified explicitly
by storage_args/runtime_args. This makes boilerplate code smaller.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>