Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Since recently, debug() messages are by default not visible. This means
they are not a good way to print error messages :-)

This patch fixes two error messages from loader.cc, that were no longer
visible. For example, "scripts/run.py -e xyz" completed without any message
instead of telling the user that "xyz" doesn't exist.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Output to stdout normally goes to file descriptor 1. However, during
early boot, before the console is opened on file descriptor 1, we also
want printf() to work. So this case is specially treated in
libc/stdio/__stdout_write.cc, where in this early case, we used debug_write
directly, instead of write to the file descriptor.

However, recent patches changed debug_write to write to a memory buffer,
instead of to screen (unless "--verbose" option is given to the loader).
This made early printf() break too.

This patch changes __stdout_write to use console::write(), not
debug_write(). To use console::write(), I had to convert that source
file to C++.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Add a 'trace extract' command that hides the gdb magic and is
future-proof when we change the trace transport protocol.

Reviewed-by: Tomasz Grabiec <tgrabiec@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This adds a '--graphics' option to run.py that enables QEMU's graphics
mode using SDL.  This is needed for testing OSv's VGA support.

Requested-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Drop echoing: "SCRIPT, tap0"

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The timeout test sets a 2 second timeout and a 1 second alarm, and expects
the timeout to happen first.

Change the timeout to 0.5 seconds.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Microsecond values were passed as is instead of being scaled by hz.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Make the JVM heap autotuning message silent by default.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Change some printf calls on boot messages, so it will call
the apropriate debug function. This will enable OSv
to operate on silent mode.

Added debug.h header so we can link debug functions to C files.

Fixes #118

Signed-off-by: Eduardo Piva <efpiva@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Add --verbose options to OSv, so it will flush buffer after
handling the kernel option and set verbose boolean flag in
debug code.

Signed-off-by: Eduardo Piva <efpiva@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Create a circular buffer that stored all debug messages accordingly.
If the debug buffer is full, reuse it. A method called flush_debug_buffer
is added to enable printing all messages to console if verbose mode is
configured.

The global variable debug_buffer_full is used to track if, when
flushing debug buffer to console, we need to flush both buffer sides.

If verbose boolean variable is set, all messages are printed to the
console after beeing stored in the buffer.

The size of the buffer is 50Kb, defined in debug.hh.

A function debugf that received a variable list of arguments
is defined so we can change some printf from boot sequence
to debugf call. A different name is used to prevent C overload.

Signed-off-by: Eduardo Piva <efpiva@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Print the thread pointer, not the address of the thread pointer.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Add a type-safe std::chrono interface to the wall-clock time. E.g., use
use osv::clock::wall::now() to get an std::chrono::time_point instead
of clock::get()->time() which returns an s64.

This patch also changes clock_gettime(CLOCK_REALTIME) and gettimeofday()
to use the new osv::clock::wall::now() interface.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

In <drivers/clock.hh> we had support for time literals looking like 100_ns,
100_us, 100_ms, 100_s. These simply translated to an integer number of
nanoseconds.

Added to <osv/clock.hh> similar literals, but using strongly-typed
std::chrono::duration, so that 100_ms translates to
std::chrono::milliseconds(100), and this is, as usual, automatically
scaled by the compiler to other units when other units (e.g., nanoseconds)
are required.

Because the literals of <drivers/clock.hh> and <osv/clock.hh> have the
same name, to use the latter you need to explicitly enable them in a scope
using the statement

    using namespace osv::clock::literals;

One the entire codebase is converted to use <osv/clock.hh> and std::chrono,
the old literals from <drivers/clock.hh> should be removed.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The timestamp typically will come from 'trace.py list' output.  This
change makes it easier to paste the timestamp. Listing prints the time
with '.' separating seconds from nanoseconds.  The options '--since'
and '--until' accept timestamp in nanoseconds so the dot must have
been removed. Now one only needs to append the 's' suffix which
changes the unit to seconds.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This:

  --since 10.0 --period 100ms

is now equivalent to:

  --since 10.0 --until 10.1

This option makes it easier to slice the time based on the output of
'trace.py list-timed' which prints trace timestamp and duration.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Removing 'osv trace summary' and 'osv trace duration' commands from
loader.py. The first is added as 'trace.py summary --timed', the
second one is added as 'trace.py list-timed'.

It was easier to move both in one commit as they are coupled with each
other.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The call-graph tree is already using call-sites, this makes also
`trace.py list -b` also use it.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Prints output like this:

Collected 36843 samples spanning 8.85 s

Tracepoint statistics:

  name              count
  ----              -----
  sched_wait        18420
  sched_wait_ret    18423

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

When output is not fully consumed because of piping, eg to `head`,
we get an ugly exception:

 Traceback (most recent call last):
   File "scripts/trace.py", line 274, in <module>
     args.func(args)
   File "scripts/trace.py", line 227, in list_timed
     bt_formatter(t.backtrace),
 IOError: [Errno 32] Broken pipe

Let's ignore it, it's normal condition.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

 - added command descriptions
 - improved clarity of options
 - extracted argument groups
 - changed command help to start with lower case letter

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

It's a left over which is not used any more.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Our clock::time() function returns the current wall-clock time (number of
nanoseconds since the Unix epoch). We never clearly documented this, so this
patch adds this documentation.

Moreover, in kvmclock, we assumed that the host never adjusts its
wall-clock time, which is not a good assumption because it prevents us
from benefiting from NTP running on the host. As Avi Kivity explains:

   "For the wall clock, you need to sample the wall clock base every time,
    since it can be changed by the host."

So this patch changes kvmclock::time() to sample the wall-clock-at-boot
reported by the paravirtual clock, on every time() call, and not just once.

The downsite of this patch is that clock::get::time(), and therefore
nanotime(), become a bit slower on kvmclock, as they need to sample
the wall_clock_boot() every time, require a couple of barriers and
arithmetic operations. But note that we shouldn't even be using the
non-monotonic clock::get::time() for our timers - we should be using the
monotonic clock::get::uptime() - and when we switch to using that we'll
get back the bit of performance lost in this patch.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

httpserver.so does not build propery for many:

  LINK httpserver
  /usr/bin/ld: cannot find -lboost_system-mt
  /usr/bin/ld: cannot find -lboost_filesystem-mt
  /usr/bin/ld: cannot find -lboost_program_options-mt
  collect2: error: ld returned 1 exit status

drop it from osv.git build until it does.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The httpserver is part of the compilation of the mgmt.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This reverts commit a39cde78. I applied
the broken version.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The httpserver.so is added to the image and will be
placed under /usr/mgmt

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The httpserver is part of the compilation of the mgmt.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

In preparation for adding httpserver.so to mgmt module, update to latest
mgmt.git which fixes unsupported boost_program_options library
dependency to boost_program_options-mt.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Instead of always putting the tests into all images, this patch adds the
option of putting them only in some of the images, by making them into a
new module, modules/tests.

The default image (images/default.py) continues to require the tests
modules, so tests are included in the default image. Building with
"make image=tests" makes an image with only the tests and nothing else.
Other images (e.g., memcached, cassandra,...) currently do not require
the tests module, so the generated image does will not include the tests.

With this patch, "make image=memcached", for example, contains only the
bare minimum needed for memcached, and the resulting qcow image is just
18 MB, down from 69 MB when we included all the tests.

Fixes #160

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This reverts commit d914ee0d because it breaks
the build:

  I'm trying to compile the latest source code from git HEAD and I'm heaving
  the following error:

  make -r -C build/release/ all
  make[1]: Entering directory `/home/eduardo/workspace/osv/build/release'
  make[1]: *** No rule to make target
  `/home/eduardo/workspace/osv/build/release/gen/include/osv/version.h',
  needed by `/home/eduardo/workspace/osv/build.mk'.  Stop.
  make[1]: *** Waiting for unfinished jobs....
    GEN gen/include/bits/alltypes.h
  make[1]: Leaving directory `/home/eduardo/workspace/osv/build/release'
  make: *** [all] Error 2

Reported-by: Eduardo Piva <efpiva@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Our need for "make external" was plagued with problems. First and formost,
it was annoying - people need to be told to run "make all external" instead
of just "make". And as a Vlad recently discovered, it strangely relied on a
debug-mode build to have happened, and also "make clean" doesn't undo what
"make external" did.

"make external" had a glorious past, but today all that was left of it
is 3 measly tests which we compile from external/glibc-testsuite: malloc,
getcwd and clock. These tiny tests - 3 out of 796 tests (!!) in the full
glibc test suite - aren't really important enough to cause all this
"make external" mess. So this patch removes them, and the need for a
"make external".

If we'll ever want to bring back these tests, I think we need to delete the
partial external/glibc-testsuite directory, and bring the *full* glibc
test suite of 796 tests, into a separate module. The glibc test suite can be
downloaded together with the glibc source code - they are all files named
tst-*.c in the glibc source tree.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

After the creation of the "java" and "mgmt" modules, we still left with a
bunch of files in the main usr.manifest.skel which really belong in the
modules/{java,mgmt}/usr.manifest.

This patch moves various shared libraries and fonts which Java needs
(the fonts are needed for AWT, including headless AWT), as well as
various JNI shared objects and our java.so and runjava.jar, into the
"java" module, and moves a couple of tools (ifconfig, lsroute) and
configuration file (/etc/inputrc used by the management module into
the "mgmt" module.

This reduces small OSv appliances, like "make image=memcached", which
don't need Java, by 13 MB. We are still left with 69 MB, most of which
are tests, so the next item on the todo is to move the tests into a
"test" module.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>