v2: Let's convert everything to std::chrono::timepoint (Avi Kivity)
v3: Use the to_timeptr approach suggested by Nadav Har'El

This test checks the functionality of the utimes support.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Start up shell and management web in parallel to make boot faster.  Note
that we also switch to latest mgmt.git which decouples JRuby and CRaSH
startup.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This adds a simple mmap microbenchmark that can be run on both OSv and
Linux.  The benchmark mmaps memory for various sizes and touches the
mmap'd memory in 4K increments to fault in memory.  The benchmark also
repeats the same tests using MAP_POPULATE for reference.

OSv page faults are slightly slower than Linux on first iteration but
faster on subsequent iterations after host operating system has faulted
in memory for the guest.

I've included full numbers on 2-core Sandy Bridge i7 for a OSv guest,
Linux guest, and Linux host below:

  OSv guest
  ---------

  Iteration 1

       time (seconds)
   MiB demand populate
     1 0.004  0.000
     2 0.000  0.000
     4 0.000  0.000
     8 0.001  0.000
    16 0.003  0.000
    32 0.007  0.000
    64 0.013  0.000
   128 0.024  0.000
   256 0.052  0.001
   512 0.229  0.002
  1024 0.587  0.005

  Iteration 2

       time (seconds)
   MiB demand populate
     1 0.001  0.000
     2 0.000  0.000
     4 0.000  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.004  0.000
    64 0.010  0.000
   128 0.019  0.001
   256 0.036  0.001
   512 0.069  0.002
  1024 0.137  0.005

  Iteration 3

       time (seconds)
   MiB demand populate
     1 0.001  0.000
     2 0.000  0.000
     4 0.000  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.005  0.000
    64 0.010  0.000
   128 0.020  0.000
   256 0.039  0.001
   512 0.087  0.002
  1024 0.138  0.005

  Iteration 4

       time (seconds)
   MiB demand populate
     1 0.001  0.000
     2 0.000  0.000
     4 0.000  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.004  0.000
    64 0.012  0.000
   128 0.025  0.001
   256 0.040  0.001
   512 0.082  0.002
  1024 0.138  0.005

  Iteration 5

       time (seconds)
   MiB demand populate
     1 0.001  0.000
     2 0.000  0.000
     4 0.000  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.004  0.000
    64 0.012  0.000
   128 0.028  0.001
   256 0.040  0.001
   512 0.082  0.002
  1024 0.166  0.005

  Linux guest
  -----------

  Iteration 1

       time (seconds)
   MiB demand populate
     1 0.001  0.000
     2 0.001  0.000
     4 0.002  0.000
     8 0.003  0.000
    16 0.005  0.000
    32 0.008  0.000
    64 0.015  0.000
   128 0.151  0.001
   256 0.090  0.001
   512 0.266  0.003
  1024 0.401  0.006

  Iteration 2

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.000  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.005  0.000
    64 0.009  0.000
   128 0.019  0.001
   256 0.037  0.001
   512 0.072  0.003
  1024 0.144  0.006

  Iteration 3

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.001  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.005  0.000
    64 0.010  0.000
   128 0.019  0.001
   256 0.037  0.001
   512 0.072  0.003
  1024 0.143  0.006

  Iteration 4

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.001  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.003  0.000
    32 0.005  0.000
    64 0.010  0.000
   128 0.020  0.001
   256 0.038  0.001
   512 0.073  0.003
  1024 0.143  0.006

  Iteration 5

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.001  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.003  0.000
    32 0.005  0.000
    64 0.010  0.000
   128 0.020  0.001
   256 0.037  0.001
   512 0.072  0.003
  1024 0.144  0.006

  Linux host
  ----------

  Iteration 1

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.001  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.005  0.000
    64 0.009  0.000
   128 0.019  0.001
   256 0.035  0.001
   512 0.152  0.003
  1024 0.286  0.011

  Iteration 2

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.000  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.004  0.000
    64 0.010  0.000
   128 0.018  0.001
   256 0.035  0.001
   512 0.192  0.003
  1024 0.334  0.011

  Iteration 3

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.000  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.004  0.000
    64 0.010  0.000
   128 0.018  0.001
   256 0.035  0.001
   512 0.194  0.003
  1024 0.329  0.011

  Iteration 4

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.000  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.004  0.000
    64 0.010  0.000
   128 0.018  0.001
   256 0.036  0.001
   512 0.138  0.003
  1024 0.341  0.011

  Iteration 5

       time (seconds)
   MiB demand populate
     1 0.000  0.000
     2 0.000  0.000
     4 0.001  0.000
     8 0.001  0.000
    16 0.002  0.000
    32 0.004  0.000
    64 0.010  0.000
   128 0.018  0.001
   256 0.035  0.001
   512 0.135  0.002
  1024 0.324  0.011

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The test creates and destroys threads, each of which creates a random number
of connections, each transferring a random number of bytes to an echo server.

This is used to stress the tcp/ip stack.

The test is portable, and builds on the host with the command

  g++ -O2  -g3 -pthread -std=gnu++11 -lboost_program_options -lboost_system tests/tst-tcp.cc

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Commit 0dcf1f8f ("OSv module support") didn't add a dependency to
bootfs.manifest.skel and usr.manifest.skel which causes image not to be
rebuilt if the files are changed.

Fix that up.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The idea of the patch is basically described in prevoius post:

https://groups.google.com/d/msg/osv-dev/RL2S3AL9TNE/l4XZJo3-lI0J

Whis this patch, you will be able to install OSv apps into disk image on
"make all" stage.

These apps does not require to exist in OSv repository, you can install
apps which is on any git repository or svn repository, or on local
directory.

You'll need to write a config file to add apps, format of the file is
JSON.

Here's a sample of the file:
{
   "modules":[
      {
	 "name":"osv-mruby",
         "type":"git",
         "path":"https://github.com/syuu1228/osv-mruby.git",
         "branch":"master"
      }
   ]
}

If you add "module" on config file, make all calls script/module.py.

This scripts perform "git clone" to fetch repository to $(out)/module,
and invoke "make module" on each module.

"make module" should outputs bootfs.manifest/usr.manifest on module
directory, the script merge bootfs.manifest.skel/usr.manifest.skel and
module local manifests to single file
$(out)/bootfs.manifest/$(out)/usr.manifest.

Here's app Makefile example:

  https://github.com/syuu1228/osv-mruby/blob/master/Makefile



It have "module" target, and the target builds all binaries and
generates *.manifest.

Signed-off-by: Takuya ASADA <syuu@dokukino.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Default image format stays QCOW2 as before.  May be changed via img_format
variable:

  make img_fomat=raw all

This option is required by release-ec2 script as EC2 does not support QCOW2.
Also will be useful for other hypervisors like VMWARE.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Dead code, and boost::signals2::signal<> does the job better.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Make use of the python nbd client.

Signed-off-by: Benoit Canet <benoit@irqsave.net>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Consolidates generated files, and makes it easier to maintain the eclipse
index.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

I dismissed the idea to configure the bridge in default JVM
configuration because having a default like this would force
all apps to use particular slf4j-api version.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>

This patch simplifies the host-side work in the new /usr zfs filesystem
build process.

Previously, we copied the files to a temporary directory, used "cpio"
to archive them and sent its output to the guest with "netcat".

With this patch, we no longer have a temporary directory, and do not
need either cpio or netcat on the build machine.

Rather, mkzfs.py itself, using python (instead of a separate "nc" process),
connects to the guest and sends it the files - still using the CPIO format.

Rather than arbitrarily sleep for 3 seconds before the host tries to
connect to the guest (which might not be enough for some, or a waste
of time for others), with this patch the host looks at the guest's output
and connects when it sees the message "Waiting for connection".

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>

The tst-zfs-simple.so test case has serverd its purpose for bringup.  As
OSv build relies on working ZFS now, there's no need to run the tests.

Furthermore, we have the full ztest stress test in the tree:

  bsd/cddl/contrib/opensolaris/cmd/ztest/ztest.c

which we can use if needed.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

The call to namei("/dir/file/") currently fails with ENOENT when
"/dir/file" exists. A more standard way is to return ENOTDIR
instead. This way calls to stat, open, rename, etc. will be in
line with the POSIX spec.

It is also useful to rename() implementation which needs to
differentiate behaviour between the case in which target does not
exist and the case in which it does but the path has trailing slash
and the last component is not a directory.

In addition to that the check was performed in an inconsistent matter
- only when dentry lookup failed. This change makes the check
performed always.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>

When converting a source file from .c to .cc, make will complain that the
old source file was missing.  This is annoying, especially when bisecting
or compile-testing a patch set.

Fix by removing the source file dependency.  Since the dependency is alread
specified explicitly by the makefile, no information is lost.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Add 'werror' option to Makefile that can be used to disable the -Werror
GCC option temporarily as follows:

  make werror=0

This is useful when fixing up build breakage caused by addition of new
default compiler warnings.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Some distributions have enabled gcc -Wformat-security by default.
This causes the build to raise format warnings even though -Wformat=0
is used.

Setting no-format-security for build consistency between distros.

Signed-off-by: Or Cohen <orc@fewbytes.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Benoit Canet <benoit@irqsave.net>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Benoit Canet <benoit@irqsave.net>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

For example, assuming the following file exists:

  /tmp/dir/x

..the following was sondiered absent:

  /tmp/dir/

..this was considered present:

  /tmp/dir

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>

Needed to link with boost_filesystem.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>

This patch adds version numbering to the loader. We use "git tag" for
generating an unique version number with a little bit of shell script
magic to make sure version number is always up-to-date.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

strlcpy() expects buffer length not string length.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>

Not supported by older qemus.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Drop sudo and global /zfs temporary directory.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

This drops the superuser requirement from the build process, and reduces
the build prerequisite list.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

This allows using the boost libraries in shared objects.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Dump registers on general protection fault for debugging purposes.  Even
if you have gdb available, getting to the exception frame is not always
possible after OSv has crashed.

Example output looks as follows:

registers:
RIP: 0x0000100000b7e913  RFL: 0x0000000000010202  CS:  0x0000000000000008  SS:  0x0000000000000010
RAX: 0xffffc000418ed278  RBX: 0xffffc00041b2c050  RCX: 0x0000000000000004  RDX: 0x0000000000000000
RSI: 0x0000000000000001  RDI: 0x43e0000000000000  RBP: 0x0000200008548d10  R8:  0xffffc000426e3010
R9:  0x0000000000000004  R10: 0x43e0000000000000  R11: 0xffffc00041b2c050  R12: 0xffffc000418ed1e8
R13: 0x0000000000000004  R14: 0x43e0000000000000  R15: 0xffffc00041b2c050  RSP: 0x0000200008548aa0
general protection fault

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Create a file system on top of the pool.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

For the moment, it only creates a zpool, not a filesystem.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Related changes previously made in java/build.xml
were reverted and moved to java/runjava/build.xml

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>

Building with ant requires no changes in the build scripts
when new classes are added. It is a standrd tool to build
java programs.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>