Currently, when we try to run an invalid shared object (e.g, run.py -e aaa)
loader.cc calls abort(). This patch changes it to use osv::poweroff().

This is useful, for example, to measure how much time our boot/poweroff
cycle takes, without running any payload, by doing

	time scripts/run.py -e aaa

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

No reason at all for page_size to be in mmu.hh but huge_page_size in mmu.cc.
Move it, so we can also use huge_page_size outside the mmu.cc scope.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Commit ed808267 used malloc() to allocate
the per-cpu variables area. As Avi pointed out, we need this area to be
aligned like the strictest alignment of any per-cpu variable. The strictest
alignment we need is probably CACHELINE_ALIGNED (64 bytes), but it's easiest
just to require 4096-byte alignment, and this is what the code prior to the
above patch did.

The above commit worked because luckily enough, our malloc() does return
page-aligned memory for large allocations. But it's possible that this will
not be the case in the future. So this patch switches to use aligned_alloc()
instead, explicitly requesting a 4096-byte-aligned block of memory.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Reviewed-by: Pekka Enberg <penberg@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

This patch provides a trivial implementation of two similar functions for
allocating aligned memory blocks: aligned_alloc() (from the C11 standard)
and posix_memaligned() (from POSIX). Memory returned by either function
can be freed with the ordinary free().

This trivial implementation just calls malloc(), and assert()s that it got
the desired alignment, aborting if not. In many cases this is good enough
because malloc() already returns 4096-byte-aligned blocks for large
allocations. In particular we'll use these functions in the next patch for
allocating the large page-aligned per-cpu areas.

If we ever fail on this assertion, we can replace these functions by a
full implementation (see issue #87).

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Reviewed-by: Pekka Enberg <penberg@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Some older version of qemu-nbd cuases error exit with nbd_client.py.
(Look at: https://groups.google.com/d/msg/osv-dev/EW5BtNFNfzs/I33BeFXg2f0J

)
This is because nbd_client.py is sending FLUSH command unconditionally, but it's extended feature, nbd client should check nbd server has the capability to accept FLUSH.
nbd server sends capability flags on negotiation stage, it sends HAS_FLAGS(0x1) and SEND_FLUSH(0x4) when server supports FLUSH.

This patch adds these capability check, and skips to send FLUSH if server doesn't support it.

Signed-off-by: Takuya ASADA <syuu@dokukino.com>
Reviewed-by: Benoît Canet <benoit.canet@irqsave.net>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

This patch reduces the size of the .percpu section 64-fold from about
5 MB to 70 KB, and solves issue #95.

The ".percpu" section is part of the .data section of our executable
(loader-stripped.elf). In our 15 MB executable, roughly 7 MB is text
(code), and 7 MB is data, and out of that, a whopping 5 MB is the
".percpu" section. The executable is read in real mode, and this is
especially slow on Amazon EC2, hence our wish to make the executable
as small as possible.

The percpu section starts with all the PERCPU variables defined in the
program. We have about 70 KB of those, and believe it or not, most of
this 70 KB is just a single variable, the 65K dynamic_percpu_buffer
(see percpu.cc).

But then, we need a copy of these variables for each CPU. The unpatched
code duplicated this 70KB section 64 times in the executable file (!),
and then used these memory locations for up-to-64 cpus. But there is
no reason to duplicate this data in the executable! All we need to do
is to dynamically allocate a copy of this section for each CPU, and
this is what this patch does.

This patch removes about 5 MB from our executable: After this patch,
our loader-stripped.elf is just 9.7 MB, and its data section's size is
just 2.8 MB.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

vop_eperm allows more code reuse (suggested by Glauber Costa)

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Use four spaces for indentation and use UNIX linefeeds.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Show running test case name.  Makes debugging test failures less
painful...

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Use the new pivot_root() functionality to switch to ZFS root filesystem
once OSv is up and running.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Needed by pivot_root() to unmount the initial rootfs.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This adds a simple pivot_root() system call that works on mountpoints
and simply removes 'put_old' from mount list so that VFS doesn't know
about it and adds renames 'new_root' ->m_path to '/'.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Previously, fill only supported small-page-size chunks.  However, it's
possible to avoid calling fill multiple times simply by allowing
variable page sizes.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Commit 0dcf1f8f ("OSv module support") didn't add a dependency to
bootfs.manifest.skel and usr.manifest.skel which causes image not to be
rebuilt if the files are changed.

Fix that up.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Add pthread_kill() stub. Needed by Cassandra when its stopped with
Ctrl-C.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Add the actual test case that was forgotten from commit a9f8092a
("Introduce test for libc locking").

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

They are temporary files and should be ignored.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

I committed the file accidentally.  It's a temporary file that shouldn't
be there.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Spotted by Nadav: libc.threaded field is not set but is used in
several 'if' statements when setting the lock_owner field.

When 'libc.threaded' is false then 'lock_owner' of a FILE is set to a
special value which indicates no locking. This field is initially set
to 0 and the original musl code had a logic which upon creation of the
first thread set it to true and adjusted 'lock_owner' field of all
open files to the value of libc.main_thread.  In OSv we had no such
logic which resulted in no locking of the FILE structure.

This patch fixes the issue by using threaded mode from the very
beginning. We also do not rely anymore on posix thread existence so
that stdlib can be used very early in the boot process without
unexpected behavior. It is used (rightfully or not) for example in
ramdisk_init(). We do not have to hold the pthread id in the
'lock_owner' field because the mutex already tracks the owner and we
can do the check using 'mutex_owned()' function.

This patch also gets rid of a magic value STDIO_SINGLETHREADED, which
is of type pthread_t and was used to disable locking when it was known
to be not necessary. A new field is introduced named 'no_locking'
which serves this purpose.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The idea of the patch is basically described in prevoius post:

https://groups.google.com/d/msg/osv-dev/RL2S3AL9TNE/l4XZJo3-lI0J

Whis this patch, you will be able to install OSv apps into disk image on
"make all" stage.

These apps does not require to exist in OSv repository, you can install
apps which is on any git repository or svn repository, or on local
directory.

You'll need to write a config file to add apps, format of the file is
JSON.

Here's a sample of the file:
{
   "modules":[
      {
	 "name":"osv-mruby",
         "type":"git",
         "path":"https://github.com/syuu1228/osv-mruby.git",
         "branch":"master"
      }
   ]
}

If you add "module" on config file, make all calls script/module.py.

This scripts perform "git clone" to fetch repository to $(out)/module,
and invoke "make module" on each module.

"make module" should outputs bootfs.manifest/usr.manifest on module
directory, the script merge bootfs.manifest.skel/usr.manifest.skel and
module local manifests to single file
$(out)/bootfs.manifest/$(out)/usr.manifest.

Here's app Makefile example:

  https://github.com/syuu1228/osv-mruby/blob/master/Makefile



It have "module" target, and the target builds all binaries and
generates *.manifest.

Signed-off-by: Takuya ASADA <syuu@dokukino.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

It simply tells QEMU not to start OSv at startup.

To continue the execution, it's possible to use either the QEMU monitor
or a GDB remote connection.

Signed-off-by: Raphael S. Carvalho <raphael.scarv@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Make sure stack pointer is 16-byte aligned in fault handler as required
by x86-64 ABI. This is needed for the page fault handler to be able to
use stack for FPU state save/restore.

Spotted by Nadav.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

For an unknown reason, the current calculation of .tls_template_size
yields 0x10 instead of the correct value.  This results in part of the
initial tls block being freed by arch::setup(), and subsequent
corruption.

Fix by switching to the ld SIZEOF() operator.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Use four spaces, not tabs for indentation.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

The -j must have been included by mistake, otherwise make uses unlimited
jobs.

Signed-off-by: David Leifker <dlei...@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

do not need initialize variable. (or use memset(3) for zero clear.)
Because set of values by ioctl(SIOCGIFFLAGS)

```
(snip)
  CC bsd/sys/kern/sys_socket.o
  CC bsd/sys/kern/subr_disk.o
  CC bsd/porting/route.o
  CXX bsd/porting/networking.o
../../bsd/porting/networking.cc: In function ‘int osv::ifup(std::string)’:
 ../../bsd/porting/networking.cc:99:30: error: missing braces around
initializer for ‘char [16]’ [-Werror=missing-braces]
cc1plus: all warnings being treated as errors
make[1]: *** [bsd/porting/networking.o] Error 1
make[1]: Leaving directory
`/home/kouki-o/work/kaishuu0123-osv/build/release'
make: *** [all] Error 2
(snip)
```

Signed-off-by: Kouki Ooyatsu <kaishuu0123@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Current limit of fragments number for IP packet is 16.
This is not enough for packets bigger than 24K on standard MTU.
This patch increases this number up to theoretical maximum.

The problem found during UDP RX performance testing - throughput
dropped to 0 for 32K UDP packets.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

EC2 does not support QCOW2. Force build to create RAW image.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Default image format stays QCOW2 as before.  May be changed via img_format
variable:

  make img_fomat=raw all

This option is required by release-ec2 script as EC2 does not support QCOW2.
Also will be useful for other hypervisors like VMWARE.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Xen requires explicit image format specification.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Fixed the following abort when build was running mkzfs.py.

Done!
eth0: ethernet address: 52:54:0:12:34:56
[I/20 dhcp]: Waiting for IP...
[I/26 dhcp]: Server acknowledged IP for interface eth0
[I/26 dhcp]: Configuring eth0: ip 192.168.122.15 subnet mask 255.255.255.0
gateway 192.168.122.1
Aborted

Signed-off-by: David Leifker <dlei...@gmail.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

In libc.h there was an optimization which disabled locking when there
is only one thread. It was using 'threads_minus_1' field from struct
__libc. In OSv this field was always 0, so there was no locking.

It was causing memory corruption when running apache benchmark against
tomcat in OSv on SMP. This patch makes them go away.

The fix is to always lock as in OSv we always have more than one
thread.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Attempt to reassign IP for lo0 fails and leaves it without an address.

This breaks tomcat:

  SEVERE: StandardServer.await: create[localhost:8005]:
  java.net.BindException: Address not available

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Saves a bit of manual work.

Example:

  gdb$ osv info threads
  0 (0x1de9e08) cpu0x0 waiting    sched::thread::join() at core/sched.cc (SNIP)
	joining on 0xffffc0003e41e010

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Xen guests do not support user mode networking.
Fall back to qemu when hypervisor detection used and no external
networking configured.

This logic is required for build to support Xen hosts properly.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch adds to support multiple NIC initialization on loader.cc.

Signed-off-by: Takuya ASADA <syuu@dokukino.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Dead code, and boost::signals2::signal<> does the job better.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>