Signed-off-by: Benoit Canet <benoit@irqsave.net>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

The initialization is done a this time so everything needed is ready.

Signed-off-by: Benoit Canet <benoit@irqsave.net>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

This patch adds version numbering to the loader. We use "git tag" for
generating an unique version number with a little bit of shell script
magic to make sure version number is always up-to-date.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Useful when a filesystem is not yet available.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

This is patch v2, incorporating most of the comments from the previous round.

Solves issue #47:

elf::program's API - add_object() and remove_object() was problematic in
two respects:

1. It did not do reference-counting on the loaded object, so if add_object()
   was done 5 times, a single remove_object() would unmap the object and
  its 4 other users will crash.

2. It is un-C++-like.

This patch replaces these two functions by a single function get_library():

  std::shared_ptr<elf::object>
   get_library(std::string lib, std::vector<std::string> extra_path = {});

get_library() returns a shared_ptr, which is reference counting and does
proper C++-style RAII. For example in the code:

	auto lib = elf::get_program()->get_library(path);
	auto main = lib->lookup<int (int, char**)>("main");
	int rc = main(argc, argv);

once lib out of scope, the reference count of the elf::object automatically
goes down, and if nobody else holds another shared-pointer to the same
library, the object is destroyed (causing the shared library to be unloaded).

This patch also documents, with Doxygen, all the functions it touches.

IMPORTANT NOTE: This reference count is completely unrelated to the issue
of concurrent use of dlopen()/dlclose()/dl_iterate_phdr(), which is still
buggy and we need to fix (I have a patch for that, but it's too long to fit
in the margin).

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>

Sorry, previous commit left loader.cc with only a copyright statement.
Our copyright is important, but not *that* important!
I'll be more careful next time...

Add copyright statement to the source files in the top-level directory (except the
trivial dummy-shlib.cc).

By the way, I don't think this code really belongs in the top-level directory.

As I have stated previously, what is true for qemu (that we always have
a user-provided network interface) is not true for Xen. It is quite possible
that we boot with no network interface at all. In that case, we will get stuck
asking for an IP that will never come.

This patch takes care to call for dhcp only if our interface is really up. Since
networking is such a core service, we'll print a message if we can't do that.

Some time ago I have moved the console initialization a bit earlier, so
messages could be seen earlier. This has been, however, creating spurious
problems (1 at each 10 - 15 boots) on Xen HVM. The reason is that the isa
serial reset code enables interrupts upon reset, and the isa irq interrupt
will call wake() in the pool thread, which at this point is not yet started.

Since these days we already have simple_write() dealing with the early stuff,
move it back to where it used to be.

P.S: Dima found a way to make this problem 100 % reproduceable, by queueing
data in the input line before the console starts. With this patch, the problem
is gone even if Dima's method is used.

Command line option: --trace-backtraces

It doesn't exist there.

[ glommer: use the alternative system ]

Now that we have everything in place, ask the loader to be able to probe the xenbus.

This adds fairly basic support for rcu.

Declaring:

   mutex mtx;
   rcu_ptr<my_object> my_ptr;

Read-side:

   WITH_LOCK(rcu_read_lock) {
      const my_object* p = my_ptr.read();
      // do things with *p
      // but don't block!
   }

Write-side:

  WITH_LOCK(mtx) {
    my_object* old = my_ptr.read_by_owner();
    my_object* p = new my_object;
    // ...
    my_ptr.assign(p);
    rcu_dispose(old);  // or rcu_defer(some_func, old);
  }

tls is needed for per-cpu storage, so initialize it before the rest of the
scheduler.

Avoid a #include loop with later patches.

the elf is mapped in a 1:1 fashion, so this patch allows addresses
of static variables to be translated as well (needed for next patch).

console:init() calls wake() but the scheduler is not fully initialized at this
point, disable preemption as soon as main_cont starts and disable it after smp_launch()

Now that we can actually see the debug message, print our name on it.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>

We could benefit from the console being ready a bit earlier. The only
dependency that I see to it are the interrupts that need to be working.  So as
soon as we initialize the ioapic, we should be able to initialize the console.

This is not the end of story: we still need an even earlier console to debug the
driver initialization functions, and I was inclined to just leave console_init
where it is, for now.

But additionally, I felt that loader is really a more appropriate place for
that than vfs_init... So I propose we switch. In the mean time, it might help
debug things that happen between ioapic init and the old vfs_init (mem
initialization, smp bring up, etc)

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>

the convention in linux is that argv[0] holds the program executable.
I had an attempt to run netserver not from the CLI and it didn't work because
its argument parsing got broken.

Java doesn't trust pthread_getattr_np() to work for the main thread in
Linux, so instead it relies on a global variable __libc_stack_end, which
we didn't implement and therefore causing an annoying message.

This patch implements __libc_stack_end. Pardoxically, this shouldn't point
to the real stack end (we can easily find this our sched::thread interfaces)
but a little below it, because Java expects the address to be in the
stack, not one byte above it. So we use __builtin_frame_address(0) to
find the current frame's address.

Unfortunately, while this elliminates one warning message, another one
remains - because Java later expects to read /proc/self/maps and doesn't
find it.

Added a "--noshutdown" option to loader which prevents a shutdown
(and a poweroff), after main() returns.

Note that this doesn't just replace poweroff() by halt() - with this
option, the system *isn't* halted after main() returns, but rather
continues as usual - possibly still running other threads that main()
didn't wait for, running various system threads and services, and so on.

Call the new poweroff() function after the payload finishes running.
Makes sense on a cloud (why would you want to pay the provider after
your workload is done?) as well as for our benchmarking, where we want
qemu to exit after running the benchmark.

When the "--leak" option is used, instead of calling poweroff(), we
call hang(), so that QEMU continues to run and we can attach the
debugger to run "osv leak show".

Note that before this patch, if the payload spawned threads, they could
continue running after the payload's main() return. This is no longer
the case - after main() returns, the entire virtual machine is shut down
(or just hung). This is reasonable behavior, though: If the payload needs
some threads to continue running, it should join() them before returning.
The behavior on Linux (and Posix threads in general) is identical to our
new behavior: When main() of a multithreaded program returns, all threads
are killed.

Use the generic one instead; the cleanup function allows destroying
the pthread object.

Somehow this worked by accident, but we need to zero the .tbss section
or random data will be used to initialize it.

fontconfig.so is loaded without a fully qualified path; it expects the
search path to be used.

Added to the loader a command-line option "--leak" to enable the leak
detector immediately before running the payload's main(). For example, to
look for leaks in tst-fpu.so (there are none, by the way ;-)), do

	scripts/imgedit.py setargs build/release/loader.img --leak tests/tst-fpu.so

and when it ends, look at the leak detection results:
	$ gdb build/release/loader.elf
	(gdb) connect
	(gdb) osv leak show

Unfortunately, this doesn't work when the payload is Java - I'm still trying
to figure out why.

A pthread created with the attribute
	pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);
Does not need to be pthread_join()ed - and its resources, most importantly
its stack, should be deleted as soon as it is done.

Previously, we ignored this mode, causing its users (Java uses it for all
its threads!) which do not ever pthread_join() on these threads, to leak
the threads' stacks.

This patch adds support for this mode, and adds a test to tst_leak.cc
which verifies that before the patch, we had a leak, and now we don't.

Unfortunately, this patch is a bit ugly, and I was surprised how much time
it took me to actually get it to work :( Because of the convoluted
relationships between pthread_*, pthread_private::pthread, sched::thread
and sched::detached_thread, I ended up, after many rewrites (!) duplicating
the "reaper" code from sched::detached_thread to do something very similar
for detached pthreads. I.e., when a detached pthread finishes, it can't
pthread_join() itself (which would cause a mess as it destroys its own
stack) so instead it tells a different thread, the "reaper" thread, to
run this pthread_join().

In the future it would be nice to create one generic "reaper" which
is used by both detached sched::threads and pthreads, or even find
a way to allow a thread to pthread_join itself as its last action.

This allocator works by giving each allocation its own virtual address
range which is not reused for later allocations.  After a free(), the
range is made inaccessible, forever, so use-after-free will result in a
page fault.

Sub-page overruns are also detected by filling unallocated space with a
pattern, and checking whether the pattern has been altered during free().

Early code may enable interrupts and get hit by a spurious interrupt.

Without adding this check, if you change the command line with to run
file.so which doesn't exist, it would start using random garbage in
memory and fail in mysterious ways. Better just let the user know we can't
open the specified program.

OSv stalls when using romfs on certain machines so revert back to
using bootfs until romfs get fixed

Edge-triggered interrupts only, at present.

Minor code simplification.

Avoids a zombie.

This allows running either testrunner.so, or one of the tests (e.g.
test/tst-fpu.so), as they now both export the same entry point.