We use sched::thread::attr to pass parameters to sched::thread creation,
i.e., create a thread with non-default stack parameters, pinned to a
particular CPU, or a detached thread.

Previously we had constructors taking many combinations of stack size
(integer), pinned cpu (cpu*) and detached (boolean), and doing "the
right thing". However, this makes the code hard to read (what does
attr(4096) specify?) and the constructors hard to expand with new
parameters.

Replace the attr() constructors with the so-called "named parameter"
idiom: attr now only has a null constructor attr(), and one modifies
it with calls to pin(cpu*), detach(), or stack(size).

For example,
    attr()                                  // default attributes
    attr().pin(sched::cpus[0])              // pin to cpu 0
    attr().stack(4096).pin(sched::cpus[0])  // pin and non-default stack
    and so on.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

It's just to painful to do this piecemeal.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Over the process of creating a zpool, history_str_get() will be eventually
called by zfs_ioc_pool_create(). history_str_get() will not work as expected as
it fully relies on the function copyinstr() which is currently unimplemented.

After that, spa_create() will be called with history_str storing trash, thus
making the spa_history_log filled with completely wrong entries.

Let's fix this problem simply by implementing copyinstr. By the way, copystr
has basically the same semantics as copyinstr, so let's implement it as well.
It's not being used anywhere, but who knows? It might be needed in the future
by some bsd compliant application.

As far as I know, this problem didn't manifest *visibly* anywhere as the spa
history log wasn't intentionally used by us yet. I found the problem while
auditing code.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

bsd defines some m_ macros, for example m_flags, to save some typing.  However
if you have a variable of the same name in another header, for example
m_flags, have fun trying to compile your code.

Expand the code in place and eliminate the macros.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Use four spaces, not tabs for indentation.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

do not need initialize variable. (or use memset(3) for zero clear.)
Because set of values by ioctl(SIOCGIFFLAGS)

```
(snip)
  CC bsd/sys/kern/sys_socket.o
  CC bsd/sys/kern/subr_disk.o
  CC bsd/porting/route.o
  CXX bsd/porting/networking.o
../../bsd/porting/networking.cc: In function ‘int osv::ifup(std::string)’:
 ../../bsd/porting/networking.cc:99:30: error: missing braces around
initializer for ‘char [16]’ [-Werror=missing-braces]
cc1plus: all warnings being treated as errors
make[1]: *** [bsd/porting/networking.o] Error 1
make[1]: Leaving directory
`/home/kouki-o/work/kaishuu0123-osv/build/release'
make: *** [all] Error 2
(snip)
```

Signed-off-by: Kouki Ooyatsu <kaishuu0123@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

This patch adds to support multiple NIC initialization on loader.cc.

Signed-off-by: Takuya ASADA <syuu@dokukino.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Wanted by zfs_ioctl.c.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Strangely, it still works even though it's not initialized.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

Needed by libzfs.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

We have _KERNEL defines scattered throughout the code, which makes
understanding it difficult.

Define it just once, and adjust the source to build.

We define it in an overridable variable, so that non-kernel imported code
can undo it.

Signed-off-by: Benoit Canet <benoit@irqsave.net>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>

bsd's ifconf conflicts with osv's; rename it.

We use the bsd version in <osv/ioctl.h>, since we currently don't support
the Linux-ABI variants of these ioctls.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Tested-By: Benoit Canet <benoit@irqsave.net>

The bsd ifaddr struct conflicts with the osv ifaddr struct, which is a
public interface.  Rename the bsd struct to avoid conflict.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Tested-By: Benoit Canet <benoit@irqsave.net>

All the other fields of the pcpu structure BSD expects are initialized by
the event channel. Except for the cpu id, which the code expects to be already
initialized.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>

Right now we send route messages with MTUs zeroed out. This can lead
to the following assert in ip_output.c (~line 308) triggering:

    KASSERT(mtu > 0, ("%s: mtu %d <= 0, rte=%p (rt_flags=0x%08x) ifp=%p",
        __func__, mtu, rte, (rte != NULL) ? rte->rt_flags : 0, ifp));

This happens because the code will assume that if there is a valid route, that
route will have a valid MTU. And in this case, will always use the route MTU
instead of the interface one.

When we allocate the route it has a valid MTU. But when we send the route
message, we will overwrite it with the value we see in the route message.  This
is done in rtsock.c:rt_setmetrics.

With this patch, those assertion stops happening.

A note: this wasn't been seen in local instalations, only on EC2. Looking at
it, there is nothing Xen specific. The reason it was not happening on local, is
that local traffic does not go through the default route, but rather through
the local 192.168.100.0/24 route. That one seems to take a different
configuration path, and thus sets the MTU correctly.

Added our copyright statements to some of the files in the top bsd/
directory, and in bsd/porting.

I only added our copyright to files which were completely by us - I did
not attempt to hunt which bsd or solaris files we modified to add our
copyright to them, I don't think this is important (or, we can do this
later).

I also found one header file (uma_stub.h) that had large chunks copied from
freebsd, so I added both the freebsd copyright and ours.

Even Lords make brown paper bag mistakes. This is a left over code from my
initial testing, where the buffer where set with pre existing values to make
sure they were going through.  I forgot to remove them. As a result reads were
fine, but writes would just wipe the previous data from the buffer.
Incidentally, the "write-then-read-the-data-back" test I was doing would also
obviously pass, so I haven't noticed this so far.

Fix is to just leave the buffer alone.

msleep() measure times in units of 1/hz seconds. We had hz = 1,000,000,
which gives excellent resolution (microsecond) but a terible range
(limits msleep()'s timeout to 35 minutes).

We had a program (Cassandra) doing poll() with a timeout of 2 hours,
which caused msleep to think we gave a negative timeout.

This patch reduces hz to 1,000, i.e., have msleep() operate in the same units
as poll(). Looking at the code, I don't believe this change will have any
ill-effects - we don't need higher resolution (freebsd code is used to
hz=1,000, which is the default there), and the code converts time units to
hz's correctly, always using the hz macro. The allowed range for timeouts will
grow to over 24 days - and match poll()'s allowed range.

Xen has hard requirements on page transfers, and how to feed the grant tables.
The address need to be page aligned, since the pfns and not addresses are used,
and we need to provide at least a full page per buffer, since the hypervisor is
free to fill any data within the page.

To achieve that, the netfront driver will use m_cljget to attach an extended
buffer to the mbuf, from the jumbop zone, since they are page-sized. However,
two problems arise from this:

1) Allocating a page goes through malloc_large. Our implementation of malloc_large
is currently terribly inefficient, and that creates a very heavy contention site.

What I am doing with this patch is to switch our uma implementation to
alloc_page / free_page instead of malloc if the caller of zcreate so specified
(and then of course, specify it for the jumbop cache)

2) The refcount that is attached in the end of the buffer would either extend the
buffer to 4100 bytes - defeating our purpose, or then the buffer would have to be
PAGE_SIZE - 4, to accomodate for the refcount. But since the hypervisor will write
to the whole page, it will eventually overwrite the refcount.

To address that, I am allocating an external reference counter. BSD already
have some infrastructure to do that, and I am taking advantage of this.
However, I have found no way of implementing this in a way in which the
reference count can be easily deduceable from the address of the extended
buffer, without having the supporting mbuf to start from. Any external data
structure such as hashes would probably make freeing way too slow. Thankfully,
uma_find_refcnt and the UMA_ZONE_REFCNT seems to be used mostly in the
setup/destruction phase (the mbuf refcount is used directly, open coded). So my
proposal here is to remove the UMA_ZONE_REFCNT for that zone.

SIOCAIFADDR appends an address to the interface's address list instead of
replacing it.  This causes 'ifconfig' to display 0.0.0.0 (the first address
configured) instead of the correct address obtained by dhcp.

Fix by also deleting the existing address, if it exists.

Wastes memory, esp. with power-of-two allocations.

Simple implementation of BSD's bus_dma interface. Since we are constrained by virtual
environments, we are able to cut out most of the things.

We won't implement interface media change routines - at least for now, so stub
them.

This contains interrupts, devices and bus definitions. Most of them are is bus
files in BSD anyway.

This is for the lack of a better place.

msleep() will fault if mtx is NULL.

The init and fini functions are fairly expensive (for networking).  Cache
initialized objects in percpu pools to save this cost.

The implementation is imperfect since if we're allocating on one cpu and
freeing on another, reuse is low.  This can be improved in the future, or
made unnecessary with VJ rings.

Increases netperf from ~14.6Gbps to ~17.8Gbps on my machine.

M_ZERO requests zeroing of the object regardless of any constructor; honor it.

It works now because we bzero() all objects unconditionally, but we soon
won't.

We're going to make zones more expensive to allocate, so allocate only
as many as we need.

Allows integrating with our mempools.

M_ZERO requests zeroing of the entire mbuf, which clears the fields initialized
by the init function.  It only works now because we don't honor M_ZERO.

Remove M_ZERO and replace with bzero() for the packet data only.

So after all the BSD code is not buggy, it is just their semantics that is
slightly different from our version of mlock (Thanks Christoph). Whether or not
we will drop the lock will be controlled by the value of the PDROP flag.

Our msleep queues are protected by an internal lock which is not the same lock
the user passed to the msleep call. Therefore, we can just a version of
wait_until that does not take a lock argument, and do the locking manually
ourselves. We may then lock it back or not, depending on the presence of the
PDROP flag.

This provides a simple msleep implementation that simply calls our own, and
an empty MTX_INIT macro

BSD register a structure with its per cpu data. We can do the same, using
just the fields we need.

Xen BSD code will attempt to create processes to serve as deamons for xenstore.
We can basically emulate this by creating threads.

The only thing I am doing differently from the already existent thread creation
layer that we have, is that callers will expect a struct proc to be returned. We
had this as a stub, now I am creating a small struct with just the PID to serve
as a return placeholder. The listener processes in xenstore never dies, so I am
not implementing a deallocate routine for them