Skip to content
Snippets Groups Projects
Commit 8ebb1693 authored by Nadav Har'El's avatar Nadav Har'El
Browse files

Fix hang in virtio_driver::wait_for_queue

virtio_driver::wait_for_queue() would often hang in a memcached and
mc_benchmark workload, waiting forever for received packets although
these *do* arrive.

As part of the virtio protocol, we need to set the host notification
flag (we call this, somewhat confusingly, queue->enable_interrupts())
and then check if there's anything in the queue, and if not, wait
for the interrupt.

This order is important: If we check the queue and only then set the
notification flag, and data came in between those, the check will be
empty and an interrupt never sent - and we can wait indefinitely for
data that has already arrived.

We did this in the right order, but the host code, running on a
different CPU, might see memory accesses in a different order!
We need a memory fence to ensure that the same order is also seen
on other processors.

This patch adds a memory fence to the end of the enable_interrupts()
function itself, so we can continue to use it as before in
wait_for_queue(). Note that we do *not* add a memory fence to
disable_interrupts() - because no current use (and no expected use)
cares about the ordering of disable_interrupts() vs other memory
accesses.
parent fd28e12d
No related branches found
No related tags found
No related merge requests found
...@@ -90,6 +90,7 @@ namespace virtio { ...@@ -90,6 +90,7 @@ namespace virtio {
trace_virtio_enable_interrupts(this); trace_virtio_enable_interrupts(this);
_avail->enable_interrupt(); _avail->enable_interrupt();
set_used_event(_used_ring_host_head, std::memory_order_relaxed); set_used_event(_used_ring_host_head, std::memory_order_relaxed);
std::atomic_thread_fence(std::memory_order_seq_cst);
} }
bool bool
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment