zfs: Enable compression on zfs dataset when creating the image

This patch enables LZ4 compression on the ZFS dataset right after its
insertion in the pool. Then the image creation process will go through
all the steps with compression enabled, and when it's done, compression
is disabled. From that moment on, compression stops taking effect, and
files previously compressed will be still supported.

Why disabling compression after image creation?
There seems to be corner-cases where setting compression by default
would affect applications performance.
For example, applications that compress data themselves (e.g. Cassandra)
might end up slower as ZFS would be duplicating the compression process
that was previously done, and consequently wasting CPU cycles.
It's worth mentioning that LZ4 is ~300% faster than LZJB when compressing
'in-compressible' data, so it might be good even for Cassandra.

Additional information: The first version of this patch used the LZJB
algorithm, however, it slowed down read operations on compressed files.
On the other hand, LZ4 improves read on compressed files, improves boot
time, and still provides a good compression ratio.

RESULTS
=====

- UNCOMPRESSED:
* Image size
-rw-r--r--. 1 root root 154533888 May 19 23:02 build/release/usr.img

* Read benchmark
REPORT
-----
Files:    552
Read:    127399kb
Time:    1069.90ms
MBps:    115.90

* Boot time
1)
    ZFS mounted: 426.57ms, (+157.75ms)
2)
    ZFS mounted: 439.13ms, (+156.24ms)

- COMPRESSED (LZ4):
* Image size
-rw-r--r--. 1 root root 81002496 May 19 23:33 build/release/usr.img

* Read benchmark
REPORT
-----
Files:    552
Read:    127399kb
Time:    957.96ms
MBps:    129.44

* Boot time
1)
    ZFS mounted: 414.55ms, (+145.47ms)
2)
    ZFS mounted: 403.72ms, (+142.82ms)

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>

scripts/upload_manifest.py

@@ -166,6 +166,10 @@ def main():
 
     osv.wait()
 
+    # Disable ZFS compression; it stops taking effect from this point on.
+    osv = subprocess.Popen('cd ../..; scripts/run.py -m 512 -c1 -i %s -u -s -e "/zfs.so set compression=off osv"' % image_path, shell = True, stdout=subprocess.PIPE)
+    osv.wait()
+
     depends.write(u'\n\n')
     depends.close()
 

tools/mkfs/mkfs.cc

@@ -41,6 +41,10 @@ void mkfs()
 
     // Create a zfs dataset within the pool named osv.
     run_cmd("/zfs.so", {"zfs", "create", "osv/zfs"});
+
+    // Enable lz4 compression on the created zfs dataset
+    // NOTE: Compression is disabled after image creation.
+    run_cmd("/zfs.so", {"zfs", "set", "compression=lz4", "osv"});
 }
 
 int main(int ac, char** av)