Loop vs. NBD benchmark =============================================================================== The benchmark compares the performance of the loop block device driver with the network block device driver using qemu-nbd version 4.0.0 and Linux kernel version 5.1.2. The performance was determined by the data throughput of write and read operations to preallocated disk image files (raw, qcow2) using either the loop device or the nbd driver. The results were mesured on an Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz with SSD storage. =============================================================================== Preparation: ~$ qemu-img create -f raw -o preallocation=full nbd-raw_test.img 100M ~$ qemu-img create -f qcow2 -o preallocation=full nbd-qcow2_test.qcow2 100M ~$ qemu-img create -f raw -o preallocation=full loop-raw_test.img 100M ~$ modprobe nbd ~$ qemu-nbd -c /dev/nbd0 -f raw nbd-raw_test.img ~$ qemu-nbd -c /dev/nbd1 -f qcow2 nbd-qcow2_test.qcow2 ~$ modprobe loop ~$ losetup /dev/loop0 loop-raw_test.img =============================================================================== Write perfromance results: ~$ dd if=/dev/zero of=/dev/loop0 bs=100M count=1 oflag=direct 1+0 records in 1+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.0942829 s, 1.1 GB/s ~$ dd if=/dev/zero of=/dev/nbd0 bs=100M count=1 oflag=direct 1+0 records in 1+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.50101 s, 69.9 MB/s ~$ dd if=/dev/zero of=/dev/nbd1 bs=100M count=1 oflag=direct 1+0 records in 1+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.02387 s, 51.8 MB/s ------------------------------------------------------------------------------- ~$ dd if=/dev/zero of=/dev/loop0 bs=4096 count=1000 oflag=direct 1000+0 records in 1000+0 records out 4096000 bytes (4.1 MB, 3.9 MiB) copied, 0.0431999 s, 94.8 MB/s ~$ dd if=/dev/zero of=/dev/nbd0 bs=4096 count=1000 oflag=direct 1000+0 records in 1000+0 records out 4096000 bytes (4.1 MB, 3.9 MiB) copied, 0.937009 s, 4.4 MB/s ~$ dd if=/dev/zero of=/dev/nbd1 bs=4096 count=1000 oflag=direct 1000+0 records in 1000+0 records out 4096000 bytes (4.1 MB, 3.9 MiB) copied, 0.927286 s, 4.4 MB/s ------------------------------------------------------------------------------- ~$ dd if=/dev/zero of=/dev/loop0 bs=512 count=1000 oflag=direct 1000+0 records in 1000+0 records out 512000 bytes (512 kB, 500 KiB) copied, 0.0328669 s, 15.6 MB/s ~$ dd if=/dev/zero of=/dev/nbd0 bs=512 count=1000 oflag=direct 1000+0 records in 1000+0 records out 512000 bytes (512 kB, 500 KiB) copied, 0.877605 s, 583 kB/s ~$ dd if=/dev/zero of=/dev/nbd1 bs=512 count=1000 oflag=direct 1000+0 records in 1000+0 records out 512000 bytes (512 kB, 500 KiB) copied, 0.887766 s, 577 kB/s ------------------------------------------------------------------------------- ~$ dd if=/dev/zero of=/dev/loop0 bs=100M count=1 oflag=dsync 1+0 records in 1+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.480756 s, 218 MB/s ~$ dd if=/dev/zero of=/dev/nbd0 bs=100M count=1 oflag=dsync 1+0 records in 1+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.14004 s, 92.0 MB/s ~$ dd if=/dev/zero of=/dev/nbd1 bs=100M count=1 oflag=dsync 1+0 records in 1+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.77562 s, 59.1 MB/s ------------------------------------------------------------------------------- ~$ dd if=/dev/zero of=/dev/loop0 bs=4096 count=1000 oflag=dsync 1000+0 records in 1000+0 records out 4096000 bytes (4.1 MB, 3.9 MiB) copied, 1.1858 s, 3.5 MB/s ~$ dd if=/dev/zero of=/dev/nbd0 bs=4096 count=1000 oflag=dsync 1000+0 records in 1000+0 records out 4096000 bytes (4.1 MB, 3.9 MiB) copied, 1.0901 s, 3.8 MB/s ~$ dd if=/dev/zero of=/dev/nbd1 bs=4096 count=1000 oflag=dsync 1000+0 records in 1000+0 records out 4096000 bytes (4.1 MB, 3.9 MiB) copied, 0.997281 s, 4.1 MB/s ------------------------------------------------------------------------------- ~$ dd if=/dev/zero of=/dev/loop0 bs=512 count=1000 oflag=dsync 1000+0 records in 1000+0 records out 512000 bytes (512 kB, 500 KiB) copied, 1.27318 s, 402 kB/s ~$ dd if=/dev/zero of=/dev/nbd0 bs=512 count=1000 oflag=dsync 1000+0 records in 1000+0 records out 512000 bytes (512 kB, 500 KiB) copied, 0.88749 s, 577 kB/ ~$ dd if=/dev/zero of=/dev/nbd1 bs=512 count=1000 oflag=dsync 1000+0 records in 1000+0 records out 512000 bytes (512 kB, 500 KiB) copied, 0.934727 s, 548 kB/s =============================================================================== Read performance results: ~$ dd of=/dev/null if=/dev/loop0 bs=100M count=1 iflag=dsync 1+0 records in 1+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.0727356 s, 1.4 GB/s ~$ dd of=/dev/null if=/dev/nbd0 bs=100M count=1 iflag=dsync 1+0 records in 1+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.0378285 s, 2.8 GB/s ~$ dd of=/dev/null if=/dev/nbd1 bs=100M count=1 iflag=dsync 1+0 records in 1+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.0388594 s, 2.7 GB/s ------------------------------------------------------------------------------- ~$ dd of=/dev/null if=/dev/loop0 bs=4096 count=1000 iflag=dsync 1000+0 records in 1000+0 records out 4096000 bytes (4.1 MB, 3.9 MiB) copied, 0.0173998 s, 235 MB/s ~$ dd of=/dev/null if=/dev/nbd0 bs=4096 count=1000 iflag=dsync 1000+0 records in 1000+0 records out 4096000 bytes (4.1 MB, 3.9 MiB) copied, 0.00280802 s, 1.5 GB/s ~$ dd of=/dev/null if=/dev/nbd1 bs=4096 count=1000 iflag=dsync 1000+0 records in 1000+0 records out 4096000 bytes (4.1 MB, 3.9 MiB) copied, 0.00285353 s, 1.4 GB/s ------------------------------------------------------------------------------- ~$ dd of=/dev/null if=/dev/loop0 bs=512 count=1000 iflag=dsync 1000+0 records in 1000+0 records out 512000 bytes (512 kB, 500 KiB) copied, 0.0177883 s, 28.8 MB/s ~$ dd of=/dev/null if=/dev/nbd0 bs=512 count=1000 iflag=dsync 1000+0 records in 1000+0 records out 512000 bytes (512 kB, 500 KiB) copied, 0.00234241 s, 219 MB/s ~$ dd of=/dev/null if=/dev/nbd1 bs=512 count=1000 iflag=dsync 1000+0 records in 1000+0 records out 512000 bytes (512 kB, 500 KiB) copied, 0.00245659 s, 208 MB/s ------------------------------------------------------------------------------- ~$ dd of=/dev/null if=/dev/loop0 bs=100M count=1 iflag=direct 1+0 records in 1+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.0402695 s, 2.6 GB/s ~$ dd of=/dev/null if=/dev/nbd0 bs=100M count=1 iflag=direct 1+0 records in 1+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.0350349 s, 3.0 GB/s ~$ dd of=/dev/null if=/dev/nbd1 bs=100M count=1 iflag=direct 1+0 records in 1+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.0363636 s, 2.9 GB/s ------------------------------------------------------------------------------- ~$ dd of=/dev/null if=/dev/loop0 bs=4096 count=1000 iflag=direct 1000+0 records in 1000+0 records out 4096000 bytes (4.1 MB, 3.9 MiB) copied, 0.0296045 s, 138 MB/s ~$ dd of=/dev/null if=/dev/nbd0 bs=4096 count=1000 iflag=direct 1000+0 records in 1000+0 records out 4096000 bytes (4.1 MB, 3.9 MiB) copied, 0.0695273 s, 58.9 MB/s ~$ dd of=/dev/null if=/dev/nbd1 bs=4096 count=1000 iflag=direct 1000+0 records in 1000+0 records out 4096000 bytes (4.1 MB, 3.9 MiB) copied, 0.0696438 s, 58.8 MB/s ------------------------------------------------------------------------------- ~$ dd of=/dev/null if=/dev/loop0 bs=512 count=1000 iflag=direct 1000+0 records in 1000+0 records out 512000 bytes (512 kB, 500 KiB) copied, 0.0288066 s, 17.8 MB/s ~$ dd of=/dev/null if=/dev/nbd0 bs=512 count=1000 iflag=direct 1000+0 records in 1000+0 records out 512000 bytes (512 kB, 500 KiB) copied, 0.0452972 s, 11.3 MB/s ~$ dd of=/dev/null if=/dev/nbd1 bs=512 count=1000 iflag=direct 1000+0 records in 1000+0 records out 512000 bytes (512 kB, 500 KiB) copied, 0.0485223 s, 10.6 MB/s