From 03d2b412aaf2078425f8472f31c8a9c2340969eb Mon Sep 17 00:00:00 2001 From: Stefan Hajnoczi Date: Tue, 2 Mar 2021 14:27:46 +0000 Subject: qemu-storage-daemon: add --pidfile option Daemons often have a --pidfile option where the pid is written to a file so that scripts can stop the daemon by sending a signal. The pid file also acts as a lock to prevent multiple instances of the daemon from launching for a given pid file. QEMU, qemu-nbd, qemu-ga, virtiofsd, and qemu-pr-helper all support the --pidfile option. Add it to qemu-storage-daemon too. Reported-by: Richard W.M. Jones Signed-off-by: Stefan Hajnoczi Message-Id: <20210302142746.170535-1-stefanha@redhat.com> Reviewed-by: Richard W.M. Jones Signed-off-by: Kevin Wolf --- docs/tools/qemu-storage-daemon.rst | 14 ++++++++++++++ 1 file changed, 14 insertions(+) (limited to 'docs') diff --git a/docs/tools/qemu-storage-daemon.rst b/docs/tools/qemu-storage-daemon.rst index c05b3d3811..6ce85f2f7d 100644 --- a/docs/tools/qemu-storage-daemon.rst +++ b/docs/tools/qemu-storage-daemon.rst @@ -118,6 +118,20 @@ Standard options: List object properties with ``,help``. See the :manpage:`qemu(1)` manual page for a description of the object properties. +.. option:: --pidfile PATH + + is the path to a file where the daemon writes its pid. This allows scripts to + stop the daemon by sending a signal:: + + $ kill -SIGTERM $(: https://gitlab.com/nbdkit/libnbd/-/commit/89113f484effb0e6c322314ba75c1cbe07a04543 Thanks to Daniel P. Berrangé for suggestions on how to get this working. Now let's document it! Reported-by: Richard W.M. Jones Cc: Kevin Wolf Cc: Daniel P. Berrangé Signed-off-by: Stefan Hajnoczi Message-Id: <20210301172728.135331-2-stefanha@redhat.com> Reviewed-by: Daniel P. Berrangé Reviewed-by: Richard W.M. Jones Signed-off-by: Kevin Wolf --- docs/tools/qemu-storage-daemon.rst | 42 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 40 insertions(+), 2 deletions(-) (limited to 'docs') diff --git a/docs/tools/qemu-storage-daemon.rst b/docs/tools/qemu-storage-daemon.rst index 6ce85f2f7d..5714794775 100644 --- a/docs/tools/qemu-storage-daemon.rst +++ b/docs/tools/qemu-storage-daemon.rst @@ -101,10 +101,12 @@ Standard options: .. option:: --nbd-server addr.type=inet,addr.host=,addr.port=[,tls-creds=][,tls-authz=][,max-connections=] --nbd-server addr.type=unix,addr.path=[,tls-creds=][,tls-authz=][,max-connections=] + --nbd-server addr.type=fd,addr.str=[,tls-creds=][,tls-authz=][,max-connections=] is a server for NBD exports. Both TCP and UNIX domain sockets are supported. - TLS encryption can be configured using ``--object`` tls-creds-* and authz-* - secrets (see below). + A listen socket can be provided via file descriptor passing (see Examples + below). TLS encryption can be configured using ``--object`` tls-creds-* and + authz-* secrets (see below). To configure an NBD server on UNIX domain socket path ``/tmp/nbd.sock``:: @@ -141,6 +143,42 @@ QMP commands:: --chardev socket,path=qmp.sock,server=on,wait=off,id=char1 \ --monitor chardev=char1 +Launch the daemon from Python with a QMP monitor socket using file descriptor +passing so there is no need to busy wait for the QMP monitor to become +available:: + + #!/usr/bin/env python3 + import subprocess + import socket + + sock_path = '/var/run/qmp.sock' + + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as listen_sock: + listen_sock.bind(sock_path) + listen_sock.listen() + + fd = listen_sock.fileno() + + subprocess.Popen( + ['qemu-storage-daemon', + '--chardev', f'socket,fd={fd},server=on,id=char1', + '--monitor', 'chardev=char1'], + pass_fds=[fd], + ) + + # listen_sock was automatically closed when leaving the 'with' statement + # body. If the daemon process terminated early then the following connect() + # will fail with "Connection refused" because no process has the listen + # socket open anymore. Launch errors can be detected this way. + + qmp_sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + qmp_sock.connect(sock_path) + ...QMP interaction... + +The same socket spawning approach also works with the ``--nbd-server +addr.type=fd,addr.str=`` and ``--export +type=vhost-user-blk,addr.type=fd,addr.str=`` options. + Export raw image file ``disk.img`` over NBD UNIX domain socket ``nbd.sock``:: $ qemu-storage-daemon \ -- cgit v1.2.3-55-g7522 From e246bf3ddc4d61d03227373fecfdcd4fec3508db Mon Sep 17 00:00:00 2001 From: Stefan Hajnoczi Date: Mon, 1 Mar 2021 17:27:28 +0000 Subject: docs: replace insecure /tmp examples in qsd docs World-writeable directories have security issues. Avoid showing them in the documentation since someone might accidentally use them in situations where they are insecure. There tend to be 3 security problems: 1. Denial of service. An adversary may be able to create the file beforehand, consume all space/inodes, etc to sabotage us. 2. Impersonation. An adversary may be able to create a listen socket and accept incoming connections that were meant for us. 3. Unauthenticated client access. An adversary may be able to connect to us if we did not set the uid/gid and permissions correctly. These can be prevented or mitigated with private /tmp, carefully setting the umask, etc but that requires special action and does not apply to all situations. Just avoid using /tmp in examples. Reported-by: Richard W.M. Jones Reported-by: Daniel P. Berrangé Signed-off-by: Stefan Hajnoczi Message-Id: <20210301172728.135331-3-stefanha@redhat.com> Reviewed-by: Daniel P. Berrangé Reviewed-by: Richard W.M. Jones Signed-off-by: Kevin Wolf --- docs/tools/qemu-storage-daemon.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) (limited to 'docs') diff --git a/docs/tools/qemu-storage-daemon.rst b/docs/tools/qemu-storage-daemon.rst index 5714794775..fe3042d609 100644 --- a/docs/tools/qemu-storage-daemon.rst +++ b/docs/tools/qemu-storage-daemon.rst @@ -69,7 +69,7 @@ Standard options: a description of character device properties. A common character device definition configures a UNIX domain socket:: - --chardev socket,id=char1,path=/tmp/qmp.sock,server=on,wait=off + --chardev socket,id=char1,path=/var/run/qsd-qmp.sock,server=on,wait=off .. option:: --export [type=]nbd,id=,node-name=[,name=][,writable=on|off][,bitmap=] --export [type=]vhost-user-blk,id=,node-name=,addr.type=unix,addr.path=[,writable=on|off][,logical-block-size=][,num-queues=] @@ -108,9 +108,10 @@ Standard options: below). TLS encryption can be configured using ``--object`` tls-creds-* and authz-* secrets (see below). - To configure an NBD server on UNIX domain socket path ``/tmp/nbd.sock``:: + To configure an NBD server on UNIX domain socket path + ``/var/run/qsd-nbd.sock``:: - --nbd-server addr.type=unix,addr.path=/tmp/nbd.sock + --nbd-server addr.type=unix,addr.path=/var/run/qsd-nbd.sock .. option:: --object help --object ,help -- cgit v1.2.3-55-g7522 From 67ae4ace9bce25d37be8dd97630ed336c29d6b72 Mon Sep 17 00:00:00 2001 From: Vladimir Sementsov-Ogievskiy Date: Wed, 24 Feb 2021 13:47:03 +0300 Subject: parallels.txt: fix bitmap L1 table description Actually L1 table entry offset is in 512 bytes sectors. Fix the spec. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20210224104707.88430-3-vsementsov@virtuozzo.com> Reviewed-by: Denis V. Lunev Signed-off-by: Kevin Wolf --- docs/interop/parallels.txt | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) (limited to 'docs') diff --git a/docs/interop/parallels.txt b/docs/interop/parallels.txt index f15bf35bd1..bb3fadf369 100644 --- a/docs/interop/parallels.txt +++ b/docs/interop/parallels.txt @@ -208,21 +208,25 @@ of its data area are: 28 - 31: l1_size The number of entries in the L1 table of the bitmap. - variable: l1_table (8 * l1_size bytes) - L1 offset table (in bytes) + variable: L1 offset table (l1_table), size: 8 * l1_size bytes -A dirty bitmap is stored using a one-level structure for the mapping to host -clusters - an L1 table. +The dirty bitmap described by this feature extension is stored in a set of +clusters inside the Parallels image file. The offsets of these clusters are +saved in the L1 offset table specified by the feature extension. Each L1 table +entry is a 64 bit integer as described below: -Given an offset in bytes into the bitmap data, the offset in bytes into the -image file can be obtained as follows: +Given an offset in bytes into the bitmap data, corresponding L1 entry is - offset = l1_table[offset / cluster_size] + (offset % cluster_size) + l1_table[offset / cluster_size] -If an L1 table entry is 0, the corresponding cluster of the bitmap is assumed -to be zero. +If an L1 table entry is 0, all bits in the corresponding cluster of the bitmap +are assumed to be 0. -If an L1 table entry is 1, the corresponding cluster of the bitmap is assumed -to have all bits set. +If an L1 table entry is 1, all bits in the corresponding cluster of the bitmap +are assumed to be 1. -If an L1 table entry is not 0 or 1, it allocates a cluster from the data area. +If an L1 table entry is not 0 or 1, it contains the corresponding cluster +offset (in 512b sectors). Given an offset in bytes into the bitmap data the +offset in bytes into the image file can be obtained as follows: + + offset = l1_table[offset / cluster_size] * 512 + (offset % cluster_size) -- cgit v1.2.3-55-g7522 From ef809f709de81aef01bbb7403b87cbe2ac7e0c10 Mon Sep 17 00:00:00 2001 From: Kevin Wolf Date: Fri, 5 Mar 2021 10:48:56 +0100 Subject: docs: qsd: Explain --export nbd,name=... default The 'name' option for NBD exports is optional. Add a note that the default for the option is the node name (people could otherwise expect that it's the empty string like for qemu-nbd). Signed-off-by: Kevin Wolf Message-Id: <20210305094856.18964-1-kwolf@redhat.com> Reviewed-by: Max Reitz Reviewed-by: Eric Blake Signed-off-by: Kevin Wolf --- docs/tools/qemu-storage-daemon.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) (limited to 'docs') diff --git a/docs/tools/qemu-storage-daemon.rst b/docs/tools/qemu-storage-daemon.rst index fe3042d609..086493ebb3 100644 --- a/docs/tools/qemu-storage-daemon.rst +++ b/docs/tools/qemu-storage-daemon.rst @@ -80,8 +80,9 @@ Standard options: requests for modifying data (the default is off). The ``nbd`` export type requires ``--nbd-server`` (see below). ``name`` is - the NBD export name. ``bitmap`` is the name of a dirty bitmap reachable from - the block node, so the NBD client can use NBD_OPT_SET_META_CONTEXT with the + the NBD export name (if not specified, it defaults to the given + ``node-name``). ``bitmap`` is the name of a dirty bitmap reachable from the + block node, so the NBD client can use NBD_OPT_SET_META_CONTEXT with the metadata context name "qemu:dirty-bitmap:BITMAP" to inspect the bitmap. The ``vhost-user-blk`` export type takes a vhost-user socket address on which -- cgit v1.2.3-55-g7522