From 25956af3fe5dd0385ad8017bc768a6afe41e2a74 Mon Sep 17 00:00:00 2001 From: Eric Blake Date: Mon, 6 Jul 2020 15:39:46 -0500 Subject: block: Finish deprecation of 'qemu-img convert -n -o' It's been two releases since we started warning; time to make the combination an error as promised. There was no iotest coverage, so add some. While touching the documentation, tweak another section heading for consistent style. Signed-off-by: Eric Blake Message-Id: <20200706203954.341758-3-eblake@redhat.com> Signed-off-by: Kevin Wolf --- docs/system/deprecated.rst | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) (limited to 'docs/system') diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst index 58a9aeb851..aa9fdc8c53 100644 --- a/docs/system/deprecated.rst +++ b/docs/system/deprecated.rst @@ -427,14 +427,6 @@ kernel in 2018, and has also been dropped from glibc. Related binaries ---------------- -``qemu-img convert -n -o`` (since 4.2.0) -'''''''''''''''''''''''''''''''''''''''' - -All options specified in ``-o`` are image creation options, so -they have no effect when used with ``-n`` to skip image creation. -Silently ignored options can be confusing, so this combination of -options will be made an error in future versions. - Backwards compatibility ----------------------- @@ -540,8 +532,8 @@ spec you can use the ``-cpu rv64gcsu,priv_spec=v1.10.0`` command line argument. Related binaries ---------------- -``qemu-nbd --partition`` (removed in 5.0.0) -''''''''''''''''''''''''''''''''''''''''''' +``qemu-nbd --partition`` (removed in 5.0) +''''''''''''''''''''''''''''''''''''''''' The ``qemu-nbd --partition $digit`` code (also spelled ``-P``) could only handle MBR partitions, and never correctly handled logical @@ -557,6 +549,12 @@ can be rewritten as:: qemu-nbd -t --image-opts driver=raw,offset=1M,size=100M,file.driver=qcow2,file.file.driver=file,file.file.filename=file.qcow2 +``qemu-img convert -n -o`` (removed in 5.1) +''''''''''''''''''''''''''''''''''''''''''' + +All options specified in ``-o`` are image creation options, so +they are now rejected when used with ``-n`` to skip image creation. + Command line options -------------------- -- cgit v1.2.3-55-g7522 From add8200dd14041d059cc376eff91461fadd93ec5 Mon Sep 17 00:00:00 2001 From: Eric Blake Date: Mon, 6 Jul 2020 15:39:50 -0500 Subject: block: Error if backing file fails during creation without -u Back in commit 6e6e55f5 (Jul 2017, v2.10), we tweaked the code to warn if the backing file could not be opened but the user gave a size, unless the user also passes the -u option to bypass the open of the backing file. As one common reason for failure to open the backing file is when there is mismatch in the requested backing format in relation to what the backing file actually contains, we actually want to open the backing file and ensure that it has the right format in as many cases as possible. iotest 301 for qcow demonstrates how detecting explicit format mismatch is useful to prevent the creation of an image that would probe differently than the user requested. Now is the time to finally turn the warning an error, as promised. Note that the original warning was added prior to our documentation of an official deprecation policy (eb22aeca, also Jul 2017), and because the warning didn't mention the word "deprecated", we never actually remembered to document it as such. But the warning has been around long enough that I don't see prolonging it another two releases. Signed-off-by: Eric Blake Message-Id: <20200706203954.341758-7-eblake@redhat.com> Signed-off-by: Kevin Wolf --- block.c | 12 ++---------- docs/system/deprecated.rst | 12 ++++++++++++ tests/qemu-iotests/111.out | 2 +- tests/qemu-iotests/301.out | 13 +++++-------- 4 files changed, 20 insertions(+), 19 deletions(-) (limited to 'docs/system') diff --git a/block.c b/block.c index 98cad57cda..6925e57d7c 100644 --- a/block.c +++ b/block.c @@ -6128,16 +6128,8 @@ void bdrv_img_create(const char *filename, const char *fmt, bs = bdrv_open(full_backing, NULL, backing_options, back_flags, &local_err); g_free(full_backing); - if (!bs && size != -1) { - /* Couldn't open BS, but we have a size, so it's nonfatal */ - warn_reportf_err(local_err, - "Could not verify backing image. " - "This may become an error in future versions.\n"); - local_err = NULL; - } else if (!bs) { - /* Couldn't open bs, do not have size */ - error_append_hint(&local_err, - "Could not open backing image to determine size.\n"); + if (!bs) { + error_append_hint(&local_err, "Could not open backing image.\n"); goto out; } else { if (size == -1) { diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst index aa9fdc8c53..c014e049c3 100644 --- a/docs/system/deprecated.rst +++ b/docs/system/deprecated.rst @@ -555,6 +555,18 @@ can be rewritten as:: All options specified in ``-o`` are image creation options, so they are now rejected when used with ``-n`` to skip image creation. + +``qemu-img create -b bad file $size`` (removed in 5.1) +'''''''''''''''''''''''''''''''''''''''''''''''''''''' + +When creating an image with a backing file that could not be opened, +``qemu-img create`` used to issue a warning about the failure but +proceed with the image creation if an explicit size was provided. +However, as the ``-u`` option exists for this purpose, it is safer to +enforce that any failure to open the backing image (including if the +backing file is missing or an incorrect format was specified) is an +error when ``-u`` is not used. + Command line options -------------------- diff --git a/tests/qemu-iotests/111.out b/tests/qemu-iotests/111.out index 5279c462fc..ba034e5c58 100644 --- a/tests/qemu-iotests/111.out +++ b/tests/qemu-iotests/111.out @@ -1,4 +1,4 @@ QA output created by 111 qemu-img: TEST_DIR/t.IMGFMT: Could not open 'TEST_DIR/t.IMGFMT.inexistent': No such file or directory -Could not open backing image to determine size. +Could not open backing image. *** done diff --git a/tests/qemu-iotests/301.out b/tests/qemu-iotests/301.out index adaf11d42d..281a16d87a 100644 --- a/tests/qemu-iotests/301.out +++ b/tests/qemu-iotests/301.out @@ -17,18 +17,15 @@ backing file: TEST_DIR/t.IMGFMT.base == mismatched command line detection == qemu-img: TEST_DIR/t.IMGFMT: invalid VMDK image descriptor -Could not open backing image to determine size. -qemu-img: warning: Could not verify backing image. This may become an error in future versions. -invalid VMDK image descriptor -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=vmdk +Could not open backing image. +qemu-img: TEST_DIR/t.IMGFMT: invalid VMDK image descriptor +Could not open backing image. qemu-img: TEST_DIR/t.IMGFMT: Image creation needs a size parameter Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=vmdk -qemu-img: warning: Could not verify backing image. This may become an error in future versions. -Unknown driver 'garbage' -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=garbage -qemu-img: TEST_DIR/t.IMGFMT: unrecognized backing format 'garbage' +qemu-img: TEST_DIR/t.IMGFMT: Unknown driver 'garbage' +Could not open backing image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=garbage qemu-img: TEST_DIR/t.IMGFMT: unrecognized backing format 'garbage' image: TEST_DIR/t.IMGFMT -- cgit v1.2.3-55-g7522 From bc5ee6da7122f6fe93ed07241a44315a331487e9 Mon Sep 17 00:00:00 2001 From: Eric Blake Date: Mon, 6 Jul 2020 15:39:51 -0500 Subject: qcow2: Deprecate use of qemu-img amend to change backing file The use of 'qemu-img amend' to change qcow2 backing files is not tested very well. In particular, our implementation has a bug where if a new backing file is provided without a format, then the prior format is blindly reused, even if this results in data corruption, but this is not caught by iotests. There are also situations where amending other options needs access to the original backing file (for example, on a downgrade to a v2 image, knowing whether a v3 zero cluster must be allocated or may be left unallocated depends on knowing whether the backing file already reads as zero), but the command line does not have a nice way to tell us both the backing file to use for opening the image as well as the backing file to install after the operation is complete. Even if we do allow changing the backing file, it is redundant with the existing ability to change backing files via 'qemu-img rebase -u'. It is time to deprecate this support (leaving the existing behavior intact, even if it is buggy), and at a point in the future, require the use of only 'qemu-img rebase' for adjusting backing chain relations, saving 'qemu-img amend' for changes unrelated to the backing chain. Signed-off-by: Eric Blake Message-Id: <20200706203954.341758-8-eblake@redhat.com> Signed-off-by: Kevin Wolf --- block/qcow2.c | 5 +++++ docs/system/deprecated.rst | 12 ++++++++++++ docs/tools/qemu-img.rst | 4 ++++ tests/qemu-iotests/061.out | 1 + tests/qemu-iotests/082.out | 2 ++ 5 files changed, 24 insertions(+) (limited to 'docs/system') diff --git a/block/qcow2.c b/block/qcow2.c index ea33673c55..f3fc2707cd 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -5511,6 +5511,11 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, } if (backing_file || backing_format) { + if (g_strcmp0(backing_file, s->image_backing_file) || + g_strcmp0(backing_format, s->image_backing_format)) { + warn_report("Deprecated use of amend to alter the backing file; " + "use qemu-img rebase instead"); + } ret = qcow2_change_backing_file(bs, backing_file ?: s->image_backing_file, backing_format ?: s->image_backing_format); diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst index c014e049c3..c1f019b9d2 100644 --- a/docs/system/deprecated.rst +++ b/docs/system/deprecated.rst @@ -427,6 +427,18 @@ kernel in 2018, and has also been dropped from glibc. Related binaries ---------------- +qemu-img amend to adjust backing file (since 5.1) +''''''''''''''''''''''''''''''''''''''''''''''''' + +The use of ``qemu-img amend`` to modify the name or format of a qcow2 +backing image is deprecated; this functionality was never fully +documented or tested, and interferes with other amend operations that +need access to the original backing image (such as deciding whether a +v3 zero cluster may be left unallocated when converting to a v2 +image). Rather, any changes to the backing chain should be performed +with ``qemu-img rebase -u`` either before or after the remaining +changes being performed by amend, as appropriate. + Backwards compatibility ----------------------- diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst index e33f5575e3..c35bd64822 100644 --- a/docs/tools/qemu-img.rst +++ b/docs/tools/qemu-img.rst @@ -258,6 +258,10 @@ Command description: Amends the image format specific *OPTIONS* for the image file *FILENAME*. Not all file formats support this operation. + The set of options that can be amended are dependent on the image + format, but note that amending the backing chain relationship should + instead be performed with ``qemu-img rebase``. + --force allows some unsafe operations. Currently for -f luks, it allows to erase the last encryption key, and to overwrite an active encryption key. diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out index b0f8befe30..44e3c624f9 100644 --- a/tests/qemu-iotests/061.out +++ b/tests/qemu-iotests/061.out @@ -370,6 +370,7 @@ wrote 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +qemu-img: warning: Deprecated use of amend to alter the backing file; use qemu-img rebase instead read 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) No errors were found on the image. diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out index f7b3d54b28..a38a26fc57 100644 --- a/tests/qemu-iotests/082.out +++ b/tests/qemu-iotests/082.out @@ -783,10 +783,12 @@ Amend options for 'qcow2': size= - Virtual disk size Testing: amend -f qcow2 -o backing_file=TEST_DIR/t.qcow2,,help TEST_DIR/t.qcow2 +qemu-img: warning: Deprecated use of amend to alter the backing file; use qemu-img rebase instead Testing: rebase -u -b -f qcow2 TEST_DIR/t.qcow2 Testing: amend -f qcow2 -o backing_file=TEST_DIR/t.qcow2,,? TEST_DIR/t.qcow2 +qemu-img: warning: Deprecated use of amend to alter the backing file; use qemu-img rebase instead Testing: rebase -u -b -f qcow2 TEST_DIR/t.qcow2 -- cgit v1.2.3-55-g7522 From d9f059aa6cfccefaffa3532556e966df4a99ece2 Mon Sep 17 00:00:00 2001 From: Eric Blake Date: Mon, 6 Jul 2020 15:39:54 -0500 Subject: qemu-img: Deprecate use of -b without -F Creating an image that requires format probing of the backing image is potentially unsafe (we've had several CVEs over the years based on probes leaking information to the guest on a subsequent boot, although these days tools like libvirt are aware of the issue enough to prevent the worst effects). For example, if our probing algorithm ever changes, or if other tools like libvirt determine a different probe result than we do, then subsequent use of that backing file under a different format will present corrupted data to the guest. Fortunately, the worst effects occur only when the backing image is originally raw, and we at least prevent commit into a probed raw backing file that would change its probed type. Still, it is worth starting a deprecation clock so that future qemu-img can refuse to create backing chains that would rely on probing, to encourage clients to avoid unsafe practices. Most warnings are intentionally emitted from bdrv_img_create() in the block layer, but qemu-img convert uses bdrv_create() which cannot emit its own warning without causing spurious warnings on other code paths. In the end, all command-line image creation or backing file rewriting now performs a check. Furthermore, if we probe a backing file as non-raw, then it is safe to explicitly record that result (rather than relying on future probes); only where we probe a raw image do we care about further warnings to the user when using such an image (for example, commits into a probed-raw backing file are prevented), to help them improve their tooling. But whether or not we make the probe results explicit, we still warn the user to remind them to upgrade their workflow to supply -F always. iotest 114 specifically wants to create an unsafe image for later amendment rather than defaulting to our new default of recording a probed format, so it needs an update. While touching it, expand it to cover all of the various warnings enabled by this patch. iotest 301 also shows a change to qcow messages. Signed-off-by: Eric Blake Message-Id: <20200706203954.341758-11-eblake@redhat.com> Signed-off-by: Kevin Wolf --- block.c | 27 ++++++++++++++++++++++++++- docs/system/deprecated.rst | 20 ++++++++++++++++++++ qemu-img.c | 9 ++++++++- tests/qemu-iotests/114 | 14 ++++++++++++++ tests/qemu-iotests/114.out | 9 +++++++++ tests/qemu-iotests/301.out | 4 +++- 6 files changed, 80 insertions(+), 3 deletions(-) (limited to 'docs/system') diff --git a/block.c b/block.c index 4acfebf0e8..35a372df57 100644 --- a/block.c +++ b/block.c @@ -6139,6 +6139,26 @@ void bdrv_img_create(const char *filename, const char *fmt, error_append_hint(&local_err, "Could not open backing image.\n"); goto out; } else { + if (!backing_fmt) { + warn_report("Deprecated use of backing file without explicit " + "backing format (detected format of %s)", + bs->drv->format_name); + if (bs->drv != &bdrv_raw) { + /* + * A probe of raw deserves the most attention: + * leaving the backing format out of the image + * will ensure bs->probed is set (ensuring we + * don't accidentally commit into the backing + * file), and allow more spots to warn the users + * to fix their toolchain when opening this image + * later. For other images, we can safely record + * the format that we probed. + */ + backing_fmt = bs->drv->format_name; + qemu_opt_set(opts, BLOCK_OPT_BACKING_FMT, backing_fmt, + NULL); + } + } if (size == -1) { /* Opened BS, have no size */ size = bdrv_getlength(bs); @@ -6152,7 +6172,12 @@ void bdrv_img_create(const char *filename, const char *fmt, } bdrv_unref(bs); } - } /* (backing_file && !(flags & BDRV_O_NO_BACKING)) */ + /* (backing_file && !(flags & BDRV_O_NO_BACKING)) */ + } else if (backing_file && !backing_fmt) { + warn_report("Deprecated use of unopened backing file without " + "explicit backing format, use of this image requires " + "potentially unsafe format probing"); + } if (size == -1) { error_setg(errp, "Image creation needs a size parameter"); diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst index c1f019b9d2..971b65be75 100644 --- a/docs/system/deprecated.rst +++ b/docs/system/deprecated.rst @@ -439,6 +439,26 @@ image). Rather, any changes to the backing chain should be performed with ``qemu-img rebase -u`` either before or after the remaining changes being performed by amend, as appropriate. +qemu-img backing file without format (since 5.1) +'''''''''''''''''''''''''''''''''''''''''''''''' + +The use of ``qemu-img create``, ``qemu-img rebase``, or ``qemu-img +convert`` to create or modify an image that depends on a backing file +now recommends that an explicit backing format be provided. This is +for safety: if QEMU probes a different format than what you thought, +the data presented to the guest will be corrupt; similarly, presenting +a raw image to a guest allows a potential security exploit if a future +probe sees a non-raw image based on guest writes. + +To avoid the warning message, or even future refusal to create an +unsafe image, you must pass ``-o backing_fmt=`` (or the shorthand +``-F`` during create) to specify the intended backing format. You may +use ``qemu-img rebase -u`` to retroactively add a backing format to an +existing image. However, be aware that there are already potential +security risks to blindly using ``qemu-img info`` to probe the format +of an untrusted backing image, when deciding what format to add into +an existing image. + Backwards compatibility ----------------------- diff --git a/qemu-img.c b/qemu-img.c index a6df64a949..efb6ca139e 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -2517,6 +2517,13 @@ static int img_convert(int argc, char **argv) goto out; } + if (out_baseimg_param) { + if (!qemu_opt_get(opts, BLOCK_OPT_BACKING_FMT)) { + warn_report("Deprecated use of backing file without explicit " + "backing format"); + } + } + /* Check if compression is supported */ if (s.compressed) { bool encryption = @@ -3797,7 +3804,7 @@ static int img_rebase(int argc, char **argv) * doesn't change when we switch the backing file. */ if (out_baseimg && *out_baseimg) { - ret = bdrv_change_backing_file(bs, out_baseimg, out_basefmt, false); + ret = bdrv_change_backing_file(bs, out_baseimg, out_basefmt, true); } else { ret = bdrv_change_backing_file(bs, NULL, NULL, false); } diff --git a/tests/qemu-iotests/114 b/tests/qemu-iotests/114 index 26104fff6c..5a7b0a4998 100755 --- a/tests/qemu-iotests/114 +++ b/tests/qemu-iotests/114 @@ -39,12 +39,21 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _unsupported_proto vxhs +# At least OpenBSD doesn't seem to have truncate +_supported_os Linux # qcow2.py does not work too well with external data files _unsupported_imgopts data_file +# Intentionally specify backing file without backing format; demonstrate +# the difference in warning messages when backing file could be probed. +# Note that only a non-raw probe result will affect the resulting image. +truncate -s $((64 * 1024 * 1024)) "$TEST_IMG.orig" +_make_test_img -b "$TEST_IMG.orig" 64M TEST_IMG="$TEST_IMG.base" _make_test_img 64M +$QEMU_IMG convert -O qcow2 -B "$TEST_IMG.orig" "$TEST_IMG.orig" "$TEST_IMG" _make_test_img -b "$TEST_IMG.base" 64M +_make_test_img -u -b "$TEST_IMG.base" 64M # Set an invalid backing file format $PYTHON qcow2.py "$TEST_IMG" add-header-ext 0xE2792ACA "foo" @@ -55,6 +64,11 @@ _img_info $QEMU_IO -c "open $TEST_IMG" -c "read 0 4k" 2>&1 | _filter_qemu_io | _filter_testdir $QEMU_IO -c "open -o backing.driver=$IMGFMT $TEST_IMG" -c "read 0 4k" | _filter_qemu_io +# Rebase the image, to show that omitting backing format triggers a warning, +# but probing now lets us use the backing file. +$QEMU_IMG rebase -u -b "$TEST_IMG.base" "$TEST_IMG" +$QEMU_IO -c "open $TEST_IMG" -c "read 0 4k" 2>&1 | _filter_qemu_io | _filter_testdir + # success, all done echo '*** done' rm -f $seq.full diff --git a/tests/qemu-iotests/114.out b/tests/qemu-iotests/114.out index 67adef37a4..0a37d20c82 100644 --- a/tests/qemu-iotests/114.out +++ b/tests/qemu-iotests/114.out @@ -1,5 +1,11 @@ QA output created by 114 +qemu-img: warning: Deprecated use of backing file without explicit backing format (detected format of raw) +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.orig Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864 +qemu-img: warning: Deprecated use of backing file without explicit backing format +qemu-img: warning: Deprecated use of backing file without explicit backing format (detected format of IMGFMT) +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT +qemu-img: warning: Deprecated use of unopened backing file without explicit backing format, use of this image requires potentially unsafe format probing Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base image: TEST_DIR/t.IMGFMT file format: IMGFMT @@ -11,4 +17,7 @@ qemu-io: can't open device TEST_DIR/t.qcow2: Could not open backing file: Unknow no file open, try 'help open' read 4096/4096 bytes at offset 0 4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +qemu-img: warning: Deprecated use of backing file without explicit backing format, use of this image requires potentially unsafe format probing +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) *** done diff --git a/tests/qemu-iotests/301.out b/tests/qemu-iotests/301.out index 281a16d87a..9004dad639 100644 --- a/tests/qemu-iotests/301.out +++ b/tests/qemu-iotests/301.out @@ -2,7 +2,8 @@ QA output created by 301 == qcow backed by qcow == Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=33554432 -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432 backing_file=TEST_DIR/t.IMGFMT.base +qemu-img: warning: Deprecated use of backing file without explicit backing format (detected format of IMGFMT) +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 32 MiB (33554432 bytes) @@ -35,6 +36,7 @@ cluster_size: 512 backing file: TEST_DIR/t.IMGFMT.base == qcow backed by raw == +qemu-img: warning: Deprecated use of backing file without explicit backing format (detected format of raw) Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432 backing_file=TEST_DIR/t.IMGFMT.base image: TEST_DIR/t.IMGFMT file format: IMGFMT -- cgit v1.2.3-55-g7522