bwlp/qemu.git/tests/qemu-iotests/030, branch master

iotests: rebase qemu_io() on top of qemu_tool()

2022-04-25T12:30:27+00:00

Rework qemu_io() to be analogous to qemu_img(); a function that requires a return code of zero by default unless disabled explicitly. Tests that use qemu_io(): 030 040 041 044 055 056 093 124 129 132 136 148 149 151 152 163 165 205 209 219 236 245 248 254 255 257 260 264 280 298 300 302 304 image-fleecing migrate-bitmaps-postcopy-test migrate-bitmaps-test migrate-during-backup migration-permissions Test that use qemu_io_log(): 242 245 255 274 303 307 nbd-reconnect-on-open Copy-pastables for testing/verification: ./check -qcow2 030 040 041 044 055 056 124 129 132 151 152 163 165 209 \ 219 236 242 245 248 254 255 257 260 264 274 \ 280 298 300 302 303 304 307 image-fleecing \ migrate-bitmaps-postcopy-test migrate-bitmaps-test \ migrate-during-backup nbd-reconnect-on-open ./check -raw 093 136 148 migration-permissions ./check -nbd 205 # ./configure configure --disable-gnutls --enable-gcrypt # this ALSO requires passwordless sudo. ./check -luks 149 # Just the tests that were edited in this commit: ./check -qcow2 030 040 242 245 ./check -raw migration-permissions ./check -nbd 205 ./check -luks 149 Signed-off-by: John Snow Message-Id: <20220418211504.943969-8-jsnow@redhat.com> Reviewed-by: Eric Blake Signed-off-by: Hanna Reitz

iotests: Fix status checks

2022-03-29T14:30:55+00:00

An iotest's 'paused' condition is fickle; it will be reported as true whenever the job is drained, for example, or when it is in the process of completing. 030 and 041 contain such checks, we should replace them by checking the job status instead. (As was done for 129 in commit f9a6256b48f29c2816 for the 'busy' condition.) Additionally, when we want to test that a job is paused on error, we might want to give it some time to actually switch to the paused state. Do that by waiting on the corresponding JOB_STATUS_CHANGE event. (But only if they are not already paused; the loops these places are in fetch all VM events, so they may have already fetched that event from the queue.) Signed-off-by: Hanna Reitz Message-Id: <20220324180221.24508-1-hreitz@redhat.com> Reviewed-by: Eric Blake

block/stream: Drain subtree around graph change

2022-03-29T14:30:55+00:00

When the stream block job cuts out the nodes between top and base in stream_prepare(), it does not drain the subtree manually; it fetches the base node, and tries to insert it as the top node's backing node with bdrv_set_backing_hd(). bdrv_set_backing_hd() however will drain, and so the actual base node might change (because the base node is actually not part of the stream job) before the old base node passed to bdrv_set_backing_hd() is installed. This has two implications: First, the stream job does not keep a strong reference to the base node. Therefore, if it is deleted in bdrv_set_backing_hd()'s drain (e.g. because some other block job is drained to finish), we will get a use-after-free. We should keep a strong reference to that node. Second, even with such a strong reference, the problem remains that the base node might change before bdrv_set_backing_hd() actually runs and as a result the wrong base node is installed. Both effects can be seen in 030's TestParallelOps.test_overlapping_5() case, which has five nodes, and simultaneously streams from the middle node to the top node, and commits the middle node down to the base node. As it is, this will sometimes crash, namely when we encounter the above-described use-after-free. Taking a strong reference to the base node, we no longer get a crash, but the resuling block graph is less than ideal: The expected result is obviously that all middle nodes are cut out and the base node is the immediate backing child of the top node. However, if stream_prepare() takes a strong reference to its base node (the middle node), and then the commit job finishes in bdrv_set_backing_hd(), supposedly dropping that middle node, the stream job will just reinstall it again. Therefore, we need to keep the whole subtree drained in stream_prepare(), so that the graph modification it performs is effectively atomic, i.e. that the base node it fetches is still the base node when bdrv_set_backing_hd() sets it as the top node's backing node. Verify this by asserting in said 030's test case that the base node is always the top node's immediate backing child when both jobs are done. Signed-off-by: Hanna Reitz Message-Id: <20220324140907.17192-1-hreitz@redhat.com> Reviewed-by: Eric Blake Acked-by: Vladimir Sementsov-Ogievskiy

iotests/030: Unthrottle parallel jobs in reverse

2021-11-16T08:43:48+00:00

See the comment for why this is necessary. Signed-off-by: Hanna Reitz Message-Id: <20211111120829.81329-11-hreitz@redhat.com> Signed-off-by: Kevin Wolf Message-Id: <20211115145409.176785-11-kwolf@redhat.com> Signed-off-by: Hanna Reitz

block: Clarify error messages pertaining to 'node-name'

2021-03-08T13:56:55+00:00

Some error messages contain ambiguous representations of the 'node-name' parameter. This can be particularly confusing when exchanging QMP messages (C = client, S = server): C: {"execute": "block_resize", "arguments": { "device": "my_file", "size": 26843545600 }} S: {"error": {"class": "GenericError", "desc": "Cannot find device=my_file nor node_name="}} ^^^^^^^^^ This error message suggests one could send a message with a key called 'node_name': C: {"execute": "block_resize", "arguments": { "node_name": "my_file", "size": 26843545600 }} ^^^^^^^^^ but using the underscore is actually incorrect, the parameter should be 'node-name': S: {"error": {"class": "GenericError", "desc": "Parameter 'node_name' is unexpected"}} This behavior was uncovered in bz1651437, but I ended up going down a rabbit hole looking for other areas where this miscommunication might occur and changing those accordingly as well. Fixes: https://bugzilla.redhat.com/1651437 Signed-off-by: Connor Kuehl Message-Id: <20210305151929.1947331-2-ckuehl@redhat.com> Signed-off-by: Kevin Wolf

iotests: 30: drop from auto group (and effectively from make check)

2021-02-05T15:16:13+00:00

I reproduced the following crash fast enough: 0 raise () at /lib64/libc.so.6 1 abort () at /lib64/libc.so.6 2 _nl_load_domain.cold () at /lib64/libc.so.6 3 annobin_assert.c_end () at /lib64/libc.so.6 4 bdrv_reopen_multiple (bs_queue=0x55de75fa9b70, errp=0x0) at ../block.c:3820 5 bdrv_reopen_set_read_only (bs=0x55de760fc020, read_only=true, errp=0x0) at ../block.c:3870 6 stream_clean (job=0x55de75fa9410) at ../block/stream.c:99 7 job_clean (job=0x55de75fa9410) at ../job.c:680 8 job_finalize_single (job=0x55de75fa9410) at ../job.c:696 9 job_txn_apply (job=0x55de75fa9410, fn=0x55de741eee27 ) at ../job.c:158 10 job_do_finalize (job=0x55de75fa9410) at ../job.c:805 11 job_completed_txn_success (job=0x55de75fa9410) at ../job.c:855 12 job_completed (job=0x55de75fa9410) at ../job.c:868 13 job_exit (opaque=0x55de75fa9410) at ../job.c:888 14 aio_bh_call (bh=0x55de76b9b4e0) at ../util/async.c:136 15 aio_bh_poll (ctx=0x55de75bc5300) at ../util/async.c:164 16 aio_dispatch (ctx=0x55de75bc5300) at ../util/aio-posix.c:381 17 aio_ctx_dispatch (source=0x55de75bc5300, callback=0x0, user_data=0x0) at ../util/async.c:306 18 g_main_context_dispatch () at /lib64/libglib-2.0.so.0 19 glib_pollfds_poll () at ../util/main-loop.c:232 20 os_host_main_loop_wait (timeout=0) at ../util/main-loop.c:255 21 main_loop_wait (nonblocking=0) at ../util/main-loop.c:531 22 qemu_main_loop () at ../softmmu/runstate.c:722 23 main (argc=20, argv=0x7ffe218f0268, envp=0x7ffe218f0310) at ../softmmu/main.c:50 (gdb) fr 4 4 bdrv_reopen_multiple (bs_queue=0x55de75fa9b70, errp=0x0) at ../block.c:3820 3820 assert(perm == state->perm); (gdb) list 3815 3816 if (ret == 0) { 3817 uint64_t perm, shared; 3818 3819 bdrv_get_cumulative_perm(state->bs, &perm, &shared); 3820 assert(perm == state->perm); 3821 assert(shared == state->shared_perm); 3822 3823 bdrv_set_perm(state->bs); 3824 } else { (gdb) p perm $1 = 1 (gdb) p state->perm $2 = 0 Then I had 38 successful iterations and another crash: 0 bdrv_check_update_perm (bs=0x5631ac97bc50, q=0x0, new_used_perm=1, new_shared_perm=31, ignore_children=0x0, errp=0x7ffd9d477cf8) at ../block.c:2197 1 bdrv_root_attach_child (child_bs=0x5631ac97bc50, child_name=0x5631aaf6b1f9 "backing", child_class=0x5631ab280ca0 , child_role=8, ctx=0x5631ab757300, perm=1, shared_perm=31, opaque=0x5631abb8c020, errp=0x7ffd9d477cf8) at ../block.c:2642 2 bdrv_attach_child (parent_bs=0x5631abb8c020, child_bs=0x5631ac97bc50, child_name=0x5631aaf6b1f9 "backing", child_class=0x5631ab280ca0 , child_role=8, errp=0x7ffd9d477cf8) at ../block.c:2719 3 bdrv_set_backing_hd (bs=0x5631abb8c020, backing_hd=0x5631ac97bc50, errp=0x7ffd9d477cf8) at ../block.c:2854 4 stream_prepare (job=0x5631ac751eb0) at ../block/stream.c:74 5 job_prepare (job=0x5631ac751eb0) at ../job.c:784 6 job_txn_apply (job=0x5631ac751eb0, fn=0x5631aacb1156 ) at ../job.c:158 7 job_do_finalize (job=0x5631ac751eb0) at ../job.c:801 8 job_completed_txn_success (job=0x5631ac751eb0) at ../job.c:855 9 job_completed (job=0x5631ac751eb0) at ../job.c:868 10 job_exit (opaque=0x5631ac751eb0) at ../job.c:888 11 aio_bh_call (bh=0x7f3d9c007680) at ../util/async.c:136 12 aio_bh_poll (ctx=0x5631ab757300) at ../util/async.c:164 13 aio_dispatch (ctx=0x5631ab757300) at ../util/aio-posix.c:381 14 aio_ctx_dispatch (source=0x5631ab757300, callback=0x0, user_data=0x0) at ../util/async.c:306 15 g_main_context_dispatch () at /lib64/libglib-2.0.so.0 16 glib_pollfds_poll () at ../util/main-loop.c:232 17 os_host_main_loop_wait (timeout=0) at ../util/main-loop.c:255 18 main_loop_wait (nonblocking=0) at ../util/main-loop.c:531 19 qemu_main_loop () at ../softmmu/runstate.c:722 20 main (argc=20, argv=0x7ffd9d478198, envp=0x7ffd9d478240) at ../softmmu/main.c:50 (gdb) list 2192 QLIST_FOREACH(c, &bs->parents, next_parent) { 2193 if (g_slist_find(ignore_children, c)) { 2194 continue; 2195 } 2196 2197 if ((new_used_perm & c->shared_perm) != new_used_perm) { 2198 char *user = bdrv_child_user_desc(c); 2199 char *perm_names = bdrv_perm_names(new_used_perm & ~c->shared_perm); 2200 2201 error_setg(errp, "Conflicts with use by %s as '%s', which does not " (gdb) p c $1 = (BdrvChild *) 0x8585858585858585 Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20210205111021.715240-1-vsementsov@virtuozzo.com Reviewed-by: Eric Blake [PMM: trimmed the part of the commit message referring to as-yet-unapplied patchseries] Signed-off-by: Peter Maydell

block: apply COR-filter to block-stream jobs

2021-01-26T13:36:37+00:00

This patch completes the series with the COR-filter applied to block-stream operations. Adding the filter makes it possible in future implement discarding copied regions in backing files during the block-stream job, to reduce the disk overuse (we need control on permissions). Also, the filter now is smart enough to do copy-on-read with specified base, so we have benefit on guest reads even when doing block-stream of the part of the backing chain. Several iotests are slightly modified due to filter insertion. Signed-off-by: Andrey Shinkevich Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20201216061703.70908-14-vsementsov@virtuozzo.com> Reviewed-by: Max Reitz Signed-off-by: Max Reitz

iotests: 30: prepare to COR filter insertion by stream job

2021-01-26T13:36:37+00:00

test_stream_parallel run parallel stream jobs, intersecting so that top of one is base of another. It's OK now, but it would be a problem if insert the filter, as one job will want to use another job's filter as above_base node. Correct thing to do is move to new interface: "bottom" argument instead of base. This guarantees that jobs don't intersect by their actions. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20201216061703.70908-12-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz

iotests: define group in each iotest

2021-01-20T20:53:22+00:00

We are going to drop group file. Define group in tests as a preparatory step. The patch is generated by cd tests/qemu-iotests grep '^[0-9]\{3\} ' group | while read line; do file=$(awk '{print $1}' <<< "$line"); groups=$(sed -e 's/^... //' <<< "$line"); awk "NR==2{print \"# group: $groups\"}1" $file > tmp; cat tmp > $file; done Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Message-Id: <20210116134424.82867-7-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake

iotests: Skip test_stream_parallel in test 030 when doing "make check"

2020-09-15T09:05:13+00:00

The test_stream_parallel test still occasionally fails in the CI. Thus let's disable it during "make check" for now so that it does not cause trouble during merge tests. We can enable it again once the problem has been resolved. Signed-off-by: Thomas Huth Message-Id: <20200907113824.134788-1-thuth@redhat.com> Signed-off-by: Max Reitz