summaryrefslogtreecommitdiffstats
path: root/tests/qemu-iotests
Commit message (Collapse)AuthorAgeFilesLines
...
* iotests/118: Drop 'change' testMax Reitz2021-01-262-21/+3Star
| | | | | | | | | | | Commit 0afec75734331 removed the 'change' QMP command, so we can no longer test it in 118. Fixes: 0afec75734331a0b52fa3aa4235220eda8c7846f ('qmp: remove deprecated "change" command') Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210126104833.57026-1-mreitz@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
* iotests: Add test for the regression fixed in c8bf9a9169Alberto Garcia2021-01-263-0/+134
| | | | | | | | | Signed-off-by: Alberto Garcia <berto@igalia.com> Suggested-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com> Message-Id: <20210112170540.2912-1-berto@igalia.com> [mreitz: Add "# group:" line] Signed-off-by: Max Reitz <mreitz@redhat.com>
* iotests: 257: prepare for backup over block-copyVladimir Sementsov-Ogievskiy2021-01-262-153/+154
| | | | | | | | | | | | | | Iotest 257 dumps a lot of in-progress information of backup job, such as offset and bitmap dirtiness. Further commit will move backup to be one block-copy call, which will introduce async parallel requests instead of plain cluster-by-cluster copying. To keep things deterministic, allow only one worker (only one copy request at a time) for this test. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210116214705.822267-15-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* iotests: 219: prepare for backup over block-copyVladimir Sementsov-Ogievskiy2021-01-261-6/+7
| | | | | | | | | | | | | The further change of moving backup to be a one block-copy call will make copying chunk-size and cluster-size two separate things. So, even with 64k cluster sized qcow2 image, default chunk would be 1M. Test 219 depends on specified chunk-size. Update it for explicit chunk-size for backup as for mirror. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210116214705.822267-14-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* iotests: 185: prepare for backup over block-copyVladimir Sementsov-Ogievskiy2021-01-262-2/+4
| | | | | | | | | | | | | | | | | | | The further change of moving backup to be a one block-copy call will make copying chunk-size and cluster-size two separate things. So, even with 64k cluster sized qcow2 image, default chunk would be 1M. 185 test however assumes, that with speed limited to 64K, one iteration would result in offset=64K. It will change, as first iteration would result in offset=1M independently of speed. So, let's explicitly specify, what test wants: set max-chunk to 64K, so that one iteration is 64K. Note, that we don't need to limit max-workers, as block-copy rate limiter will handle the situation and wouldn't start new workers when speed limit is obviously reached. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210116214705.822267-13-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* iotests/129: Limit backup's max-chunk/max-workersMax Reitz2021-01-261-1/+6
| | | | | | | | | | | Right now, this does not change anything, because backup ignores max-chunk and max-workers. However, as soon as backup is switched over to block-copy for the background copying process, we will need it to keep 129 passing. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210120102043.28346-1-mreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* iotests: 56: prepare for backup over block-copyVladimir Sementsov-Ogievskiy2021-01-261-2/+7
| | | | | | | | | | | | After introducing parallel async copy requests instead of plain cluster-by-cluster copying loop, we'll have to wait for paused status, as we need to wait for several parallel request. So, let's gently wait instead of just asserting that job already paused. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210116214705.822267-12-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* job: call job_enter from job_pauseVladimir Sementsov-Ogievskiy2021-01-261-0/+24
| | | | | | | | | | | | | | | | | | | | If main job coroutine called job_yield (while some background process is in progress), we should give it a chance to call job_pause_point(). It will be used in backup, when moved on async block-copy. Note, that job_user_pause is not enough: we want to handle child_job_drained_begin() as well, which call job_pause(). Still, if job is already in job_do_yield() in job_pause_point() we should not enter it. iotest 109 output is modified: on stop we do bdrv_drain_all() which now triggers job pause immediately (and pause after ready is standby). Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20210116214705.822267-10-vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* iotests/300: Clean up pylint and mypy complaintsMax Reitz2021-01-262-4/+16
| | | | | | | | | And consequentially drop it from 297's skip list. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210118105720.14824-11-mreitz@redhat.com>
* iotests/129: Clean up pylint and mypy complaintsMax Reitz2021-01-262-3/+3
| | | | | | | | | And consequentially drop it from 297's skip list. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20210118105720.14824-10-mreitz@redhat.com>
* iotests/129: Limit mirror job's buffer sizeMax Reitz2021-01-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | Issuing 'stop' on the VM drains all nodes. If the mirror job has many large requests in flight, this may lead to significant I/O that looks a bit like 'stop' would make the job try to complete (which is what 129 should verify not to happen). We can limit the I/O in flight by limiting the buffer size, so mirror will make very little progress during the 'stop' drain. (We do not need to do anything about commit, which has a buffer size of 512 kB by default; or backup, which goes cluster by cluster. Once we have asynchronous requests for backup, that will change, but then we can fine-tune the backup job to only perform a single request on a very small chunk, too.) Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210118105720.14824-9-mreitz@redhat.com>
* iotests/129: Actually test a commit jobMax Reitz2021-01-261-2/+25
| | | | | | | | | | | | | | | | Before this patch, test_block_commit() performs an active commit, which under the hood is a mirror job. If we want to test various different block jobs, we should perhaps run an actual commit job instead. Doing so requires adding an overlay above the source node before the commit is done (and then specifying the source node as the top node for the commit job). Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210118105720.14824-8-mreitz@redhat.com>
* iotests/129: Use throttle nodeMax Reitz2021-01-261-24/+13Star
| | | | | | | | | | | | Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210118105720.14824-7-mreitz@redhat.com>
* iotests/129: Do not check @busyMax Reitz2021-01-261-1/+1
| | | | | | | | | | | | | | | | | @busy is false when the job is paused, which happens all the time because that is how jobs yield (e.g. for mirror at least since commit 565ac01f8d3). Back when 129 was added (2015), perhaps there was no better way of checking whether the job was still actually running. Now we have the @status field (as of 58b295ba52c, i.e. 2018), which can give us exactly that information. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210118105720.14824-6-mreitz@redhat.com>
* iotests/129: Remove test images in tearDown()Max Reitz2021-01-261-0/+2
| | | | | | | | Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210118105720.14824-5-mreitz@redhat.com>
* iotests: Move try_remove to iotests.pyMax Reitz2021-01-262-11/+8Star
| | | | | | | | Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210118105720.14824-4-mreitz@redhat.com>
* iotests/297: Rewrite in Python and extend reachMax Reitz2021-01-262-25/+92
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of checking iotests.py only, check all Python files in the qemu-iotests/ directory. Of course, most of them do not pass, so there is an extensive skip list for now. (The only files that do pass are 209, 254, 283, and iotests.py.) (Alternatively, we could have the opposite, i.e. an explicit list of files that we do want to check, but I think it is better to check files by default.) Unless started in debug mode (./check -d), the output has no information on which files are tested, so we will not have a problem e.g. with backports, where some files may be missing when compared to upstream. Besides the technical rewrite, some more things are changed: - For the pylint invocation, PYTHONPATH is adjusted. This mirrors setting MYPYPATH for mypy. - Also, MYPYPATH is now derived from PYTHONPATH, so that we include paths set by the environment. Maybe at some point we want to let the check script add '../../python/' to PYTHONPATH so that iotests.py does not need to do that. - Passing --notes=FIXME,XXX to pylint suppresses warnings for TODO comments. TODO is fine, we do not need 297 to complain about such comments. - The "Success" line from mypy's output is suppressed, because (A) it does not add useful information, and (B) it would leak information about the files having been tested to the reference output, which we decidedly do not want. Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210118105720.14824-3-mreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* iotests.py: Assume a couple of variables as givenMax Reitz2021-01-262-14/+13Star
| | | | | | | | | | | | | | | | | | | | | | | | There are a couple of environment variables that we fetch with os.environ.get() without supplying a default. Clearly they are required and expected to be set by the ./check script (as evidenced by execute_setup_common(), which checks for test_dir and qemu_default_machine to be set, and aborts if they are not). Using .get() this way has the disadvantage of returning an Optional[str] type, which mypy will complain about when tests just assume these values to be str. Use [] instead, which raises a KeyError for environment variables that are not set. When this exception is raised, catch it and move the abort code from execute_setup_common() there. Drop the 'assert iotests.sock_dir is not None' from iotest 300, because that sort of thing is precisely what this patch wants to prevent. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210118105720.14824-2-mreitz@redhat.com>
* block: apply COR-filter to block-stream jobsAndrey Shinkevich2021-01-263-12/+18
| | | | | | | | | | | | | | | | | | | | | This patch completes the series with the COR-filter applied to block-stream operations. Adding the filter makes it possible in future implement discarding copied regions in backing files during the block-stream job, to reduce the disk overuse (we need control on permissions). Also, the filter now is smart enough to do copy-on-read with specified base, so we have benefit on guest reads even when doing block-stream of the part of the backing chain. Several iotests are slightly modified due to filter insertion. Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201216061703.70908-14-vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* iotests: 30: prepare to COR filter insertion by stream jobVladimir Sementsov-Ogievskiy2021-01-261-1/+3
| | | | | | | | | | | | | | | test_stream_parallel run parallel stream jobs, intersecting so that top of one is base of another. It's OK now, but it would be a problem if insert the filter, as one job will want to use another job's filter as above_base node. Correct thing to do is move to new interface: "bottom" argument instead of base. This guarantees that jobs don't intersect by their actions. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20201216061703.70908-12-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* iotests: add #310 to test bottom node in COR driverAndrey Shinkevich2021-01-263-0/+133
| | | | | | | | | | | | The test case #310 is similar to #216 by Max Reitz. The difference is that the test #310 involves a bottom node to the COR filter driver. Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> [vsementsov: detach backing to test reads from top, limit to qcow2] Message-Id: <20201216061703.70908-7-vsementsov@virtuozzo.com> [mreitz: Add "# group:" line] Signed-off-by: Max Reitz <mreitz@redhat.com>
* iotests: fix _check_o_directVladimir Sementsov-Ogievskiy2021-01-261-3/+4
| | | | | | | | | | | | | | | | | | | Unfortunately commit "iotests: handle tmpfs" breaks running iotests with -nbd -nocache, as _check_o_direct tries to create $TEST_IMG.test_o_direct, but in case of nbd TEST_IMG is something like nbd+unix:///... , and test fails with message qemu-img: nbd+unix:///?socket[...]test_o_direct: Protocol driver 'nbd' does not support image creation, and opening the image failed: Failed to connect to '/tmp/tmp.[...]/nbd/test_o_direct': No such file or directory Use TEST_DIR instead. Fixes: cfdca2b9f9d4ca26bb2b2dfe8de3149092e39170 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201218182012.47607-1-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* iotests.py: qemu_io(): reuse qemu_tool_pipe_and_status()Vladimir Sementsov-Ogievskiy2021-01-211-8/+1Star
| | | | | | | | | Just drop code duplication. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201130134024.19212-5-vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
* iotests.py: fix qemu_tool_pipe_and_status()Vladimir Sementsov-Ogievskiy2021-01-211-3/+2Star
| | | | | | | | | qemu_img_args variable is unrelated here. We should print just args. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201130134024.19212-4-vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
* iotests/264: fix styleVladimir Sementsov-Ogievskiy2021-01-211-6/+5Star
| | | | | | | | | | Fix long line, extra import and one mypy complaint about incompatible int and float. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201118180433.11931-7-vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
* iotests: define group in each iotestVladimir Sementsov-Ogievskiy2021-01-20291-0/+291
| | | | | | | | | | | | | | | | | | | | | We are going to drop group file. Define group in tests as a preparatory step. The patch is generated by cd tests/qemu-iotests grep '^[0-9]\{3\} ' group | while read line; do file=$(awk '{print $1}' <<< "$line"); groups=$(sed -e 's/^... //' <<< "$line"); awk "NR==2{print \"# group: $groups\"}1" $file > tmp; cat tmp > $file; done Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20210116134424.82867-7-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
* iotests/294: add shebang lineVladimir Sementsov-Ogievskiy2021-01-201-0/+1
| | | | | | | Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20210116134424.82867-6-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
* iotests: make tests executableVladimir Sementsov-Ogievskiy2021-01-203-0/+0
| | | | | | | | | All other test files are executable. Fix these. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20210116134424.82867-5-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
* iotests: fix some whitespaces in test output filesVladimir Sementsov-Ogievskiy2021-01-203-12/+12
| | | | | | | | | | We are going to be stricter about comparing test result with .out files. So, fix some whitespaces now. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20210116134424.82867-4-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
* iotests/303: use dot slash for qcow2.py runningVladimir Sementsov-Ogievskiy2021-01-201-1/+1
| | | | | | | | | | | | | | If you run './check 303', check includes common.config which adjusts $PATH to include '.' first, and therefore finds qcow2.py on PATH. But if you run './303' directly, there is nothing to adjust PATH, and if '.' is not already on your PATH by other means, the test fails because the executable is not found. Adjust how we invoke the helper executable to avoid needing a PATH search in the first place. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20210116134424.82867-3-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
* iotests/277: use dot slash for nbd-fault-injector.py runningVladimir Sementsov-Ogievskiy2021-01-201-1/+1
| | | | | | | | | | | | | | If you run './check 277', check includes common.config which adjusts $PATH to include '.' first, and therefore finds nbd-fault-injector.py on PATH. But if you run './277' directly, there is nothing to adjust PATH, and if '.' is not already on your PATH by other means, the test fails because the executable is not found. Adjust how we invoke the helper executable to avoid needing a PATH search in the first place. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20210116134424.82867-2-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
* tests/iotests: drop test 312 from auto groupAlex Bennée2021-01-061-1/+1
| | | | | | | | | | | | | | | | The "auto" documentation states: That means they should run with every QEMU binary (also non-x86) which is not the case as the check-system-fedora build which only includes a rag tag group of rare and deprecated targets doesn't support the virtio device required. Fixes: ef9bba1484b ("quorum: Implement bdrv_co_block_status()") Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20210105100402.12350-1-alex.bennee@linaro.org>
* configure: move tests/qemu-iotests/common.env generation to mesonPaolo Bonzini2021-01-062-0/+6
| | | | Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-12-18' ↵Peter Maydell2021-01-0131-201/+1169
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | into staging Block patches: - New block filter: preallocate (which, on writes beyond an image file's end, allocates big chunks of data so that such post-EOF writes will occur less frequently) - write-zeroes and block-status support for Quorum - Implementation of truncate for the nvme block driver similarly to the existing implementations for host block devices and iscsi devices - Block layer refactoring: Drop the tighten_restrictions concept in the block permission functions - iotest fixes # gpg: Signature made Fri 18 Dec 2020 14:45:30 GMT # gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40 # gpg: issuer "mreitz@redhat.com" # gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full] # Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40 * remotes/maxreitz/tags/pull-block-2020-12-18: (30 commits) iotests: Fix _send_qemu_cmd with bash 5.1 iotests/102: Pass $QEMU_HANDLE to _send_qemu_cmd block/nvme: Implement fake truncate() coroutine quorum: Implement bdrv_co_pwrite_zeroes() quorum: Implement bdrv_co_block_status() scripts/simplebench: add bench_prealloc.py simplebench/results_to_text: make executable simplebench/results_to_text: add difference line to the table simplebench/results_to_text: improve view of the table simplebench: move results_to_text() into separate file simplebench: rename ascii() to results_to_text() scripts/simplebench: use standard deviation for +- error scripts/simplebench: support iops scripts/simplebench: fix grammar: s/successed/succeeded/ iotests: add 298 to test new preallocate filter driver iotests.py: execute_setup_common(): add required_fmts argument iotests: qemu_io_silent: support --image-opts qemu-io: add preallocate mode parameter for truncate command block: introduce preallocate filter block: bdrv_check_perm(): process children anyway ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * iotests: Fix _send_qemu_cmd with bash 5.1Max Reitz2020-12-1824-199/+728
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With bash 5.1, the output of the following script changes: a=("double space") a=${a[@]:0:1} echo "$a" from "double space" to "double space", i.e. all white space is preserved as-is. This is probably what we actually want here (judging from the "...to accommodate pathnames with spaces" comment), but before 5.1, we would have to quote the ${} slice to get the same behavior. In any case, without quoting, the reference output of many iotests is different between bash 5.1 and pre-5.1, which is not very good. The output of 5.1 is what we want, so whatever we do to get pre-5.1 to the same result, it means we have to fix the reference output of basically all tests that invoke _send_qemu_cmd (except the ones that only use single spaces in the commands they invoke). Instead of quoting the ${} slice (cmd="${$@: 1:...}"), we can also just not use array slicing and replace the whole thing with a simple "cmd=$1; shift", which works because all callers quote the whole $cmd argument anyway. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20201217153803.101231-3-mreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
| * iotests/102: Pass $QEMU_HANDLE to _send_qemu_cmdMax Reitz2020-12-182-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | The first parameter passed to _send_qemu_cmd is supposed to be the $QEMU_HANDLE. 102 does not do so here, fix it. As a result, the output changes: Now we see the prompt this command is supposedly waiting for before the resize message - as it should be. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20201217153803.101231-2-mreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
| * quorum: Implement bdrv_co_pwrite_zeroes()Alberto Garcia2020-12-182-0/+19
| | | | | | | | | | | | | | | | | | | | | | | | This simply calls bdrv_co_pwrite_zeroes() in all children. bs->supported_zero_flags is also set to the flags that are supported by all children. Signed-off-by: Alberto Garcia <berto@igalia.com> Message-Id: <2f09c842781fe336b4c2e40036bba577b7430190.1605286097.git.berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * quorum: Implement bdrv_co_block_status()Alberto Garcia2020-12-183-0/+216
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The quorum driver does not implement bdrv_co_block_status() and because of that it always reports to contain data even if all its children are known to be empty. One consequence of this is that if we for example create a quorum with a size of 10GB and we mirror it to a new image the operation will write 10GB of actual zeroes to the destination image wasting a lot of time and disk space. Since a quorum has an arbitrary number of children of potentially different formats there is no way to report all possible allocation status flags in a way that makes sense, so this implementation only reports when a given region is known to contain zeroes (BDRV_BLOCK_ZERO) or not (BDRV_BLOCK_DATA). If all children agree that a region contains zeroes then we can return BDRV_BLOCK_ZERO using the smallest size reported by the children (because all agree that a region of at least that size contains zeroes). If at least one child disagrees we have to return BDRV_BLOCK_DATA. In this case we use the largest of the sizes reported by the children that didn't return BDRV_BLOCK_ZERO (because we know that there won't be an agreement for at least that size). Signed-off-by: Alberto Garcia <berto@igalia.com> Tested-by: Tao Xu <tao3.xu@intel.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <db83149afcf0f793effc8878089d29af4c46ffe1.1605286097.git.berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * iotests: add 298 to test new preallocate filter driverVladimir Sementsov-Ogievskiy2020-12-183-0/+192
| | | | | | | | | | | | | | Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201021145859.11201-13-vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * iotests.py: execute_setup_common(): add required_fmts argumentVladimir Sementsov-Ogievskiy2020-12-181-1/+8
| | | | | | | | | | | | | | | | | | | | Add a parameter to skip test if some needed additional formats are not supported (for example filter drivers). Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201021145859.11201-12-vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * iotests: qemu_io_silent: support --image-optsVladimir Sementsov-Ogievskiy2020-12-181-1/+6
| | | | | | | | | | | | | | Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20201021145859.11201-11-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* | iotests:172: use _filter_qom_pathVladimir Sementsov-Ogievskiy2020-12-182-77/+77
| | | | | | | | | | | | | | | | | | | | device[NUMBER] thing in QOM path is not stable and tracking it during code modifications is not fun. Let's filter it like it's already done in iotest 186. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201216095205.526235-3-vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* | iotests: make _filter_qom_path more strictVladimir Sementsov-Ogievskiy2020-12-182-29/+29
| | | | | | | | | | | | | | | | | | | | | | According to original commit, that added this filter (627f607e3dddb2), the problematic thing in qom path is device[NUMBER], not the whole path. Seems that tracking the other parts of the path in iotest output is not bad. Let's make _filter_qom_path stricter. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201216095205.526235-2-vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* | iotests/210: Fix reference outputMax Reitz2020-12-181-1/+1
|/ | | | | | | | | | | | | Commit 8b1170012b1 has added a global maximum disk length for the block layer, so the error message when creating an overly large disk has changed. Fixes: 8b1170012b1de6649c66ac1887f4df7e312abf3b ("block: introduce BDRV_MAX_LENGTH") Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20201214175158.299919-1-mreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* block: introduce BDRV_MAX_LENGTHVladimir Sementsov-Ogievskiy2020-12-112-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We are going to modify block layer to work with 64bit requests. And first step is moving to int64_t type for both offset and bytes arguments in all block request related functions. It's mostly safe (when widening signed or unsigned int to int64_t), but switching from uint64_t is questionable. So, let's first establish the set of requests we want to work with. First signed int64_t should be enough, as off_t is signed anyway. Then, obviously offset + bytes should not overflow. And most interesting: (offset + bytes) being aligned up should not overflow as well. Aligned to what alignment? First thing that comes in mind is bs->bl.request_alignment, as we align up request to this alignment. But there is another thing: look at bdrv_mark_request_serialising(). It aligns request up to some given alignment. And this parameter may be bdrv_get_cluster_size(), which is often a lot greater than bs->bl.request_alignment. Note also, that bdrv_mark_request_serialising() uses signed int64_t for calculations. So, actually, we already depend on some restrictions. Happily, bdrv_get_cluster_size() returns int and bs->bl.request_alignment has 32bit unsigned type, but defined to be a power of 2 less than INT_MAX. So, we may establish, that INT_MAX is absolute maximum for any kind of alignment that may occur with the request. Note, that bdrv_get_cluster_size() is not documented to return power of 2, still bdrv_mark_request_serialising() behaves like it is. Also, backup uses bdi.cluster_size and is not prepared to it not being power of 2. So, let's establish that Qemu supports only power-of-2 clusters and alignments. So, alignment can't be greater than 2^30. Finally to be safe with calculations, to not calculate different maximums for different nodes (depending on cluster size and request_alignment), let's simply set QEMU_ALIGN_DOWN(INT64_MAX, 2^30) as absolute maximum bytes length for Qemu. Actually, it's not much less than INT64_MAX. OK, then, let's apply it to block/io. Let's consider all block/io entry points of offset/bytes: 4 bytes/offset interface functions: bdrv_co_preadv_part(), bdrv_co_pwritev_part(), bdrv_co_copy_range_internal() and bdrv_co_pdiscard() and we check them all with bdrv_check_request(). We also have one entry point with only offset: bdrv_co_truncate(). Check the offset. And one public structure: BdrvTrackedRequest. Happily, it has only three external users: file-posix.c: adopted by this patch write-threshold.c: only read fields test-write-threshold.c: sets obviously small constant values Better is to make the structure private and add corresponding interfaces.. Still it's not obvious what kind of interface is needed for file-posix.c. Let's keep it public but add corresponding assertions. After this patch we'll convert functions in block/io.c to int64_t bytes and offset parameters. We can assume that offset/bytes pair always satisfy new restrictions, and make corresponding assertions where needed. If we reach some offset/bytes point in block/io.c missing bdrv_check_request() it is considered a bug. As well, if block/io.c modifies a offset/bytes request, expanding it more then aligning up to request_alignment, it's a bug too. For all io requests except for discard we keep for now old restriction of 32bit request length. iotest 206 output error message changed, as now test disk size is larger than new limit. Add one more test case with new maximum disk size to cover too-big-L1 case. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201203222713.13507-5-vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* iotests/221: Discard image before qemu-img mapMax Reitz2020-12-112-8/+13
| | | | | | | | | | | | | | See the new comment for why this should be done. I do not have a reproducer on master, but when using FUSE block exports, this test breaks depending on the underlying filesystem (for me, it works on tmpfs, but fails on xfs, because the block allocated by file-posix has 16 kB there instead of 4 kB). Suggested-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20201207152245.66987-1-mreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* iotests/308: Add test for FUSE exportsMax Reitz2020-12-113-0/+437
| | | | | | | | | | | | We have good coverage of the normal I/O paths now, but what remains is a test that tests some more special cases: Exporting an image on itself (thus turning a formatted image into a raw one), some error cases, and non-writable and non-growable exports. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20201027190600.192171-21-mreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* iotests: Enable fuse for many testsMax Reitz2020-12-1175-75/+75
| | | | | | | | | | | | | | | | Many tests (that do not support generic protocols) can run just fine with FUSE-exported images, so allow them to. Note that this is no attempt at being definitely complete. There are some tests that might be modified to run on FUSE, but this patch still skips them. This patch only tries to pick the rather low-hanging fruits. Note that 221 and 250 only pass when .lseek is correctly implemented, which is only possible with a libfuse that is 3.8 or newer. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20201027190600.192171-20-mreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* iotests: Allow testing FUSE exportsMax Reitz2020-12-113-1/+134
| | | | | | | | | | | This pretends FUSE exports are a kind of protocol. As such, they are always tested under the format node. This is probably the best way to test them, actually, because this will generate more I/O load and more varied patterns. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20201027190600.192171-19-mreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* iotests: Give access to the qemu-storage-daemonMax Reitz2020-12-112-0/+28
| | | | | | Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20201027190600.192171-18-mreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
generated by cgit v1.2.3-55-g7522 (git 2.39.0) at 2024-12-18 12:17:29 +0100