summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* s390x/tcg: Check vector register instructions at central pointDavid Hildenbrand2019-03-112-0/+19
| | | | | | | | | | | | Check them at a central point. We'll use a new instruction flag to flag all vector instructions (IF_VEC) and handle it very similar to AFP, whereby we use another unused position in the PSW mask to store the state of vector register enablement per translation block. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com> Message-Id: <20190307121539.12842-3-david@redhat.com> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
* s390x/tcg: Define vector instruction formatsDavid Hildenbrand2019-03-112-1/+63
| | | | | | | | | | | | | | | | | | | | These are the new instruction formats related to vector instructions as up to the z14 (a.k.a. latest PoP). As v2 appeares (like x2 in VRX) with d2/b2 in VRV, we have to assign it a higher field number to avoid collisions. Properly take care of the MSB (to be able to address 32 registers) for each vector register field stored in the RXB field (Bit 36 - 30 for all vector instructions). As we have 32 bit vector registers and the "v" fields are only 4 bit in size, the 5th bit is stored in the RXB. We use a new type to indicate that the MSB has to be fetched from the RXB. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com> Message-Id: <20190307121539.12842-2-david@redhat.com> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
* target/s390x: Remove non-architected entries from struct LowCoreThomas Huth2019-03-111-39/+2Star
| | | | | | | | | | | | | | | | | | | There are some fields in our struct LowCore which apparently have been copied from a very old version of the Linux kernel. These fields are not architected in the "Principles of Operation", and only used on these memory locations in Linux kernels older than 2.6.29. Newer Linux kernels moved the entries to different locations or are not using them at all anymore. Thus we should never access these fields from the QEMU side, so they should be removed. While we're at it, also add a QEMU_BUILD_BUG_ON() statement to assert that struct LowCore has the right size. Reviewed-by: David Hildenbrand <david@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <1551775581-27989-1-git-send-email-thuth@redhat.com> Acked-by: Christian Borntraeger <borntraeger@de.ibm.com> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
* Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-03-08' into ↵Peter Maydell2019-03-0911-19/+93
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | staging nbd patches for 2019-03-08 - support TLS client authorization in NBD servers - iotest 223 race fix # gpg: Signature made Fri 08 Mar 2019 17:37:59 GMT # gpg: using RSA key A7A16B4A2527436A # gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full] # gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full] # gpg: aka "[jpeg image of size 6874]" [full] # Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A * remotes/ericb/tags/pull-nbd-2019-03-08: iotests: Wait for qemu to end in 223 nbd: fix outdated qapi docs syntax for tls-creds nbd: allow authorization with nbd-server-start QMP command qemu-nbd: add support for authorization of TLS clients Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * iotests: Wait for qemu to end in 223Eric Blake2019-03-062-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When iotest 223 was first written, it didn't matter if we waited for the qemu process to clean up. But with the introduction of a later qemu-nbd process trying to reuse the same file, there is a race where even though the asynchronous qemu process has responded to "quit", it has not yet had time to unlock the file and exit, resulting in: -[{ "start": 0, "length": 65536, "depth": 0, "zero": false, "data": false}, -{ "start": 65536, "length": 2031616, "depth": 0, "zero": false, "data": true}, -{ "start": 2097152, "length": 2097152, "depth": 0, "zero": false, "data": false}] +qemu-nbd: Failed to blk_new_open 'tests/qemu-iotests/scratch/t.qcow2': Failed to get shared "write" lock +Is another process using the image [tests/qemu-iotests/scratch/t.qcow2]? +qemu-img: Could not open 'driver=nbd,server.type=unix,server.path=tests/qemu-iotests/scratch/qemu-nbd.sock,x-dirty-bitmap=qemu:dirty-bitmap:b': Failed to connect socket tests/qemu-iotests/scratch/qemu-nbd.sock: Connection refused +./common.nbd: line 33: kill: (11122) - No such process Fixes: ddd09448 Reported-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20190305182908.13557-1-eblake@redhat.com> Tested-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
| * nbd: fix outdated qapi docs syntax for tls-credsDaniel P. Berrangé2019-03-061-1/+1
| | | | | | | | | | | | | | Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20190227162035.18543-4-berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
| * nbd: allow authorization with nbd-server-start QMP commandDaniel P. Berrange2019-03-064-6/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As with the previous patch to qemu-nbd, the nbd-server-start QMP command also needs to be able to specify authorization when enabling TLS encryption. First the client must create a QAuthZ object instance using the 'object-add' command: { 'execute': 'object-add', 'arguments': { 'qom-type': 'authz-list', 'id': 'authz0', 'parameters': { 'policy': 'deny', 'rules': [ { 'match': '*CN=fred', 'policy': 'allow' } ] } } } They can then reference this in the new 'tls-authz' parameter when executing the 'nbd-server-start' command: { 'execute': 'nbd-server-start', 'arguments': { 'addr': { 'type': 'inet', 'host': '127.0.0.1', 'port': '9000' }, 'tls-creds': 'tls0', 'tls-authz': 'authz0' } } Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <20190227162035.18543-3-berrange@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
| * qemu-nbd: add support for authorization of TLS clientsDaniel P. Berrange2019-03-066-12/+73
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently any client which can complete the TLS handshake is able to use the NBD server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the NBD server. This is still a fairly low bar to cross. This adds a '--tls-authz OBJECT-ID' option to the qemu-nbd command which takes the ID of a previously added 'QAuthZ' object instance. This will be used to validate the client's x509 distinguished name. Clients failing the authorization check will not be permitted to use the NBD server. For example to setup authorization that only allows connection from a client whose x509 certificate distinguished name is CN=laptop.example.com,O=Example Org,L=London,ST=London,C=GB escape the commas in the name and use: qemu-nbd --object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\ endpoint=server,verify-peer=yes \ --object 'authz-simple,id=auth0,identity=CN=laptop.example.com,,\ O=Example Org,,L=London,,ST=London,,C=GB' \ --tls-creds tls0 \ --tls-authz authz0 \ ....other qemu-nbd args... NB: a real shell command line would not have leading whitespace after the line continuation, it is just included here for clarity. Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <20190227162035.18543-2-berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [eblake: split long line in --help text, tweak 233 to show that whitespace after ,, in identity= portion is actually okay] Signed-off-by: Eric Blake <eblake@redhat.com>
* | Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into ↵Peter Maydell2019-03-094-57/+57
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | staging Pull request # gpg: Signature made Fri 08 Mar 2019 16:53:34 GMT # gpg: using RSA key 9CA4ABB381AB73C8 # gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full] # gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full] # Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8 * remotes/stefanha/tags/block-pull-request: iothread: document about why we need explicit aio_poll() iothread: push gcontext earlier in the thread_fn iothread: create main loop unconditionally iothread: create the gcontext unconditionally iothread: replace init_done_cond with a semaphore hw/block/virtio-blk: Clean req->dev repetitions MAINTAINERS: add missing support status fields Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * | iothread: document about why we need explicit aio_poll()Peter Xu2019-03-081-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After consulting Paolo I know why we'd better keep the explicit aio_poll() in iothread_run(). Document it directly into the code so that future readers will know the answer from day one. Signed-off-by: Peter Xu <peterx@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-id: 20190306115532.23025-6-peterx@redhat.com Message-Id: <20190306115532.23025-6-peterx@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
| * | iothread: push gcontext earlier in the thread_fnPeter Xu2019-03-081-3/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We were pushing the context until right before running the gmainloop. Now since we have everything unconditionally, we can move this earlier. One benefit is that now it's done even before init_done_sem, so as long as the iothread user calls iothread_create() and completes, we know that the thread stack is ready. Signed-off-by: Peter Xu <peterx@redhat.com> Message-id: 20190306115532.23025-5-peterx@redhat.com Message-Id: <20190306115532.23025-5-peterx@redhat.com> [Tweaked comment wording as discussed with Peter Xu. --Stefan] Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
| * | iothread: create main loop unconditionallyPeter Xu2019-03-081-9/+3Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since we've have the gcontext always there, create the main loop altogether. The iothread_run() is even cleaner. Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Signed-off-by: Peter Xu <peterx@redhat.com> Message-id: 20190306115532.23025-4-peterx@redhat.com Message-Id: <20190306115532.23025-4-peterx@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
| * | iothread: create the gcontext unconditionallyPeter Xu2019-03-082-23/+22Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In existing code we create the gcontext dynamically at the first access of the gcontext from caller. That can bring some complexity and potential races during using iothread. Since the context itself is not that big a resource, and we won't have millions of iothread, let's simply create the gcontext unconditionally. This will also be a preparation work further to move the thread context push operation earlier than before (now it's only pushed right before we want to start running the gmainloop). Removing the g_once since it's not necessary, while introducing a new run_gcontext boolean to show whether we want to run the gcontext. Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Signed-off-by: Peter Xu <peterx@redhat.com> Message-id: 20190306115532.23025-3-peterx@redhat.com Message-Id: <20190306115532.23025-3-peterx@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
| * | iothread: replace init_done_cond with a semaphorePeter Xu2019-03-082-15/+5Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Only sending an init-done message using lock+cond seems an overkill to me. Replacing it with a simpler semaphore. Meanwhile, init the semaphore unconditionally, then we can destroy it unconditionally too in finalize which seems cleaner. Signed-off-by: Peter Xu <peterx@redhat.com> Message-id: 20190306115532.23025-2-peterx@redhat.com Message-Id: <20190306115532.23025-2-peterx@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
| * | hw/block/virtio-blk: Clean req->dev repetitionsAnastasiia Rusakova2019-03-071-7/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some functions sometimes uses req->dev even though a local variable VirtIOBlock* s = req->dev has already been defined. Updated places to use s everywhere in the file. Signed-off-by: Anastasiia Rusakova <arusakova917@gmail.com> Message-id: 20190307161925.4158-1-rusakova.nastasia@icloud.com Message-Id: <20190307161925.4158-1-rusakova.nastasia@icloud.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
| * | MAINTAINERS: add missing support status fieldsStefan Hajnoczi2019-03-071-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds the "S:" line for areas of the codebase that currently lack a support status field. Note that there are a few more areas that are more abstract and do not correspond to a specific set of files. They have not been modified. Cc: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20190301163518.20702-1-stefanha@redhat.com Message-Id: <20190301163518.20702-1-stefanha@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
* | | Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into stagingPeter Maydell2019-03-09215-422/+1584
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Block layer patches: - qcow2: Support for external data files - qcow2: Default to 4KB for the qcow2 cache entry size - Apply block driver whitelist for -drive format=help - Several qemu-iotests improvements # gpg: Signature made Fri 08 Mar 2019 12:54:27 GMT # gpg: using RSA key 7F09B272C88F2FD6 # gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full] # Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6 * remotes/kevin/tags/for-upstream: (33 commits) qcow2 spec: Describe string header extensions qemu-iotests: Add dependency to qemu-nbd tool ahci-test: Add dependency to qemu-img tool qemu-iotests: amend with external data file qemu-iotests: General tests for qcow2 with external data file qemu-iotests: Preallocation with external data file qcow2: Implement data-file-raw create option qcow2: Store data file name in the image qcow2: Creating images with external data file qcow2: Add basic data-file infrastructure qcow2: Support external data file in qemu-img check qcow2: Return error for snapshot operation with data file qcow2: External file I/O qcow2: Prepare qcow2_co_block_status() for data file qcow2: Return 0/-errno in qcow2_alloc_compressed_cluster_offset() qcow2: Don't assume 0 is an invalid cluster offset qcow2: Prepare count_contiguous_clusters() for external data file qcow2: Prepare qcow2_get_cluster_type() for external data file qcow2: Pass bs to qcow2_get_cluster_type() qcow2: Basic definitions for external data files ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * | | qcow2 spec: Describe string header extensionsKevin Wolf2019-03-081-2/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Be more specific about the string representation in header extensions. Suggested-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
| * | | qemu-iotests: Add dependency to qemu-nbd toolPhilippe Mathieu-Daudé2019-03-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since a9660664fde, some iotests use qemu-nbd. Add a dependency to build it before using it. This fixes: $ make check-block GEN qemu-img-cmds.h CC qemu-img.o LINK qemu-img CC qemu-io.o LINK qemu-io CC tests/qemu-iotests/socket_scm_helper.o LINK tests/qemu-iotests/socket_scm_helper tests/qemu-iotests-quick.sh check: qemu-nbd not found make: *** [tests/Makefile.include:1059: check-tests/qemu-iotests-quick.sh] Error 1 Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | ahci-test: Add dependency to qemu-img toolPhilippe Mathieu-Daudé2019-03-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since the ahci-test uses qemu-img, add a dependency to build it before using it. This fixes: $ gmake check-qtest V=1 QTEST_QEMU_BINARY=x86_64-softmmu/qemu-system-x86_64 QTEST_QEMU_IMG=qemu-img tests/ahci-test Failed to execute child process "/tmp/qemu-test.19tMRF/qemu-img" (No such file or directory) ERROR:tests/libqos/libqos.c:192:mkimg: assertion failed: (ret && !err) Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qemu-iotests: amend with external data fileKevin Wolf2019-03-082-1/+133
| | | | | | | | | | | | | | | | Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qemu-iotests: General tests for qcow2 with external data fileKevin Wolf2019-03-083-0/+326
| | | | | | | | | | | | | | | | Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qemu-iotests: Preallocation with external data fileKevin Wolf2019-03-082-0/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Test that preallocating metadata results in a somewhat larger qcow2 file, but preallocating data only affects the disk usage of the data file and the qcow2 file stays small. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: Implement data-file-raw create optionKevin Wolf2019-03-086-3/+109
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Provide an option to force QEMU to always keep the external data file consistent as a standalone read-only raw image. At the moment, this means making sure that write_zeroes requests are forwarded to the data file instead of just updating the metadata, and checking that no backing file is used. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: Store data file name in the imageKevin Wolf2019-03-084-2/+128
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rather than requiring that the external data file node is passed explicitly when creating the qcow2 node, store the filename in the designated header extension during .bdrv_create and read it from there as a default during .bdrv_open. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: Creating images with external data fileKevin Wolf2019-03-083-0/+31
| | | | | | | | | | | | | | | | | | | | | | | | This adds a .bdrv_create option to use an external data file. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: Add basic data-file infrastructureKevin Wolf2019-03-083-4/+41
| | | | | | | | | | | | | | | | | | | | | | | | This adds a .bdrv_open option to specify the external data file node. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: Support external data file in qemu-img checkKevin Wolf2019-03-081-11/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For external data files, data clusters must be excluded from the refcount calculations. Instead, an implicit refcount of 1 is assumed for the COPIED flag. Compressed clusters and internal snapshots are incompatible with external data files, so print an error if they are in use for images with an external data file. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: Return error for snapshot operation with data fileKevin Wolf2019-03-081-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Internal snapshots and an external data file are incompatible because snapshots require refcounting and non-linear mapping. Return an error for all of the snapshot operations if an external data file is in use. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: External file I/OKevin Wolf2019-03-087-37/+122
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This changes the qcow2 implementation to direct all guest data I/O to s->data_file rather than bs->file, while metadata I/O still uses bs->file. At the moment, this is still always the same, but soon we'll add options to set s->data_file to an external data file. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: Prepare qcow2_co_block_status() for data fileKevin Wolf2019-03-081-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Offset 0 cannot be assumed to mean an unallocated cluster any more. Instead, the cluster type needs to be checked. *file must refer to the data file instead of the image file if a valid offset is returned from qcow2_co_block_status(). Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: Return 0/-errno in qcow2_alloc_compressed_cluster_offset()Kevin Wolf2019-03-084-30/+26Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | qcow2_alloc_compressed_cluster_offset() used to return the cluster offset for success and 0 for error. This doesn't only conflict with 0 as a valid host offset, but also loses the error code. Similar to the change made to qcow2_alloc_cluster_offset() for uncompressed clusters in commit 148da7ea9d6, make the function return 0/-errno and return the allocated cluster offset in a by-reference parameter. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: Don't assume 0 is an invalid cluster offsetKevin Wolf2019-03-082-32/+29Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The cluster allocation code uses 0 as an invalid offset that is used in case of errors or as "offset not yet determined". With external data files, a host cluster offset of 0 becomes valid, though. Define a constant INV_OFFSET (which is not cluster aligned and will therefore never be a valid offset) that can be used for such purposes. This removes the additional host_offset == 0 check that commit ff52aab2df5 introduced; the confusion between an invalid offset and (erroneous) allocation at offset 0 is removed with this change. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: Prepare count_contiguous_clusters() for external data fileKevin Wolf2019-03-081-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Offset 0 can be valid for normal (allocated) clusters now, so use qcow2_get_cluster_type() instead. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: Prepare qcow2_get_cluster_type() for external data fileKevin Wolf2019-03-081-1/+9
| | | | | | | | | | | | | | | | Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: Pass bs to qcow2_get_cluster_type()Kevin Wolf2019-03-083-24/+26
| | | | | | | | | | | | | | | | Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: Basic definitions for external data filesKevin Wolf2019-03-085-23/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds basic constants, struct fields and helper function for external data file support to the implementation. QCOW2_INCOMPAT_MASK and QCOW2_AUTOCLEAR_MASK are not updated yet so that opening images with an external data file still fails (we don't handle them correctly yet). Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: Extend spec for external data filesKevin Wolf2019-03-081-4/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds external data file to the qcow2 spec as a new incompatible feature. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: Simplify preallocation codeKevin Wolf2019-03-081-27/+1Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Image creation already involves a bdrv_co_truncate() call, which allows to specify a preallocation mode. Just pass the right mode there and remove the code that is made redundant by this. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qemu-iotests: Test qcow2 preallocation modesKevin Wolf2019-03-083-0/+90
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | | qemu-iotests: Ensure GNU sed is usedPhilippe Mathieu-Daudé2019-03-082-18/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Various sed regexp from common.filter use sed GNU extensions. Instead of spending time to write these regex to be POSIX compliant, verify the GNU sed is available and use it. Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qemu-iotests: Improve portability by searching bash in the $PATHPhilippe Mathieu-Daudé2019-03-08185-185/+185
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bash is not always installed as /bin/bash. In particular on OpenBSD, the package installs it in /usr/local/bin. Use the 'env' shebang to search bash in the $PATH. Patch created mechanically by running: $ git grep -lE '#! ?/bin/bash' -- tests/qemu-iotests \ | while read f; do \ sed -i 's|^#!.\?/bin/bash$|#!/usr/bin/env bash|' $f; \ done Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | tests/bios-tables: Improve portability by searching bash in the $PATHPhilippe Mathieu-Daudé2019-03-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bash is not always installed as /bin/bash. In particular on OpenBSD, the package installs it in /usr/local/bin. Use the 'env' shebang to search bash in the $PATH. Reviewed-by: Kamil Rytarowski <n54@gmx.com> Reviewed-by: Igor Mammedov <imammedo@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Acked-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | tests/multiboot: Improve portability by searching bash in the $PATHPhilippe Mathieu-Daudé2019-03-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bash is not always installed as /bin/bash. In particular on OpenBSD, the package installs it in /usr/local/bin. Use the 'env' shebang to search bash in the $PATH. Reviewed-by: Kamil Rytarowski <n54@gmx.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | iotests: check whitelisted formatsAndrey Shinkevich2019-03-081-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some test cases require specific formats. The method decorator skip_if_unsupported() checks if requested formats are whitelisted. The test #139 was selected for a sample output, after running $ ./check -qcow2 131-140 137 3s ... 138 0s ... 139 2s ... [case not run] testBlkDebug (__main__.TestBlockdevDel): formats ['blkdebug'] are not whitelisted [case not run] testBlkVerify (__main__.TestBlockdevDel): formats ['blkverify'] are not whitelisted [case not run] testQuorum (__main__.TestBlockdevDel): formats ['quorum'] are not whitelisted 140 0s ... Not run: 131 135 136 Some cases not run in: 139 Passed all 7 tests Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | iotests: ask QEMU for supported formatsAndrey Shinkevich2019-03-082-1/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Supported formats listed by 'qemu' may differ from those listed by 'qemu-img' due to whitelists. Some test cases require specific formats that may be used with qemu. They can be inquired directly by running 'qemu -drive format=help'. The response takes whitelists into account. The method supported_formats() serves for that. The method decorator skip_if_unsupported() checks if all requested formats are whitelisted. If not, the test case will be skipped. That has been implemented in the 'check' file in the way similar to the 'test notrun' mechanism. Suggested-by: Roman Kagan <rkagan@virtuozzo.com> Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Suggested-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | block: iterate_format with account of whitelistingAndrey Shinkevich2019-03-084-7/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | bdrv_iterate_format (which is currently only used for printing out the formats supported by the block layer) doesn't take format whitelisting into account. This creates a problem for tests: they enumerate supported formats to decide which tests to enable, but then discover that QEMU doesn't let them actually use some of those formats. To avoid that, exclude formats that are not whitelisted from enumeration, if whitelisting is in use. Since we have separate whitelists for r/w and r/o, take this a parameter to bdrv_iterate_format, and print two lists of supported formats (r/w and r/o) in main qemu. Signed-off-by: Roman Kagan <rkagan@virtuozzo.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | iotests: open notrun files in text modeAndrey Shinkevich2019-03-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Replace the binary mode with the default text one when *.notrun files are opened for skipped tests. That change is made for the compatibility with Python 3 which returns error otherwise. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qcow2: Default to 4KB for the qcow2 cache entry sizeAlberto Garcia2019-03-082-6/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | QEMU 2.12 (commit 1221fe6f636754ab5f2c1c87caa77633e9123622) introduced a new setting called l2-cache-entry-size that allows making entries on the qcow2 L2 cache smaller than the cluster size. I have been performing several tests with different cluster and entry sizes and all of them show that reducing the entry size (aka L2 slice) consistently improves I/O performance, notably during random I/O (all tests done with sequential I/O show similar results). This is to be expected because loading and evicting an L2 slice is more expensive the larger the slice is. Here are some numbers on fully populated 40GB qcow2 images. The rightmost column represents the maximum L2 cache size in both cases. Cluster size = 64 KB |-------------+--------------+--------------+--------------| | | 1MB L2 cache | 3MB L2 cache | 5MB L2 cache | |-------------+--------------+--------------+--------------| | 4KB slices | 6545 IOPS | 12045 IOPS | 55680 IOPS | | 16KB slices | 5177 IOPS | 9798 IOPS | 56278 IOPS | | 64KB slices | 2718 IOPS | 5326 IOPS | 57355 IOPS | |-------------+--------------+--------------+--------------| Cluster size = 256 KB |--------------+----------------+--------------+-----------------| | | 512KB L2 cache | 1MB L2 cache | 1280KB L2 cache | |--------------+----------------+--------------+-----------------| | 4KB slices | 8539 IOPS | 21071 IOPS | 55417 IOPS | | 64KB slices | 3598 IOPS | 9772 IOPS | 57687 IOPS | | 256KB slices | 1415 IOPS | 4120 IOPS | 58001 IOPS | |--------------+----------------+--------------+-----------------| As can be seen in the numbers, the only exception to the rule is when the cache is large enough to hold all L2 tables. This is also to be expected because in this case no cache entry is ever evicted so reducing its size doesn't bring any benefit. This patch sets the default L2 cache entry size to 4KB except when the cache is large enough for the whole disk. Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | iotests: use iotests.VM in 238Stefan Hajnoczi2019-03-081-6/+1Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Test 238 does not require the kvm accelerator. Using the qtest accelerator allows the test to run in both non-kvm and non-tcg environments. iotests.VM implicitly uses the qtest accelerator and is really the class that this test should be using. Switch to that instead of qemu.QEMUMachine. Suggested-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
generated by cgit v1.2.3-55-g7522 (git 2.39.0) at 2024-10-27 08:29:59 +0100