summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Acceptance tests: add Linux kernel boot and console checking testCleber Rosa2018-06-151-0/+47
| | | | | | | | | | | | | | | | | | | | | | | | This test boots a Linux kernel, and checks that the given command line was effective in two ways: * It makes the kernel use the set "console device" as a console * The kernel records the command line as expected in the console Given that way too many error conditions may occur, and detecting the kernel boot progress status may not be trivial, this test relies on a timeout to handle unexpected situations. Also, it's *not* tagged as a quick test for obvious reasons. It may be useful, while interactively running/debugging this test, or tests similar to this one, to show some of the logging channels. Example: $ avocado --show=QMP,console run boot_linux_console.py Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20180530184156.15634-6-crosa@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
* scripts/qemu.py: introduce set_console() methodCleber Rosa2018-06-151-1/+96
| | | | | | | | | | | | | | | | | | | | | | | The set_console() method is intended to ease higher level use cases that require a console device. The amount of intelligence is limited on purpose, requiring either the device type explicitly, or the existence of a machine (pattern) definition. Because of the console device type selection criteria (by machine type), users should also be able to define that. It'll then be used for both '-machine' and for the console device type selection. Users of the set_console() method will certainly be interested in accessing the console device, and for that a console_socket property has been added. Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20180530184156.15634-5-crosa@redhat.com> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
* Acceptance tests: add quick VNC testsCleber Rosa2018-06-151-0/+60
| | | | | | | | | | | This patch adds a few simple behavior tests for VNC. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20180530184156.15634-4-crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
* scripts/qemu.py: allow adding to the list of extra argumentsCleber Rosa2018-06-151-0/+6
| | | | | | | | | | | | Tests will often need to add extra arguments to QEMU command line arguments. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20180530184156.15634-3-crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
* Add functional/acceptance tests infrastructureCleber Rosa2018-06-154-0/+280
| | | | | | | | | | | | | | | | | | | | This patch adds the very minimum infrastructure necessary for writing and running functional/acceptance tests, including: * Documentation * The avocado_qemu.Test base test class * One example tests (version.py) Additional functionality is expected to be added along the tests that require them. Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20180530184156.15634-2-crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> [ehabkost: fix typo on testing.rst] Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
* Remove COPYING.PYTHONEduardo Habkost2018-06-151-270/+0Star
| | | | | | | | | | | | | | | | The COPYING.PYTHON file was added when we added the compatibility argparse.py module, which was licensed under the Python Software Foundation License Version 2. Now the compatibility argparse.py module was removed, and we are not carrying any code under that license anymore. Remove COPYING.PYTHON. Signed-off-by: Eduardo Habkost <ehabkost@redhat.com> Message-Id: <20180611180152.2681-1-ehabkost@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
* Merge remote-tracking branch ↵Peter Maydell2018-06-1512-33/+146
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 'remotes/dgilbert/tags/pull-migration-20180615a' into staging Migration pull 2018-06-15 # gpg: Signature made Fri 15 Jun 2018 16:13:17 BST # gpg: using RSA key 0516331EBC5BFDE7 # gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" # Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A 9FA9 0516 331E BC5B FDE7 * remotes/dgilbert/tags/pull-migration-20180615a: migration: calculate expected_downtime with ram_bytes_remaining() migration/postcopy: Wake rate limit sleep on postcopy request migration: Wake rate limiting for urgent requests migration/postcopy: Add max-postcopy-bandwidth parameter migration: introduce migration_update_rates migration: fix counting xbzrle cache_miss_rate migration/block-dirty-bitmap: fix dirty_bitmap_load migration: Poison ramblock loops in migration migration: Fixes for non-migratable RAMBlocks typedefs: add QJSON Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * migration: calculate expected_downtime with ram_bytes_remaining()Balamuruhan S2018-06-152-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | expected_downtime value is not accurate with dirty_pages_rate * page_size, using ram_bytes_remaining() would yeild it resonable. consider to read the remaining ram just after having updated the dirty pages count later migration_bitmap_sync_range() in migration_bitmap_sync() and reuse the `remaining` field in ram_counters to hold ram_bytes_remaining() for calculating expected_downtime. Reported-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Balamuruhan S <bala24@linux.vnet.ibm.com> Signed-off-by: Laurent Vivier <lvivier@redhat.com> Message-Id: <20180612085009.17594-2-bala24@linux.vnet.ibm.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
| * migration/postcopy: Wake rate limit sleep on postcopy requestDr. David Alan Gilbert2018-06-151-1/+8
| | | | | | | | | | | | | | | | | | | | | | Use the 'urgent request' mechanism added in the previous patch for entries added to the postcopy request queue for RAM. Ignore the rate limiting while we have requests. Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <20180613102642.23995-4-dgilbert@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
| * migration: Wake rate limiting for urgent requestsDr. David Alan Gilbert2018-06-153-4/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Rate limiting sleeps the migration thread for a while when it runs out of bandwidth; but sometimes we want to wake up to get on with something more urgent (like a postcopy request). Here we use a semaphore with a timedwait instead of a simple sleep; Incrementing the sempahore will wake it up sooner. Anything that consumes these urgent events must decrement the sempahore. Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <20180613102642.23995-3-dgilbert@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
| * migration/postcopy: Add max-postcopy-bandwidth parameterDr. David Alan Gilbert2018-06-153-4/+57
| | | | | | | | | | | | | | | | | | | | | | Limit the background transfer bandwidth during the postcopy phase to the value set on this new parameter. The default, 0, corresponds to the existing behaviour which is unlimited bandwidth. Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <20180613102642.23995-2-dgilbert@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
| * migration: introduce migration_update_ratesXiao Guangrong2018-06-151-13/+22
| | | | | | | | | | | | | | | | | | It is used to slightly clean the code up, no logic is changed Signed-off-by: Xiao Guangrong <xiaoguangrong@tencent.com> Message-Id: <20180604095520.8563-5-xiaoguangrong@tencent.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
| * migration: fix counting xbzrle cache_miss_rateXiao Guangrong2018-06-151-1/+1
| | | | | | | | | | | | | | | | | | | | Sync up xbzrle_cache_miss_prev only after migration iteration goes forward Signed-off-by: Xiao Guangrong <xiaoguangrong@tencent.com> Message-Id: <20180604095520.8563-4-xiaoguangrong@tencent.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
| * migration/block-dirty-bitmap: fix dirty_bitmap_loadVladimir Sementsov-Ogievskiy2018-06-151-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | dirty_bitmap_load_header return code is obtained but not handled. Fix this. Bug was introduced in b35ebdf076d697bc "migration: add postcopy migration of dirty bitmaps" with the whole function. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20180530112424.204835-1-vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
| * migration: Poison ramblock loops in migrationDr. David Alan Gilbert2018-06-153-2/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | The migration code should be using the RAMBLOCK_FOREACH_MIGRATABLE and qemu_ram_foreach_block_migratable not the all-block versions; poison them so that we can't accidentally use them. Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <20180605162545.80778-3-dgilbert@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Reviewed-by: Cédric Le Goater <clg@kaod.org> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
| * migration: Fixes for non-migratable RAMBlocksDr. David Alan Gilbert2018-06-152-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | There are still a few cases where migration code is using the macros and functions that do all RAMBlocks rather than just the migratable blocks; fix those up. Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <20180605162545.80778-2-dgilbert@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Reviewed-by: Cédric Le Goater <clg@kaod.org> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
| * typedefs: add QJSONGreg Kurz2018-06-153-4/+1Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since commit 83ee768d6247b, we now have two places that define the QJSON type: $ git grep 'typedef struct QJSON QJSON' include/migration/vmstate.h:typedef struct QJSON QJSON; migration/qjson.h:typedef struct QJSON QJSON; This breaks docker-test-build@centos6: In file included from /tmp/qemu-test/src/migration/savevm.c:59: /tmp/qemu-test/src/migration/qjson.h:16: error: redefinition of typedef 'QJSON' /tmp/qemu-test/src/include/migration/vmstate.h:30: note: previous declaration of 'QJSON' was here make: *** [migration/savevm.o] Error 1 This happens because CentOS 6 has an old GCC 4.4.7. Even if redefining a typedef with the same type is permitted since GCC 4.6, unless -pedantic is passed, we don't really need to do that on purpose. Let's have a single definition in <qemu/typedefs.h> instead. Signed-off-by: Greg Kurz <groug@kaod.org> Message-Id: <152844714981.11789.3657734445739553287.stgit@bahia.lan> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* | Merge remote-tracking branch ↵Peter Maydell2018-06-152-12/+4Star
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 'remotes/edgar/tags/edgar/xilinx-next-2018-06-15.for-upstream' into staging xilinx-next-2018-06-15.for-upstream # gpg: Signature made Fri 15 Jun 2018 15:32:47 BST # gpg: using RSA key 29C596780F6BCA83 # gpg: Good signature from "Edgar E. Iglesias (Xilinx key) <edgar.iglesias@xilinx.com>" # gpg: aka "Edgar E. Iglesias <edgar.iglesias@gmail.com>" # Primary key fingerprint: AC44 FEDC 14F7 F1EB EDBF 4151 29C5 9678 0F6B CA83 * remotes/edgar/tags/edgar/xilinx-next-2018-06-15.for-upstream: target-microblaze: Rework NOP/zero instruction handling target-microblaze: mmu: Correct masking of output addresses Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * | target-microblaze: Rework NOP/zero instruction handlingEdgar E. Iglesias2018-06-151-11/+4Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove the abort on a sequence of NOP/zero instructions. Always return early and avoid decoding NOP/zero instructions. This fixes Coverity CID 1391443. Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
| * | target-microblaze: mmu: Correct masking of output addressesEdgar E. Iglesias2018-06-151-1/+0Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Correct the masking of output addresses. This fixes Coverity CID 1391441. Fixes: commit 3924a9aa02 Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reported-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
* | | Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into stagingPeter Maydell2018-06-1555-1668/+1692
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Block layer patches: - Fix options that work only with -drive or -blockdev, but not with both, because of QDict type confusion - rbd: Add options 'auth-client-required' and 'key-secret' - Remove deprecated -drive options serial/addr/cyls/heads/secs/trans - rbd, iscsi: Remove deprecated 'filename' option - Fix 'qemu-img map' crash with unaligned image size - Improve QMP documentation for jobs # gpg: Signature made Fri 15 Jun 2018 15:20:03 BST # gpg: using RSA key 7F09B272C88F2FD6 # gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" # Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6 * remotes/kevin/tags/for-upstream: (26 commits) block: Remove dead deprecation warning code block: Remove deprecated -drive option serial block: Remove deprecated -drive option addr block: Remove deprecated -drive geometry options rbd: New parameter key-secret rbd: New parameter auth-client-required block: Fix -blockdev / blockdev-add for empty objects and arrays check-block-qdict: Cover flattening of empty lists and dictionaries check-block-qdict: Rename qdict_flatten()'s variables for clarity block-qdict: Simplify qdict_is_list() some block-qdict: Clean up qdict_crumple() a bit block-qdict: Tweak qdict_flatten_qdict(), qdict_flatten_qlist() block-qdict: Simplify qdict_flatten_qdict() block: Make remaining uses of qobject input visitor more robust block: Factor out qobject_input_visitor_new_flat_confused() block: Clean up a misuse of qobject_to() in .bdrv_co_create_opts() block: Fix -drive for certain non-string scalars block: Fix -blockdev for certain non-string scalars qobject: Move block-specific qdict code to block-qdict.c block: Add block-specific QDict header ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * | | block: Remove dead deprecation warning codeKevin Wolf2018-06-151-12/+0Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We removed all options from the 'deprecated' array, so the code is dead and can be removed as well. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com>
| * | | block: Remove deprecated -drive option serialKevin Wolf2018-06-1514-48/+8Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The -drive option serial was deprecated in QEMU 2.10. It's time to remove it. Tests need to be updated to set the serial number with -global instead of using the -drive option. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com>
| * | | block: Remove deprecated -drive option addrKevin Wolf2018-06-155-30/+2Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The -drive option addr was deprecated in QEMU 2.10. It's time to remove it. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com>
| * | | block: Remove deprecated -drive geometry optionsKevin Wolf2018-06-157-131/+9Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The -drive options cyls, heads, secs and trans were deprecated in QEMU 2.10. It's time to remove them. hd-geo-test tested both the old version with geometry options in -drive and the new one with -device. Therefore the code using -drive doesn't have to be replaced there, we just need to remove the -drive test cases. This in turn allows some simplification of the code. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com>
| * | | rbd: New parameter key-secretMarkus Armbruster2018-06-152-16/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Legacy -drive supports "password-secret" parameter that isn't available with -blockdev / blockdev-add. That's because we backed out our first try to provide it there due to interface design doubts, in commit 577d8c9a811, v2.9.0. This is the second try. It brings back the parameter, except it's named "key-secret" now. Let's review our reasons for backing out the first try, as stated in the commit message: * BlockdevOptionsRbd member @password-secret isn't actually a password, it's a key generated by Ceph. Addressed by the rename. * We're not sure where member @password-secret belongs (see the previous commit). See previous commit. * How @password-secret interacts with settings from a configuration file specified with @conf is undocumented. Not actually true, the documentation for @conf says "Values in the configuration file will be overridden by options specified via QAPI", and we've tested this. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | rbd: New parameter auth-client-requiredMarkus Armbruster2018-06-152-10/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Parameter auth-client-required lets you configure authentication methods. We tried to provide that in v2.9.0, but backed out due to interface design doubts (commit 464444fcc16). This commit is similar to what we backed out, but simpler: we use a list of enumeration values instead of a list of objects with a member of enumeration type. Let's review our reasons for backing out the first try, as stated in the commit message: * The implementation uses deprecated rados_conf_set() key "auth_supported". No biggie. Fixed: we use "auth-client-required". * The implementation makes -drive silently ignore invalid parameters "auth" and "auth-supported.*.X" where X isn't "auth". Fixable (in fact I'm going to fix similar bugs around parameter server), so again no biggie. That fix is commit 2836284db60. This commit doesn't bring the bugs back. * BlockdevOptionsRbd member @password-secret applies only to authentication method cephx. Should it be a variant member of RbdAuthMethod? We've had time to ponder, and we decided to stick to the way Ceph configuration works: the key configured separately, and silently ignored if the authentication method doesn't use it. * BlockdevOptionsRbd member @user could apply to both methods cephx and none, but I'm not sure it's actually used with none. If it isn't, should it be a variant member of RbdAuthMethod? Likewise. * The client offers a *set* of authentication methods, not a list. Should the methods be optional members of BlockdevOptionsRbd instead of members of list @auth-supported? The latter begs the question what multiple entries for the same method mean. Trivial question now that RbdAuthMethod contains nothing but @type, but less so when RbdAuthMethod acquires other members, such the ones discussed above. Again, we decided to stick to the way Ceph configuration works, except we make auth-client-required a list of enumeration values instead of a string containing keywords separated by delimiters. * How BlockdevOptionsRbd member @auth-supported interacts with settings from a configuration file specified with @conf is undocumented. I suspect it's untested, too. Not actually true, the documentation for @conf says "Values in the configuration file will be overridden by options specified via QAPI", and we've tested this. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | block: Fix -blockdev / blockdev-add for empty objects and arraysMarkus Armbruster2018-06-152-29/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | -blockdev and blockdev-add silently ignore empty objects and arrays in their argument. That's because qmp_blockdev_add() converts the argument to a flat QDict, and qdict_flatten() eats empty QDict and QList members. For instance, we ignore an empty BlockdevOptions member @cache. No real harm, as absent means the same as empty there. Thus, the flaw puts an artificial restriction on the QAPI schema: we can't have potentially empty objects and arrays within BlockdevOptions, except when they're optional and "empty" has the same meaning as "absent". Our QAPI schema satisfies this restriction (I checked), but it's a trap for the unwary, and a temptation to employ awkward workarounds for the wary. Let's get rid of it. Change qdict_flatten() and qdict_crumple() to treat empty dictionaries and lists exactly like scalars. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | check-block-qdict: Cover flattening of empty lists and dictionariesMarkus Armbruster2018-06-151-1/+13
| | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | check-block-qdict: Rename qdict_flatten()'s variables for clarityMarkus Armbruster2018-06-151-28/+29
| | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | block-qdict: Simplify qdict_is_list() someMarkus Armbruster2018-06-151-16/+11Star
| | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | block-qdict: Clean up qdict_crumple() a bitMarkus Armbruster2018-06-151-16/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When you mix scalar and non-scalar keys, whether you get an "already set as scalar" or an "already set as dict" error depends on qdict iteration order. Neither message makes much sense. Replace by ""Cannot mix scalar and non-scalar keys". This is similar to the message we get for mixing list and non-list keys. I find qdict_crumple()'s first loop hard to understand. Rearrange it and add a comment. Signed-off-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | block-qdict: Tweak qdict_flatten_qdict(), qdict_flatten_qlist()Markus Armbruster2018-06-151-5/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | qdict_flatten_qdict() skips copying scalars from @qdict to @target when the two are the same. Fair enough, but it uses a non-obvious test for "same". Replace it by the obvious one. While there, improve comments. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | block-qdict: Simplify qdict_flatten_qdict()Markus Armbruster2018-06-151-15/+3Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There's no need to restart the loop. We don't elsewhere, e.g. in qdict_extract_subqdict(), qdict_join() and qemu_opts_absorb_qdict(). Simplify accordingly. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | block: Make remaining uses of qobject input visitor more robustMarkus Armbruster2018-06-152-5/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remaining uses of qobject_input_visitor_new_keyval() in the block subsystem: * block_crypto_open_opts_init() Currently doesn't visit any non-string scalars, thus safe. It's called from - block_crypto_open_luks() Creates the QDict with qemu_opts_to_qdict_filtered(), which creates only string scalars, but has a TODO asking for other types. - qcow_open() - qcow2_open(), qcow2_co_invalidate_cache(), qcow2_reopen_prepare() * block_crypto_create_opts_init(), called from - block_crypto_co_create_opts_luks() Also creates the QDict with qemu_opts_to_qdict_filtered(). * vdi_co_create_opts() Also creates the QDict with qemu_opts_to_qdict_filtered(). Replace these uses by qobject_input_visitor_new_flat_confused() for robustness. This adds crumpling. Right now, that's a no-op, but if we ever extend these things in non-flat ways, crumpling will be needed. Signed-off-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | block: Factor out qobject_input_visitor_new_flat_confused()Markus Armbruster2018-06-1513-62/+53Star
| | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | block: Clean up a misuse of qobject_to() in .bdrv_co_create_opts()Markus Armbruster2018-06-156-30/+24Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The following pattern occurs in the .bdrv_co_create_opts() methods of parallels, qcow, qcow2, qed, vhdx and vpc: qobj = qdict_crumple_for_keyval_qiv(qdict, errp); qobject_unref(qdict); qdict = qobject_to(QDict, qobj); if (qdict == NULL) { ret = -EINVAL; goto done; } v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); [...] ret = 0; done: qobject_unref(qdict); [...] return ret; If qobject_to() fails, we return failure without setting errp. That's wrong. As far as I can tell, it cannot fail here. Clean it up anyway, by removing the useless conversion. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | block: Fix -drive for certain non-string scalarsMarkus Armbruster2018-06-153-30/+6Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The previous commit fixed -blockdev breakage due to misuse of the qobject input visitor's keyval flavor in bdrv_file_open(). The commit message explain why using the plain flavor would be just as wrong; it would break -drive. Turns out we break it in three places: nbd_open(), sd_open() and ssh_file_open(). They are even marked FIXME. Example breakage: $ qemu-system-x86 -drive node-name=n1,driver=nbd,server.type=inet,server.host=localhost,server.port=1234,server.numeric=off qemu-system-x86: -drive node-name=n1,driver=nbd,server.type=inet,server.host=localhost,server.port=1234,server.numeric=off: Invalid parameter type for 'numeric', expected: boolean Fix it the same way: replace qdict_crumple() by qdict_crumple_for_keyval_qiv(), and switch from plain to the keyval flavor. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | block: Fix -blockdev for certain non-string scalarsMarkus Armbruster2018-06-1511-9/+67
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Configuration flows through the block subsystem in a rather peculiar way. Configuration made with -drive enters it as QemuOpts. Configuration made with -blockdev / blockdev-add enters it as QAPI type BlockdevOptions. The block subsystem uses QDict, QemuOpts and QAPI types internally. The precise flow is next to impossible to explain (I tried for this commit message, but gave up after wasting several hours). What I can explain is a flaw in the BlockDriver interface that leads to this bug: $ qemu-system-x86_64 -blockdev node-name=n1,driver=nfs,server.type=inet,server.host=localhost,path=/foo/bar,user=1234 qemu-system-x86_64: -blockdev node-name=n1,driver=nfs,server.type=inet,server.host=localhost,path=/foo/bar,user=1234: Internal error: parameter user invalid QMP blockdev-add is broken the same way. Here's what happens. The block layer passes configuration represented as flat QDict (with dotted keys) to BlockDriver methods .bdrv_file_open(). The QDict's members are typed according to the QAPI schema. nfs_file_open() converts it to QAPI type BlockdevOptionsNfs, with qdict_crumple() and a qobject input visitor. This visitor comes in two flavors. The plain flavor requires scalars to be typed according to the QAPI schema. That's the case here. The keyval flavor requires string scalars. That's not the case here. nfs_file_open() uses the latter, and promptly falls apart for members @user, @group, @tcp-syn-count, @readahead-size, @page-cache-size, @debug. Switching to the plain flavor would fix -blockdev, but break -drive, because there the scalars arrive in nfs_file_open() as strings. The proper fix would be to replace the QDict by QAPI type BlockdevOptions in the BlockDriver interface. Sadly, that's beyond my reach right now. Next best would be to fix the block layer to always pass correctly typed QDicts to the BlockDriver methods. Also beyond my reach. What I can do is throw another hack onto the pile: have nfs_file_open() convert all members to string, so use of the keyval flavor actually works, by replacing qdict_crumple() by new function qdict_crumple_for_keyval_qiv(). The pattern "pass result of qdict_crumple() to qobject_input_visitor_new_keyval()" occurs several times more: * qemu_rbd_open() Same issue as nfs_file_open(), but since BlockdevOptionsRbd has only string members, its only a latent bug. Fix it anyway. * parallels_co_create_opts(), qcow_co_create_opts(), qcow2_co_create_opts(), bdrv_qed_co_create_opts(), sd_co_create_opts(), vhdx_co_create_opts(), vpc_co_create_opts() These work, because they create the QDict with qemu_opts_to_qdict_filtered(), which creates only string scalars. The function sports a TODO comment asking for better typing; that's going to be fun. Use qdict_crumple_for_keyval_qiv() to be safe. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qobject: Move block-specific qdict code to block-qdict.cMarkus Armbruster2018-06-157-1271/+1302
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Pure code motion, except for two brace placements and a comment tweaked to appease checkpatch. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | block: Add block-specific QDict headerMax Reitz2018-06-1526-17/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are numerous QDict functions that have been introduced for and are used only by the block layer. Move their declarations into an own header file to reflect that. While qdict_extract_subqdict() is in fact used outside of the block layer (in util/qemu-config.c), it is still a function related very closely to how the block layer works with nested QDicts, namely by sometimes flattening them. Therefore, its declaration is put into this header as well and util/qemu-config.c includes it with a comment stating exactly which function it needs. Suggested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20180509165530.29561-7-mreitz@redhat.com> [Copyright note tweaked, superfluous includes dropped] Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | iscsi: Drop deprecated -drive parameter "filename"Markus Armbruster2018-06-151-21/+2Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Parameter "filename" is deprecated since commit 5c3ad1a6a8f, v2.10.0. Time to get rid of it. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | rbd: Drop deprecated -drive parameter "filename"Markus Armbruster2018-06-151-16/+0Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Parameter "filename" is deprecated since commit 91589d9e5ca, v2.10.0. Time to get rid of it. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | jobs: fix verb references in docsJohn Snow2018-06-151-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These point to the job versions now, not the blockjob versions which don't really exist anymore. Except set-speed, which does. It sticks out like a sore thumb. This patch doesn't fix that, but it doesn't make it any worse, either. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | jobs: fix stale wordingJohn Snow2018-06-151-5/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During the design for manual completion, we decided not to use the "manual" property as a shorthand for both auto-dismiss and auto-finalize. Fix the wording. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | iotests: Add test 221 to catch qemu-img map regressionEric Blake2018-06-153-0/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Although qemu-img creates aligned files (by rounding up), it must also gracefully handle files that are not sector-aligned. Test that the bug fixed in the previous patch does not recur. It's a bit annoying that we can see the (implicit) hole past the end of the file on to the next sector boundary, so if we ever reach the point where we report a byte-accurate size rather than our current behavior of always rounding up, this test will probably need a slight modification. Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * | | qemu-img: Fix assert when mapping unaligned raw fileEric Blake2018-06-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit a290f085 exposed a latent bug in qemu-img map introduced during the conversion of block status to be byte-based. Earlier in commit 5e344dd8, the internal interface get_block_status() switched to take byte-based parameters, but still called a sector-based block layer function; as such, rounding was added in the lone caller to obey the contract. However, commit 237d78f8 changed get_block_status() to truly be byte-based, at which point rounding to sector boundaries can result in calling bdrv_block_status() with 'bytes == 0' (a coding error) when the boundary between data and a hole falls mid-sector (true for the past-EOF implicit hole present in POSIX files). Fix things by removing the rounding that is now no longer necessary. See also https://bugzilla.redhat.com/1589738 Fixes: 237d78f8 Reported-by: Dan Kenigsberg <danken@redhat.com> Reported-by: Nir Soffer <nsoffer@redhat.com> Reported-by: Maor Lipchuk <mlipchuk@redhat.com> CC: qemu-stable@nongnu.org Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* | | | Merge remote-tracking branch ↵Peter Maydell2018-06-1548-363/+4114
|\ \ \ \ | |_|_|/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 'remotes/pmaydell/tags/pull-target-arm-20180615' into staging target-arm and miscellaneous queue: * fix KVM state save/restore for GICv3 priority registers for high IRQ numbers * hw/arm/mps2-tz: Put ethernet controller behind PPC * hw/sh/sh7750: Convert away from old_mmio * hw/m68k/mcf5206: Convert away from old_mmio * hw/block/pflash_cfi02: Convert away from old_mmio * hw/watchdog/wdt_i6300esb: Convert away from old_mmio * hw/input/pckbd: Convert away from old_mmio * hw/char/parallel: Convert away from old_mmio * armv7m: refactor to get rid of armv7m_init() function * arm: Don't crash if user tries to use a Cortex-M CPU without an NVIC * hw/core/or-irq: Support more than 16 inputs to an OR gate * cpu-defs.h: Document CPUIOTLBEntry 'addr' field * cputlb: Pass cpu_transaction_failed() the correct physaddr * CODING_STYLE: Define our preferred form for multiline comments * Add and use new stn_*_p() and ldn_*_p() memory access functions * target/arm: More parts of the upcoming SVE support * aspeed_scu: Implement RNG register * m25p80: add support for two bytes WRSR for Macronix chips * exec.c: Handle IOMMUs being in the path of TCG CPU memory accesses * target/arm: Allow ARMv6-M Thumb2 instructions # gpg: Signature made Fri 15 Jun 2018 15:24:03 BST # gpg: using RSA key 3C2525ED14360CDE # gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>" # gpg: aka "Peter Maydell <pmaydell@gmail.com>" # gpg: aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>" # Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83 15CF 3C25 25ED 1436 0CDE * remotes/pmaydell/tags/pull-target-arm-20180615: (43 commits) target/arm: Allow ARMv6-M Thumb2 instructions exec.c: Handle IOMMUs in address_space_translate_for_iotlb() iommu: Add IOMMU index argument to translate method iommu: Add IOMMU index argument to notifier APIs iommu: Add IOMMU index concept to IOMMU API m25p80: add support for two bytes WRSR for Macronix chips aspeed_scu: Implement RNG register target/arm: Implement SVE Floating Point Arithmetic - Unpredicated Group target/arm: Implement SVE Integer Wide Immediate - Unpredicated Group target/arm: Implement FDUP/DUP target/arm: Implement SVE Integer Compare - Scalars Group target/arm: Implement SVE Predicate Count Group target/arm: Implement SVE Partition Break Group target/arm: Implement SVE Integer Compare - Immediate Group target/arm: Implement SVE Integer Compare - Vectors Group target/arm: Implement SVE Select Vectors Group target/arm: Implement SVE vector splice (predicated) target/arm: Implement SVE reverse within elements target/arm: Implement SVE copy to vector (predicated) target/arm: Implement SVE conditionally broadcast/extract element ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * | | target/arm: Allow ARMv6-M Thumb2 instructionsJulia Suvorova2018-06-151-5/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ARMv6-M supports 6 Thumb2 instructions. This patch checks for these instructions and allows their execution. Like Thumb2 cores, ARMv6-M always interprets BL instruction as 32-bit. This patch is required for future Cortex-M0 support. Signed-off-by: Julia Suvorova <jusual@mail.ru> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20180612204632.28780-1-jusual@mail.ru [PMM: move armv6m_insn[] and armv6m_mask[] closer to point of use, and mark 'const'. Check for M-and-not-v7 rather than M-and-6.] Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * | | exec.c: Handle IOMMUs in address_space_translate_for_iotlb()Peter Maydell2018-06-154-4/+140
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently we don't support board configurations that put an IOMMU in the path of the CPU's memory transactions, and instead just assert() if the memory region fonud in address_space_translate_for_iotlb() is an IOMMUMemoryRegion. Remove this limitation by having the function handle IOMMUs. This is mostly straightforward, but we must make sure we have a notifier registered for every IOMMU that a transaction has passed through, so that we can flush the TLB appropriately when any of the IOMMUs change their mappings. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 20180604152941.20374-5-peter.maydell@linaro.org