summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* aspeed/smc: Add some tracingCédric Le Goater2020-03-123-0/+27
| | | | | | | | | Signed-off-by: Cédric Le Goater <clg@kaod.org> Reviewed-by: Andrew Jeffery <andrew@aj.id.au> Reviewed-by: Joel Stanley <joel@jms.id.au> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20200206112645.21275-2-clg@kaod.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
* target/arm: Fix some comment typosPeter Maydell2020-03-122-2/+2
| | | | | | | | Fix a couple of comment typos. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200303174950.3298-5-peter.maydell@linaro.org
* target/arm: Recalculate hflags correctly after writes to CONTROLPeter Maydell2020-03-123-4/+16
| | | | | | | | | | | | | | | | | | A write to the CONTROL register can change our current EL (by writing to the nPRIV bit). That means that we can't assume that s->current_el is still valid in trans_MSR_v7m() when we try to rebuild the hflags. Add a new helper rebuild_hflags_m32_newel() which, like the existing rebuild_hflags_a32_newel(), recalculates the current EL from scratch, and use it in trans_MSR_v7m(). This fixes an assertion about an hflags mismatch when the guest changes privilege by writing to CONTROL. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200303174950.3298-4-peter.maydell@linaro.org
* target/arm: Update hflags in trans_CPS_v7m()Peter Maydell2020-03-121-1/+4
| | | | | | | | | | | For M-profile CPUs, the FAULTMASK value affects the CPU's MMU index (it changes the NegPri bit). We update the hflags after calls to the v7m_msr helper in trans_MSR_v7m() but forgot to do so in trans_CPS_v7m(). Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200303174950.3298-3-peter.maydell@linaro.org
* hw/intc/armv7m_nvic: Rebuild hflags on resetPeter Maydell2020-03-121-0/+6
| | | | | | | | | | | | | | | Some of an M-profile CPU's cached hflags state depends on state that's in our NVIC object. We already do an hflags rebuild when the NVIC registers are written, but we also need to do this on NVIC reset, because there's no guarantee that this will happen before the CPU reset. This fixes an assertion due to mismatched hflags which happens if the CPU is reset from inside a HardFault handler. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200303174950.3298-2-peter.maydell@linaro.org
* Merge remote-tracking branch 'remotes/pmaydell/tags/pull-docs-20200312' into ↵Peter Maydell2020-03-1231-378/+369Star
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | staging docs queue: * Remove some no longer needed texinfo infrastructure * Reorder the top level index docs to put most useful manuals first * Split the Arm target-specific info into sub-pages * Improve the Arm documentation a bit with info previously only on the wiki page # gpg: Signature made Thu 12 Mar 2020 11:42:10 GMT # gpg: using RSA key E1A5C593CD419DE28E8315CF3C2525ED14360CDE # gpg: issuer "peter.maydell@linaro.org" # gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>" [ultimate] # gpg: aka "Peter Maydell <pmaydell@gmail.com>" [ultimate] # gpg: aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>" [ultimate] # Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83 15CF 3C25 25ED 1436 0CDE * remotes/pmaydell/tags/pull-docs-20200312: docs: Be consistent about capitalization of 'Arm' docs: Move arm-cpu-features.rst into the system manual docs/system/target-arm.rst: Add some introductory text docs/system: Split target-arm.rst into sub-documents Makefile: Allow for subdirectories in Sphinx manual dependencies docs/qemu-option-trace.rst.inc: Remove redundant comment docs/index.rst, docs/index.html.in: Reorder manuals Makefile: Make all Sphinx documentation depend on the extensions docs/sphinx/hxtool.py: Remove STEXI/ETEXI support hxtool: Remove Texinfo generation support Update comments in .hx files that mention Texinfo Makefile: Remove redundant Texinfo related code Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * docs: Be consistent about capitalization of 'Arm'Peter Maydell2020-03-1219-29/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The company 'Arm' went through a rebranding some years back involving a recapitalization from 'ARM' to 'Arm'. As a result our documentation is a bit inconsistent between the two forms. It's not worth trying to update everywhere in QEMU, but it's easy enough to make docs/ consistent. Note that "ARMv8" and similar architecture names, and older CPU names like "ARM926" still retain all-caps. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com> Message-id: 20200309215818.2021-6-peter.maydell@linaro.org
| * docs: Move arm-cpu-features.rst into the system manualPeter Maydell2020-03-122-7/+7
| | | | | | | | | | | | | | | | | | | | | | Now we have somewhere to put arm-specific rst documentation, we can move arm-cpu-features.rst from the docs/ top level directory into docs/system/arm/. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com> Message-id: 20200309215818.2021-5-peter.maydell@linaro.org
| * docs/system/target-arm.rst: Add some introductory textPeter Maydell2020-03-121-2/+64
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now we've moved the various bits of per-board documentation into their own files, the top level document is a little bare. Add some introductory information, including a note that many of the board models we support are currently undocumented. (Most sections of this new text were originally written by me for the wiki page https://wiki.qemu.org/Documentation/Platforms/ARM) Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com> Message-id: 20200309215818.2021-4-peter.maydell@linaro.org
| * docs/system: Split target-arm.rst into sub-documentsPeter Maydell2020-03-1211-212/+249
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently the documentation for Arm system emulator targets is in a single target-arm.rst. This describes only some of the boards and often in a fairly abbreviated fashion. Restructure it so that each board has its own documentation file in the docs/system/arm/ subdirectory. This will hopefully encourage us to write board documentation that describes the board in detail, rather than a few brief paragraphs in a single long page. The table of contents should also help users to find the board they care about faster. Once the structure is in place we'll be able to move microvm.rst from the top-level docs/ directory. All the text from the old page is retained, except for the final paragraph ("A Linux 2.6 test image is available on the QEMU web site. More information is available in the QEMU mailing-list archive."), which is deleted. The git history shows this was originally added in reference to the integratorcp board (at that time the only Arm board that was supported), and has subsequently gradually been further and further separated from the integratorcp documentation by the insertion of other board documentation sections. It's extremely out of date and no longer accurate, since AFAICT there isn't an integratorcp kernel on the website any more; so better deleted than retained. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com> Message-id: 20200309215818.2021-3-peter.maydell@linaro.org
| * Makefile: Allow for subdirectories in Sphinx manual dependenciesPeter Maydell2020-03-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently we put 'docs/foo/*.rst' in the Make list of dependencies for the Sphinx 'foo' manual, which means all the files must be in the top level of that manual's directory. We'd like to be able to have subdirectories inside some of the manuals, so add 'docs/foo/*/*.rst' to the dependencies too. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200309215818.2021-2-peter.maydell@linaro.org
| * docs/qemu-option-trace.rst.inc: Remove redundant commentPeter Maydell2020-03-121-4/+0Star
| | | | | | | | | | | | | | | | | | | | | | The Texinfo version of the tracing options documentation has now been deleted, so we can remove the now-redundant comment at the top of the rST version that was reminding us that the two should be kept in sync. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200306171749.10756-8-peter.maydell@linaro.org
| * docs/index.rst, docs/index.html.in: Reorder manualsPeter Maydell2020-03-122-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now that qemu-doc.html is no longer present, the ordering of manuals within the top-level index page looks a bit odd. Reshuffle so that the manuals the user is most likely to be interested in are at the top of the list, and the reference material is at the bottom. Similarly, we reorder the index.rst file used as the base of the "all manuals in one" documentation for readthedocs. The new order is: * system * user * tools * interop * specs * QMP reference (if present) * Guest agent protocol reference (if present) * devel (if present) Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200306171749.10756-7-peter.maydell@linaro.org
| * Makefile: Make all Sphinx documentation depend on the extensionsPeter Maydell2020-03-121-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add the Python source files of our Sphinx extensions to the dependencies of the Sphinx manuals, so that if we edit the extension source code the manuals get rebuilt. Adding this dependency unconditionally means that we'll rebuild a manual even if it happens to not use the extension whose source file was changed, but this is simpler and less error prone, and it's unlikely that we'll be making frequent changes to the extensions. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200306171749.10756-6-peter.maydell@linaro.org
| * docs/sphinx/hxtool.py: Remove STEXI/ETEXI supportPeter Maydell2020-03-121-24/+4Star
| | | | | | | | | | | | | | | | | | | | Now that none of our input .hx files have STEXI/ETEXI blocks, we can remove the code in the Sphinx hxtool extension that supported parsing them. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200306171749.10756-5-peter.maydell@linaro.org
| * hxtool: Remove Texinfo generation supportPeter Maydell2020-03-121-77/+1Star
| | | | | | | | | | | | | | | | | | | | All the STEXI/ETEXI blocks and the Makfile rules that use them have now been removed from the codebase. We can remove the code from the hxtool script which handles the STEXI/ETEXI directives and the '-t' option. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200306171749.10756-4-peter.maydell@linaro.org
| * Update comments in .hx files that mention TexinfoPeter Maydell2020-03-123-12/+12
| | | | | | | | | | | | | | | | | | | | Update the header comments in .hx files that mention STEXI/ETEXI markup; this is now SRST/ERST as all these files have been converted to rST. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200306171749.10756-3-peter.maydell@linaro.org
| * Makefile: Remove redundant Texinfo related codePeter Maydell2020-03-121-10/+1Star
| | | | | | | | | | | | | | | | | | The recent conversion of qemu-doc.texi to rST forgot a few stray bits of makefile code that are now redundant. Remove them. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20200306171749.10756-2-peter.maydell@linaro.org
* | Merge remote-tracking branch 'remotes/gkurz/tags/9p-next-2020-03-10' into ↵Peter Maydell2020-03-121-2/+2
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | staging Fix the proxy fsdev so that it honours "readonly" and "writeout". # gpg: Signature made Tue 10 Mar 2020 15:16:39 GMT # gpg: using RSA key B4828BAF943140CEF2A3491071D4D5E5822F73D6 # gpg: Good signature from "Greg Kurz <groug@kaod.org>" [full] # gpg: aka "Gregory Kurz <gregory.kurz@free.fr>" [full] # gpg: aka "[jpeg image of size 3330]" [full] # Primary key fingerprint: B482 8BAF 9431 40CE F2A3 4910 71D4 D5E5 822F 73D6 * remotes/gkurz/tags/9p-next-2020-03-10: 9p/proxy: Fix export_flags Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * | 9p/proxy: Fix export_flagsGreg Kurz2020-03-101-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The common fsdev options are set by qemu_fsdev_add() before it calls the backend specific option parsing code. In the case of "proxy" this means "writeout" or "readonly" were simply ignored. This has been broken from the beginning. Reported-by: Stéphane Graber <stgraber@ubuntu.com> Signed-off-by: Greg Kurz <groug@kaod.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Christian Schoenebeck <qemu_oss@crudebyte.com> Message-Id: <158349633705.1237488.8895481990204796135.stgit@bahia.lan>
* | | tests: Disable dbus-vmstate-testPeter Maydell2020-03-121-1/+2
| |/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The dbus-vmstate-test has been failing in some Patchew configs since about the 6th March: dbus-daemon[9321]: Could not get password database information for UID of current process: User "???" unknown or no memory to allocate password entry ** ERROR:/tmp/qemu-test/src/tests/qtest/dbus-vmstate-test.c:114:get_connection: assertion failed (err == NULL): The connection is closed (g-io-error-quark, 18) cleaning up pid 9321 ERROR - Bail out! ERROR:/tmp/qemu-test/src/tests/qtest/dbus-vmstate-test.c:114:get_connection: assertion failed (err == NULL): The connection is closed (g-io-error-quark, 18) make: *** [/tmp/qemu-test/src/tests/Makefile.include:632: check-qtest-x86_64] Error 1 make: *** Waiting for unfinished jobs.... It's not clear why this is happening (perhaps a recently revealed race condition or a change in the patchew build environment?). For the moment, disable this test so that patchew test runs are useful and don't email the list with spurious failure mails. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20200310152141.13959-1-peter.maydell@linaro.org
* | Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-03-11' ↵Peter Maydell2020-03-1124-278/+749
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | into staging Block patches for the 5.0 softfreeze: - qemu-img measure for LUKS - Improve block-copy's performance by reducing inter-request dependencies - Make curl's detection of accept-ranges more robust - Memleak fixes - iotest fix # gpg: Signature made Wed 11 Mar 2020 13:19:01 GMT # gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40 # gpg: issuer "mreitz@redhat.com" # gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full] # Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40 * remotes/maxreitz/tags/pull-block-2020-03-11: block/block-copy: hide structure definitions block/block-copy: reduce intersecting request lock block/block-copy: rename start to offset in interfaces block/block-copy: refactor interfaces to use bytes instead of end block/block-copy: factor out find_conflicting_inflight_req block/block-copy: use block_status block/block-copy: specialcase first copy_range request block/block-copy: fix progress calculation job: refactor progress to separate object block/qcow2-threads: fix qcow2_decompress qemu-img: free memory before re-assign block/qcow2: do free crypto_opts in qcow2_close() iotests: Fix nonportable use of od --endian block/curl: HTTP header field names are case insensitive block/curl: HTTP header fields allow whitespace around values iotests: add 288 luks qemu-img measure test qemu-img: allow qemu-img measure --object without a filename luks: implement .bdrv_measure() luks: extract qcrypto_block_calculate_payload_offset() Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * | block/block-copy: hide structure definitionsVladimir Sementsov-Ogievskiy2020-03-114-62/+80
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Hide structure definitions and add explicit API instead, to keep an eye on the scope of the shared fields. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200311103004.7649-10-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | block/block-copy: reduce intersecting request lockVladimir Sementsov-Ogievskiy2020-03-111-24/+105
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, block_copy operation lock the whole requested region. But there is no reason to lock clusters, which are already copied, it will disturb other parallel block_copy requests for no reason. Let's instead do the following: Lock only sub-region, which we are going to operate on. Then, after copying all dirty sub-regions, we should wait for intersecting requests block-copy, if they failed, we should retry these new dirty clusters. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Message-Id: <20200311103004.7649-9-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | block/block-copy: rename start to offset in interfacesVladimir Sementsov-Ogievskiy2020-03-112-43/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | offset/bytes pair is more usual naming in block layer, let's use it. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200311103004.7649-8-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | block/block-copy: refactor interfaces to use bytes instead of endVladimir Sementsov-Ogievskiy2020-03-112-40/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We have a lot of "chunk_end - start" invocations, let's switch to bytes/cur_bytes scheme instead. While being here, improve check on block_copy_do_copy parameters to not overflow when calculating nbytes and use int64_t for bytes in block_copy for consistency. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200311103004.7649-7-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | block/block-copy: factor out find_conflicting_inflight_reqVladimir Sementsov-Ogievskiy2020-03-111-12/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Split find_conflicting_inflight_req to be used separately. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200311103004.7649-6-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | block/block-copy: use block_statusVladimir Sementsov-Ogievskiy2020-03-112-13/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use bdrv_block_status_above to chose effective chunk size and to handle zeroes effectively. This substitutes checking for just being allocated or not, and drops old code path for it. Assistance by backup job is dropped too, as caching block-status information is more difficult than just caching is-allocated information in our dirty bitmap, and backup job is not good place for this caching anyway. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200311103004.7649-5-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | block/block-copy: specialcase first copy_range requestVladimir Sementsov-Ogievskiy2020-03-111-10/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In block_copy_do_copy we fallback to read+write if copy_range failed. In this case copy_size is larger than defined for buffered IO, and there is corresponding commit. Still, backup copies data cluster by cluster, and most of requests are limited to one cluster anyway, so the only source of this one bad-limited request is copy-before-write operation. Further patch will move backup to use block_copy directly, than for cases where copy_range is not supported, first request will be oversized in each backup. It's not good, let's change it now. Fix is simple: just limit first copy_range request like buffer-based request. If it succeed, set larger copy_range limit. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200311103004.7649-4-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | block/block-copy: fix progress calculationVladimir Sementsov-Ogievskiy2020-03-113-25/+19Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Assume we have two regions, A and B, and region B is in-flight now, region A is not yet touched, but it is unallocated and should be skipped. Correspondingly, as progress we have total = A + B current = 0 If we reset unallocated region A and call progress_reset_callback, it will calculate 0 bytes dirty in the bitmap and call job_progress_set_remaining, which will set total = current + 0 = 0 + 0 = 0 So, B bytes are actually removed from total accounting. When job finishes we'll have total = 0 current = B , which doesn't sound good. This is because we didn't considered in-flight bytes, actually when calculating remaining, we should have set (in_flight + dirty_bytes) as remaining, not only dirty_bytes. To fix it, let's refactor progress calculation, moving it to block-copy itself instead of fixing callback. And, of course, track in_flight bytes count. We still have to keep one callback, to maintain backup job bytes_read calculation, but it will go on soon, when we turn the whole backup process into one block_copy call. Cc: qemu-stable@nongnu.org Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Message-Id: <20200311103004.7649-3-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | job: refactor progress to separate objectVladimir Sementsov-Ogievskiy2020-03-116-25/+76
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We need it in separate to pass to the block-copy object in the next commit. Cc: qemu-stable@nongnu.org Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200311103004.7649-2-vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | block/qcow2-threads: fix qcow2_decompressVladimir Sementsov-Ogievskiy2020-03-111-5/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On success path we return what inflate() returns instead of 0. And it most probably works for Z_STREAM_END as it is positive, but is definitely broken for Z_BUF_ERROR. While being here, switch to errno return code, to be closer to qcow2_compress API (and usual expectations). Revert condition in if to be more positive. Drop dead initialization of ret. Cc: qemu-stable@nongnu.org # v4.0 Fixes: 341926ab83e2b Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20200302150930.16218-1-vsementsov@virtuozzo.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | qemu-img: free memory before re-assignPan Nengyuan2020-03-111-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | collect_image_check() is called twice in img_check(), the filename/format will be alloced without free the original memory. It is not a big deal since the process will exit anyway, but seems like a clean code and it will remove the warning spotted by asan. Reported-by: Euler Robot <euler.robot@huawei.com> Signed-off-by: Pan Nengyuan <pannengyuan@huawei.com> Message-Id: <20200227012950.12256-3-pannengyuan@huawei.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | block/qcow2: do free crypto_opts in qcow2_close()Pan Nengyuan2020-03-111-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 'crypto_opts' forgot to free in qcow2_close(), this patch fix the bellow leak stack: Direct leak of 24 byte(s) in 1 object(s) allocated from: #0 0x7f0edd81f970 in __interceptor_calloc (/lib64/libasan.so.5+0xef970) #1 0x7f0edc6d149d in g_malloc0 (/lib64/libglib-2.0.so.0+0x5249d) #2 0x55d7eaede63d in qobject_input_start_struct /mnt/sdb/qemu-new/qemu_test/qemu/qapi/qobject-input-visitor.c:295 #3 0x55d7eaed78b8 in visit_start_struct /mnt/sdb/qemu-new/qemu_test/qemu/qapi/qapi-visit-core.c:49 #4 0x55d7eaf5140b in visit_type_QCryptoBlockOpenOptions qapi/qapi-visit-crypto.c:290 #5 0x55d7eae43af3 in block_crypto_open_opts_init /mnt/sdb/qemu-new/qemu_test/qemu/block/crypto.c:163 #6 0x55d7eacd2924 in qcow2_update_options_prepare /mnt/sdb/qemu-new/qemu_test/qemu/block/qcow2.c:1148 #7 0x55d7eacd33f7 in qcow2_update_options /mnt/sdb/qemu-new/qemu_test/qemu/block/qcow2.c:1232 #8 0x55d7eacd9680 in qcow2_do_open /mnt/sdb/qemu-new/qemu_test/qemu/block/qcow2.c:1512 #9 0x55d7eacdc55e in qcow2_open_entry /mnt/sdb/qemu-new/qemu_test/qemu/block/qcow2.c:1792 #10 0x55d7eacdc8fe in qcow2_open /mnt/sdb/qemu-new/qemu_test/qemu/block/qcow2.c:1819 #11 0x55d7eac3742d in bdrv_open_driver /mnt/sdb/qemu-new/qemu_test/qemu/block.c:1317 #12 0x55d7eac3e990 in bdrv_open_common /mnt/sdb/qemu-new/qemu_test/qemu/block.c:1575 #13 0x55d7eac4442c in bdrv_open_inherit /mnt/sdb/qemu-new/qemu_test/qemu/block.c:3126 #14 0x55d7eac45c3f in bdrv_open /mnt/sdb/qemu-new/qemu_test/qemu/block.c:3219 #15 0x55d7ead8e8a4 in blk_new_open /mnt/sdb/qemu-new/qemu_test/qemu/block/block-backend.c:397 #16 0x55d7eacde74c in qcow2_co_create /mnt/sdb/qemu-new/qemu_test/qemu/block/qcow2.c:3534 #17 0x55d7eacdfa6d in qcow2_co_create_opts /mnt/sdb/qemu-new/qemu_test/qemu/block/qcow2.c:3668 #18 0x55d7eac1c678 in bdrv_create_co_entry /mnt/sdb/qemu-new/qemu_test/qemu/block.c:485 #19 0x55d7eb0024d2 in coroutine_trampoline /mnt/sdb/qemu-new/qemu_test/qemu/util/coroutine-ucontext.c:115 Reported-by: Euler Robot <euler.robot@huawei.com> Signed-off-by: Pan Nengyuan <pannengyuan@huawei.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200227012950.12256-2-pannengyuan@huawei.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | iotests: Fix nonportable use of od --endianEric Blake2020-03-111-5/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Tests 261 and 272 fail on RHEL 7 with coreutils 8.22, since od --endian was not added until coreutils 8.23. Fix this by manually constructing the final value one byte at a time. Fixes: fc8ba423 Reported-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200226125424.481840-1-eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | block/curl: HTTP header field names are case insensitiveDavid Edmondson2020-03-111-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | RFC 7230 section 3.2 indicates that HTTP header field names are case insensitive. Signed-off-by: David Edmondson <david.edmondson@oracle.com> Message-Id: <20200224101310.101169-3-david.edmondson@oracle.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | block/curl: HTTP header fields allow whitespace around valuesDavid Edmondson2020-03-111-4/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | RFC 7230 section 3.2 indicates that whitespace is permitted between the field name and field value and after the field value. Signed-off-by: David Edmondson <david.edmondson@oracle.com> Message-Id: <20200224101310.101169-2-david.edmondson@oracle.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | iotests: add 288 luks qemu-img measure testStefan Hajnoczi2020-03-113-0/+124
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This test exercises the block/crypto.c "luks" block driver .bdrv_measure() code. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200221112522.1497712-5-stefanha@redhat.com> [mreitz: Renamed test from 282 to 288] Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | qemu-img: allow qemu-img measure --object without a filenameStefan Hajnoczi2020-03-114-13/+11Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In most qemu-img sub-commands the --object option only makes sense when there is a filename. qemu-img measure is an exception because objects may be referenced from the image creation options instead of an existing image file. Allow --object without a filename. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200221112522.1497712-4-stefanha@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | luks: implement .bdrv_measure()Stefan Hajnoczi2020-03-111-0/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | | Add qemu-img measure support in the "luks" block driver. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200221112522.1497712-3-stefanha@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
| * | luks: extract qcrypto_block_calculate_payload_offset()Stefan Hajnoczi2020-03-113-55/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The qcow2 .bdrv_measure() code calculates the crypto payload offset. This logic really belongs in crypto/block.c where it can be reused by other image formats. The "luks" block driver will need this same logic in order to implement .bdrv_measure(), so extract the qcrypto_block_calculate_payload_offset() function now. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200221112522.1497712-2-stefanha@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* | | Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into ↵Peter Maydell2020-03-1111-313/+915
|\ \ \ | |/ / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | staging Pull request # gpg: Signature made Wed 11 Mar 2020 12:40:36 GMT # gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8 # gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full] # gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full] # Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8 * remotes/stefanha/tags/block-pull-request: aio-posix: remove idle poll handlers to improve scalability aio-posix: support userspace polling of fd monitoring aio-posix: add io_uring fd monitoring implementation aio-posix: simplify FDMonOps->update() prototype aio-posix: extract ppoll(2) and epoll(7) fd monitoring aio-posix: move RCU_READ_LOCK() into run_poll_handlers() aio-posix: completely stop polling when disabled aio-posix: remove confusing QLIST_SAFE_REMOVE() qemu/queue.h: clear linked list pointers on remove Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * | aio-posix: remove idle poll handlers to improve scalabilityStefan Hajnoczi2020-03-094-7/+98
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When there are many poll handlers it's likely that some of them are idle most of the time. Remove handlers that haven't had activity recently so that the polling loop scales better for guests with a large number of devices. This feature only takes effect for the Linux io_uring fd monitoring implementation because it is capable of combining fd monitoring with userspace polling. The other implementations can't do that and risk starving fds in favor of poll handlers, so don't try this optimization when they are in use. IOPS improves from 10k to 105k when the guest has 100 virtio-blk-pci,num-queues=32 devices and 1 virtio-blk-pci,num-queues=1 device for rw=randread,iodepth=1,bs=4k,ioengine=libaio on NVMe. [Clarified aio_poll_handlers locking discipline explanation in comment after discussion with Paolo Bonzini <pbonzini@redhat.com>. --Stefan] Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Link: https://lore.kernel.org/r/20200305170806.1313245-8-stefanha@redhat.com Message-Id: <20200305170806.1313245-8-stefanha@redhat.com>
| * | aio-posix: support userspace polling of fd monitoringStefan Hajnoczi2020-03-095-3/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Unlike ppoll(2) and epoll(7), Linux io_uring completions can be polled from userspace. Previously userspace polling was only allowed when all AioHandler's had an ->io_poll() callback. This prevented starvation of fds by userspace pollable handlers. Add the FDMonOps->need_wait() callback that enables userspace polling even when some AioHandlers lack ->io_poll(). For example, it's now possible to do userspace polling when a TCP/IP socket is monitored thanks to Linux io_uring. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Link: https://lore.kernel.org/r/20200305170806.1313245-7-stefanha@redhat.com Message-Id: <20200305170806.1313245-7-stefanha@redhat.com>
| * | aio-posix: add io_uring fd monitoring implementationStefan Hajnoczi2020-03-096-5/+376
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The recent Linux io_uring API has several advantages over ppoll(2) and epoll(2). Details are given in the source code. Add an io_uring implementation and make it the default on Linux. Performance is the same as with epoll(7) but later patches add optimizations that take advantage of io_uring. It is necessary to change how aio_set_fd_handler() deals with deleting AioHandlers since removing monitored file descriptors is asynchronous in io_uring. fdmon_io_uring_remove() marks the AioHandler deleted and aio_set_fd_handler() will let it handle deletion in that case. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Link: https://lore.kernel.org/r/20200305170806.1313245-6-stefanha@redhat.com Message-Id: <20200305170806.1313245-6-stefanha@redhat.com>
| * | aio-posix: simplify FDMonOps->update() prototypeStefan Hajnoczi2020-03-094-23/+22Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The AioHandler *node, bool is_new arguments are more complicated to think about than simply being given AioHandler *old_node, AioHandler *new_node. Furthermore, the new Linux io_uring file descriptor monitoring mechanism added by the new patch requires access to both the old and the new nodes. Make this change now in preparation. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Link: https://lore.kernel.org/r/20200305170806.1313245-5-stefanha@redhat.com Message-Id: <20200305170806.1313245-5-stefanha@redhat.com>
| * | aio-posix: extract ppoll(2) and epoll(7) fd monitoringStefan Hajnoczi2020-03-097-276/+366
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The ppoll(2) and epoll(7) file descriptor monitoring implementations are mixed with the core util/aio-posix.c code. Before adding another implementation for Linux io_uring, extract out the existing ones so there is a clear interface and the core code is simpler. The new interface is AioContext->fdmon_ops, a pointer to a FDMonOps struct. See the patch for details. Semantic changes: 1. ppoll(2) now reflects events from pollfds[] back into AioHandlers while we're still on the clock for adaptive polling. This was already happening for epoll(7), so if it's really an issue then we'll need to fix both in the future. 2. epoll(7)'s fallback to ppoll(2) while external events are disabled was broken when the number of fds exceeded the epoll(7) upgrade threshold. I guess this code path simply wasn't tested and no one noticed the bug. I didn't go out of my way to fix it but the correct code is simpler than preserving the bug. I also took some liberties in removing the unnecessary AioContext->epoll_available (just check AioContext->epollfd != -1 instead) and AioContext->epoll_enabled (it's implicit if our AioContext->fdmon_ops callbacks are being invoked) fields. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Link: https://lore.kernel.org/r/20200305170806.1313245-4-stefanha@redhat.com Message-Id: <20200305170806.1313245-4-stefanha@redhat.com>
| * | aio-posix: move RCU_READ_LOCK() into run_poll_handlers()Stefan Hajnoczi2020-03-091-10/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now that run_poll_handlers_once() is only called by run_poll_handlers() we can improve the CPU time profile by moving the expensive RCU_READ_LOCK() out of the polling loop. This reduces the run_poll_handlers() from 40% CPU to 10% CPU in perf's sampling profiler output. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Link: https://lore.kernel.org/r/20200305170806.1313245-3-stefanha@redhat.com Message-Id: <20200305170806.1313245-3-stefanha@redhat.com>
| * | aio-posix: completely stop polling when disabledStefan Hajnoczi2020-03-091-7/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | One iteration of polling is always performed even when polling is disabled. This is done because: 1. Userspace polling is cheaper than making a syscall. We might get lucky. 2. We must poll once more after polling has stopped in case an event occurred while stopping polling. However, there are downsides: 1. Polling becomes a bottleneck when the number of event sources is very high. It's more efficient to monitor fds in that case. 2. A high-frequency polling event source can starve non-polling event sources because ppoll(2)/epoll(7) is never invoked. This patch removes the forced polling iteration so that poll_ns=0 really means no polling. IOPS increases from 10k to 60k when the guest has 100 virtio-blk-pci,num-queues=32 devices and 1 virtio-blk-pci,num-queues=1 device because the large number of event sources being polled slows down the event loop. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Link: https://lore.kernel.org/r/20200305170806.1313245-2-stefanha@redhat.com Message-Id: <20200305170806.1313245-2-stefanha@redhat.com>
| * | aio-posix: remove confusing QLIST_SAFE_REMOVE()Stefan Hajnoczi2020-03-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | QLIST_SAFE_REMOVE() is confusing here because the node must be on the list. We actually just wanted to clear the linked list pointers when removing it from the list. QLIST_REMOVE() now does this, so switch to it. Suggested-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Link: https://lore.kernel.org/r/20200224103406.1894923-3-stefanha@redhat.com Message-Id: <20200224103406.1894923-3-stefanha@redhat.com>
generated by cgit v1.2.3-55-g7522 (git 2.39.0) at 2025-02-04 12:43:59 +0100