summaryrefslogtreecommitdiffstats
path: root/python
Commit message (Collapse)AuthorAgeFilesLines
* misc: fix commonly doubled up wordsDaniel P. Berrangé2022-08-012-2/+2
| | | | | | | Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20220707163720.1421716-5-berrange@redhat.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
* python/qemu/qmp/legacy: Replace 'returns-whitelist' with the correct typeThomas Huth2022-07-181-1/+1
| | | | | | | | | 'returns-whitelist' has been renamed to 'command-returns-exceptions' in commit b86df3747848 ("qapi: Rename pragma *-whitelist to *-exceptions"). Message-Id: <20220711095721.61280-1-thuth@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* Fix 'writeable' typosPeter Maydell2022-06-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | We have about 30 instances of the typo/variant spelling 'writeable', and over 500 of the more common 'writable'. Standardize on the latter. Change produced with: sed -i -e 's/\([Ww][Rr][Ii][Tt]\)[Ee]\([Aa][Bb][Ll][Ee]\)/\1\2/g' $(git grep -il writeable) and then hand-undoing the instance in linux-headers/linux/kvm.h. Most of these changes are in comments or documentation; the exceptions are: * a local variable in accel/hvf/hvf-accel-ops.c * a local variable in accel/kvm/kvm-all.c * the PMCR_WRITABLE_MASK macro in target/arm/internals.h * the EPT_VIOLATION_GPA_WRITABLE macro in target/i386/hvf/vmcs.h (which is never used anywhere) * the AR_TYPE_WRITABLE_MASK macro in target/i386/hvf/vmx.h (which is never used anywhere) Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Stefan Weil <sw@weilnetz.de> Message-id: 20220505095015.2714666-1-peter.maydell@linaro.org
* python: update for mypy 0.950John Snow2022-06-062-1/+4
| | | | | | | | | | | | | | | typeshed (included in mypy) recently updated to improve the typing for WriteTransport objects. I was working around this, but now there's a version where I shouldn't work around it. Unfortunately this creates some minor ugliness if I want to support both pre- and post-0.950 versions. For now, for my sanity, just disable the unused-ignores warning. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220526000921.1581503-2-jsnow@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* python/qmp: remove pylint workaround from legacy.pyJohn Snow2022-04-211-2/+0Star
| | | | | | | | | | | | | Pylint upgraded recently (2.13.z) and having a pylint: disable comment in the middle of an argument field causes it some grief (It appears to stop parsing when it encounters it, causing some syntax problems). Since the duplicate line threshold was bumped up in 22305c2a081b, we don't need this workaround anymore. Drop it. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@openvz.org> Message-id: 20220330172812.3427355-10-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python: rename 'aqmp-tui' to 'qmp-tui'John Snow2022-04-212-9/+9
| | | | | | | | | | This is the last vestige of the "aqmp" moniker surviving in the tree; remove it. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@openvz.org> Message-id: 20220330172812.3427355-9-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python: rename qemu.aqmp to qemu.qmpJohn Snow2022-04-2122-60/+60
| | | | | | | | | | | | | | | | | | | | | Now that we are fully switched over to the new QMP library, move it back over the old namespace. This is being done primarily so that we may upload this package simply as "qemu.qmp" without introducing confusion over whether or not "aqmp" is a new protocol or not. The trade-off is increased confusion inside the QEMU developer tree. Sorry! Note: the 'private' member "_aqmp" in legacy.py also changes to "_qmp"; not out of necessity, but just to remove any traces of the "aqmp" name. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Acked-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@openvz.org> Message-id: 20220330172812.3427355-8-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python: re-enable pylint duplicate-code warningsJohn Snow2022-04-211-1/+0Star
| | | | | | | | | | | With the old library gone, there's nothing duplicated in the tree, so the warning suppression can be removed. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Message-id: 20220330172812.3427355-7-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python: remove the old QMP packageJohn Snow2022-04-216-410/+4Star
| | | | | | | | | | Thank you for your service! Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Message-id: 20220330172812.3427355-6-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp: copy qmp docstrings to qemu.aqmp.legacyJohn Snow2022-04-211-8/+90
| | | | | | | | | | | | | | Copy the docstrings out of qemu.qmp, adjusting them as necessary to more accurately reflect the current state of this class. (Licensing: This is copying and modifying GPLv2-only licensed docstrings into a GPLv2-only file.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Message-id: 20220330172812.3427355-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp: fully separate from qmp.QEMUMonitorProtocolJohn Snow2022-04-211-6/+31
| | | | | | | | | | | | | | | After this patch, qemu.aqmp.legacy.QEMUMonitorProtocol no longer inherits from qemu.qmp.QEMUMonitorProtocol. To do this, several inherited methods need to be explicitly re-defined. (Licensing: This is copying and modifying GPLv2-only code into a GPLv2-only file.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Message-id: 20220330172812.3427355-4-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp: take QMPBadPortError and parse_address from qemu.qmpJohn Snow2022-04-213-32/+27Star
| | | | | | | | | | | | | Shift these definitions over from the qmp package to the async qmp package. (Licensing: this is a lateral move, from GPLv2 (only) to GPLv2 (only)) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Message-id: 20220330172812.3427355-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python: temporarily silence pylint duplicate-code warningsJohn Snow2022-04-211-0/+1
| | | | | | | | | | | | The next several commits copy some code from qemu.qmp to qemu.aqmp, then delete qemu.qmp. In the interim, to prevent test failures, the duplicate code detection needs to be silenced to prevent bisect problems with CI testing. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20220330172812.3427355-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp-tui: relicense as LGPLv2+John Snow2022-04-211-1/+1
| | | | | | | | | | | | | | | | | | | | | | aqmp-tui, the async QMP text user interface tool, is presently licensed as GPLv2+. I intend to include this tool as an add-on to an LGPLv2+ library package hosted on PyPI.org. I've selected LGPLv2+ to maximize compatibility with other licenses while retaining a copyleft license. To keep licensing matters simple, I'd like to relicense this tool as LGPLv2+ as well in order to keep the resultant license of the hosted release files simple -- even if library users won't "link against" this command line tool. Therefore, I am asking permission to loosen the license. Niteesh is effectively the sole author of this code, with scattered lines from myself. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: G S Niteesh Babu <niteesh.gs@gmail.com> Message-id: 20220325200438.2556381-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/qmp-shell: relicense as LGPLv2+John Snow2022-04-211-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | qmp-shell is presently licensed as GPLv2 (only). I intend to include this tool as an add-on to an LGPLv2+ library package hosted on PyPI.org. I've selected LGPLv2+ to maximize compatibility with other licenses while retaining a copyleft license. To keep licensing matters simple, I'd like to relicense this tool as LGPLv2+ as well in order to keep the resultant license of the hosted release files simple -- even if library users won't "link against" this command line tool. Therefore, I am asking permission from the current authors of this tool to loosen the license. At present, those people are: - John Snow (me!), 411/609 - Luiz Capitulino, Author, 97/609 - Daniel Berrangé, 81/609 - Eduardo Habkost, 10/609 - Marc-André Lureau, 6/609 - Fam Zheng, 3/609 - Cleber Rosa, 1/609 (All of which appear to have been written under redhat.com addresses.) Eduardo's fixes are largely automated from 2to3 conversion tools and may not necessarily constitute authorship, but his signature would put to rest any questions. Cleber's changes concern a single import statement change. Also won't hurt to ask. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Acked-by: Fam Zheng <fam@euphon.net> Acked-by: Luiz Capitulino <lcapitulino@redhat.com> Acked-by: Eduardo Habkost <eduardo@habkost.net> Acked-by: Daniel P. Berrangé <berrange@redhat.com> Acked-by: Cleber Rosa <crosa@redhat.com> Message-id: 20220325200438.2556381-4-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp: relicense as LGPLv2+John Snow2022-04-211-3/+3
| | | | | | | | | | | | | | | | | | | | | | | I am the sole author of all of the async QMP code (python/qemu/aqmp) with the following exceptions: python/qemu/aqmp/qmp_shell.py and python/qemu/aqmp/legacy.py were written by Luiz Capitulino (et al) and are already licensed separately as GPLv2 (only). aqmp_tui.py was written by Niteesh Babu G S and is licensed as GPLv2+. I wish to relicense as LGPLv2+ in order to provide as much flexibility as I reasonably can, while retaining a copyleft license. It is my belief that LGPLv2+ is a suitable license for the Python ecosystem that aligns with the goals and philosophy of the QEMU project. The intent is to eventually drop legacy.py, leaving only library code that is LGPLv2+. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20220325200438.2556381-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp: add explicit GPLv2 license to legacy.pyJohn Snow2022-04-211-0/+11
| | | | | | | | | | | | | The legacy.py module is heavily based on the QMP module by Luiz Capitulino (et al) which is licensed as explicit GPLv2-only. The async QMP package is currently licensed similarly, but I intend to relicense the async package to the more flexible LGPLv2+. In preparation for that change, make the license on legacy.py explicit. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20220325200438.2556381-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/machine: permanently switch to AQMPJohn Snow2022-04-212-12/+8Star
| | | | | | | | | | | | | Remove the QEMU_PYTHON_LEGACY_QMP environment variable, making the switch from sync qmp to async qmp permanent. Update exceptions and import paths as necessary. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Acked-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20220321203315.909411-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/utils: add VerboseProcessErrorJohn Snow2022-03-221-0/+39
| | | | | | | | | | | | | | | | This adds an Exception that extends the Python stdlib subprocess.CalledProcessError. The difference is that the str() method of this exception also adds the stdout/stderr logs. In effect, if this exception goes unhandled, Python will print the output in a visually distinct wrapper to the terminal so that it's easy to spot in a sea of traceback information. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20220321201618.903471-3-jsnow@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* python/utils: add add_visual_margin() text decoration utilityJohn Snow2022-03-221-0/+78
| | | | | | | | | | | | | | | | >>> print(add_visual_margin(msg, width=72, name="Commit Message")) ┏━ Commit Message ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ┃ add_visual_margin() takes a chunk of text and wraps it in a visual ┃ container that force-wraps to a specified width. An optional title ┃ label may be given, and any of the individual glyphs used to draw the ┃ box may be replaced or specified as well. ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Acked-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20220321201618.903471-2-jsnow@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* python/aqmp: drop _bind_hack()John Snow2022-03-072-39/+4Star
| | | | | | | | | | | | | | | | | _bind_hack() was a quick fix to allow async QMP to call bind(2) prior to calling listen(2) and accept(2). This wasn't sufficient to fully address the race condition present in synchronous clients. With the race condition in legacy.py fixed (see the previous commit), there are no longer any users of _bind_hack(). Drop it. Fixes: b0b662bb2b3 Signed-off-by: John Snow <jsnow@redhat.com> Acked-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20220225205948.3693480-11-jsnow@redhat.com [Expanded commit message. --js] Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp: fix race condition in legacy.pyJohn Snow2022-03-071-5/+2Star
| | | | | | | | | | | | | | | | | | | | | | | legacy.py provides a synchronous model. iotests frequently uses this paradigm: - create QMP client object - start QEMU process - await connection from QEMU process In the switch from sync to async QMP, the QMP client object stopped calling bind() and listen() during the QMP object creation step, which creates a race condition if the QEMU process dials in too quickly. With refactoring out of the way, restore the former behavior of calling bind() and listen() during __init__() to fix this race condition. Signed-off-by: John Snow <jsnow@redhat.com> Acked-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20220225205948.3693480-10-jsnow@redhat.com [Expanded commit message. --js] Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp: add start_server() and accept() methodsJohn Snow2022-03-072-5/+69
| | | | | | | | | | | | | | | | | | | | | Add start_server() and accept() methods that can be used instead of start_server_and_accept() to allow more fine-grained control over the incoming connection process. (Eagle-eyed reviewers will surely notice that it's a bit weird that "CONNECTING" is a state that's shared between both the start_server() and connect() states. That's absolutely true, and it's very true that checking on the presence of _accepted as an indicator of state is a hack. That's also very certainly true. But ... this keeps client code an awful lot simpler, as it doesn't have to care exactly *how* the connection is being made, just that it *is*. Is it worth disrupting that simplicity in order to provide a better state guard on `accept()`? Hm.) Signed-off-by: John Snow <jsnow@redhat.com> Acked-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20220225205948.3693480-9-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp: stop the server during disconnect()John Snow2022-03-071-1/+5
| | | | | | | | | | | | Before we allow the full separation of starting the server and accepting new connections, make sure that the disconnect cleans up the server and its new state, too. Signed-off-by: John Snow <jsnow@redhat.com> Acked-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20220225205948.3693480-8-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp: refactor _do_accept() into two distinct stepsJohn Snow2022-03-072-7/+26
| | | | | | | | | | | | | | | | Refactor _do_accept() into _do_start_server() and _do_accept(). As of this commit, the former calls the latter, but in subsequent commits they'll be split apart. (So please forgive the misnomer for _do_start_server(); it will live up to its name shortly, and the docstring will be updated then too. I'm just cutting down on some churn.) Signed-off-by: John Snow <jsnow@redhat.com> Acked-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20220225205948.3693480-7-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp: squelch pylint warning for too many linesJohn Snow2022-03-071-0/+3
| | | | | | | | | | | I would really like to keep this under 1000 lines, I promise. Doesn't look like it's gonna happen. Signed-off-by: John Snow <jsnow@redhat.com> Acked-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20220225205948.3693480-6-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp: split _client_connected_cb() out as _incoming()John Snow2022-03-071-25/+58
| | | | | | | | | | | | | As part of disentangling the monolithic nature of _do_accept(), split out the incoming callback to prepare for factoring out the "wait for a peer" step. Namely, this means using an event signal we can wait on from outside of this method. Signed-off-by: John Snow <jsnow@redhat.com> Acked-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20220225205948.3693480-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp: remove _new_session and _establish_connectionJohn Snow2022-03-072-74/+53Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | These two methods attempted to entirely envelop the logic of establishing a connection to a peer start to finish. However, we need to break apart the incoming connection step into more granular steps. We will no longer be able to reasonably constrain the logic inside of these helper functions. So, remove them - with _session_guard(), they no longer serve a real purpose. Although the public API doesn't change, the internal API does. Now that there are no intermediary methods between e.g. connect() and _do_connect(), there's no hook where the runstate is set. As a result, the test suite changes a little to cope with the new semantics of _do_accept() and _do_connect(). Lastly, take some pieces of the now-deleted docstrings and move them up to the public interface level. They were a little more detailed, and it won't hurt to keep them. Signed-off-by: John Snow <jsnow@redhat.com> Acked-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20220225205948.3693480-4-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp: rename 'accept()' to 'start_server_and_accept()'John Snow2022-03-073-15/+17
| | | | | | | | | | | | | | | | | | | Previously, I had a method named "accept()" that under-the-hood calls bind(2), listen(2) *and* accept(2). I meant this as a simplification and counterpart to the one-shot "connect()" method. This is confusing to readers who expect accept() to mean *just* accept(2). Since I need to split apart the "accept()" method into multiple methods anyway (one of which strongly resembling accept(2)), it feels pertinent to rename this method *now*. Rename this all-in-one method "start_server_and_accept()" instead. Signed-off-by: John Snow <jsnow@redhat.com> Acked-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20220225205948.3693480-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp: add _session_guard()John Snow2022-03-071-27/+62
| | | | | | | | | | | | | | | | | | | | | In _new_session, there's a fairly complex except clause that's used to give semantic errors to callers of accept() and connect(). We need to create a new two-step replacement for accept(), so factoring out this piece of logic will be useful. Bolster the comments and docstring here to try and demystify what's going on in this fairly delicate piece of Python magic. (If we were using Python 3.7+, this would be an @asynccontextmanager. We don't have that very nice piece of magic, however, so this must take an Awaitable to manage the Exception contexts properly. We pay the price for platform compatibility.) Signed-off-by: John Snow <jsnow@redhat.com> Acked-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20220225205948.3693480-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* Revert "python: pin setuptools below v60.0.0"John Snow2022-02-232-3/+0Star
| | | | | | | | This reverts commit 1e4d8b31be35e54b6429fea54f5ecaa0083f91e7. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20220204221804.2047468-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* Python: add setuptools v60.0 workaroundJohn Snow2022-02-232-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Setuptools v60 and later include a bundled version of distutils, a deprecated standard library scheduled for removal in future versions of Python. Setuptools v60 is only possible to install for Python 3.7 and later. Python has a distutils.sysconfig.get_python_lib() function that returns '/usr/lib/pythonX.Y' on posix systems. RPM-based systems actually use '/usr/lib64/pythonX.Y' instead, so Fedora patches stdlib distutils for Python 3.7 and Python 3.8 to return the correct value. Python 3.9 and later introduce a sys.platlibdir property, which returns the correct value on RPM-based systems. The change to a distutils package not provided by Fedora on Python 3.7 and 3.8 causes a regression in distutils.sysconfig.get_python_lib() that ultimately causes false positives to be emitted by pylint, because it can no longer find the system source libraries. Many Python tools are fairly aggressive about updating setuptools packages, and so even though this package is a fair bit newer than Python 3.7/3.8, it's not entirely unreasonable for a given user to have such a modern package with a fairly old Python interpreter. Updates to Python 3.7 and Python 3.8 are being produced for Fedora which will fix the problem on up-to-date systems. Until then, we can force the loading of platform-provided distutils when running the pylint test. This is the least-invasive yet most comprehensive fix. References: https://github.com/pypa/setuptools/pull/2896 https://github.com/PyCQA/pylint/issues/5704 https://github.com/pypa/distutils/issues/110 Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20220204221804.2047468-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* Python: discourage direct setup.py installJohn Snow2022-02-231-1/+18
| | | | | | | | | | | | | | | | | | | | | | When invoking setup.py directly, the default behavior for 'install' is to run the bdist_egg installation hook, which is ... actually deprecated by setuptools. It doesn't seem to work quite right anymore. By contrast, 'pip install' will invoke the bdist_wheel hook instead. This leads to differences in behavior for the two approaches. I advocate using pip in the documentation in this directory, but the 'setup.py' which has been used for quite a long time in the Python world may deceptively appear to work at first glance. Add an error message that will save a bit of time and frustration that points the user towards using the supported installation invocation. Reported-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Message-id: 20220207213039.2278569-1-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python: support recording QMP session to a fileDaniel P. Berrangé2022-02-232-7/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When running QMP commands with very large response payloads, it is often not easy to spot the info you want. If we can save the response to a file then tools like 'grep' or 'jq' can be used to extract information. For convenience of processing, we merge the QMP command and response dictionaries together: { "arguments": {}, "execute": "query-kvm", "return": { "enabled": false, "present": true } } Example usage $ ./scripts/qmp/qmp-shell-wrap -l q.log -p -- ./build/qemu-system-x86_64 -display none Welcome to the QMP low-level shell! Connected (QEMU) query-kvm { "return": { "enabled": false, "present": true } } (QEMU) query-mice { "return": [ { "absolute": false, "current": true, "index": 2, "name": "QEMU PS/2 Mouse" } ] } $ jq --slurp '. | to_entries[] | select(.value.execute == "query-kvm") | .value.return.enabled' < q.log false Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20220128161157.36261-3-berrange@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python: introduce qmp-shell-wrap convenience toolDaniel P. Berrangé2022-02-232-4/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | With the current 'qmp-shell' tool developers must first spawn QEMU with a suitable -qmp arg and then spawn qmp-shell in a separate terminal pointing to the right socket. With 'qmp-shell-wrap' developers can ignore QMP sockets entirely and just pass the QEMU command and arguments they want. The program will listen on a UNIX socket and tell QEMU to connect QMP to that. For example, this: # qmp-shell-wrap -- qemu-system-x86_64 -display none Is roughly equivalent of running: # qemu-system-x86_64 -display none -qmp qmp-shell-1234 & # qmp-shell qmp-shell-1234 Except that 'qmp-shell-wrap' switches the socket peers around so that it is the UNIX socket server and QEMU is the socket client. This makes QEMU reliably go away when qmp-shell-wrap exits, closing the server socket. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20220128161157.36261-2-berrange@redhat.com [Edited for rebase. --js] Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp: add socket bind step to legacy.pyJohn Snow2022-02-022-3/+41
| | | | | | | | | | | | | | | | | | | | | | | | The synchronous QMP library would bind to the server address during __init__(). The new library delays this to the accept() call, because binding occurs inside of the call to start_[unix_]server(), which is an async method -- so it cannot happen during __init__ anymore. Python 3.7+ adds the ability to create the server (and thus the bind() call) and begin the active listening in separate steps, but we don't have that functionality in 3.6, our current minimum. Therefore ... Add a temporary workaround that allows the synchronous version of the client to bind the socket in advance, guaranteeing that there will be a UNIX socket in the filesystem ready for the QEMU client to connect to without a race condition. (Yes, it's a bit ugly. Fixing it more nicely will have to wait until our minimum Python version is 3.7+.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20220201041134.1237016-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python: upgrade mypy to 0.780John Snow2022-02-022-28/+40
| | | | | | | | | | | | | | | We need a slightly newer version of mypy in order to use some features of the asyncio server functions in the next commit. (Note: pipenv is not really suited to upgrading individual packages; I need to replace this tool with something better for the task. For now, the miscellaneous updates not related to the mypy upgrade are simply beyond my control. It's on my list to take care of soon.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20220201041134.1237016-4-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/machine: raise VMLaunchFailure exception from launch()John Snow2022-02-021-6/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | This allows us to pack in some extra information about the failure, which guarantees that if the caller did not *intentionally* cause a failure (by capturing this Exception), some pretty good clues will be printed at the bottom of the traceback information. This will help make failures in the event of a non-negative return code more obvious when they go unhandled; the current behavior in _post_shutdown() is to print a warning message only in the event of signal-based terminations (for negative return codes). (Note: In Python, catching BaseException instead of Exception catches a broader array of Exception events, including SystemExit and KeyboardInterrupt. We do not want to "wrap" such exceptions as a VMLaunchFailure, because that will 'downgrade' the exception from a BaseException to a regular Exception. We do, however, want to perform cleanup in either case, so catch on the broadest scope and wrap-and-re-raise only in the more targeted scope.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20220201041134.1237016-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/aqmp: Fix negotiation with pre-"oob" QEMUJohn Snow2022-02-021-2/+2
| | | | | | | | | | | | QEMU versions prior to the "oob" capability *also* can't accept the "enable" keyword argument at all. Fix the handshake process with older QEMU versions. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20220201041134.1237016-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python: move qmp-shell under the AQMP packageJohn Snow2022-01-213-2/+2
| | | | | | Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Beraldo Leal <bleal@redhat.com>
* python: move qmp utilities to python/qemu/utilsJohn Snow2022-01-215-8/+8
| | | | | | | | | | | | | In order to upload a QMP package to PyPI, I want to remove any scripts that I am not 100% confident I want to support upstream, beyond our castle walls. Move most of our QMP utilities into the utils package so we can split them out from the PyPI upload. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Beraldo Leal <bleal@redhat.com>
* python/qmp: switch qmp-shell to AQMPJohn Snow2022-01-212-14/+20
| | | | | | | | We have a replacement for async QMP, but it doesn't have feature parity yet. For now, then, port the old tool onto the new backend. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* python/qmp: switch qom tools to AQMPJohn Snow2022-01-213-8/+11
| | | | | | Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Beraldo Leal <bleal@redhat.com>
* python/qmp: switch qemu-ga-client to AQMPJohn Snow2022-01-211-11/+11
| | | | | | | | | | | Async QMP always raises a "ConnectError" on any connection error which houses the cause in a second exception. We can check if this root cause was python's ConnectionError to determine a fairly similar condition to the original error check here. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Beraldo Leal <bleal@redhat.com>
* python/qemu-ga-client: don't use deprecated CLI syntax in usage commentJohn Snow2022-01-211-1/+1
| | | | | | | | | Cleanup related to commit ccd3b3b8112b670f, "qemu-option: warn for short-form boolean options". Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
* python/aqmp: rename AQMPError to QMPErrorJohn Snow2022-01-216-21/+21
| | | | | | | | | This is in preparation for renaming qemu.aqmp to qemu.qmp. I should have done this from this from the very beginning, but it's a convenient time to make sure this churn is taken care of. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* python/aqmp: add SocketAddrT to package rootJohn Snow2022-01-211-1/+9
| | | | | | | | It's a commonly needed definition, it can be re-exported by the root. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Beraldo Leal <bleal@redhat.com>
* python/aqmp: copy type definitions from qmpJohn Snow2022-01-212-8/+30
| | | | | | | | | | Copy the remaining type definitions from QMP into the qemu.aqmp.legacy module. Now, users that require the legacy interface don't need to import anything else but qemu.aqmp.legacy wrapper. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Beraldo Leal <bleal@redhat.com>
* python/aqmp: handle asyncio.TimeoutError on execute()John Snow2022-01-211-2/+6
| | | | | | | | | | This exception can be injected into any await statement. If we are canceled via timeout, we want to clear the pending execution record on our way out. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* python/aqmp: add __del__ method to legacy interfaceJohn Snow2022-01-211-0/+18
| | | | | | | | | | | | | | | | | | | | | | | asyncio can complain *very* loudly if you forget to back out of things gracefully before the garbage collector starts destroying objects that contain live references to asyncio Tasks. The usual fix is just to remember to call aqmp.disconnect(), but for the sake of the legacy wrapper and quick, one-off scripts where a graceful shutdown is not necessarily of paramount imporance, add a courtesy cleanup that will trigger prior to seeing screenfuls of confusing asyncio tracebacks. Note that we can't *always* save you from yourself; depending on when the GC runs, you might just seriously be out of luck. The best we can do in this case is to gently remind you to clean up after yourself. (Still much better than multiple pages of incomprehensible python warnings for the crime of forgetting to put your toys away.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Beraldo Leal <bleal@redhat.com>
generated by cgit v1.2.3-55-g7522 (git 2.39.0) at 2024-06-05 11:31:56 +0200