summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Remove superfluous .gitignore filesThomas Huth2020-10-139-59/+0Star
| | | | | | | | Since we are now always doing out-of-tree builds, these gitignore files should not be necessary anymore. Message-Id: <20200919133637.72744-1-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* MAINTAINERS: Ignore bios-tables-test in the qtest sectionThomas Huth2020-10-131-1/+1
| | | | | | | | | | | | | | | I'm very often getting CC: on rather large patch series that modify the ACPI stuff of either ARM or x86, just because the bios-table-test is often slightly involved here. I can't say much about ACPI, and the bios-table-test is already covered by the ACPI section in MAINTAINERS, so I'd rather prefer to not getting automatically CC-ed on such patch series anymore. If people want my opinion about qtest-related changes, they can still put me on CC manually. Message-Id: <20201001042717.136033-1-thuth@redhat.com> Acked-by: Laszlo Ersek <lersek@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* Add a comment in bios-tables-test.c to clarify the reason behind approachAni Sinha2020-10-131-1/+6
| | | | | | | | | | | | | A comment is added in bios-tables-test.c that explains the reasoning behind the process of updating the ACPI table blobs when new tests are added or old tests are modified or code is committed that affect tests. The explanation would help future contributors follow the correct process when making code changes that affect ACPI tables. Signed-off-by: Ani Sinha <ani@anisinha.ca> Acked-by: Igor Mammedov <imammedo@redhat.com> Message-Id: <20200929142501.1057-1-ani@anisinha.ca> Signed-off-by: Thomas Huth <thuth@redhat.com>
* softmmu/vl: Be less verbose about missing KVM when running the qtestsThomas Huth2020-10-131-7/+12
| | | | | | | | | | | | | | | | Some of the qtests use "-accel kvm -accel tcg" to run real guest code. This causes some error messages when kvm is not available. We do not really care about these messages since the fallback to tcg is expected here. So let's silence them to avoid that they spoil the output of the tests. Unfortunately, we can not use the qtest_enabled() wrapper in this case, since the qtest accelerator itself is not initialized. Thus we have to test for the qtest_chrdev variable instead. Message-Id: <20200710085020.28222-1-thuth@redhat.com> Reviewed-by: Alexander Bulekov <alxndr@bu.edu> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/migration: Allow longer timeoutsDr. David Alan Gilbert2020-10-131-10/+11
| | | | | | | | | | | | | | | | | | | | | | | | | In travis, with gcov and gprof we're seeing timeouts; hopefully fix this by increasing the test timeouts a bit, but for xbzrle ensure it really does get a couple of cycles through to test the cache. I think the problem in travis is we have about 2 host CPU threads, in the test we have at least 3: a) The vCPU thread (100% flat out) b) The source migration thread c) The destination migration thread if (b) & (c) are slow for any reason - gcov+gperf or a slow host - then they're sharing one host CPU thread so limit the migration bandwidth. Tested on my laptop with: taskset -c 0,1 ./tests/qtest/migration-test -p /x86_64/migration Reported-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <20201008160330.130431-1-dgilbert@redhat.com> [thuth: Move the #define to the right location] Signed-off-by: Thomas Huth <thuth@redhat.com>
* qtest: add fuzz test caseLi Qiang2020-10-132-0/+50
| | | | | | | | | | | | | | | | | | | | | | Currently the device fuzzer finds more and more issues. For every fuzz case, we need not only the fixes but also the corresponding test case. We can analysis the reproducer for every case and find what happened in where and write a beautiful test case. However the raw data of reproducer is not friendly to analysis. It will take a very long time, even far more than the fixes itself. So let's create a new file to hold all of the fuzz test cases and just use the raw data to act as the test case. This way nobody will be afraid of writing a test case for the fuzz reproducer. This patch adds the issue LP#1878263 test case. Signed-off-by: Li Qiang <liq3ea@163.com> Message-Id: <20200921160605.19329-1-liq3ea@163.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Alexander Bulekov <alxndr@bu.edu> [thuth: Slightly adjusted commit message, removed empty lines] Signed-off-by: Thomas Huth <thuth@redhat.com>
* Acceptance tests: show test report on GitLab CICleber Rosa2020-10-131-0/+5
| | | | | | | | | | | Avocado will, by default, produce JUnit files. Let's ask GitLab to present those in the web UI. Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009205513.751968-4-crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* Acceptance tests: do not show canceled test logs on GitLab CICleber Rosa2020-10-131-1/+1
| | | | | | | | | | | | | | | | | Tests resulting in "CANCEL" in Avocado are usually canceled on purpose, and are almost identical to "SKIP". The logs for canceled tests are adding a lot of noise to the logs being shown on GitLab CI, and causing distraction from real failures. As a side note, this "after script" is scheduled for removal once the feature is implemented within Avocado itself. Reference: https://github.com/avocado-framework/avocado/issues/4266 Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009205513.751968-3-crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* Acceptance tests: bump pycdlib version for easier installationCleber Rosa2020-10-131-1/+1
| | | | | | | | | | | | | | | | | | | On with certain versions of "pip", package installations will attempt to create wheels. And, on environments without a "complete" Python installation (as described in the acceptance tests requirements docs), that will fail. pycdlib, starting with version 1.11.0, is now being made available as wheels, so its instalation on those constrained environments is now possible. Buglink: https://bugs.launchpad.net/qemu/+bug/1897783 Reported-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009205513.751968-2-crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* gitlab-ci.yml: Only run one test-case per fuzzerAlexander Bulekov2020-10-131-1/+1
| | | | | | | | | | | | | | | With 1000 runs, there is a non-negligible chance that the fuzzer can trigger a crash. With this CI job, we care about catching build/runtime issues in the core fuzzing code. Actual device fuzzing takes place on oss-fuzz. For these purposes, only running one input should be sufficient. Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Suggested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20201002143524.56930-1-alxndr@bu.edu> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/qtest: Replace magic value by NANOSECONDS_PER_SECOND definitionPhilippe Mathieu-Daudé2020-10-131-1/+1
| | | | | | | | | | Use self-explicit NANOSECONDS_PER_SECOND definition instead of a magic value. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20201011194918.3219195-5-f4bug@amsat.org> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* Merge remote-tracking branch ↵Peter Maydell2020-10-127-37/+57
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 'remotes/dgilbert/tags/pull-migration-20201012a' into staging v3 Migration+ virtiofsd pull 2020-10-12 V3 Remove the postcopy recovery changes Migration: Dirtyrate measurement API cleanup Virtiofsd: Missing qemu_init_exec_dir call Support for setting the group on socket creation Stop a gcc warning Avoid tempdir in sandboxing # gpg: Signature made Mon 12 Oct 2020 12:43:30 BST # gpg: using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7 # gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full] # Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A 9FA9 0516 331E BC5B FDE7 * remotes/dgilbert/tags/pull-migration-20201012a: migration/dirtyrate: present dirty rate only when querying the rate has completed migration/dirtyrate: record start_time and calc_time while at the measuring state virtiofsd: avoid /proc/self/fd tempdir virtiofsd: Call qemu_init_exec_dir tools/virtiofsd: add support for --socket-group virtiofsd: Silence gcc warning Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * migration/dirtyrate: present dirty rate only when querying the rate has ↵Chuan Zheng2020-10-122-7/+4Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | completed Make dirty_rate field optional, present dirty rate only when querying the rate has completed. The qmp results is shown as follow: @unstarted: {"return":{"status":"unstarted","start-time":0,"calc-time":0},"id":"libvirt-12"} @measuring: {"return":{"status":"measuring","start-time":102931,"calc-time":1},"id":"libvirt-85"} @measured: {"return":{"status":"measured","dirty-rate":4,"start-time":150146,"calc-time":1},"id":"libvirt-15"} Signed-off-by: Chuan Zheng <zhengchuan@huawei.com> Reviewed-by: David Edmondson <david.edmondson@oracle.com> Message-Id: <1601350938-128320-3-git-send-email-zhengchuan@huawei.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
| * migration/dirtyrate: record start_time and calc_time while at the measuring ↵Chuan Zheng2020-10-121-4/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | state Querying could include both the start-time and the calc-time while at the measuring state, allowing a caller to determine when they should expect to come back looking for a result. Signed-off-by: Chuan Zheng <zhengchuan@huawei.com> Message-Id: <1601350938-128320-2-git-send-email-zhengchuan@huawei.com> Reviewed-by: David Edmondson <david.edmondson@oracle.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
| * virtiofsd: avoid /proc/self/fd tempdirStefan Hajnoczi2020-10-121-23/+11Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In order to prevent /proc/self/fd escapes a temporary directory is created where /proc/self/fd is bind-mounted. This doesn't work on read-only file systems. Avoid the temporary directory by bind-mounting /proc/self/fd over /proc. This does not affect other processes since we remounted / with MS_REC | MS_SLAVE. /proc must exist and virtiofsd does not use it so it's safe to do this. Path traversal can be tested with the following function: static void test_proc_fd_escape(struct lo_data *lo) { int fd; int level = 0; ino_t last_ino = 0; fd = lo->proc_self_fd; for (;;) { struct stat st; if (fstat(fd, &st) != 0) { perror("fstat"); return; } if (last_ino && st.st_ino == last_ino) { fprintf(stderr, "inode number unchanged, stopping\n"); return; } last_ino = st.st_ino; fprintf(stderr, "Level %d dev %lu ino %lu\n", level, (unsigned long)st.st_dev, (unsigned long)last_ino); fd = openat(fd, "..", O_PATH | O_DIRECTORY | O_NOFOLLOW); level++; } } Before and after this patch only Level 0 is displayed. Without /proc/self/fd bind-mount protection it is possible to traverse parent directories. Fixes: 397ae982f4df4 ("virtiofsd: jail lo->proc_self_fd") Cc: Miklos Szeredi <mszeredi@redhat.com> Cc: Jens Freimann <jfreimann@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20201006095826.59813-1-stefanha@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Tested-by: Jens Freimann <jfreimann@redhat.com> Reviewed-by: Jens Freimann <jfreimann@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
| * virtiofsd: Call qemu_init_exec_dirDr. David Alan Gilbert2020-10-121-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Since fcb4f59c879 qemu_get_local_state_pathname relies on the init_exec_dir, and virtiofsd asserts because we never set it. Set it. Reported-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <20201002124015.44820-1-dgilbert@redhat.com> Tested-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
| * tools/virtiofsd: add support for --socket-groupAlex Bennée2020-10-124-2/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | If you like running QEMU as a normal user (very common for TCG runs) but you have to run virtiofsd as a root user you run into connection problems. Adding support for an optional --socket-group allows the users to keep using the command line. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20200925125147.26943-2-alex.bennee@linaro.org> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> dgilbert: Split long line
| * virtiofsd: Silence gcc warningDr. David Alan Gilbert2020-10-121-1/+1
|/ | | | | | | | | | | Gcc worries fd might be used unset, in reality it's always set if fi is set, and only used if fi is set so it's safe. Initialise it to -1 just to keep gcc happy for now. Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <20200827153657.111098-2-dgilbert@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2020-10-10' into ↵Peter Maydell2020-10-1219-344/+767
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | staging QAPI patches patches for 2020-10-10 # gpg: Signature made Sat 10 Oct 2020 10:43:14 BST # gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653 # gpg: issuer "armbru@redhat.com" # gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full] # gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full] # Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653 * remotes/armbru/tags/pull-qapi-2020-10-10: (34 commits) qapi/visit.py: add type hint annotations qapi/visit.py: remove unused parameters from gen_visit_object qapi/visit.py: assert tag_member contains a QAPISchemaEnumType qapi/types.py: remove one-letter variables qapi/types.py: add type hint annotations qapi/gen.py: delint with pylint qapi/gen.py: update write() to be more idiomatic qapi/gen.py: Remove unused parameter qapi/gen.py: add type hint annotations qapi/gen: Make _is_user_module() return bool qapi/source.py: delint with pylint qapi/source.py: add type hint annotations qapi/commands.py: add type hint annotations qapi/commands.py: Don't re-bind to variable of different type qapi/events.py: Move comments into docstrings qapi/events.py: add type hint annotations qapi: establish mypy type-checking baseline qapi/common.py: move build_params into gen.py qapi/common.py: Convert comments into docstrings, and elaborate qapi/common.py: add type hint annotations ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * qapi/visit.py: add type hint annotationsJohn Snow2020-10-102-22/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | Annotations do not change runtime behavior. This commit *only* adds annotations. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-37-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/visit.py: remove unused parameters from gen_visit_objectJohn Snow2020-10-102-3/+2Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | And this fixes the pylint report for this file, so make sure we check this in the future, too. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20201009161558.107041-36-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/visit.py: assert tag_member contains a QAPISchemaEnumTypeJohn Snow2020-10-101-5/+7
| | | | | | | | | | | | | | | | | | | | | | | | This is true by design, but not presently able to be expressed in the type system. An assertion helps mypy understand our constraints. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-35-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/types.py: remove one-letter variablesJohn Snow2020-10-102-15/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | "John, if pylint told you to jump off a bridge, would you?" Hey, if it looked like fun, I might. Now that this file is clean, enable pylint checks on this file. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-34-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/types.py: add type hint annotationsJohn Snow2020-10-102-27/+64
| | | | | | | | | | | | | | | | | | | | | | | | Annotations do not change runtime behavior. This commit *only* adds annotations. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-33-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/gen.py: delint with pylintJohn Snow2020-10-102-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 'fp' and 'fd' are self-evident in context, add them to the list of OK names. _top and _bottom also need to stay standard methods because some users override the method and need to use `self`. Tell pylint to shush. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-32-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/gen.py: update write() to be more idiomaticJohn Snow2020-10-101-14/+11Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make the file handling here just a tiny bit more idiomatic. (I realize this is heavily subjective.) Use exist_ok=True for os.makedirs and remove the exception, use fdopen() to wrap the file descriptor in a File-like object, and use a context manager for managing the file pointer. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20201009161558.107041-31-jsnow@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/gen.py: Remove unused parameterJohn Snow2020-10-101-2/+2
| | | | | | | | | | | | | | | | | | | | | | _module_dirname doesn't use the 'what' argument, so remove it. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-30-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/gen.py: add type hint annotationsJohn Snow2020-10-102-52/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Annotations do not change runtime behavior. This commit *only* adds annotations. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-28-jsnow@redhat.com> Message-Id: <20201009161558.107041-29-jsnow@redhat.com> [mypy.ini update squashed in] Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/gen: Make _is_user_module() return boolJohn Snow2020-10-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | _is_user_module() returns thruth values. The next commit wants it to return bool. Make it so. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20201009161558.107041-27-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Commit message rewritten] Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/source.py: delint with pylintJohn Snow2020-10-102-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | Shush an error and leave a hint for future cleanups when we're allowed to use Python 3.7+. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-26-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/source.py: add type hint annotationsJohn Snow2020-10-102-18/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Annotations do not change runtime behavior. This commit *only* adds annotations. A note on typing of __init__: mypy requires init functions with no parameters to document a return type of None to be considered fully typed. In the case when there are input parameters, None may be omitted. Since __init__ may never return any value, it is preferred to omit the return annotation whenever possible. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-25-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/commands.py: add type hint annotationsJohn Snow2020-10-102-23/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Annotations do not change runtime behavior. This commit *only* adds annotations. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-23-jsnow@redhat.com> Message-Id: <20201009161558.107041-24-jsnow@redhat.com> [mypy.ini update squashed in] Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/commands.py: Don't re-bind to variable of different typeJohn Snow2020-10-101-3/+1Star
| | | | | | | | | | | | | | | | | | | | | | | | Mypy isn't a fan of rebinding a variable with a new data type. It's easy enough to avoid. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20201009161558.107041-22-jsnow@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/events.py: Move comments into docstringsJohn Snow2020-10-101-1/+5
| | | | | | | | | | | | | | | | | | | | | | Clarify them while we're here. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-21-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/events.py: add type hint annotationsJohn Snow2020-10-102-16/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Annotations do not change runtime behavior. This commit *only* adds annotations. Note: __init__ does not need its return type annotated, as it is special. https://mypy.readthedocs.io/en/stable/class_basics.html#annotating-init-methods Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-20-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi: establish mypy type-checking baselineJohn Snow2020-10-102-1/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix a minor typing issue, and then establish a mypy type-checking baseline. Like pylint, this should be run from the folder above: > mypy --config-file=qapi/mypy.ini qapi/ This is designed and tested for mypy 0.770 or greater. Signed-off-by: John Snow <jsnow@redhat.com> Tested-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-19-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/common.py: move build_params into gen.pyJohn Snow2020-10-104-34/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Including it in common.py creates a circular import dependency; schema relies on common, but common.build_params requires a type annotation from schema. To type this properly, it needs to be moved outside the cycle. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-18-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/common.py: Convert comments into docstrings, and elaborateJohn Snow2020-10-101-14/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As docstrings, they'll show up in documentation and IDE help. The docstring style being targeted is the Sphinx documentation style. Sphinx uses an extension of ReST with "domains". We use the (implicit) Python domain, which supports a number of custom "info fields". Those info fields are documented here: https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#info-field-lists Primarily, we use `:param X: descr`, `:return[s]: descr`, and `:raise[s] Z: when`. Everything else is the Sphinx dialect of ReST. (No, nothing checks or enforces this style that I am aware of. Sphinx either chokes or succeeds, but does not enforce a standard of what is otherwise inside the docstring. Pycharm does highlight when your param fields are not aligned with the actual fields present. It does not highlight missing return or exception statements. There is no existing style guide I am aware of that covers a standard for a minimally acceptable docstring. I am debating writing one.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-17-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/common.py: add type hint annotationsJohn Snow2020-10-101-11/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Annotations do not change runtime behavior. This commit *only* adds annotations. Note that build_params() cannot be fully annotated due to import dependency issues. The commit after next will take care of it. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-16-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/common.py: check with pylintJohn Snow2020-10-101-2/+1Star
| | | | | | | | | | | | | | | | | | | | | | | | Remove qapi/common.py from the pylintrc ignore list. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Tested-by: Eduardo Habkost <ehabkost@redhat.com> Message-Id: <20201009161558.107041-15-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/common.py: Replace one-letter 'c' variableJohn Snow2020-10-101-4/+4
| | | | | | | | | | | | | | | | | | | | Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20201009161558.107041-14-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/common.py: delint with pylintJohn Snow2020-10-102-17/+15Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | At this point, that just means using a consistent strategy for constant names. constants get UPPER_CASE and names not used externally get a leading underscore. As a preference, while renaming constants to be UPPERCASE, move them to the head of the file. Generally, it's nice to be able to audit the code that runs on import in one central place. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-13-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/common.py: Add indent managerJohn Snow2020-10-102-20/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Code style tools really dislike the use of global keywords, because it generally involves re-binding the name at runtime which can have strange effects depending on when and how that global name is referenced in other modules. Make a little indent level manager instead. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-12-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi/common.py: Remove python compatibility workaroundJohn Snow2020-10-101-4/+1Star
| | | | | | | | | | | | | | | | | | Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-11-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi: add pylintrcJohn Snow2020-10-101-0/+73
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Using `pylint --generate-rcfile > pylintrc`, generate a skeleton pylintrc file. Sections that are not presently relevant (by the end of this series) are removed leaving just the empty section as a search engine / documentation hint to future authors. I am targeting pylint 2.6.0. In the future (and hopefully before 5.2 is released), I aim to have gitlab CI running the specific targeted versions of pylint, mypy, flake8, etc in a job. 2.5.x will work if you additionally pass --disable=bad-whitespace. This warning was removed from 2.6.x, for lack of consistent support. Right now, quite a few modules are ignored as they are known to fail as of this commit. modules will be removed from the known-bad list throughout this and following series as they are repaired. Note: Normally, pylintrc would go in the folder above the module, but as that folder is shared by many things, it is going inside the module folder (for now). Due to a bug in pylint 2.5+, pylint does not correctly recognize when it is being run from "inside" a package, and must be run *outside* of the package. Therefore, to run it, you must: > pylint scripts/qapi/ --rcfile=scripts/qapi/pylintrc Signed-off-by: John Snow <jsnow@redhat.com> Tested-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-10-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi: delint using flake8John Snow2020-10-104-9/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Petty style guide fixes and line length enforcement. Not a big win, not a big loss, but flake8 passes 100% on the qapi module, which gives us an easy baseline to enforce hereafter. A note on the flake8 exception: flake8 will warn on *any* bare except, but pylint's is context-aware and will suppress the warning if you re-raise the exception. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-9-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi: enforce import order/styling with isortJohn Snow2020-10-105-5/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | While we're mucking around with imports, we might as well formalize the style we use. Let's use isort to do it for us. lines_after_imports=2: Use two lines after imports, to match PEP8's desire to have "two lines before and after" class definitions, which are likely to start immediately after imports. force_sort_within_sections: Intermingles "from x" and "import x" style statements, such that sorting is always performed strictly on the module name itself. force_grid_wrap=4: Four or more imports from a single module will force the one-per-line style that's more git-friendly. This will generally happen for 'typing' imports. multi_line_output=3: Uses the one-per-line indented style for long imports. include_trailing_comma: Adds a comma to the last import in a group, which makes git conflicts nicer to deal with, generally. line_length: 72 is chosen to match PEP8's "docstrings and comments" line length limit. If you have a single line import that exceeds 72 characters, your names are too long! Suggested-by: Cleber Rosa <crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20201009161558.107041-8-jsnow@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi: Remove wildcard includesJohn Snow2020-10-106-8/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Wildcard includes become hard to manage when refactoring and dealing with circular dependencies with strictly typed mypy. flake8 also flags each one as a warning, as it is not smart enough to know which names exist in the imported file. Remove them and include things explicitly by name instead. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20201009161558.107041-7-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi: Prefer explicit relative importsJohn Snow2020-10-1010-33/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | All of the QAPI include statements are changed to be package-aware, as explicit relative imports. A quirk of Python packages is that the name of the package exists only *outside* of the package. This means that to a module inside of the qapi folder, there is inherently no such thing as the "qapi" package. The reason these imports work is because the "qapi" package exists in the context of the caller -- the execution shim, where sys.path includes a directory that has a 'qapi' folder in it. When we write "from qapi import sibling", we are NOT referencing the folder 'qapi', but rather "any package named qapi in sys.path". If you should so happen to have a 'qapi' package in your path, it will use *that* package. When we write "from .sibling import foo", we always reference explicitly our sibling module; guaranteeing consistency in *where* we are importing these modules from. This can be useful when working with virtual environments and packages in development mode. In development mode, a package is installed as a series of symlinks that forwards to your same source files. The problem arises because code quality checkers will follow "import qapi.x" to the "installed" version instead of the sibling file and -- even though they are the same file -- they have different module paths, and this causes cyclic import problems, false positive type mismatch errors, and more. It can also be useful when dealing with hierarchical packages, e.g. if we allow qemu.core.qmp, qemu.qapi.parser, etc. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20201009161558.107041-6-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
| * qapi: move generator entrypoint into packageJohn Snow2020-10-102-88/+101
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As part of delinting and adding type hints to the QAPI generator, it's helpful for the entrypoint to be part of the package, only leaving a very tiny entrypoint shim outside of the package. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009161558.107041-5-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [invalid_char() renamed to invalid_prefix_char()] Signed-off-by: Markus Armbruster <armbru@redhat.com>
generated by cgit v1.2.3-55-g7522 (git 2.39.0) at 2025-04-26 19:14:21 +0200