summaryrefslogtreecommitdiffstats
path: root/tests/qapi-schema
Commit message (Collapse)AuthorAgeFilesLines
* qapi: merge QInt and QFloat in QNumMarc-André Lureau2017-06-2010-10/+10
| | | | | | | | | | | | | | | We would like to use a same QObject type to represent numbers, whether they are int, uint, or floats. Getters will allow some compatibility between the various types if the number fits other representations. Add a few more tests while at it. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20170607163635.17635-7-marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [parse_stats_intervals() simplified a bit, comment in test_visitor_in_int_overflow() tidied up, suppress bogus warnings] Signed-off-by: Markus Armbruster <armbru@redhat.com>
* tests: Remove test cases for alternates of 'number' and 'int'Marc-André Lureau2017-06-192-10/+0Star
| | | | | | | | | | | | Alternates with both a 'number' and an 'int' branch will become invalid when the next patch merges of QFloat and QInt into QNum. More sophisticated alternate code could keep them valid, but since we have no users outside tests, simply drop the tests. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20170607163635.17635-4-marcandre.lureau@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Reject alternates that can't work with keyval_parse()Markus Armbruster2017-05-3113-9/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Alternates are sum types like unions, but use the JSON type on the wire / QType in QObject instead of an explicit tag. That's why we require alternate members to have distinct QTypes. The recently introduced keyval_parse() (commit d454dbe) can only produce string scalars. The qobject_input_visitor_new_keyval() input visitor mostly hides the difference, so code using a QObject input visitor doesn't have to care whether its input was parsed from JSON or KEY=VALUE,... The difference leaks for alternates, as noted in commit 0ee9ae7: a non-string, non-enum scalar alternate value can't currently be expressed. In part, this is just our insufficiently sophisticated implementation. Consider alternate type 'GuestFileWhence'. It has an integer member and a 'QGASeek' member. The latter is an enumeration with values 'set', 'cur', 'end'. The meaning of b=set, b=cur, b=end, b=0, b=1 and so forth is perfectly obvious. However, our current implementation falls apart at run time for b=0, b=1, and so forth. Fixable, but not today; add a test case and a TODO comment. Now consider an alternate type with a string and an integer member. What's the meaning of a=42? Is it the string "42" or the integer 42? Whichever meaning you pick makes the other inexpressible. This isn't just an implementation problem, it's fundamental. Our current implementation will pick string. So far, we haven't needed such alternates. To make sure we stop and think before we add one that cannot sanely work with keyval_parse(), let's require alternate members to have sufficiently distinct representation in KEY=VALUE,... syntax: * A string member clashes with any other scalar member * An enumeration member clashes with bool members when it has value 'on' or 'off'. * An enumeration member clashes with numeric members when it has a value that starts with '-', '+', or a decimal digit. This is a rather lazy approximation of the actual number syntax accepted by the visitor. Note that enumeration values starting with '-' and '+' are rejected elsewhere already, but better safe than sorry. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <1495471335-23707-5-git-send-email-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
* tests/qapi-schema: Avoid 'str' in alternate test casesMarkus Armbruster2017-05-317-22/+27
| | | | | | | | | | | | The next commit is going to make alternate members of type 'str' conflict with other scalar types. Would break a few test cases that don't actually require 'str'. Flip them from 'str' to 'bool' or 'EnumOne'. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <1495471335-23707-4-git-send-email-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
* qapi2texi: Fix translation of *strong* and _emphasized_Markus Armbruster2017-03-211-4/+4
| | | | | | Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <1490015515-25851-7-git-send-email-armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
* tests/qapi-schema: Systematic positive doc comment testsMarkus Armbruster2017-03-215-0/+528
| | | | | | | | | We have a number of negative tests, but we don't have systematic positive coverage. Fix that. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <1490015515-25851-6-git-send-email-armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
* tests/qapi-schema: Make test-qapi.py print docs againMarkus Armbruster2017-03-211-0/+11
| | | | | | | | | | | test-qapi.py used to print the internal representation of doc comments (commit 3313b61). This went away when we dropped the doc comments in positive tests (commit 87c16dc). Bring it back, because I'm going to add real positive doc comment tests. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <1490015515-25851-5-git-send-email-armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
* qapi: Fix a misleading parser error messageMarkus Armbruster2017-03-161-1/+1
| | | | | | | | | | | | When choking on a token where an expression is expected, we report 'Expected "{", "[" or string'. Close, but no cigar. Fix it to Expected '"{", "[", string, boolean or "null"'. Missed in commit e53188a. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-48-git-send-email-armbru@redhat.com>
* qapi: Fix detection of bogus member documentationMarkus Armbruster2017-03-163-12/+2Star
| | | | | | | | | | | | | check_definition_doc() checks for member documentation without a matching member. It laboriously second-guesses what members QAPISchema._def_exprs() will create. That's a stupid game. Move the check into QAPISchema.check(), where the members are known. Delegate the actual checking to new QAPIDoc.check(). Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-38-git-send-email-armbru@redhat.com>
* tests/qapi-schema: Improve coverage of bogus member docsMarkus Armbruster2017-03-168-0/+42
| | | | | | | | | New test doc-bad-union-member.json shows we can fail to reject documentation for nonexistent members. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-37-git-send-email-armbru@redhat.com>
* tests/qapi-schema: Rename doc-bad-args to doc-bad-command-argMarkus Armbruster2017-03-165-1/+1
| | | | | | Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-36-git-send-email-armbru@redhat.com>
* qapi: Move empty doc section checking to doc parserMarkus Armbruster2017-03-161-1/+1
| | | | | | | | | Results in a more precise error location, but the real reason is emptying out check_docs() step by step. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-35-git-send-email-armbru@redhat.com>
* qapi: Improve error message on @NAME: in free-form docMarkus Armbruster2017-03-161-1/+1
| | | | | | Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-34-git-send-email-armbru@redhat.com>
* qapi: Move detection of doc / expression name mismatchMarkus Armbruster2017-03-161-1/+1
| | | | | | | | | | | | | Move the check whether the doc matches the expression name from check_definition_doc() to check_exprs(). This changes the error location from the comment to the expression. Makes sense as the message talks about the expression: "Definition of '%s' follows documentation for '%s'". It's also a step towards getting rid of check_docs(). Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-33-git-send-email-armbru@redhat.com>
* qapi: Fix detection of doc / expression mismatchMarkus Armbruster2017-03-1611-15/+6Star
| | | | | | | | This fixes the errors uncovered by the previous commit. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-32-git-send-email-armbru@redhat.com>
* tests/qapi-schema: Improve doc / expression mismatch coverageMarkus Armbruster2017-03-1612-0/+35
| | | | | | | | | | | New tests doc-before-include.json and doc-before-pragma.json show we fail to reject a misplaced expression comment. New test doc-no-symbol.json shows a bad error message. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <1489582656-31133-31-git-send-email-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
* qapi: The #optional tag is redundant, dropMarkus Armbruster2017-03-164-9/+0Star
| | | | | | | | | | | | | | | | | | | | | | | | | We traditionally mark optional members #optional in the doc comment. Before commit 3313b61, this was entirely manual. Commit 3313b61 added some automation because its qapi2texi.py relied on #optional to determine whether a member is optional. This is no longer the case since the previous commit: the only thing qapi2texi.py still does with #optional is stripping it out. We still reject bogus qapi-schema.json and six places for qga/qapi-schema.json. Thus, you can't actually rely on #optional to see whether something is optional. Yet we still make people add it manually. That's just busy-work. Drop the code to check, fix up and strip out #optional, along with all instances of #optional. To keep it out, add code to reject it, to be dropped again once the dust settles. No change to generated documentation. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-18-git-send-email-armbru@redhat.com>
* qapi: Fix to reject empty union base gracefullyMarkus Armbruster2017-03-161-10/+1Star
| | | | | | | | | | Common Python pitfall: 'assert base_members' fires on [] in addition to None. Correct to 'assert base_members is not None'. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-10-git-send-email-armbru@redhat.com>
* tests/qapi-schema: Cover empty union baseMarkus Armbruster2017-03-164-0/+20
| | | | | | | | | The new test case shows off qapi.py choking on an empty union base. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-9-git-send-email-armbru@redhat.com>
* qapi: Have each QAPI schema declare its name rule violationsMarkus Armbruster2017-03-166-1/+7
| | | | | | | | | | | qapi.py has a hardcoded white-list of type names that may violate the rule on use of upper and lower case. Add a new pragma directive 'name-case-whitelist', and use it to replace the hard-coded white-list. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <1489582656-31133-7-git-send-email-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
* qapi: Have each QAPI schema declare its returns white-listMarkus Armbruster2017-03-167-1/+17
| | | | | | | | | | qapi.py has a hardcoded white-list of command names that may violate the rules on permitted return types. Add a new pragma directive 'returns-whitelist', and use it to replace the hard-coded white-list. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <1489582656-31133-6-git-send-email-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
* qapi: Back out doc comments added just to please qapi.pyMarkus Armbruster2017-03-16200-1074/+92Star
| | | | | | | | | | | | | | | This reverts commit 3313b61's changes to tests/qapi-schema/, except for tests/qapi-schema/doc-*. We could keep some of these doc comments to serve as positive test cases. However, they don't actually add to what we get from doc comment use in actual schemas, as we we don't test output matches expectations, and don't systematically cover doc comment features. Proper positive test coverage would be nice. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-4-git-send-email-armbru@redhat.com>
* qapi: Make doc comments optional where we don't need themMarkus Armbruster2017-03-1620-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | Since we added the documentation generator in commit 3313b61, doc comments are mandatory. That's a very good idea for a schema that needs to be documented, but has proven to be annoying for testing. Make doc comments optional again, but add a new directive { 'pragma': { 'doc-required': true } } to let a QAPI schema require them. Add test cases for the new pragma directive. While there, plug a minor hole in includ directive test coverage. Require documentation in the schemas we actually want documented: qapi-schema.json and qga/qapi-schema.json. We could probably make qapi2texi.py cope with incomplete documentation, but for now, simply make it refuse to run unless the schema has 'doc-required': true. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <1489582656-31133-3-git-send-email-armbru@redhat.com> [qapi-code-gen.txt wording tweaked] Reviewed-by: Eric Blake <eblake@redhat.com>
* qapi: add qapi2texi scriptMarc-André Lureau2017-01-16272-92/+1238
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As the name suggests, the qapi2texi script converts JSON QAPI description into a texi file suitable for different target formats (info/man/txt/pdf/html...). It parses the following kind of blocks: Free-form: ## # = Section # == Subsection # # Some text foo with *emphasis* # 1. with a list # 2. like that # # And some code: # | $ echo foo # | -> do this # | <- get that # ## Symbol description: ## # @symbol: # # Symbol body ditto ergo sum. Foo bar # baz ding. # # @param1: the frob to frobnicate # @param2: #optional how hard to frobnicate # # Returns: the frobnicated frob. # If frob isn't frobnicatable, GenericError. # # Since: version # Notes: notes, comments can have # - itemized list # - like this # # Example: # # -> { "execute": "quit" } # <- { "return": {} } # ## That's roughly following the following EBNF grammar: api_comment = "##\n" comment "##\n" comment = freeform_comment | symbol_comment freeform_comment = { "# " text "\n" | "#\n" } symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment } member = "# @" name ':' [ text ] "\n" freeform_comment tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment text = free text with markup Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed both as freeform_comment and as symbol_comment. The actual parser recognizes symbol_comment. See docs/qapi-code-gen.txt for more details. Deficiencies and limitations: - the generated QMP documentation includes internal types - union type support is lacking - type information is lacking in generated documentation - doc comment error message positions are imprecise, they point to the beginning of the comment. - a few minor issues, all marked TODO/FIXME in the code Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [test-qapi.py tweaked to avoid trailing empty lines in .out] Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Implement boxed types for commands/eventsEric Blake2016-07-1924-3/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Turn on the ability to pass command and event arguments in a single boxed parameter, which must name a non-empty type (although the type can be a struct with all optional members). For structs, it makes it possible to pass a single qapi type instead of a breakout of all struct members (useful if the arguments are already in a struct or if the number of members is large); for other complex types, it is now possible to use a union or alternate as the data for a command or event. The empty type may be technically feasible if needed down the road, but it's easier to forbid it now and relax things to allow it later, than it is to allow it now and have to special case how the generated 'q_empty' type is handled (see commit 7ce106a9 for reasons why nothing is generated for the empty type). An alternate type is never considered empty, but now that a boxed type can be either an object or an alternate, we have to provide a trivial QAPISchemaAlternateType.is_empty(). The new call to arg_type.is_empty() during QAPISchemaCommand.check() requires that we first check the type in question; but there is no chance of introducing a cycle since objects do not refer back to commands. We still have a split in syntax checking between ad-hoc parsing up front (merely validates that 'boxed' has a sane value) and during .check() methods (if 'boxed' is set, then 'data' must name a non-empty user-defined type). Generated code is unchanged, as long as no client uses the new feature. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1468468228-27827-10-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Test files renamed to *-boxed-*] Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Plumb in 'boxed' to qapi generator lower levelsEric Blake2016-07-195-13/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | The next patch will add support for passing a qapi union type as the 'data' of a command. But to do that, the user function for implementing the command, as called by the generated marshal command, must take the corresponding C struct as a single boxed pointer, rather than a breakdown into one parameter per member. Even without a union, being able to use a C struct rather than a list of parameters can make it much easier to handle coding with QAPI. This patch adds the internal plumbing of a 'boxed' flag associated with each command and event. In several cases, this means adding indentation, with one new dead branch and the remaining branch being the original code more deeply nested; this was done so that the new implementation in the next patch is easier to review without also being mixed with indentation changes. For this patch, no behavior or generated output changes, other than the testsuite outputting the value of the new flag (always False for now). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1468468228-27827-9-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Identifier box renamed to boxed in two places] Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Hide tag_name data member of variantsEric Blake2016-07-192-2/+11
| | | | | | | | | | | | | | Clean up the only remaining external use of the tag_name field of QAPISchemaObjectTypeVariants, by explicitly listing the generated 'type' tag for all variants in the testsuite (you can still tell simple unions by the -wrapper types). Then we can mark the tag_name field as private by adding a leading underscore to prevent any further use. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1468468228-27827-5-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Require all branches of flat union enum to be coveredEric Blake2016-07-194-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | We were previously enforcing that all flat union branches were found in the corresponding enum, but not that all enum values were covered by branches. The resulting generated code would abort() if the user passes the uncovered enum value. We don't automatically treat non-present branches in a flat union as empty types, for symmetry with simple unions (there, the enum type is generated from the list of all branches, so there is no way to omit a branch but still have it be part of the union). A later patch will add shorthand so that branches that are empty in flat unions can be declared as 'branch':{} instead of 'branch':'Empty', to avoid the need for an otherwise useless explicit empty type. [Such shorthand for simple unions is a bit harder to justify, since we would still have to generate a wrapper type that parses 'data':{}, rather than truly being an empty branch with no additional siblings to the 'type' member.] Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1468468228-27827-3-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Allow anonymous base for flat unionEric Blake2016-03-184-14/+9Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rather than requiring all flat unions to explicitly create a separate base struct, we can allow the qapi schema to specify the common members via an inline dictionary. This is similar to how commands can specify an inline anonymous type for its 'data'. We already have several struct types that only exist to serve as a single flat union's base; the next commit will clean them up. In particular, this patch's change to the BlockdevOptions example in qapi-code-gen.txt will actually be done in the real QAPI schema. Now that anonymous bases are legal, we need to rework the flat-union-bad-base negative test (as previously written, it forms what is now valid QAPI; tweak it to now provide coverage of a new error message path), and add a positive test in qapi-schema-test to use an anonymous base (making the integer argument optional, for even more coverage). Note that this patch only allows anonymous bases for flat unions; simple unions are already enough syntactic sugar that we do not want to burden them further. Meanwhile, while it would be easy to also allow an anonymous base for structs, that would be quite redundant, as the members can be put right into the struct instead. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1458254921-17042-15-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Adjust names of implicit typesEric Blake2016-03-189-88/+88
| | | | | | | | | | | | | | | | | | | | | | | | | | The original choice of ':obj-' as the prefix for implicit types made it obvious that we weren't going to clash with any user-defined names, which cannot contain ':'. But now we want to create structs for implicit types, to get rid of special cases in the generators, and our use of ':' in implicit names needs a tweak to produce valid C code. We could transliterate ':' to '_', except that C99 mandates that "identifiers that begin with an underscore are always reserved for use as identifiers with file scope in both the ordinary and tag name spaces". So it's time to change our naming convention: we can instead use the 'q_' prefix that we reserved for ourselves back in commit 9fb081e0. Technically, since we aren't planning on exposing the empty type in generated code, we could keep the name ':empty', but renaming it to 'q_empty' makes the check for startswith('q_') cover all implicit types, whether or not code is generated for them. As long as we don't declare 'empty' or 'obj' ticklish, it shouldn't clash with c_name() prepending 'q_' to the user's ticklish names. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1458254921-17042-5-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Fix command with named empty argument typeEric Blake2016-03-182-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | The generator special-cased { 'command':'foo', 'data': {} } to avoid emitting a visitor variable, but failed to see that { 'struct':'NamedEmptyType, 'data': {} } { 'command':'foo', 'data':'NamedEmptyType' } needs the same treatment. There, the generator happily generates a visitor to get no arguments, and a visitor to destroy no arguments; and the compiler isn't happy with that, as demonstrated by the updated qapi-schema-test.json: tests/test-qmp-marshal.c: In function ‘qmp_marshal_user_def_cmd0’: tests/test-qmp-marshal.c:264:14: error: variable ‘v’ set but not used [-Werror=unused-but-set-variable] Visitor *v; ^ No change to generated code except for the testsuite addition. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1458254921-17042-3-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Drop useless 'data' member of unionsEric Blake2016-03-054-17/+0Star
| | | | | | | | | | | | | | We started moving away from the use of the 'void *data' member in the C union corresponding to a QAPI union back in commit 544a373; recent commits have gotten rid of other uses. Now that it is completely unused, we can remove the member itself as well as the FIXME comment. Update the testsuite to drop the negative test union-clash-data. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <1457021813-10704-11-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Rename 'fields' to 'members' in generatorEric Blake2016-03-051-1/+1
| | | | | | | | | | | | | | C types and JSON objects don't have fields, but members. We shouldn't gratuitously invent terminology. This patch is a strict renaming of generator code internals (including testsuite comments), before later patches rename C interfaces. No change to generated code with this patch. Suggested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1457021813-10704-2-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Add tests of complex objects within alternateEric Blake2016-02-192-2/+6
| | | | | | | | | | | | | | | | | | Upcoming patches will adjust how we visit an object branch of an alternate; but we were completely lacking testsuite coverage. Rectify this, so that the future patches will be able to highlight the changes and still prove that we avoided regressions. In particular, the use of a flat union UserDefFlatUnion rather than a simple struct UserDefA as the branch will give us coverage of an object with variants. And visiting an alternate as both the top level and as a nested member gives confidence in correct memory allocation handling, especially if the test is run under valgrind. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1455778109-6278-5-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Forbid 'any' inside an alternateEric Blake2016-02-194-0/+6
| | | | | | | | | | | | | | | | | The whole point of an alternate is to allow some type-safety while still accepting more than one JSON type. Meanwhile, the 'any' type exists to bypass type-safety altogether. The two are incompatible: you can't accept every type, and still tell which branch of the alternate to use for the parse; fix this to give a sane error instead of a Python stack trace. Note that other types that can't be alternate members are caught earlier, by check_type(). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1455778109-6278-4-git-send-email-eblake@redhat.com> [Commit message tweaked] Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Forbid empty unions and useless alternatesEric Blake2016-02-1912-26/+9Star
| | | | | | | | | | | | | | | | | | | | | | Empty unions serve no purpose, and while we compile with gcc which permits them, strict C99 forbids them. We happen to inject a dummy 'void *data' member into the C unions that represent QAPI unions and alternates, but we want to get rid of that member (it pollutes the namespace for no good reason), which would leave us with an empty union if the user didn't provide any branches. While empty structs make sense in QAPI, empty unions don't add any expressiveness to the QMP language. So prohibit them at parse time. Update the documentation and testsuite to match. Note that the documentation already mentioned that alternates should have "two or more JSON data types"; so this also fixes the code to enforce that. However, we have existing uses of a union type with only one branch, so the 2-or-more strictness is intentionally limited to alternates. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1455778109-6278-3-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi-visit: Honor prefix of discriminator enumEric Blake2016-02-192-5/+11
| | | | | | | | | | | | | | | When we added support for a user-specified prefix for an enum type (commit 351d36e), we forgot to teach the qapi-visit code to honor that prefix in the case of using a prefixed enum as the discriminator for a flat union. While there is still some on-list debate on whether we want to keep prefixes, we should at least make it work as long as it is still part of the code base. Reported-by: Daniel P. Berrange <berrange@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1455665965-27638-1-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Detect base class loopsEric Blake2015-12-178-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It should be fairly obvious that qapi base classes need to form an acyclic graph, since QMP cannot specify the same key more than once, while base classes are included as flat members alongside other members added by the child. But the old check_member_clash() parser function was not prepared to check for this, and entered an infinite recursion (at least until Python gives up, complaining about nesting too deep). Now that check_member_clash() has been recently removed, attempts at self-inheritance trigger an assertion failure introduced by commit ac88219a. The obvious fix is to turn the assertion into a conditional. This patch includes both the tests (base-cycle-direct and base-cycle-indirect) and the fix, since the .err file output for the unfixed case is not useful (particularly when it was warning about unbounded recursion, as that limit may be platform-specific). We don't need to worry about cycles in flat unions (neither the base type nor the type of a variant can be a union) nor in alternates (alternate branches cannot themselves be an alternate). But if we later allow a union type as a variant, we will still be okay, as QAPISchemaObjectTypeVariants.check() triggers the same QAPISchemaObjectType.check() that will detect any loops. Likewise, we need not worry about the case of diamond inheritance where the same class is used for a flat union base class and one of its variants; either both uses will introduce a collision in trying to insert the same member name twice, or the shared type is empty and changes nothing. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1449033659-25497-16-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Move duplicate collision checks to schema check()Eric Blake2015-12-1712-18/+8Star
| | | | | | | | | | | | | | | | | | | | | | | | | | With the recent commit 'qapi: Detect collisions in C member names', we have two different locations for detecting clashes - one at parse time, and another at QAPISchema*.check() time. Remove all of the ad hoc parser checks, and delete associated code (for example, the global check_member_clash() method is no longer needed). Testing this showed that the test union-bad-branch wasn't adding much: union-clash-branches also exposes the error message when branches collide, and we've recently fixed things to avoid an implicit collision with max. Likewise, the error for enum-clash-member changes to report our new detection of upper case in a value name, unless we modify the test to use all lower case. The wording of several error messages has changed, but the change is generally an improvement rather than a regression. No change to generated code. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1449033659-25497-15-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Enforce (or whitelist) case conventions on qapi membersEric Blake2015-12-1712-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We document that members of enums and objects should be 'lower-case', although we were not enforcing it. We have to whitelist a few pre-existing entities that violate the norms. Add three new tests to expose the new error message, each of which first uses the whitelisted name 'UuidInfo' to prove the whitelist works, then triggers the failure (this is the same pattern used in the existing returns-whitelist.json test). Note that by adding this check, we have effectively forbidden an entity with a case-insensitive clash of member names, for any entity that is not on the whitelist (although there is still the possibility to clash via '-' vs. '_'). Not done here: a future patch should also add naming convention support and whitelist exceptions for command, event, and type names. The additions to QAPISchemaMember.check_clash() check whether info['name'] is in the whitelist (the top-most entity name at the point 'info' tracks), rather than self.owner (the type, possibly implicit, that directly owns the member), because it is easier to maintain the whitelist by the names actually in the user's .json file, rather than worrying about the names of implicit types. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1449033659-25497-14-git-send-email-eblake@redhat.com> [Simplified a bit as per discussion with Eric] Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Simplify visiting of alternate typesEric Blake2015-12-172-9/+0Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, working with alternates required two lookup arrays and some indirection: for type Foo, we created Foo_qtypes[] which maps each qtype to a value of the generated FooKind enum, then look up that value in FooKind_lookup[] like we do for other union types. This has a couple of subtle bugs. First, the generator was creating a call with a parameter '(int *) &(*obj)->type' where type is an enum type; this is unsafe if the compiler chooses to store the enum type in a different size than int, where assigning through the wrong size pointer can corrupt data or cause a SIGBUS. Related bug, not not fixed in this patch: qapi-visit.py's gen_visit_enum() generates a cast of its enum * argument to int *. Marked FIXME. Second, since the values of the FooKind enum start at zero, all entries of the Foo_qtypes[] array that were not explicitly initialized will map to the same branch of the union as the first member of the alternate, rather than triggering a desired failure in visit_get_next_type(). Fortunately, the bug seldom bites; the very next thing the input visitor does is try to parse the incoming JSON with the wrong parser, which normally fails; the output visitor is not used with a C struct in that state, and the dealloc visitor has nothing to clean up (so there is no leak). However, the second bug IS observable in one case: parsing an integer causes unusual behavior in an alternate that contains at least a 'number' member but no 'int' member, because the 'number' parser accepts QTYPE_QINT in addition to the expected QTYPE_QFLOAT (that is, since 'int' is not a member, the type QTYPE_QINT accidentally maps to FooKind 0; if this enum value is the 'number' branch the integer parses successfully, but if the 'number' branch is not first, some other branch tries to parse the integer and rejects it). A later patch will worry about fixing alternates to always parse all inputs that a non-alternate 'number' would accept, for now this is still marked FIXME in the updated test-qmp-input-visitor.c, to merely point out that new undesired behavior of 'ans' matches the existing undesired behavior of 'asn'. This patch fixes the default-initialization bug by deleting the indirection, and modifying get_next_type() to directly assign a QTypeCode parameter. This in turn fixes the type-casting bug, as we are no longer casting a pointer to enum to a questionable size. There is no longer a need to generate an implicit FooKind enum associated with the alternate type (since the QMP wire format never uses the stringized counterparts of the C union member names). Since the updated visit_get_next_type() does not know which qtypes are expected, the generated visitor is modified to generate an error statement if an unexpected type is encountered. Callers now have to know the QTYPE_* mapping when looking at the discriminator; but so far, only the testsuite was even using the C struct of an alternate types. I considered the possibility of keeping the internal enum FooKind, but initialized differently than most generated arrays, as in: typedef enum FooKind { FOO_KIND_A = QTYPE_QDICT, FOO_KIND_B = QTYPE_QINT, } FooKind; to create nicer aliases for knowing when to use foo->a or foo->b when inspecting foo->type; but it turned out to add too much complexity, especially without a client. There is a user-visible side effect to this change, but I consider it to be an improvement. Previously, the invalid QMP command: {"execute":"blockdev-add", "arguments":{"options": {"driver":"raw", "id":"a", "file":true}}} failed with: {"error": {"class": "GenericError", "desc": "Invalid parameter type for 'file', expected: QDict"}} (visit_get_next_type() succeeded, and the error comes from the visit_type_BlockdevOptions() expecting {}; there is no mention of the fact that a string would also work). Now it fails with: {"error": {"class": "GenericError", "desc": "Invalid parameter type for 'file', expected: BlockdevRef"}} (the error when the next type doesn't match any expected types for the overall alternate). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1449033659-25497-5-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Convert QType into QAPI built-in enum typeEric Blake2015-12-1713-0/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | What's more meta than using qapi to define qapi? :) Convert QType into a full-fledged[*] builtin qapi enum type, so that a subsequent patch can then use it as the discriminator type of qapi alternate types. Fortunately, the judicious use of 'prefix' in the qapi definition avoids churn to the spelling of the enum constants. To avoid circular definitions, we have to flip the order of inclusion between "qobject.h" vs. "qapi-types.h". Back in commit 28770e0, we had the latter include the former, so that we could use 'QObject *' for our implementation of 'any'. But that usage also works with only a forward declaration, whereas the definition of QObject requires QType to be a complete type. [*] The type has to be builtin, rather than declared in qapi/common.json, because we want to use it for alternates even when common.json is not included. But since it is the first builtin enum type, we have to add special cases to qapi-types and qapi-visit to only emit definitions once, even when two qapi files are being compiled into the same binary (the way we already handled builtin list types like 'intList'). We may need to revisit how multiple qapi files share common types, but that's a project for another day. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1449033659-25497-4-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Remove obsolete tests for MAX collisionEric Blake2015-12-1712-14/+0Star
| | | | | | | | | | | Now that we no longer collide with an implicit _MAX enum member, we no longer need to reject it in the ad hoc parser, and can remove several tests that are no longer needed. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1447836791-369-24-git-send-email-eblake@redhat.com> [Commit message tweaked] Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Tighten the regex on valid namesEric Blake2015-12-178-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We already documented that qapi names should match specific patterns (such as starting with a letter unless it was an enum value or a downstream extension). Tighten that from a suggestion into a hard requirement, which frees up names beginning with a single underscore for qapi internal usage. The tighter regex doesn't forbid everything insane that a user could provide (for example, a user could name a type 'Foo-lookup' to collide with the generated 'Foo_lookup[]' for an enum 'Foo'), but does a good job at protecting the most obvious uses, and also happens to reserve single leading underscore for later use. The handling of enum values starting with a digit is tricky: commit 9fb081e introduced a subtle bug by using c_name() on a munged value, which would allow an enum to include the member 'q-int' in spite of our reservation. Furthermore, munging with a leading '_' would fail our tighter regex. So fix it by only munging for leading digits (which are never ticklish in c_name()) and by using a different prefix (I picked 'D', although any letter should do). Add new tests, reserved-member-underscore and reserved-enum-q, to demonstrate the tighter checking. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1447836791-369-22-git-send-email-eblake@redhat.com> Message-Id: <1447883135-18020-1-git-send-email-eblake@redhat.com> [Eric's fixup squashed in] Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Fix c_name() mungingEric Blake2015-12-172-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | The method c_name() is supposed to do two different actions: munge '-' into '_', and add a 'q_' prefix to ticklish names. But it did these steps out of order, making it possible to submit input that is not ticklish until after munging, where the output then lacked the desired prefix. The failure is exposed easily if you have a compiler that recognizes C11 keywords, and try to name a member '_Thread-local', as it would result in trying to compile the declaration 'uint64_t _Thread_local;' which is not valid. However, this name violates our conventions (ultimately, want to enforce that no qapi names start with single underscore), so the test is slightly weaker by instead testing 'wchar-t'; the declaration 'uint64_t wchar_t;' is valid in C (where wchar_t is only a typedef) but would fail with a C++ compiler (where it is a keyword). Fix things by reversing the order of actions within c_name(). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1447836791-369-18-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Detect collisions in C member namesEric Blake2015-12-174-10/+4Star
| | | | | | | | | | | | | | | | Detect attempts to declare two object members that would result in the same C member name, by keying the 'seen' dictionary off of the C name rather than the qapi name. It also requires passing info through the check_clash() methods. This addresses a TODO and fixes the previously-broken args-name-clash test. The resulting error message demonstrates the utility of the .describe() method added previously. No change to generated code. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1447836791-369-17-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Remove outdated tests related to QMP/branch collisionsEric Blake2015-12-1712-60/+0Star
| | | | | | | | | | | | | | | | Now that branches are in a separate C namespace, we can remove the restrictions in the parser that claim a branch name would collide with QMP, and delete the negative tests that are no longer problematic. A separate patch can then add positive tests to qapi-schema-test to test that any corner cases will compile correctly. This reverts the scripts/qapi.py portion of commit 7b2a5c2, now that the assertions that it plugged are no longer possible. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1447836791-369-15-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Track simple union tag in object.local_membersEric Blake2015-12-173-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | We were previously creating all unions with an empty list for local_members. However, it will make it easier to unify struct and union generation if we include the generated tag member in local_members. That way, we can have a common code pattern: visit the base (if any), visit the local members (if any), visit the variants (if any). The local_members of a flat union remains empty (because the discriminator is already visited as part of the base). Then, by visiting tag_member.check() during AlternateType.check(), we no longer need to call it during Variants.check(). The various front end entities now exist as follows: struct: optional base, optional local_members, no variants simple union: no base, one-element local_members, variants with tag_member from local_members flat union: base, no local_members, variants with tag_member from base alternate: no base, no local_members, variants With the new local members, we require a bit of finesse to avoid assertions in the clients. No change to generated code. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1447836791-369-2-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Use generated TestStruct machinery in testsEric Blake2015-11-092-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | Commit d88f5fd and friends first introduced the various test-qmp-* tests in 2011, with duplicated hand-rolled TestStruct machinery, to make sure the qapi visitor interface was tested. Later, commit 4f193e3 in 2013 added a .json file for further testing use by the files, but without consolidating any of the existing hand-rolled visitors. And with four copies, subtle differences have crept in, between the tests themselves (mainly whitespace differences, but also a question of whether to use NULL or "TestStruct" when calling visit_start_struct()) and from what the generator produces (the hand-rolled versions did not cater to partially-allocated objects, because they did not have a deallocation usage). Of course, just because the visitor interface is tested does not mean it is a sane interface; and future patches will be changing some of the visitor contracts. Rather than having to duplicate the cleanup work in each copy of the TestStruct visitor, and keep each hand-rolled copy in sync with what the generator supplies, we might as well just test what the generator should give us in the first place. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1446791754-23823-2-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
* qapi: Reserve 'u' member nameEric Blake2015-11-024-0/+9
| | | | | | | | | | | | | | | | | | | | | | Now that we have separated union tag values from colliding with non-variant C names, by naming the union 'u', we should reserve this name for our use. Note that we want to forbid 'u' even in a struct with no variants, because it is possible for a future qemu release to extend QMP in a backwards-compatible manner while converting from a struct to a flat union. Fortunately, no existing clients were using this member name. If we ever find the need for QMP to have a member 'u', we could at that time relax things, perhaps by having c_name() munge the QMP member to 'q_u'. Note that we cannot forbid 'u' everywhere (by adding the rejection code to check_name()), because the existing QKeyCode enum already uses it; therefore we only reserve it as a struct type member name. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1445898903-12082-24-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
generated by cgit v1.2.3-55-g7522 (git 2.39.0) at 2024-12-22 21:00:32 +0100