kconfig: tests: add framework for Kconfig unit testing

Many parts in Kconfig are so cryptic and need refactoring.  However,
its complexity prevents us from moving forward.  There are several
naive corner cases where it is difficult to notice breakage.  If
those are covered by unit tests, we will be able to touch the code
with more confidence.

Here is a simple test framework based on pytest.  The conftest.py
provides a fixture useful to run commands such as 'oldaskconfig' etc.
and to compare the resulted .config, stdout, stderr with expectations.

How to add test cases?
----------------------

For each test case, you should create a subdirectory under
scripts/kconfig/tests/ (so test cases are separated from each other).
Every test case directory should contain the following files:

 - __init__.py: describes test functions
 - Kconfig: the top level Kconfig file for the test

To do a useful job, test cases generally need additional data like
input .config and information about expected results.

How to run tests?
-----------------

You need python3 and pytest.  Then, run "make testconfig".  O= option
is supported.  If V=1 is given, detailed logs captured during tests
are displayed.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>

1 files changed, 8 insertions, 0 deletions

diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
index a9f325ada4cb..78c96aabd00f 100644
--- a/scripts/kconfig/Makefile
+++ b/scripts/kconfig/Makefile
@@ -142,6 +142,14 @@ PHONY += tinyconfig
 tinyconfig:
 	$(Q)$(MAKE) -f $(srctree)/Makefile allnoconfig tiny.config
 
+# CHECK: -o cache_dir=<path> working?
+PHONY += testconfig
+testconfig: $(obj)/conf
+	$(PYTHON3) -B -m pytest $(srctree)/$(src)/tests \
+	-o cache_dir=$(abspath $(obj)/tests/.cache) \
+	$(if $(findstring 1,$(KBUILD_VERBOSE)),--capture=no)
+clean-dirs += tests/.cache
+
 # Help text used by make help
 help:
 	@echo  '  config	  - Update current config utilising a line-oriented program'