From 38c2aa296736af202983c2d7581f5c1f71cb0063 Mon Sep 17 00:00:00 2001
From: Jonathan Bauer
Date: Tue, 9 Apr 2019 12:27:51 +0200
Subject: README.md: update to reflect current usage :)

---
 README.md | 265 +++++++++++++++++++++++++++++++++++++-------------------------
 1 file changed, 159 insertions(+), 106 deletions(-)

diff --git a/README.md b/README.md
index 6c43078..3d8e09e 100644
--- a/README.md
+++ b/README.md
@@ -1,144 +1,197 @@
-# Packer templates for OpenSLX
-
-Packer-based generation of reference systems using distro's ISOs.
+# Packer templates for network boot using OpenSLX-NG
+
+This repository provides a build system based on a Makefile and Packer/Ansible
+templates to create clean operating system images - using Packer to generate
+the base system and Ansible to further install software stacks. This repository
+contains a set of templates for CentOS and Ubuntu distributions as well as
+kickstart/preseed files to handle the initial unattended installations. A
+special Packer template further allows for the base images to be provisioned
+using custom Ansible playbooks. The provisioning playbooks are not maintained
+within this repository - but can be specified via environment variable when
+calling the Makefile (more on that later). Once provisioned, the network boot
+playbook maintained on `https://git.openslx.org/openslx-ng/ansible-dracut.git`
+can optionally be executed to generate the initramfs (and to extract the
+kernel) needed for network boot.
+
+This repository contains a Makefile coordinating all these steps. Even though
+the provided Packer templates and Ansible playbooks are usable as is, it is
+recommended to use the Makefile to build images. Beware that changing
+parameters of the provided templates (e.g. output directories or image names)
+might break the full workflow - if changes are necessary, make sure to reflect
+these changes within the Makefile.
 
 ## Dependencies
 
-Packer => 0.12.2
-Download: https://www.packer.io/downloads.html
+* Packer => 1.2.2
+	* Download: https://www.packer.io/downloads.html
 
-## Usage
+* QEMU => 3.0
+	* Download: https://www.qemu.org/download/
 
-The file 'base.json' contains default variables (VM, SSH) common to all templates,
-and should always be included using packer's '-var-file' option. The root password
-is expected to be specified as the ROOTPW environment variable!
+* Ansible => 2.5.0
+	* Installation: https://docs.ansible.com/ansible/latest/installation_guide/
 
-Basic usage:
-```shell
-ROOTPW=<password> packer build -var-file=base.json <template>
-```
-For the rest of the examples, ROOTPW is assumed to be set in the environment.
+Note: Make sure the build machine supports KVM to accelerate the build process
+substantially.  Note: VMware Player and VirtualBox are supported to build base
+images, i.e. simple unattended installations, they are not yet supported for
+the provisioning steps.
 
-## Usage: Creating base images
+## Usage
 
-#### Example
-Build Ubuntu 16.04.2 LTS template using qemu builder
+The file `base.json` contains default variables (VM, SSH) common to all
+templates and is always included during packer builds. This file controls the
+VM's hardware configuration, in particular the number of CPUs (`cpus`), the
+amount of RAM in bytes (`memory`) and the virtual disk size in bytes
+(`disk_size`). Adjust these as needed. The SSH variables specify the user
+connecting to the VM (`ssh_username`) and the time packer will try to connect
+to be VM before aborting a build (`ssh_timeout`). Note that the root password
+is expected to be specified as the `ROOTPW` environment variable, as explained
+later.
+
+### Templates for Packer, Kickstart, Preseed
+
+Distribution templates are saved in the root directory of this repository, e.g.
+`centos-7-x86_64.json`. To see a list of currently maintained distributions,
+refer to `make help` and look for 'Base image targets'.
+
+The kickstart/preseed configuration files needed for unattended installations
+(UAI) during the packer builds are stored in the `http` folder. The Packer
+builders are configured to look for UAI files in the distribution's version
+directory, e.g. `http/centos-7`.
+
+So far, only one UAI file is kept for every distribution family. The versioned
+folder thus point to the common family folders. This is to allow customized or
+version-specific UAI files to be used if needed, without having to fiddle with
+the packer templates files themselves. We think it is a good practice to try
+and keep the images resulting from UAIs as basic as possible. This way, base
+UAI images can be used for different provisioning runs. To support this, OS
+configuration or the installation of additional software should only be
+realised in Packer's provisioning step.
+
+### Playbooks for Ansible provisioning
+
+The build system also has code to automatically provision base images. It
+thereby looks for the `packer-templates/ansible-prov` directory or in the path
+specified by the environment variable `ANSIBLE_DIR_PROV`. Within the
+provisioning directory, it further looks for Ansible playbooks named
+`setup-*.yml`, e.g. `setup-flavor.yml`. Calling `make help` will list the
+playbooks detected under the section 'Provisioning images targets'.
+
+Note that the build system will *copy* the base image and only run the
+playbooks on these copies. This avoids generating the (same) base images for
+different provisioning run. However, the same `ROOTPW` must be used throughout
+various provisioning builds. Ansible will not be able to connect to the base
+images otherwise. The provisioning builds are timestamped with the last one
+always symlinked to e.g. `flavor.latest`. These build directories also contain
+a copy of the ansible directory (`ANSIBLE_DIR_PROV`) used for that particular
+build.
+
+## Usage: Makefile
+
+A Makefile to build packer templates is provided for convenience. It
+automatically detects JSON files in the root directory and assumes them to be
+packer templates. Refer to `make help` to get a list of detected templates and
+provisioning targets.
+
+Since only the `qemu` packer builder is currently fully supported and we
+internally mostly want qcow2 images, it is prioritized if detected. While
+VirtualBox and VMware builders are also supported to build base images, there
+is no support for the provisioning steps yet. If you still want to use either
+one, you can force it with the environment variable `BUILDER`.
+
+### Debugging
+
+For Packer builds on your local machine, you can use the environment variable
+`WINDOW` to control if the VM is started headlessly (default) or not
+(`WINDOW=1`).  This would allow to see how Packer is interacting with the VM by
+sending keystrokes via VNC. Be aware, that if you also send keystrokes during
+that step, the build will likely fail!
+
+Another useful way to debug is `DEBUG`. Normally, Packer automatically stops
+the VM and cleans up the output directories of builds that failed. When `DEBUG`
+is set, Packer will be started with `-on-error=ask`, causing it to just wait
+for confirmation before cleaning up a failed build. This is useful to access
+the VM of a failed build to debug the cause. Using `DEBUG=STEP` will use
+Packer's `-debug` option, which will ask the user for confirmation before each
+Packer step. You can also specify additional Packer parameters via
+`PACKER_OPTS`.
+
+#### Connect via SSH
+
+When spawning a VM, Packer automatically configures port forwarding for SSH.
+The host ports are randomized on each build, but can easily be found looking at
+the command line of the packer process, in particular the @hostfwd@ parameter:
 ```shell
-packer build -var-file=base.json -only=qemu ubuntu-16.04-amd64.json
-```
-Same with debug output:
-```shell
-PACKER_LOG=1 PACKER_LOG_FILE=/var/log/packer.log \
-	packer build -debug -only=qemu -var-file=base.json ubuntu-16.04-amd64.json
-```
-A successful build process would generate the disk image as '<builder>/<template>', e.g.:
-```shell
-output-qemu-ubuntu-16.04-amd64/ubuntu-16.04-amd64
+$ ps a | grep -e qemu-system -e 'hostfwd=tcp.*22'
+(...) hostfwd=tcp::2222-:22 (...) 
 ```
+This would highlight the host port forwarded to the VM's SSH port, in above
+example output 3836.
 
-Additionally, the manifest post-processor creates a manifest.json file in
-the working directory containing various information about the generated
-artifact, in particular the path to the disk file which can be easily extracted
-with jq:
-```shell
-jq -r '.builds[0].files[0].name' < output-qemu-ubuntu-16.04-amd64/manifest.json
-```
-## Usage: Provisioning base images
-
-The templates provided by this repository only perform minimal OS installations.
-Further customizations on base images should be done separately in
-a provisioning step. These will be developed in the ansible-roles repository and
-will not be further discussed here. The only requirement on the packer build process
-is to install the provisioning software and configure its remote access. Since the focus
-is on ansible, each provided template installs a SSH server and the ansible packages. 
-
-The example below shows how to run a playbook using a special packer template.
-The [ansible-roles](http://git.openslx.org/openslx-ng/ansible-roles.git/) submodule
-provides ansible roles and a sample playbook, to buildthe stage3 initramfs using
-[systemd-init](http://git.openslx.org/openslx-ng/systemd-init.git/).
-Refer to those repos for more information.
-
-#### Example
-Build a stage3 initramfs with dracut inside a previously created image using the
-'run-playbook-only' packer template used to only run the 'build-dracut-initramfs'
-playbook on a previously generated image
-(e.g. output-qemu-ubuntu-16.04-amd64/ubuntu-16.04-amd64):
-```shell
-packer build -var-file=base.json \
-	-var='image_dir="output-qemu-ubuntu-16.04-amd64"' \
-	-var='image_name="ubuntu-16.04-amd64"' \
-	-var='playbook="build-dracut-initramfs.yml"' \
-	ansible-roles/run-playbook-only.json
+#### Connect via VNC
+
+Similarly, Packer automatically configures VNC to access the VM. It shows the
+port in the stdout output, look for this line:
+```text
+(...)
+==> qemu: Connecting to VM via VNC (127.0.0.1:5959)
+(...)
 ```
-The packer artifact is created in 'output-<image_dir>'. This artifact
-contains all the development packages needed to build the initramfs, which
-can be discarded. Instead, the base image can be used as the main stage4 image.
+Use a VNC viewer to connect to the VM using this port.
+
+### Example for CentOS 7
+
+This example uses the template maintained in this repository
+`centos-7-x86_64.json`. This assumes `ROOTPW` is already in the environment:
 
-## Notes
+#### Generating base image
 
-Variables set in base.json can be overriden using packer's '-var' options,
-e.g. to disable the headless mode:
 ```shell
-packer build -var-file=base.json -var='headless=true' <template>
+$ make centos-7-x86_64/base
 ```
-Make sure you first include base.json before overriding a variable defined 
-in that file or it will not work.
 
-Partitioning the virtual disk is done during the kickstart/preseed installation. 
-A simple partition scheme is currently used: a dedicated boot partition and a
-main system partition labeled as SLX_SYS, using xfs as filesystem. Other 
-filesystems are also supported, tested were currently ext4 and btrfs. If you
-change the filesystem to use you need to adapt your boot configuration file 
-(SLX_MOUNT_ROOT_OPTIONS). Refer to systemd-init for more infos.
+#### Provision base image
+
+Assuming there is an Ansible playbook under `$(ANSIBLE_DIR_PROV)/setup-flavor.yml`:
 
-Generated qcow2 image can be further compressed using libvirt tools:
 ```shell
-virt-sparsify --compress <qcow2_file> <output_qcow2_file>
+$ make centos-7-x86_64/flavor
 ```
-## Makefile
 
-A simple Makefile to build packer templates is provided for convenience.
-Refer to `make help` for further instructions.
+#### Generate boot files
 
-#### Example
-Build Ubuntu 16.04 LTS with qemu:
+The ansible playbook generating the network boot files are maintained in
+another repository:
 ```shell
-make ubuntu-16.04-amd64/base
+$ git clone https://git.openslx.org/openslx-ng/ansible-dracut
 ```
-Generate boot files:
+
+The Makefile looks in this repository's root directory for `ansible-dracut`.
+The path to `ansible-dracut` can also be specified via the environment variable
+`ANSIBLE_DIR_CORE`.  Once setup, you can generate boot files by appending
+`/boot` to your target (both base and provisioned targets are supported).
+
 ```shell
-make ubuntu-16.04-amd64/base/boot
+$ make centos-7-x86_64/flavor/boot
 ```
-This generates the virtual disk image under 'ubuntu-16.04-amd64/base/image'
-and the boot files under 'ubuntu-16.04-amd64/base/boot/'.
-
-Only the qemu builder is currently fully supported, which is why it is
-always used if detected on the system. Virtualbox and Vmware can be used
-to build base images only, provisioning is not yet supported using these.
-Note they generate OVF and VMDK files respectively. Using qemu is
-currently recommended.
+This generates the disk image under `centos-7-x86_64/flavor/rootfs-image` and
+the boot files under `centos-7-x86_64/flavor/boot/`.
 
 ## TL;DR
-0) Clone repo:
+0) Export root password:
 ```shell
-git clone --recursive git://git.openslx.org/openslx-ng/packer-templates
+$ export ROOTPW="s0mer4nd0mp4$$w0rd"
 ```
 1) Build base image:
 ```shell
-make ubuntu-16.04-amd64/base
+$ make centos-7-x86_64/base
 ```
 2) Build initramfs for base image:
 ```shell
-make ubuntu-16.04-amd64/base/boot
+$ make centos-7-x86_64/base/boot
 ```
 
-## Results
-Produces:
-* kernel & initramfs in ubuntu-16.04-amd64/base/boot/
-* main qcow2 image under ubuntu-16.04-amd64/base/image
+## Results Produces:
+* kernel & initramfs in centos-7-x86_64/base/boot/
+* main qcow2 image under centos-7-x86_64/base/rootfs-image
 
-To boot:
-* deploy qcow2 image on your dnbd3 server
-* deploy kernel and initramfs on http server
-* point to the new dnbd3 image in your config file, see template:
-    *   http://git.openslx.org/openslx-ng/systemd-init.git/tree/dev-tools/example-openslx.config
-- 
cgit v1.2.3-55-g7522