# Packer templates for network boot using OpenSLX-NG

This repository provides a build system based on GNU Make and Packer/Ansible
templates to create clean operating system images - using Packer to generate
the base system and Ansible to further install software stacks. This repository
contains a set of templates for CentOS and Ubuntu distributions as well as
kickstart/preseed files to handle the initial unattended installations. A
special Packer template further allows for the base images to be provisioned
using custom Ansible playbooks. The provisioning playbooks are not maintained
within this repository - but can be specified via environment variable when
calling the Makefile (more on that later). Once provisioned, the network boot
playbook maintained on `https://git.openslx.org/openslx-ng/ansible-dracut.git`
can optionally be executed to generate the initramfs (and to extract the
kernel) needed for network boot.

This repository contains a Makefile coordinating all these steps. Even though
the provided Packer templates and Ansible playbooks are usable as is, it is
recommended to use the Makefile to build images. Beware that changing
parameters of the provided templates (e.g. output directories or image names)
might break the full workflow - if changes are necessary, make sure to reflect
these changes within the Makefile.

## Dependencies

* Packer => 1.2.2
	* Download: https://www.packer.io/downloads.html

* QEMU => 3.0
	* Download: https://www.qemu.org/download/

* Ansible => 2.5.0
	* Installation: https://docs.ansible.com/ansible/latest/installation_guide/

Note: Make sure the build machine supports KVM to accelerate the build process
substantially.  Note: VMware Player and VirtualBox are supported to build base
images, i.e. simple unattended installations, they are not yet supported for
the provisioning steps.

## Preliminaries

The file `base.json` contains default variables (VM, SSH) common to all
templates and is always included during packer builds. This file controls the
VM's hardware configuration, in particular the number of CPUs (`cpus`), the
amount of RAM in bytes (`memory`) and the virtual disk size in bytes
(`disk_size`). Adjust these as needed. The SSH variables specify the user
connecting to the VM (`ssh_username`) and how long packer will try to connect
to be VM before aborting a build (`ssh_timeout`). Note that the root password
is expected to be specified as the `ROOTPW` environment variable, as explained
later.

### Templates for Packer, Kickstart, Preseed

Distribution templates are saved in the root directory of this repository, e.g.
`centos-7-x86_64.json`. To see a list of currently maintained distributions,
refer to `make help` and look for *Base image targets*.

The kickstart/preseed configuration files needed for unattended installations
(UAI) during the packer builds are stored in the `http` folder. The Packer
builders are configured to look for UAI files in the distribution's version
directory, e.g. `http/centos-7`.

So far, only one UAI file is kept for every distribution family. The versioned
folder thus point to the common family folders. This is to allow customized or
version-specific UAI files to be used if needed, without having to fiddle with
the packer templates files themselves. We think it is a good practice to try
and keep the images resulting from UAIs as basic as possible. This way, base
UAI images can be used for different provisioning runs. To support this, OS
configuration or the installation of additional software should only be
realised in Packer's provisioning step.

### Playbooks for Ansible provisioning

The build system also has code to automatically provision base images. It
thereby looks for the `packer-templates/ansible-roles-prov` directory or in the path
specified by the environment variable `ANSIBLE_DIR_PROV`. Within the
provisioning directory, it further looks for Ansible playbooks named
`setup-*.yml`, e.g. `setup-flavor.yml`. Calling `make help` will list the
playbooks detected under the section *Provisioning images targets*.

Note that the build system will **copy** the base image and only run the
playbooks on these copies. This avoids generating the (same) base images for
different provisioning run. However, the same `ROOTPW` must be used throughout
various provisioning builds. Ansible will not be able to connect to the base
images otherwise. The provisioning builds are timestamped with the last one
always symlinked to e.g. `flavor.latest`. These build directories also contain
a copy of the ansible directory (`ANSIBLE_DIR_PROV`) used for that particular
build.

## Usage

A Makefile to build packer templates is provided for convenience. It
automatically detects JSON files in the root directory and assumes them to be
packer templates. Refer to `make help` to get a list of detected templates and
provisioning targets.

Since only the `qemu` packer builder is currently fully supported and we
internally mostly want qcow2 images, it is prioritized if detected. While
VirtualBox and VMware builders are also supported to build base images, there
is no support for the provisioning steps yet. If you still want to use either
one, you can force it with the environment variable `BUILDER`.

#### Example for CentOS 7

This example uses the template maintained in this repository
`centos-7-x86_64.json`. This assumes `ROOTPW` is already in the environment:

##### Generating base image

```shell
$ make centos-7-x86_64/base
```

##### Provision base image

Assuming there is an Ansible playbook under `$(ANSIBLE_DIR_PROV)/setup-flavor.yml`:

```shell
$ make centos-7-x86_64/flavor
```

##### Generate boot files

The ansible playbook generating the network boot files are maintained in
another repository:
```shell
$ git clone https://git.openslx.org/openslx-ng/ansible-dracut
```

The Makefile looks in this repository's root directory for `ansible-dracut`.
The path to `ansible-dracut` can also be specified via the environment variable
`ANSIBLE_DIR_CORE`.  Once setup, you can generate boot files by appending
`/boot` to your target (both base and provisioned targets are supported).

```shell
$ make centos-7-x86_64/flavor/boot
```
This generates the disk image under `centos-7-x86_64/flavor/rootfs-image` and
the boot files under `centos-7-x86_64/flavor/boot/`.


## Debugging

For Packer builds on your local machine, you can use the environment variable
`WINDOW` to control if the VM is started headlessly (default) or not
(`WINDOW=1`).  This would allow to see how Packer is interacting with the VM by
sending keystrokes via VNC. Be aware, that if you also send keystrokes during
that step, the build will likely fail!

Another useful way to debug is `DEBUG`. Normally, Packer automatically stops
the VM and cleans up the output directories of builds that failed. When `DEBUG`
is set, Packer will be started with `-on-error=ask`, causing it to just wait
for confirmation before cleaning up a failed build. This is useful to access
the VM of a failed build to debug the cause. Using `DEBUG=STEP` will use
Packer's `-debug` option, which will ask the user for confirmation before each
Packer step. You can also specify additional Packer parameters via
`PACKER_OPTS`.

#### Connect via SSH

When spawning a VM, Packer automatically configures port forwarding for SSH.
The host ports are randomized on each build, but can easily be found looking at
the command line of the packer process, in particular the @hostfwd@ parameter:
```shell
$ ps a | grep -e qemu-system -e 'hostfwd=tcp.*22'
(...) hostfwd=tcp::2222-:22 (...) 
```
This would highlight the host port forwarded to the VM's SSH port, in above
example output 3836.

#### Connect via VNC

Similarly, Packer automatically configures VNC to access the VM. It shows the
port in the stdout output, look for this line:
```text
(...)
==> qemu: Connecting to VM via VNC (127.0.0.1:5959)
(...)
```
Use a VNC viewer to connect to the VM using this port.

## TL;DR
0) Export root password:
```shell
$ export ROOTPW="s0mer4nd0mp455w0rd"
```
1) Build base image:
```shell
$ make centos-7-x86_64/base
```
2) Build initramfs for base image:
```shell
$ make centos-7-x86_64/base/boot
```
##### Generates
* kernel & initramfs in `centos-7-x86_64/base/boot/`
* root filesystem as qcow2 image in `centos-7-x86_64/base/rootfs-image`