From 8a894ef9e3c202d4d0cce0196b3298f251891c6e Mon Sep 17 00:00:00 2001 From: Sebastian Vater Date: Mon, 27 Oct 2025 16:35:59 +0100 Subject: Added documentation for iSCSI rootfs with dracut in README.md. --- README.md | 92 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 91 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index f4e8efc..7ccbe14 100644 --- a/README.md +++ b/README.md @@ -639,7 +639,7 @@ apt-get install open-iscsi ``` After successful installation, it is possible to attach _iSCSI_ devices as regular block devices like `/dev/sda`. -Since _iSCSI_ discovery is **NOT** supported yet, target nodes have to be created manually: +Without using _iSCSI_ discovery, target nodes have to be created manually: ```shell iscsiadm -m node -T -p : --op=new @@ -649,6 +649,14 @@ This will create an _iSCSI_ target node `image-name` which is attached to the po A target node configuration file will be created in _/etc/iscsi/nodes/`image-name`/`iscsi-server`,`port`_. Please note that for any images residing in sub directories of the base path, the sub directory tree has to be created manually in _/etc/iscsi/nodes_ and _/etc/iscsi/static_ before adding the target node with `iscsiadm`. +If using _iSCSI_ discovery with: + +```shell +iscsiadm -m discovery -t sendtargets -p : +``` + +This will not only show all public _DNBD3_ images currently in use (i.e. logged in) but also all alternative server addresses (from `alt-servers` configuration file) where the _DNBD3_ images can also be found which can be used as a base for `multipath` support (see below). For any image shown here, a direct login is possible without having to manually add the target node. + After the target node has been created, it can be made available as a block device using: ```shell @@ -677,6 +685,88 @@ This will delete the target node in _/etc/iscsi/nodes/`image-name`/`iscsi-server Please note that for any images residing in sub directories of the base path, the sub directory tree has to be removed manually from _/etc/iscsi/nodes_ and _/etc/iscsi/static_ after removing the target node with _iscsiadm_. +### Booting using a rootfs from iSCSI with dracut and open-iscsi. +Having a root filesystem on _iSCSI_ needs changes to _initramfs_ which can easily be applied using `dracut`. +On _Debian_ based distributions like _Ubuntu_, the following packages have to be installed: + +```shell +apt-get install dracut \ + dracut-network \ + open-iscsi \ + multipath-tools \ + multipath-tools-boot +``` + +Since the _iSCSI_ server now supports discovery, no `dracut` scripts have to be adapted. However, at least with _Ubuntu_, the _initramfs_ requires additional runtime binaries and scripts in order to work, therefore the missing files have to be added via `dracut` configuration files in `/etc/dracut.conf.d`. In order to add all missing files for _open-iscsi_, create a file named `iscsi.conf` in the previously mentioned directory: + +```ini +add_dracutmodules+=" iscsi network " +install_items+=" $(find /etc/iscsi /usr/lib/open-iscsi | while read -r FILE; do echo -n "$FILE "; done) " +hostonly="no" +``` + +`open-iscsi` has its own configuration in `/etc/iscsi` and in order to work properly with _initramfs_, at least the files `initiatorname.iscsi` and `iscsid.conf`, as well as an empty directory `nodes` have to be integrated into _initramfs_. The default content of those two files is sufficient but it is strongly recommended to at least change the _iSCSI_ initiator _IQN_ name in `initiatorname.iscsi`. If manual _target nodes_ have to be added, do so before updating the _initramfs_, so the required directories and files in `/etc/iscsi` get added as well. +Since _iSCSI_ discovery currently only shows currently logged in public _DNBD3_ images, a login into the image used for _rootfs_ should be automatically done on startup, which can be achieved with: + +```shell +iscsiadm -m node -T -p : --op=new --name=node.startup --value=onboot +``` + +This ensures that a login occurs before the actual _iSCSI_ discovery occurs. + +Since _DNBD3_ does not have write support, the rootfs needs an `overlayfs` which can be added using the `dracut` builtin plugin `overlay-root`. To do so, create a file `overlay-root.conf` in `/etc/dracut.conf.d` with the following contents: + +```ini +add_dracutmodules+=" overlay-root " +``` + +Last, but not least, `multipath` has to be added to `dracut`, which is done by creating `multipath.conf` in `/etc/dracut.conf.d`, containing: + +```ini +add_dracutmodules+=" multipath " +install_items+=" $(find /etc/multipath.conf /etc/multipath /lib/multipath | while read -r FILE; do echo -n "$FILE "; done) " +``` + +Here is a minimal `/etc/multipath.conf` for automatically adding any _DNBD3_ image to _multipath_: + +```ini +devices { + # Define configuration for HP OPEN-V storage + device { + vendor "UNI FRBG" +# product "debian-13.1.0-amd64-DVD-1.iso" + product "." + no_path_retry 18 + } +} +``` + +All _iSCSI_ _DNBD3_ images have the vendor identifier `UNI FRBG`. The product identifier equals to the first 16 characters of the _DNBD3_ image name, it however, does not hurt to specify larger names in `multipath.conf`. The example above just matches any product / _DNBD3_ image name with `"."`. Since `multipath` works with full _WWN_, it is recommended to login first into the image and retrieve the _WWN_ from `/dev/disk/by-id`. By looking at the symlink destination of `scsi_SUNI_FRBG...`, it is possible to correctly assign the symlink starting with `scsi_`, followed by the _WWN_. Note that there are two _WWN_ names, one symlink starting with `wwn-0x`, followed by 16 hexadecimal digits and the previously mentioned one starting with `scsi_` which has another additional digit in front of the actual _WWN_. `multipath` needs the one with the additional digit, which can be registered using: + +```shell +multipath -a <17-hex-digit-wwn> +``` + +The `multipath` devices don't add the partitions automatically, a proper `udev` rule invoking `kpartx` has to be created in order to do this. + +Once everything has been configured, it is time to rebuild the _initramfs_ using `dracut` by: + +```shell +dracut -f +``` + +Finally, the _iSCSI_ boot parameters have to be appended to the Linux command line using `GRUB` (the `rd.iscsi.*` parameters are used by `iscsistart` and are necessary when booting via _PXE_ directly into an _iSCSI_ target): + +```ini +GRUB_CMDLINE_LINUX_DEFAULT="... rd.iscsi.initiator= rd.iscsi.target.name= rd.iscsi.target.ip= rd.iscsi.target.port= ip=dhcp rd.neednet=1 netroot=iscsi::tcp::: ro root=UUID= ro rootovl rd.multipath=1" +``` + +Before rebooting for testing, this one should not be forgotten: +```shell +update-grub +``` + + ### Configuring and using _QEMU_ for using the iSCSI server Here is a bash script called `qemu-iscsi.sh` that invokes _QEMU_ using _iSCSI_. -- cgit v1.2.3-55-g7522