RAMBOOT

RAMBOOT is a distribution of Ubuntu Core designed for rapid development of embedded Linux systems. Such systems are commonly referred to as “network appliance” systems.


Current Release: RAMBOOT-1.0.0-beta1 (2012-12-27)

Download Archive OR Download Disk Image


News


Contents


RAMBOOT is a very early work-in-progress.
It is released under the GNU General Public License version 2 or later:

    RAMBOOT 
    Copyright (C) 2013 Ray Patrick Soucy
    
    This program is free software; you can redistribute it and/or
    modify it under the terms of the GNU General Public License
    as published by the Free Software Foundation; either version 2
    of the License, or (at your option) any later version.
    
    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.
    
    You should have received a copy of the GNU General Public License
    along with this program; if not, write to the Free Software
    Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.

If you use RAMBOOT or find it useful we would love to hear about it.


Introduction

From a user perspective RAMBOOT provides a bootable in-memory minimal install release of Ubuntu LTS 12.04.
The user can use "apt-get" to install the desired packages from Ubuntu repositories.
The installed and configured system can then be preserved by generating a new system image.

RAMBOOT is designed for use on inexpensive USB flash storage. It has a small footprint and does not make use of the bootflash once the system has booted. This avoids the performance constraints of USB storage.

An in-memory operating system provides several advantages when used for an infrastructure device:

These features are common for network infrastructure devices such as network routers and firewalls but remain uncommon with Linux-based alternatives. RAMBOOT aims to apply this model to Linux-based appliances.


Installing

There are two options for installing RAMBOOT. The first option is to manually install SYSLINUX on a storage device and then download the RAMBOOT ZIP archive and extract it to the storage device.

The second option is to download the 1G RAMBOOT disk image and write it to a USB drive directly. This method requires the use of GNU dd and gzip. This is a raw disk 1G disk image compressed using gzip.

Assuming /dev/sdb is your USB device:

wget http://ramboot.org/download/RAMBOOT-1.0.0-beta1.img.gz
gzip -dc RAMBOOT-1.0.0-beta1.img.gz | dd of=/dev/sdb
WARNING: GNU DD is a powerful tool. Make sure you write to the correct device or you may overwrite your hard drive and yield your PC unusable.


Technical Information

RAMBOOT makes use of the Ubuntu Core rootfs: https://wiki.ubuntu.com/Core
Ubuntu Core provides a base environment with functional package management (apt-get) against Ubuntu-maintained software repositories.

RAMBOOT makes use of SYSLINUX for its bootloader: http://www.syslinux.org/
SYSLINUX makes use of an MS-DOS FAT filesystem allowing for the bootflash to be mounted on most platforms. SYSLINUX was specifically chosen due to many BIOS implimentations using the presence of a FAT filesystem to determine whether or not a USB device is bootable (such as ones used by Intel Atom system boards).

RAMBOOT provides an "in-memory" OS using the Linux kernel ramdisk driver.
Once the system is booted, the bootflash is unmounted and no longer needed for the operation of the system.
A system reboot will restore the system to a known state as all system files are extracted to build the ramdisk upon each boot.
Because RAMBOOT does not make use of a storage device during operation it is more resiliant (HDD failure is the most common cause of system failure) faster (HDD IO wait is the biggest bottleneck in modern systems) and more energy efficient. Storage demands are very low and RAMBOOT is well suited (and designed for) the use of USB flash storage for its startup disk (bootflash).

RAMBOOT is implimented by adding an additional "boot=" target to initramfs-tools: the toolchain used by Debian and Debian-based Linux distributions such as Ubuntu for building the initrd for kernel upgrades. As such the same kernel and initrd can be used for both a traditional Linux system and a RAMBOOT system. This allows for easy development and intergration of RAMBOOT packages.

The ramdisk size is also specified as a kernel option within the bootloader. This allows for the ramdisk to be resized easily. The recommended minimum ramdisk size is 512MB. Systems with 4GB RAM should consider a 1GB ramdisk and systems with 6GB or more RAM should consider the use of a 2GB ramdisk.

The directory structure of the RAMBOOT bootflash:
The following directories are required:

    CONFIG/
    OS/
    SYSLINUX/
In addition, development tools are provided in:
    INSTALL/
Provided the directories above are not modified, the bootflash can be used for storage of other material (such as User Documentation).

RAMBOOT makes use of GNU tar.gz compressed archives rather than a package format such as .deb or .rpm.

For the base system, there are 3 archives used to build the ramdisk:

  1. The Ubuntu Core LTS rootfs archive.
  2. An archive of kernel modules and firmware for the kernel used.
  3. A configuration archive that maintains system-specific configuration (such as "/etc/passwd").

If RAMBOOT fails to boot, you may need to edit the SYSLINUX/SYSLINUX.CFG file on your USB device. The "root=" kernel option must be the device name that Linux will see the boot device as. If you are installing to USB and there is one HDD on the system, this is generally "/dev/sdb1" if using RAMBOOT on a system with no other storage device, use "/dev/sda1". Also note that the "ramdisk_size=" option can be modified to increase the size of the ramdisk.

Assuming a bootable SYSLINUX storage device and the latest RAMBOOT archive installed, upon boot, all files ending in ".tar.gz" within the OS directory will be extracted to the root filesystem on the ramdisk. The name of individual archives is otherwise irrelevant. To prevent an archive from being extracted it can be moved out of the OS folder or renamed to not end in ".tar.gz" (example: ".tar.gz.removed").

After the ramdisk is built using the archives in the OS directory, the archive "config.tar.gz" within the CONFIG directory is extracted to the system root. Because the configuration archive is extracted last, it can contain files that overwrite existing files within the rootfs archive. This provides a method of keeping system configuration seperate from the OS.

The modules archive is generated as an archive of "/lib/modules/" and "/lib/firmware" for the kernel used. By default, the modules archive contains all modules provided by the production Ubuntu kernel. Because the majority of hardware support is provided though the use of loadable kernel modules, this is essentially a driver pack. Using the "rebuild_modules" script within the INSTALL directory will build a new modules archive containing only the modules that have been loaded by the system. This provides a significant footprint reduction for the OS. By keep modules as a seperate archive, the modules package can be hardware-specific allowing for the same RAMBOOT rootfs to be applied to several hardware configurations without each needing a unique rootfs or having to maintain unneeded modules on the system.

[2013-12-27 rps] Note that this script is a work in progress and further refinements are necessary (such as a list of modules to always include that may not be loaded at the time the script was run).

Within the RAMBOOT boot process, the root user account is enabled (no password) and the hostname is set to "ramboot" before the configuration archive is loaded. These settings should be overwritten using a configuration archive once the system is built. The system does not require a configuration archive to boot and will not have one upon first-run. RAMBOOT also generates an initial fstab providing a "/bootflash" mountpoint which can be used to access the boot storage device: "mount /bootflash".

Once booted, the bootflash can be mounted and scripts within the INSTALL directory can be used to build your RAMBOOT system:

    mount /bootflash
    cd /bootflash/INSTALL
    ./setup_network
The "setup_network" script will provide quick IP configuration. Note that RAMBOOT is a minimal install, so only the iproute2 package is provided for network configuration (legacy "ifconfig" and "route" tools, and packages such as a DHCP client, need to be manually installed).

Once connected, issuing an "apt-get update" will build a list of available Ubuntu LTS packages.
You can then use "apt-get install " to install the desired software for your OS.

Example:

    apt-get update
    
    apt-get install sudo
    apt-get install nano
    apt-get install ssh
    apt-get install vlan
    apt-get install bridge-utils
    apt-get install apache2 php5 php5-cli
    apt-get install isc-dhcp-server
    apt-get install bind9 
Modify any configuration and system files that you wish to be part of the base OS image.
Then the "rebuild_rootfs" script within the INSTALL directory can be run to build a new rootfs image of Ubuntu Core that includes the installed packages. The script excludes UDEV persistance rules and other stateful data. It also clears the apt cache to avoid including cached packages in the image and maintaining a small footprint.

[2013-12-27 rps] This script still needs some improvment: Right now it depends on the Ubuntu Core archive being named "ubuntu-core-12.04.2-core-amd64.tar.gz". There are also further footprint reductions that should be included, such as removing MAN pages and internationalization dictionaries.

You should then be able to reboot your RAMBOOT system using the new rootfs with your desired software.
For quickly re-creating the modified rootfs image (e.g. to use updated packages) you should script the process or maintain notes on the steps taken.

For system-specific configuration, you should use the "config.tar.gz" archive.

Included within the OS directory is a scripts archive which extracts a "/scripts" folder to the rootfs. The intent of this archive is to provide system management scripts. Currently, only one script is provided as an example: config-save. The config-save script will generate a config.tar.gz archive and save it to the bootflash. This will allow system configuration to be preserved through a system reboot. You should review the files included within the configuration archive and modify the config-save script for your needs.

Note that since all archives in the OS directory are extracted on boot, you can use multiple archives (as provided using the scripts.tar.gz example) to keep system components seperate. A production OS directory may look something like:

    /base_system.tar.gz
    /modules_DELL-R720.tar.gz
    /scripts.tar.gz
    /http_root.tar.gz


Software Development

The implimentation of RAMBOOT makes use of the Debian project "initramfs-tools" package.
A "boot=" target of "ramboot" is added by creating a script in "/usr/share/initramfs-tools/scripts/ramboot" along side "nfs" and "local" target scripts.

The current script source:

# RAMBOOT filesystem mounting			-*- shell-script -*-

mountroot()
{

	while [ -z "${FSTYPE}" ]; do
		FSTYPE=$(wait-for-root "${ROOT}" ${ROOTDELAY:-30})
	done

	mkdir /bootflash
	mount -t vfat ${ROOT} /bootflash

	if [ ! -d /bootflash/OS ]; then
		panic "

ERROR: Could not find "OS" directory on ${ROOT}

The ramboot option requires the boot device contain an OS directory containing 
the tar.gz archive(s) used to build the root filesystem.  

At a minimum this must include Ubuntu Core and an archives of kernel modules 
for the kernel used.

"
	fi

	echo -n "Creating ramdisk on /dev/ram1: "
	mke2fs /dev/ram1
	mount -t ext2 /dev/ram1 ${rootmnt}
	echo "Done"

	echo "Extracting OS archive(s)"
	for i in `ls -1 /bootflash/OS/*.tar.gz`; do
		echo -n "    Extracting $i:"
		tar zxfp $i -C ${rootmnt}
		echo "Done"
	done

	mkdir ${rootmnt}/bootflash
	echo "# Generated by RAMBOOT" > ${rootmnt}/etc/fstab
	echo "/dev/ram1 / ext2 rw 0 0" >> ${rootmnt}/etc/fstab
	echo "${ROOT} /bootflash vfat noauto 0 0" >> ${rootmnt}/etc/fstab

	echo "ramboot" > ${rootmnt}/etc/hostname
	sed -i 's/root:\*:/root::/g' ${rootmnt}/etc/shadow

	echo "" > ${rootmnt}/etc/motd.tail
	echo " * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *" >> ${rootmnt}/etc/motd.tail
	echo " *                                                                           *" >> ${rootmnt}/etc/motd.tail
	echo " *  Ubuntu Core 12.04 RAMBOOT                                                *" >> ${rootmnt}/etc/motd.tail
	echo " *                                                                           *" >> ${rootmnt}/etc/motd.tail
	echo " *  Use \"mount /bootflash\" to access system flash.                           *" >> ${rootmnt}/etc/motd.tail
	echo " *  See the INSTALL directory on the flash for setup instructions.           *" >> ${rootmnt}/etc/motd.tail
	echo " *                                                                           *" >> ${rootmnt}/etc/motd.tail
	echo " * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *" >> ${rootmnt}/etc/motd.tail
	

	if [ -f /bootflash/CONFIG/config.tar.gz ]; then
		echo -n "Extracting configuration archive: "
		tar zxfp /bootflash/CONFIG/config.tar.gz -C ${rootmnt}
		echo "Done"
	else
		echo "WARNING: No configuration package found."
	fi

	echo -n "Unmounting bootflash: "
	umount /bootflash
	echo "Done"

	echo "Booting system ..."

}
By default, the copy of busybox included for initramfs-tools lacks support for GNU tar. The mke2fs utility is also absent.
The binaries available within the initrd generated are copied from "/usr/lib/initramfs-tools/bin/".

The necessary tools need to be manually added:

    cd /usr/lib/initramfs-tools/bin/
    mv busybox /root/busybox.backup
    cp /bin/busybox .
    cp /sbin/mke2fs .
For releases after 12.04, all binaries in "/usr/lib/initramfs-tools/bin/" are automatically included in the initrd. For 12.04 however, a hook script is needed to copy mke2fs into the environment.

Add a hook to import "mke2fs" by creating "/usr/share/initramfs-tools/hooks/mke2fs":

#!/bin/sh

# initramfs hook for mke2fs (ramboot)

set -e

# initramfs-tools header

PREREQ=""

prereqs()
{
	echo "${PREREQ}"
}

case "${1}" in
	prereqs)
		prereqs
		exit 0
		;;
esac

. /usr/share/initramfs-tools/hook-functions

# mke2fs hook

copy_exec /usr/lib/initramfs-tools/bin/mke2fs /bin/mke2fs

exit 0

Once these changes have been done, running "update-initramfs -u" or installing a new kernel package will generate a initrd with RAMBOOT in place.

The RAMBOOT script is only run with "boot=ramboot" is passed to the kernel from the bootloader, so the kernel can still be used for a standard installation.

A modules archive must be generated for each kernel since the Ubuntu Core rootfs archive excludes a kernel:

    tar zcvf modules-`uname -r`.tar.gz /lib/modules/`uname -r` /lib/firmware
The kernel image can then be copied into the SYSLINUX folder. You will need the kernel image and the initrd, both should be located in "/boot" on your reference system:
    vmlinuz-3.8.0-34-generic
    initrd.img-3.8.0-34-generic
Manually create a SYSLINUX bootable USB:

(assuming USB is on /deb/sdb)

fdisk /dev/sdb

	Delete partition table "o"
	Create a new partition "n" (use defaults)
	Change the partition type to FAT 32 (LBA) "t" and option "c"
	Set the partition to bootable "a" (then "1" for the partition)
	Write and exit "w"

mkfs.vfat -F 32 -n "RAMBOOT" /dev/sdb1
syslinux -i /dev/sdb1
cat /usr/lib/syslinux/mbr.bin | dd of=/dev/sdb
mount /dev/sdb1 /mnt
cd /mnt
mkdir SYSLINUX
cd SYSLINUX
nano SYSLINUX.CFG
SYSLINUX.CFG:
DISPLAY README.TXT
TIMEOUT 50
TOTALTIMEOUT 9000
ONTIMEOUT core

LABEL memtest
	KERNEL memtest

LABEL core
	KERNEL vmlinuz-3.8.0-34-generic
	INITRD initrd.img-3.8.0-34-generic
	APPEND root=/dev/sdb1 boot=ramboot ramdisk_size=520000K elevator=deadline


Copyright © 2013, Ray Patrick Soucy
Code examples are licensed under the GNU General Public License version 2 or later. Verbatim copying and distribution of this page is permitted provided this notice is preserved.