title: FreeBSD

FreeBSD

USB 3.1 Type-C to RJ45 Gigabit Ethernet adapter

The Amazon Basics Aluminum USB 3.1 Type-C to RJ45 Gigabit Ethernet Adapter works well with FreeBSD 14.1-RELEASE. It uses the AX88179 chipset from ASIX Electronics Corp.

Install the ports tree

Source: Chapter 4. Installing Applications: Packages and Ports | FreeBSD Documentation Portal

Ensure the FreeBSD source code is checked out

sudo git clone -o freebsd -b releng/14.1 https://git.FreeBSD.org/src.git /usr/src

Check out the ports tree

sudo git clone --depth 1 https://git.FreeBSD.org/ports.git -b 2024Q3 /usr/ports

To switch to a different quarterly branch:

sudo git -C /usr/ports switch 2024Q4

drm-61-kmod

Install from the ports tree.

cd /usr/ports/graphics/drm-61-kmod
sudo make install clean

Alternatively, for Alderlake GPUs:

sudo pkg install drm-kmod

Edit /etc/rc.conf:

kld_list="i915kms"

Add user to video group:

sudo pw groupmod video -m jas

Mount filesystems in single-user mode

When booted into single-user mode.

fsck
mount -u /
mount -a -t zfs
zfs mount -a

You should now be able to edit files, add/remove packages, etc.

Mount encrypted zroot in LiveCD

Boot into the LiveCD environment.

mkdir /tmp/mnt
geli attach /dev/nda0p4
zpool import -f -R /tmp/mnt zroot
zfs mount zroot/ROOT/default

The root directory of the zroot, zroot/ROOT/default, is labeled to not be automounted when imported, hence the need for the last command.

Setup Podman (FreeBSD >= 14)

The following is a condensed version of the guide found at CloudSpinx: Install Podman and run Containers in FreeBSD 14.

sudo pkg install podman-suite
sudo mount -t fdescfs fdesc /dev/fd

Add the following line to /etc/fstab:

fdesc /dev/fd fdescfs rw 0 0

Enable the Podman service.

sudo sysrc podman_enable="YES"

Container networking requires a NAT to allow the container network's packets to reach the host's network. Copy the sample pf.conf for Podman.

sudo cp -v /usr/local/etc/containers/pf.conf.sample /etc/pf.conf

Change v4egress_if and v6egress_if to the host's main network interface in /etc/pf.conf.

v4egress_if="igc0"
v6egree_if="igc0"

Enable and start PF.

sudo sysrc pf_enable="YES"
sudo service pf start

FreeBSD >= 13.3 has support for rerouting connections from the host to services inside the container. To enable this, load the PF kernel module, then use sysctl to activate PF support for this rerouting.

echo 'pf_load="YES"' | sudo tee -a /boot/loader.conf
sudo kldload pf
sudo sysctl net.pf.filter_local=1
echo 'net.pf.filter_local=1' | sudo tee -a /etc/sysctl.conf.local
sudo service pf restart

The rerouting rules will only work if the destination address is localhost. Ensure the following exists in /etc/pf.conf.

nat-anchor "cni-rdr/*"

Container images and related state is stored in /var/db/containers. Create a ZFS dataset for this with the mountpoint set to that directory.

sudo zfs create -o mountpoint=/var/db/containers zroot/containers

If the system is not using ZFS, change storage.conf to use the vfs storage driver.

sudo sed -I .bak -e 's/driver = "zfs"/driver = "vfs"/' /usr/local/etc/containers/storage.conf

If there are any errors caused by the /var/db/containers/storage database, remove it.

sudo rm -rfv /var/db/containers/storage

IMPORTANT
Note: Podman can only be run with root privileges on FreeBSD at this time.

Enable the Linux service.

sudo sysrc linux_enable="YES"
sudo service linux start

To run Linux containers, add the --os=linux argument to Podman commands.

sudo podman run --os=linux ubuntu /usr/bin/cat "/etc/os-release"

Everything else should work as expected.

Install Linux VM in Bhyve

Based on How to install Linux VM on FreeBSD using bhyve and ZFS, but condensed and collated for my use-case.

Setting up the network interfaces

Make the tap device UP by default in /etc/sysctl.conf.

echo "net.link.tap.up_on_open=1" >> /etc/sysctl.conf
sysctl net.link.tap.up_on_open=1

Load the kernel modules neeeded for bhyve.

kldload vmm
kldload nmdm

Make sure the modules are loaded at boot time.

echo 'vmm_load="YES"' >> /boot/loader.conf
echo 'nmdm_load="YES"' >> /boot/loader.conf
echo 'if_tap_load="YES"' >> /boot/loader.conf
echo 'if_bridge_load="YES"' >> /boot/loader.conf

Create the bridge and tap device. If you already have a bridge created, use that instead. We'll assume this is the case, and the bridge is called igb0bridge.

ifconfig bridge create

If a bridge is already created and the main network interface igc0 is attached to it, the following command is not necessary.

ifconfig igb0bridge addm igc0

Create tap interface and attach it to the igb0bridge.

ifconfig tap0 create
ifconfig igb0bridge addm tap0

If there wasn't a bridge already being used for jails, then /etc/rc.conf should contain the following:

cloned_interfaces="igb0bridge tap0"
ifconfig_igb0bridge="addm igc0 addm tap0 up"

If there was already a bridge used for jails, then /etc/rc.conf should contain the following:

cloned_interfaces="igb0bridge tap0"
ifconfig_igb0bridge="inet 10.0.0.8/24 addm igc0 addm tap0 up"

Setting up the ZFS volumes for Linux bhyve VM

zfs create -V128G -o volmode=dev zroot/debianvm

Downloading Debian installer iso

cd /tmp/
DEBIAN_VERSION=12.10.0
wget "https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-${DEBIAN_VERSION}-amd64-netinst.iso"

Installing Debian in VM

Install the grub-bhyve binary to allow booting of non-FreeBSD guest OSes.

pkg install grub2-bhyve bhyve-firmware

Install Debian by running bhyve with the netinstall iso image and the zvol attached.

bhyve -c 4 -m 8G -w -H \
  -s 0,hostbridge \
  -s 3,ahci-cd,/tmp/debian-12.10.0-amd64-netinst.iso \
  -s 4,virtio-blk,/dev/zvol/zroot/debianvm \
  -s 5,virtio-net,tap0 \
  -s 29,fbuf,tcp=0.0.0.0:5900,w=800,h=600,wait \
  -s 30,xhci,tablet \
  -s 31,lpc \
  -l com1,stdio \
  -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
  debianvm

Arguments	Description
`-c 4`	number of virtual CPUs
`-m 8G`	RAM size for VM
`-w`	ignore unimplemented MSRs
`-H`	host filesystem to export to the loader
`-s 3,ahci-cd,/tmp/debian-12.10.0-amd64-netinst.iso`	Configure an AHCI-CD device in virtual PCI slot 3 to hold the netinstall iso cdrom.
`-s 4,virtio-blk,/dev/zvol/zroot/debianvm`	Configure a virtio block device in virtual PCI slot 4 to install the OS onto.
`-s 5,virtio-net,tap0`	Configure a virtual network interface in virtual PCI slot 5 and attach the `tap0` interface.
`-s 29,fbuf,tcp=0.0.0.0:5900,w=800,h=600,wait`	Configure a virtual framebuffer device in virtual PCI slot 29 to enable connection from a remote VNC viewer on port 5900. The framebuffer resolution is 800x600. The `wait` argument instructs bhyve to only boot upon the initiation of a VNC connection.
`-s 30,xhci,tablet`	Provides precise cursor synchronization when using VNC.
`-s 31,lpc`	Configure a virtual LPC device on virtual PCI slot 31.
`-l com1,stdio`	Configure the TTY-class device `com1` with stdio.
`-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd`	the OS loader to use (UEFI needed for non-FreeBSD OSes).

When the command runs, use a remote VNC view to connect to and start the netinstall iso.

IMPORTANT
The following step is required to boot from UEFI.

Run the Debian installer with desired configuration. When you reach the "Finish the installation" stage, select "Go Back", then select "Execute a shell". Once in the shell, run the following commands:

mkdir /target/boot/efi/EFI/BOOT/
cp -v /target/boot/efi/EFI/debian/grubx64.efi /target/boot/efi/EFI/BOOT/bootx64.efi
exit

Now continue with "Finish the installation".

Booting Debian bhyve VM

The instance of the virtual machine needs to be destroyed before it can be started again.

bhyvectl --destroy --vm=debianvm

Boot the Debian VM.

bhyve -c 4 -m 8G -w -H \
  -s 0,hostbridge \
  -s 4,virtio-blk,/dev/zvol/zroot/debianvm \
  -s 5,virtio-net,tap0 \
  -s 29,fbuf,tcp=0.0.0.0:5900,w=1024,h=768 \
  -s 30,xhci,tablet \
  -s 31,lpc \
  -l com1,stdio \
  -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
  debianvm

Starting the Debian VM on boot with a shell script

#!/bin/sh
# Name: startdebianvm
# Purpose: Simple script to start my Debian 10 VM using bhyve on FreeBSD
# Author: Vivek Gite {https://www.cyberciti.biz} under GPL v2.x+
-------------------------------------------------------------------------
# Lazy failsafe (not needed but I will leave them here)
ifconfig tap0 create
ifconfig em0bridge addm tap0
if ! kldstat | grep -w vmm.ko 
then
 kldload -v vmm
fi
if ! kldstat | grep -w nmdm.ko
then
 kldload -v nmdm
fi
bhyve -c 1 -m 1G -w -H \
-s 0,hostbridge \
-s 4,virtio-blk,/dev/zvol/zroot/debianvm \
-s 5,virtio-net,tap0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1024,h=768 \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
debianvm

Create a crontab entry:

@reboot /path/to/startdebianvm

Installing a Linux jail

Create the ZFS datasets for the base jail and Linux jail.

sudo zfs create naspool/jails/debian
sudo zfs create naspool/jails/14.2-RELEASE

Download the base userland system for FreeBSD.

fetch https://download.freebsd.org/ftp/releases/amd64/amd64/14.2-RELEASE/base.txz

Extract the base userland into the base jail's directory.

sudo tar -xf base.txz -C /jails/14.2-RELEASE --unlink

Copy DNS and timezone files.

sudo cp -v /etc/resolv.conf /jails/14.2-RELEASE/etc/resolv.conf
sudo cp -v /etc/localtime /jails/14.2-RELEASE/etc/localtime

Update the base jail to the latest patch level.

sudo freebsd-update -b /jails/14.2-RELEASE/ fetch install

Create a ZFS snapshot from the base jail.

sudo zfs snapshot naspool/jails/14.2-RELEASE@base

Clone the base jail to create a thin jail for the Linux distribution.

sudo zfs clone naspool/jails/14.2-RELEASE@base naspool/jails/debian

Enable the Linux ABI.

sudo sysrc linux_enable="YES"
sudo service linux start

Run the jail command with a quick configuration.

sudo jail -cm \
  name=debian \
  host.hostname="debian" \
  path="/jails/debian" \
  interface="igc0" \
  ip4.addr="10.0.0.21" \
  exec.start="/bin/sh /etc/rc" \
  exec.stop="/bin/sh /etc/rc.shutdown" \
  mount.devfs \
  devfs_ruleset=11 \
  allow.mount \
  allow.mount.devfs \
  allow.mount.fdescfs \
  allow.mount.procfs \
  allow.mount.linprocfs \
  allow.mount.linsysfs \
  allow.mount.tmpfs \
  enforce_statfs=1

Access the jail.

sudo jexec -u root debian

Install the debootstrap program and prepare the Debian environment.

pkg install debootstrap
debootstrap bookworm /compat/debian

When the process finishes, stop the jail from the host system.

sudo service jail onestop debian

Add an entry in /etc/jail.conf for the Debian Linux jail.

debian {
  # STARTUP/LOGGING
  exec.start = "/bin/sh /etc/rc";
  exec.stop = "/bin/sh /etc/rc.shutdown";
  exec.consolelog = "/var/log/jail_console_${name}.log";

  # PERMISSIONS
  allow.raw_sockets;
  exec.clean;
  mount.devfs;
  devfs_ruleset = 11;

  # HOSTNAME/PATH
  host.hostname = "${name}";
  path = "/jails/${name}";

  # NETWORK
  ip4.addr = 10.0.0.21;
  interface = igc0;

  # MOUNT
  mount += "devfs       $path/compat/debian/dev       devfs     rw 0 0";
  mount += "tmpfs       $path/compat/debian/dev/shm   tmpfs     rw,size=1g,mode=1777 0 0";
  mount += "fdescfs     $path/compat/debian/dev/fd    fdescfs   rw,linrdlnk 0 0";
  mount += "linprocfs   $path/compat/debian/proc      linprocfs rw 0 0";
  mount += "linsysfs    $path/compat/debian/sys       linsysfs  rw 0 0";
  mount += "/tmp        $path/compat/debian/tmp       nullfs    rw 0 0";
  mount += "/home       $path/compat/debian/home      nullfs    rw 0 0";
}

Start the jail.

sudo service jail start debian

The Debian environment can be accessed using the following command:

sudo jexec debian chroot /compat/debian /bin/bash