FreeBSD installation
Updated on: 2025-03-18.
Initially written in russian on: 2021-01-12.
This article explains FreeBSD installation without sysinstall, allowing fine
tuning and showing how simple all of that is actually. Pay attention that this
is my personal experience exclusively. This is collection of notes.
* Load LiveCD
* Partition your hard drive and install bootloader
* Without partitioning at all, naked ZFS, CSM/BIOS mode
Naked ZFS usage is simple and trivial. Just create ZFS pool on
the disk and copy zfsboot loader to the empty reserved space
inside ZFS pool:
zpool create zroot ada0
dd if=/boot/zfsboot of=/dev/ada0 count=1
dd if=/boot/zfsboot of=/dev/ada0 iseek=1 oseek=1024
Easy, beautiful, minimalistic and working, but I do not
recommend that, because at least you have to place your swap
inside zvol, that gives considerable overhead.
If you want to create swap anyway, then I recommend the
following options for the zvol:
volblocksize=4k # amd64's native page size
sync=always # always write data to the disk, without keeping in RAM
logbias=throughput # place data on the disk directly and immediately,
# without intermediate ZIL
primarycache=metadata # keep only metadata in the ARC cache
* GPT partitioning, CSM/BIOS mode
Create GPT scheme, partitions for bootloader, swap and root
filesystem. Everything with explicit labels. 4K alignment,
because all modern drives has 4K physical sectors and unaligned
access can drastically decrease your performance.
gpart create -s GPT diskid/DISK-SERIAL
gpart add -t freebsd-boot -a 4K -s 512K -l BOOT-SERIAL diskid/DISK-SERIAL
gpart add -t freebsd-swap -s 2G -l SWAP-SERIAL diskid/DISK-SERIAL
gpart add -t freebsd-zfs -l ROOT-SERIAL diskid/DISK-SERIAL
Install MBR and ZFS bootloaders.
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 diskid/DISK-SERIAL
* GPT partitioning, UEFI mode
FreeBSD includes completely prepared FAT32 partition image with
EFI bootloader installed in it. The only difference with the
previous section is bootloader installation:
gpart create -s GPT diskid/DISK-SERIAL
gpart add -t efi -a 4K -s 64M -l BOOT-SERIAL diskid/DISK-SERIAL
gpart add -t freebsd-swap -a 4K -s 2G -l SWAP-SERIAL diskid/DISK-SERIAL
gpart add -t freebsd-zfs -a 4K -l ROOT-SERIAL diskid/DISK-SERIAL
gpart bootcode -p /boot/boot1.efifat -i 1 diskid/DISK-SERIAL
If there is no boot1.efifat, then you can create it manually:
gpart add -t efi -s XXX diskid/DISK-SERIAL
newfs_msdos [-c 1] -F 32 /dev/diskid/DISK-SERIALp1
mount -t msdosfs /dev/diskid/DISK-SERIALp1 /mnt
mkdir -p /mnt/EFI/BOOT
cp /boot/loader.efi /mnt/EFI/BOOT/BOOTX64.efi
umount /mnt
If you want to use UFS2 instead of ZFS for some reason, then replace
freebsd-zfs with freebsd-ufs, and bootloader with gptboot.
* Create filesystems
sysctl vfs.zfs.min_auto_ashift=12
zpool create zroot gpt/ROOT-SERIAL
zfs set checksum=skein compression=zstd atime=off mountpoint=/mnt zroot
If you want to create a mirror, then specify that here:
zpool create zroot mirror gpt/ROOT-SERIAL1 gpt/ROOT-SERIAL2
I always recommend to use disks with explicit labels on them,
without unstable and fragile enumeration like adaX or something
relying on disk "geography". I do not know any problems with
diskid/XXX usage, where XXX is some serial number for example.
You can use glabel command to create those labels, which will record
it to the end of the disk and make it available under label/XXX
path. But this is BSD-specific solution. GPT partition will be
visible and known in any GPT-aware OS. Moreover you can also align
your partitions on 4K sectors with it. Also some people specify
slightly smaller last partitions (1GB less for example), to
workaround possible problems with the drives from different vendors,
that can easily have slightly varying actual number of sectors. GPT
helps with all those issues.
Solaris documentation recommends to give the whole disk for ZFS, but
that is related to that OS exclusively, because only that way ZFS
can disable write caching on the drive. FreeBSD does not have that
problem and you can give partitions to ZFS without any difference.
So I strongly suggest and advice to use GPT labeled and aligned
partitions:
gpart create -s GPT diskid/SERIAL1
gpart create -s GPT diskid/SERIAL2
gpart add -t freebsd-zfs -a 4K -l STORAGE-SERIAL1 diskid/SERIAL1
gpart add -t freebsd-zfs -a 4K -l STORAGE-SERIAL2 diskid/SERIAL2
sysctl vfs.zfs.min_auto_ashift=12
zpool create storage mirror gpt/STORAGE-SERIAL1 gpt/STORAGE-SERIAL2
I specified Zstandard compression algorithm for the zroot pool
immediately. There are few cases where compression can hurt or it
will be useless. There are no reasons not to use transparent fast
compression on your root filesystem, because most data on it is
compressible, giving considerable actual performance boost.
atime is disabled, because hardly anyone met cases when it is
useful. It can result in pretty high overhead. If any of software
wants honest atime behaviour, then you should create separate
dataset for it, with atime enabled.
I always use only cryptographically secure hash functions for
checksums. For all datasets I prefer to skein because of
performance. With cryptographic hashes you can also use
deduplication feature in the future.
It is crucial to use proper ashift. Its value is the power of two,
that specified disk's sector size. You can set it only during
initial pool creation. It is immutable after that. Problem with
ashift is the fact that modern hard drives still likes to lie about
their real physical sector size, replying with 512 bytes ones.
If you plan to use encrypted GELI drives/partitions, then do not
forget about GELI's sector size too! Larger its sector, less sectors
you have to process, less burden on CPU for keys generation (8 times
less with 4K sectors, comparing to 512 ones).
geli init -s 4K ...
If you want to use UFS2 (for non-critical low-power system), then I
would make it with "newfs -Ut [-E] /dev/gpt/ROOT-SERIAL"
Soft-updates considerably increases performance. TRIM is a must for
SSDs (ZFS does TRIM automatically even when used on top of GELI).
* OS installation
for what in base kernel [src ports lib32 kernel-dbg tests] ; do
tar xfC /usr/freebsd-dist/$what.txz /mnt
done
Everything below is made inside mounted /mnt: chroot /mnt
* Bootloader configuration
# cat > /boot/loader.conf <<EOF
zfs_load=YES
vm.pmap.pti=0
hw.ibrs_disable=1
machdep.hyperthreading_allowed=0
kern.random.initial_seeding.bypass_before_seeding=0
net.isr.maxthreads=-1
net.isr.bindthreads=1
# For 10GbE
net.isr.defaultqlimit=4096
net.isr.maxqlimit=65536
net.link.ifqmaxlen=4096
# Just to remember that ipfw by default drops everything and you can
# loose control on the server after you enable it
#net.inet.ip.fw.default_to_accept=0
EOF
* Pseudo filesystems and swap setup
# cat > /etc/fstab <<EOF
tmpfs /tmp tmpfs rw,nosuid,mode=1777 0 0
fdescfs /dev/fd fdescfs rw 0 0
proc /proc procfs rw 0 0
/dev/gpt/SWAP-SERIAL.eli none swap sw 0 0
EOF
It is crucial to specify .eli at the end of the path to swap
volume -- that way it will be encrypted with a temporary onetime key.
* sysctl tuning
# cat >> /etc/sysctl.conf <<EOF
kern.msgbuf_show_timestamp=1 # timestamps in dmesg output
kern.cam.ada.write_cache=0 # with ZFS you do not want any write buffering
security.bsd.stack_guard_page=1
security.bsd.see_other_uids=0
security.bsd.see_other_gids=0
security.bsd.see_jail_proc=0
security.bsd.unprivileged_idprio=1 # personally I often use idprio
kern.randompid=1
net.inet.ip.random_id=1
security.bsd.hardlink_check_gid=1
security.bsd.hardlink_check_uid=1
#security.jail.allow_raw_sockets=1 # if you want to use ping inside jails
net.inet.ip.redirect=0
net.inet6.ip6.redirect=0
net.inet6.icmp6.rediraccept=0
net.inet.tcp.drop_synfin=1
net.inet.tcp.ecn.enable=1
net.inet6.ip6.use_tempaddr=0
net.inet.tcp.tso=0
vfs.nfsd.server_min_nfsvers=4
#net.link.tap.user_open=1
#net.link.tap.up_on_open=1
vfs.zfs.min_auto_ashift=12
vfs.zfs.dmu_offset_next_sync=0 # https://github.com/openzfs/zfs/issues/15526
#vfs.usermount=1 # Allow user to do mounts
EOF
* Basic OS configuration
# cat > /etc/rc.conf <<EOF
hostname=stargrave.org
zfs_enable=YES
clear_tmp_enable=YES # actually does not make any sense if we have got tmpfs
chronyd_enable=YES # ntpd_enable=YES
sshd_rsa_enable=no
sshd_dsa_enable=no
sshd_ecdsa_enable=no
sshd_ed25519_enable=yes
sshd_enable=YES
# Prioritize IPv6 addresses got from DNS
ip6addrctl_enable=YES
ip6addrctl_policy=ipv6_prefer
ipv6_ipv4mapping=YES # it is convenient to have [::] IPv6-listening
# daemons to be automatically available also on IPv4
nfsv4_server_only=YES
nfsv4_server_enable=YES
firewall_enable=YES
firewall_script=/etc/ipfw.rules
# Be sure IPv6 link-local addresses are up and all hardware offloading
# is off, that may hurt in many cases on routers
ifconfig_XXX="-tso -lro up"
ifconfig_XXX_ipv6="inet6 -ifdisabled"
EOF
* Timezone setup
tzsetup /usr/share/zoneinfo/Europe/Moscow
* ipfw firewall setup
# cat > /etc/ipfw.rules <<EOF
#!/bin/sh -x
ipfw -f flush
ipfw -f table all destroy
ipfw zero
ipfw disable one_pass
add="ipfw add"
\$add deny all from any to any frag
\$add allow { icmp or icmp6 } from any to any keep-state
\$add allow esp from any to any keep-state
\$add allow udp from any to any isakmp keep-state
\$add allow all from any to any via lo0
\$add deny all from any to any not verrevpath in via XXX
\$add check-state
\$add deny tcp from any to any established via XXX
\$add allow tcp from any to me ssh keep-state
\$add allow all from me to any out keep-state
#\$add deny log all from any to any
EOF
# chmod 600 /etc/ipfw.rules
* Disable excess periodic cron jobs
sed -i.tmp "/periodic/ s/^/#/" /etc/crontab
* NTP client setup
% cat > /usr/local/etc/chrony.conf <<EOF
server time.stargrave.org
driftfile /var/db/chrony/drift
mailonchange stargrave@stargrave.org 0.5
makestep 0.1 3
EOF
or
echo server time.stargrave.org iburst >> /etc/ntp.conf
* NFSv4 export ability setup
echo V4: / > /etc/exports
* Reboot
zfs umount zroot
zfs set mountpoint=none zroot
zpool export zroot
reboot
* Install and configure Postfix:
You can install that MTA and configure the host to use the relay:
# cat >> /usr/local/etc/postfix/main.cfg <<EOF
inet_interfaces = loopback-only
mynetworks_style = host
relayhost = [gw.stargrave.org]
EOF
* Turn off verbose motd:
# truncate -s 0 /etc/motd.template
* Faster and better log compression:
# perl -F'\t' -i -lanpe '($F[$#F] =~ s/J/Y/g) if $#F > 0; $_ = join "\t", @F'
/etc/newsyslog.conf /etc/newsyslog.conf.d/*
* BBR TCP congestion (actually now I do not use it):
# cd /usr/src
# cat > sys/amd64/conf/GENERIC-EXTRA <<EOF
include GENERIC
ident GENERIC-EXTRA
makeoptions WITH_EXTRA_TCP_STACKS=1
options TCPHPTS
EOF
# make -j8 buildkernel KERNCONF=GENERIC-EXTRA
# make installkernel KERNCONF=GENERIC-EXTRA
# rm -r /boot/kernel.old /usr/obj
# echo tcp_bbr_load=YES >> /boot/loader.conf
# echo net.inet.tcp.functions_default=bbr >> /etc/sysctl.conf
* Additionally enable TCP Fast Open:
# echo options TCP_RFC7413 >> /usr/src/sys/amd64/conf/GENERIC-EXTRA
[...recompile and install kernel...]
# echo net.inet.tcp.fastopen.server_enable=1 >> /etc/sysctl.conf
* CUBIC TCP congestion:
# echo kld_list=cc_cubic >> /etc/rc.conf
# echo net.inet.tcp.cc.algorithm=cubic >> /etc/sysctl.conf
* Unicode-aware locale and minimal root configuration:
# echo me:charset=UTF-8:lang=en_US.UTF-8: >~/.login_conf
# cat >~/.profile <<EOF
export HISTFILE=
export LESSHISTFILE=-
export PAGER=less
export EDITOR=vi
export LC_ALL=en_US.UTF-8
export LANG=$LC_ALL
export IFCONFIG_FORMAT=inet:cidr,inet6:cidr
umask 077
EOF
# cat >~/.vimrc <<EOF
set viminfo=
set mouse=
set paste
EOF
# cat >~/.tmux.conf <<EOF
set-option -g mode-keys vi
set-option -g base-index 1
EOF