INTRODUCTION :
___________________________________________________________
___________________________________________________________
There is many ressources online explaining how to build/update FreeBSD from sources, the main one being the
FreeBSD Handbook itself. However, if you never did it before,
I find the various explanations being daunting or confusing. Once you read all of the related pages of the Handbook and put the pieces together, you start to get the picture.
Once I upgraded my home router (which will be a story for a future article), I found that the procedure wasn't that complicated, even easy, and that I would have better
understood beforehand the whole process if it would have been presented differently. Do not misunderstand me, the Handbook is wonderful and is one extraordinary place to go
to learn about FreeBSD, and I thanks all of the people who are giving of their time to maintain it. I have read also the excellent book "Absolute FreeBSD (2nd edition)" from Michael W.LUCAS
which is a must read ! (M LUCAS, if you ever read this, you got me at the end, I fall for it, I have to spend two days out !).
Doing a binary upgrade with "freebsd-update" is far easier and faster, but is only possible with the RELEASE branch. If you either need to run STABLE or CURRENT branch, that is not possible. That was my case, unfortunately my home router's network card was not supported by FreeBSD 10-RELEASE, and I had to install the STABLE branch.
In the end, I simply wish to sum up the upgrade from sources process with my words, and provide few simple scripts to help in the process. Clearly, if you are already experienced in upgrading FreeBSD from sources, this article is not for you :-)
Doing a binary upgrade with "freebsd-update" is far easier and faster, but is only possible with the RELEASE branch. If you either need to run STABLE or CURRENT branch, that is not possible. That was my case, unfortunately my home router's network card was not supported by FreeBSD 10-RELEASE, and I had to install the STABLE branch.
In the end, I simply wish to sum up the upgrade from sources process with my words, and provide few simple scripts to help in the process. Clearly, if you are already experienced in upgrading FreeBSD from sources, this article is not for you :-)
SUMMARY :
___________________________________________________________
___________________________________________________________
1. OVERVIEW
___________________________________________________________
___________________________________________________________
To sume up quickly the whole upgrade process, it is :
1 - Get the sources of your branch -> svn checkout svn://xxx.freebsd.org/base/stable/10/ /usr/src
2 - Build the sources -> make buildworld, make buildkernel
3 - Install the result in single user mode (and reboot) : make installkernel, mergemaster, make installworld, mergemaster
4 - Upgrade your packages : portmaster -a
That's it. However do not blindly use only these commands, that is a only a brief summary of the whole process to make it easier to understand.
You have to get the sources of your version and your branch, for instance FreeBSD 10-STABLE. Then build the base and the kernel, and reboot in single user mode. There, install the base and the kernel, merge differences, and reboot. Finally, upgrade your packages, and delete the old librairies left. While doing a binary update can take two minutes (including reboot), upgrading from sources on my home router takes a solid 3h30mn for the whole thing.
1 - Get the sources of your branch -> svn checkout svn://xxx.freebsd.org/base/stable/10/ /usr/src
2 - Build the sources -> make buildworld, make buildkernel
3 - Install the result in single user mode (and reboot) : make installkernel, mergemaster, make installworld, mergemaster
4 - Upgrade your packages : portmaster -a
That's it. However do not blindly use only these commands, that is a only a brief summary of the whole process to make it easier to understand.
You have to get the sources of your version and your branch, for instance FreeBSD 10-STABLE. Then build the base and the kernel, and reboot in single user mode. There, install the base and the kernel, merge differences, and reboot. Finally, upgrade your packages, and delete the old librairies left. While doing a binary update can take two minutes (including reboot), upgrading from sources on my home router takes a solid 3h30mn for the whole thing.
2. GET THE SOURCES
___________________________________________________________
___________________________________________________________
The first step is to choose a mirror to pull the sources from. You can find a list here.
Then you need "subversion" to be installed. If you do not yet have the Ports Tree pulled, and portmaster installed, do :
Now that subversion is installed, we can finally download the sources (replace the url with your mirror):
The next times simply run :
Then you need "subversion" to be installed. If you do not yet have the Ports Tree pulled, and portmaster installed, do :
$ sudo portsnap fetch extract
$ cd /usr/ports/ports-mgmt/portmaster
$ sudo make install clean
$ sudo portmaster -d --no-confirm devel/subversion
$ cd /usr/ports/ports-mgmt/portmaster
$ sudo make install clean
$ sudo portmaster -d --no-confirm devel/subversion
Now that subversion is installed, we can finally download the sources (replace the url with your mirror):
$ sudo svn checkout svn://svn0.eu.freebsd.org/base/stable/10/ /usr/src
The next times simply run :
$ sudo svn update /usr/src
3. BUILD
___________________________________________________________
___________________________________________________________
The build process is the longest part, it can takes many hours depending on your hardware. There is still no risks at this point, the building process does not modify your system.
As per the handbook, we can first empty /usr/obj to speed things up, and then build the system and the kernel :
Before proceeding to the next step, it is advised to have backups of the system and/or your data.
$ sudo chflags -R noschg /usr/obj/*
$ sudo rm -rf /usr/obj/*
$ sudo make buildworld
$ sudo make buildkernel
$ sudo rm -rf /usr/obj/*
$ sudo make buildworld
$ sudo make buildkernel
Before proceeding to the next step, it is advised to have backups of the system and/or your data.
4. INSTALL
___________________________________________________________
___________________________________________________________
Reboot in single user mode :
At the boot screen menu, select the option 2. If at the install you followed the guided partitioning and that you have as a result a single partition (besides swap), you won't have any error or warning while arriving in single user mode. If however, you did create separate partitions (which is recommended), such as /, swap, /var, /tmp, and /usr, on FreeBSD 10 you have the following warning :
Indeed, /etc/termcap file is a link to /usr/share/misc/termcap, and as /usr is not mounted, termcap file cannot be read.
Wether I have a single partition or many, I always run the following commands when arriving in single user mode :
Now we have a smart /bin/sh shell, and we can continue to install our built kernel and system :
If all gone fine, you can check with "uname -a" that you system is at a new revision and date.
$ sudo shutdown -r now
At the boot screen menu, select the option 2. If at the install you followed the guided partitioning and that you have as a result a single partition (besides swap), you won't have any error or warning while arriving in single user mode. If however, you did create separate partitions (which is recommended), such as /, swap, /var, /tmp, and /usr, on FreeBSD 10 you have the following warning :
Cannot read termcap database.
Using dumb terminal settings.
Using dumb terminal settings.
Indeed, /etc/termcap file is a link to /usr/share/misc/termcap, and as /usr is not mounted, termcap file cannot be read.
Wether I have a single partition or many, I always run the following commands when arriving in single user mode :
# fsck -p checking partitions
# mount -uw / making / read write
# mount -a -t ufs mounting ufs partitions from /etc/fstab
# mount display mounted partitions
# kbdcontrol -l fr.iso.acc set keyboard FR(ench) language map
# mount -uw / making / read write
# mount -a -t ufs mounting ufs partitions from /etc/fstab
# mount display mounted partitions
# kbdcontrol -l fr.iso.acc set keyboard FR(ench) language map
Now we have a smart /bin/sh shell, and we can continue to install our built kernel and system :
# swapon -a turn on swap
# adjkerntz -i synchronise kernel time
# cd /usr/src
# make installkernel install the kernel
# mergemaster -p merge new configuration files with existing ones
# make installworld install the system
# mergemaster -iF update remaining configuration files
# make delete-old delete obsolete files
# reboot
# adjkerntz -i synchronise kernel time
# cd /usr/src
# make installkernel install the kernel
# mergemaster -p merge new configuration files with existing ones
# make installworld install the system
# mergemaster -iF update remaining configuration files
# make delete-old delete obsolete files
# reboot
If all gone fine, you can check with "uname -a" that you system is at a new revision and date.
5. PACKAGES UPGRADE
___________________________________________________________
___________________________________________________________
Once the system has booted, the packages may rely on librairies that have been upgraded in the process. By upgrading and rebuilding them, they should keep working as usual :
Although not necessary, reboot once again to ensure everything boots fine.
$ sudo portsnap fetch update update ports tree
$ sudo portmaster -a update all packages
$ cd /usr/src
$ sudo make delete-old-libs deleting old librairies
$ cd /usr/obj
$ sudo chflags -R noschg *
$ sudo rm -rf /usr/obj/* cleaning /usr/obj
$ sudo portmaster -a update all packages
$ cd /usr/src
$ sudo make delete-old-libs deleting old librairies
$ cd /usr/obj
$ sudo chflags -R noschg *
$ sudo rm -rf /usr/obj/* cleaning /usr/obj
Although not necessary, reboot once again to ensure everything boots fine.
6. SCRIPTS
___________________________________________________________
___________________________________________________________
The whole process is made far easier with only 3 scripts : one "build script", one "install script", and one "package upgrade script".
Below is an example of three quick made scripts, but you can come up with your own to fit your system and taste.
Build script "/root/build.sh" : can be run in multi-users mode
Install script "/root/upgrade.sh" : should be run in single-user mode
And finally the post upgrade script "/root/post_upgrade.sh" :
These scripts have been tested on FreeBSD 10-STABLE amd64 only. Always make a backup of your data before attempting an upgrade, moreover if you are using scripts from someone else.
I highly encourage you to do things manually to learn the process, and eventually make your own script afterwards. These scripts are merely an example.
Below is an example of three quick made scripts, but you can come up with your own to fit your system and taste.
Build script "/root/build.sh" : can be run in multi-users mode
#!/bin/sh
next_screen() {
echo ""
echo "[Press ENTER to continue]"
read input
clear
}
clear
echo "This script will :"
echo "1 - Update Ports sources tree with portsnap"
echo "2 - Update System sources from svn"
echo "3 - Run 'make buildworld' and 'make buildkernel'".
next_screen
echo "1/3 - Update Ports sources tree with portsnap"
echo "___________________________________________________________________________"
# If this script has been called with : ./upgrade_from_source.sh --first_launch
if [ "$1" = "--first_launch" ]; then
echo "Deleting files from /usr/src"
rm -Rf /usr/src
echo ""
echo "Updating Ports Tree..."
if [ ! -d "/usr/ports" ]; then
portsnap fetch extract update
else
portsnap fetch update
fi
echo ""
echo "2/3 - Fetching sources in /usr/src from svn :"
echo "___________________________________________________________________________"
svn checkout svn://svn0.eu.freebsd.org/base/stable/10/ /usr/src
else
# If /usr/src does not exist, it cannot be updated
if [ ! -d "/usr/src" ]; then
echo "/usr/src not found."
echo "Run './upgrade_from_source.sh --first_launch' if it's the first time"
exit
fi
# If /usr/ports does not exist, it cannot be updated
if [ ! -d "/usr/ports" ]; then
echo "/usr/ports not found."
echo "Run './upgrade_from_source.sh --first_launch' if it's the first time"
exit
fi
echo ""
echo "SVN update of /usr/src..."
svn update /usr/src
echo ""
echo "Portsnap update of /usr/ports..."
portsnap fetch update
fi
# Before building anything :
echo ""
echo "Cleaning /usr/obj"
cd /usr/obj
chflags -R noschg /usr/obj/*
rm -rf /usr/obj/*
echo ""
echo "3/3 - Buildworld and Buildkernel :"
echo "___________________________________________________________________________"
cd /usr/src
echo ""
echo "make buildworld"
make buildworld
echo ""
echo "make buildkernel"
make buildkernel
echo ""
echo "You now have to 'shutdown -r now' and reboot into Single User mode"
echo "Once there, use /root/upgrade_system.sh"
next_screen() {
echo ""
echo "[Press ENTER to continue]"
read input
clear
}
clear
echo "This script will :"
echo "1 - Update Ports sources tree with portsnap"
echo "2 - Update System sources from svn"
echo "3 - Run 'make buildworld' and 'make buildkernel'".
next_screen
echo "1/3 - Update Ports sources tree with portsnap"
echo "___________________________________________________________________________"
# If this script has been called with : ./upgrade_from_source.sh --first_launch
if [ "$1" = "--first_launch" ]; then
echo "Deleting files from /usr/src"
rm -Rf /usr/src
echo ""
echo "Updating Ports Tree..."
if [ ! -d "/usr/ports" ]; then
portsnap fetch extract update
else
portsnap fetch update
fi
echo ""
echo "2/3 - Fetching sources in /usr/src from svn :"
echo "___________________________________________________________________________"
svn checkout svn://svn0.eu.freebsd.org/base/stable/10/ /usr/src
else
# If /usr/src does not exist, it cannot be updated
if [ ! -d "/usr/src" ]; then
echo "/usr/src not found."
echo "Run './upgrade_from_source.sh --first_launch' if it's the first time"
exit
fi
# If /usr/ports does not exist, it cannot be updated
if [ ! -d "/usr/ports" ]; then
echo "/usr/ports not found."
echo "Run './upgrade_from_source.sh --first_launch' if it's the first time"
exit
fi
echo ""
echo "SVN update of /usr/src..."
svn update /usr/src
echo ""
echo "Portsnap update of /usr/ports..."
portsnap fetch update
fi
# Before building anything :
echo ""
echo "Cleaning /usr/obj"
cd /usr/obj
chflags -R noschg /usr/obj/*
rm -rf /usr/obj/*
echo ""
echo "3/3 - Buildworld and Buildkernel :"
echo "___________________________________________________________________________"
cd /usr/src
echo ""
echo "make buildworld"
make buildworld
echo ""
echo "make buildkernel"
make buildkernel
echo ""
echo "You now have to 'shutdown -r now' and reboot into Single User mode"
echo "Once there, use /root/upgrade_system.sh"
Install script "/root/upgrade.sh" : should be run in single-user mode
#!/bin/sh
next_screen() {
echo ""
echo "[Press ENTER to continue]"
read input
clear
}
echo "---------------------------------------------"
echo "Initialising environnement :"
echo "-> checking partitions"
fsck -p
echo ""
echo "-> making / read write"
mount -uw /
echo "-> mounting /etc/fstab"
mount -a -t ufs
mount
echo "-> set keyboard language map"
kbdcontrol -l fr.iso.acc
echo "-> activate swap and synchronise kernel time"
swapon -a
adjkerntz -i
echo "---------------------------------------------"
next_screen
echo "This script will :"
echo "1 - Lower securelevel at next boot to -1"
echo "2 - Install the new kernel".
echo "3 - Merge files"
echo "4 - Install the new world"
echo "5 - Merge remaining files"
next_screen
echo ""
echo "1/5 - Lower securelevel at next boot to -1"
echo "________________________________________________________________"
echo "Remove chflags from /etc/sysctl.conf"
chflags noschg /etc/sysctl.conf
ls -lo /etc/sysctl.conf
echo ""
echo "Disable kern.securelevel from /etc/sysctl.conf"
SECURELEVEL=`grep "#kern.securelevel" /etc/sysctl.conf`
# If kern.securelevel not commented, comment it
if [ -z "$SECURELEVEL" ]; then
sed -i -e "s/kern.securelevel/#kern.securelevel/g" /etc/sysctl.conf
fi
grep kern.securelevel /etc/sysctl.conf
next_screen
echo ""
echo "2/5 - Install the new kernel"
echo "________________________________________________________________"
cd /usr/src
make installkernel
next_screen
echo ""
echo "3/5 Mergemaster before installing world."
mergemaster -p
next_screen
echo ""
clear
echo "4/5 - Installworld"
make installworld
next_screen
echo ""
echo "5/5 - Update any remaining configuration files with mergemaster"
mergemaster -iF
make delete-old
echo ""
echo "You can now reboot"
echo ""
echo "After the reboot, run /root/post_upgrade.sh"
next_screen() {
echo ""
echo "[Press ENTER to continue]"
read input
clear
}
echo "---------------------------------------------"
echo "Initialising environnement :"
echo "-> checking partitions"
fsck -p
echo ""
echo "-> making / read write"
mount -uw /
echo "-> mounting /etc/fstab"
mount -a -t ufs
mount
echo "-> set keyboard language map"
kbdcontrol -l fr.iso.acc
echo "-> activate swap and synchronise kernel time"
swapon -a
adjkerntz -i
echo "---------------------------------------------"
next_screen
echo "This script will :"
echo "1 - Lower securelevel at next boot to -1"
echo "2 - Install the new kernel".
echo "3 - Merge files"
echo "4 - Install the new world"
echo "5 - Merge remaining files"
next_screen
echo ""
echo "1/5 - Lower securelevel at next boot to -1"
echo "________________________________________________________________"
echo "Remove chflags from /etc/sysctl.conf"
chflags noschg /etc/sysctl.conf
ls -lo /etc/sysctl.conf
echo ""
echo "Disable kern.securelevel from /etc/sysctl.conf"
SECURELEVEL=`grep "#kern.securelevel" /etc/sysctl.conf`
# If kern.securelevel not commented, comment it
if [ -z "$SECURELEVEL" ]; then
sed -i -e "s/kern.securelevel/#kern.securelevel/g" /etc/sysctl.conf
fi
grep kern.securelevel /etc/sysctl.conf
next_screen
echo ""
echo "2/5 - Install the new kernel"
echo "________________________________________________________________"
cd /usr/src
make installkernel
next_screen
echo ""
echo "3/5 Mergemaster before installing world."
mergemaster -p
next_screen
echo ""
clear
echo "4/5 - Installworld"
make installworld
next_screen
echo ""
echo "5/5 - Update any remaining configuration files with mergemaster"
mergemaster -iF
make delete-old
echo ""
echo "You can now reboot"
echo ""
echo "After the reboot, run /root/post_upgrade.sh"
And finally the post upgrade script "/root/post_upgrade.sh" :
#!/bin/sh
echo "Upgrading all Ports"
portsnap fetch update
portmaster -a
echo ""
echo "Deleting old librairies"
cd /usr/src
make delete-old-libs
echo "Cleaning /usr/obj"
cd /usr/obj
chflags -R noschg *
rm -rf *
echo "Upgrading all Ports"
portsnap fetch update
portmaster -a
echo ""
echo "Deleting old librairies"
cd /usr/src
make delete-old-libs
echo "Cleaning /usr/obj"
cd /usr/obj
chflags -R noschg *
rm -rf *
These scripts have been tested on FreeBSD 10-STABLE amd64 only. Always make a backup of your data before attempting an upgrade, moreover if you are using scripts from someone else.
I highly encourage you to do things manually to learn the process, and eventually make your own script afterwards. These scripts are merely an example.
CONCLUSION :
___________________________________________________________
___________________________________________________________
This article is just an example of upgrading FreeBSD 10-STABLE amd64 from sources on a UFS filesystem. However there is different cases, for instance if you have a ZFS filesystem, or
following CURRENT branch, or upgrading a 8.x or 9.x version. I am not an upgrading from sources veteran, but rather usually a "binary user" because of the convenience. If there is not enough information
on this article, I advised you to read the FreeBSD Handbook and to ask questions at the
FreeBSD forum.
I hope this article can help beginners having a better overview of the upgrading from sources process. If there is any mistakes, thanks in advance to point them to me.
Once again, make backups, test on a virtual machine and/or a test server first.
I hope this article can help beginners having a better overview of the upgrading from sources process. If there is any mistakes, thanks in advance to point them to me.
Once again, make backups, test on a virtual machine and/or a test server first.
LINKS
___________________________________________________________
___________________________________________________________
Follow me @gkweb76
