2009-07-10

FreeBSD - How to upgrade the kernel or base system

The nature of FreeBSD is so rock-solid stable that some system administrator adopt the philosophy "if it ain't broken, don't fix it", others, beg to differ. Upgrading & patching of servers should be diligently carry out so that security vulnerabilities are minimize to the least (who can be sure of their servers are 100% secure ???), introduce to new application features (which geek doesn't like new stuff ???) & performance increase (meaner & leaner :) ). Patching can only be done to a certain level then it will need kernel and base system to compile and build the latest patches or upgrades.

Regular upgrade of the FreeBSD kernel and base system is a good way to follow the support schedule. It would mean that your server would have a longer life span in the production. Not because of some latest OS "GUI" release that demand higher RAM for the "processor hungry monster" lies beneath the OS "GUI" release. But rather, it fixes the bugs of the applications, gives you more new functions and make your hardware works harder & faster.

There are some hiccups to take note of. As with all OS upgrade, the will experience up time interruption (aka server down time) . Scenario such as :

:- The performance of the server have greatly draw by the upgrade process and impact the usual services the server runs. Thus the response time of the server fail to serve the users in time and affected the normal operation of the business.

:- The upgrades break the kernel or base system and the server fail to boot.

:- After the first boot up of the upgrade, the applications fail to compile properly and resulted the services fail to start.

:- The is your first FreeBSD upgrade and you can't predict what would happen and hiccups just like to happen at these times. :p

All of the above can be avoided as long as the below points are observe :
  • plan and schedule the upgrade with adequate time so that even hiccups happen, there are ample time to solve it.
  • plan a trial server upgrade or mock run of the upgrade to anticipate what hiccups might happen and test the solutions. Use the server backup and restore to another hardware, virtual machine or sandbox to test the server upgrade. Drop down every step and retry it if time permits to minimize the hiccups.
  • The is the most important one. Always backup/restore perform a full system backup with no less then 1 day of difference. The shorter time of the difference between the backup and the upgrade, the faster and painless the restore process is. This also mean shorter down time.

First part - make sure all ports on the system are updated

portsnap fetch update ### Update the ports repository current ###
portupgrade -va     ### Either upgrade all the ports at once OR upgrade the ports one by one (portupgrade -v <ports name>) e.g. portupgrade -vr apache22 ###
*** If your FreeBSD box is using custom compile kernel, be sure to revert back to the "GENERIC" kernel then follow by a reboot to make sure it boots fine.

Second part - update the kernel & base system of freebsd

freebsd-update fetch     ### Step 1. Fetch all the files need to upgrade the kernel and base system ###
freebsd-update upgrade -r 7.2-RELEASE     ### Step 2. Compile necessary files for kernel & base system upgrade, to a specific version. In this step, it might ask to decide how to merge the config files. e.g. httpd.conf, sshd_config, /etc/hosts ###
freebsd-update install     ### Step 3. Install the binaries (take note that this step will need repeat later) ###
### Step 4. Reboot the server. After it rebooted successfully, check the log files (dmesg or less /var/log/messages), and system to make sure that all hardware drivers are loaded. ###
freebsd-update install         ### Step 5. Again. Install the outstanding binaries which cannot proceed in step 3. This step is normal to repeat as some of the binaries are not able to install in step 4 due to file or process locking. ###

Third part - rebuild all ports on the system

Perform the below steps to upgrade ruby first then all of the ports collection in order to make sure the application are working fine with the new kernel and base system.
portupgrade -vf ruby
rm /var/db/pkg/pkgdb.db
portupgrade -vf ruby18-bdb
rm /var/db/pkg/pkgdb.db /usr/ports/INDEX-*.db
portupgrade -vaf

After the long time rebuilding of all ports, make sure that all services & program are able to start/run/execute. Because, is time to re-run freebsd-update utility (again) to make sure all ports works fine with the new base system :

freebsd-update install

And this time, the "freebsd-update install" will remove the old libraries.

At this point, all of your ports should have been recompile to use the new base libraries. But if any of the ports or packages failed to re-compile in the previous step, it'll failed to start or work at this point.

If you happen to upgrade from 7.x to 8.x, then you're in luck. First, install "misc/compat7x" . This will let those ports or packages using old libraries to resume working. Next, install "sysutils/libchk". Use "sysutils/libchk" & "pkg_which" to check if a package is linking to /usr/local/lib/compat. If it is, use "portupgrade -f" to recompile it so that it use the new libraries.

Optionally, after installing "misc/compat7x", recompile all ports ("portupgrade -vaf") again so that it uses the new base libraries.

Voilla!!!

ref :
- http://www.freebsd.org/doc/handbook/updating-upgrading-freebsdupdate.html
- FreeBSD forum post suggesting how to fix packages with missing libraries

No comments: