FreeBSD - How to format & partition additional hard disk

While housekeeping the external hard disk laying around, I managed to squeeze out one spare hard disk out of this cleaning practice. The size is big enough to act as a backup media for my FreeBSD server & desktop workstations. The external hard disk partition layout preferred would be 3 partitions. The 1st (backup of OS & data) & 2nd partition (data backup) will be FreeBSD UFS & the 3rd will be FAT32 (for media transfer).

Since my workstation doesn't support FreeBSD UFS file system, I'll partition & format it using the FreeBSD server. All tools involve are command line utilities and can be done through remote secure shell. This post can also be serve as a guide for adding new/used additional hard disk to FreeBSD server.

Here goes :

FreeBSD - How to setup & configure jail with ezjail

OS virtualization is a technology that enable multiple copies of guest OS to run on top of a host OS. Why do we wanna do this?
- partition resources to share across multiple guest OS (current hardware to run multiple light guest OS. e.g. FreeBSD & Linux)
- enhance security (guest OS are isolated and thus hacking into a guest OS leaves other OS safe)
- saves power & money (hosting rack space & power are minimize)
- creating environments (application or OS testing, backup, legacy apps, admin delegation)
- improves server management (guest OS within virtualization are easier to move, copy, backup & remove)

FreeBSD Jail first appear in FreeBSD 4.x, introduced by Poul-Henning Kamp. Jail is a powerful OS virtualization that achieves security, cost, management & flexibility. Because of all these advantages, jail can some time be quite complex to setup & configure. Ezjail offers the simplicity to configure & maintain jails, without the complexity of getting our hands dirty. :p

Ezjail is a framework that controls jail. Ezjail takes care of the complicated part and makes setting up & maintenance of jail fun. Ezjail is develop by Dirk Engling, thanks Dirk. :)

FreeBSD jail does have its limitation. Here are the ones I've bump :
- resource control, e.g. ram & processor cannot be partition, they are all equally being shared. But hard disk space can be separate and control by implementing hard disk partition (or slice).
- all guest OS have to use the same kernel version as the host OS

Okay, enough of talking. Here's a simple step-by-step guide on how to setup Ezjail. Firstly, install Ezjail from sysutils/ezjail using Portmanager then proceed the below steps:

N900 - A phone running open source OS & programs

N900 is the latest offering from Nokia that answers the open source crave for software freedom, it runs Maemo, an OS based on linux, specifically Debian. Maemo also uses "apt" as package manager. Its GUI is fantastic, with accurate touch screen experience. Command line comes with it by default, to access the very core of the OS. And not like some other OS that hides hard to NOT let user touch it (even that, users managed to "jailbreak" it so to access the command line :p).

The N900 hardware runs a fast ARM cortex A8 processor (overclockable), 32GB internal storage with expandable micro MMC external storage card, touch screen with 800 × 480 pixel resolution, QWERTY slide hardware keyboard, 3.5 mm audio jack, A-GPS, WiFi, bluetooth, 3G internet connectivity and the long list goes on.

A big note here, the Maemo community is active & generous to its user that wants to have software freedom. Freedom to choose software from a community that offers alternatives, community that patches and write software just so that we all can use freely and not bow our heads to companies that charges software, even minor function. The Maemo community is active in providing patches both to the OS & 3rd party programs.

We all know that Debian is a hacker friendly linux distribution that seems more and more linux distribution derived from. e.g. Ubuntu family, Knoppix, Damn Small Linux, here & here. Maemo is based on Debian.

The price is unbeatable consider that it comes with such fast & furious hardware, OS and most importantly, the active & friendly open source community.

This post will list out my other blog post that's written based on N900. Here are some of it :

1. N900 - How to ssh into N900 using OpenSSH
2. Perl - A happy note, on N900 
3. N900 - How to overclock the processor speed of N900
4. N900 - How to clear all the phone data & settings
5. Stay tuned, this list will be update periodically.

Adios !!!

Perl - A happy note, on N900

The Nokia N900 runs perl :

~ $ perl -v

This is perl, v5.8.3 built for arm-linux-gnueabi-thread-multi

Copyright 1987-2003, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using `man perl' or `perldoc perl'. If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

Cheers !!!

N900 - How to ssh into N900 using OpenSSH

Since the Nokia N900 runs on Maemo, Debian based Linux, it also offers open source programs. The secure shell is one of open source gems. Maemo offers the OpenSSH variety of secure shell.

OpenSSH on Maemo not only offers encrypted secure remote shell, it also comes with functions that runs like the desktop version. Such as ssh socks 5 tunneling, that can be used to encrypt traffic transmit of the programs that supports socks 5 proxy.

Here's how to do it :

Perl - How to change @INC to include non-standard path

Have been playing around with local::lib and bootstrap a few directories. After switching a few local::lib directories, I tend to forget which cpan module is installed in which local::lib directory and this make perl don't know where to find the cpan module. This is how to set the environment variable @INC so that perl can quickly locate the cpan module.

Perl - How to list installed modules

A note for myself. This is how to list out what perl modules are installed.
Way 1 :

perl -MFile::Find=find -MFile::Spec::Functions -Tlwe 'find { wanted => sub { print canonpath $_ if /\.pm\z/ }, no_chdir => 1 }, @INC'

Way 2 :

perldoc perllocal

*** these are all taken from http://perldoc.perl.org/perlmodlib.html

Das Vidanya !!!

Perl - How to install perl module without root or super user

By default, perl modules installed through cpanX (cpanm, cpanp & cpan) utilities are install into system wide perl directory. Although this will make the module available through out all users on this box, it also means if there's a problem with particular module, it affects all users. Some other problem hit me as well :

Problem 1 - bloated with perl modules :
Some modules installed because it was needed to fulfill some dependencies (not necessary useful though), testing purpose or other reasons. This resulted the system wide perl directory is bloated with modules that aren't used as often as it should & should be removed. Removing modules in perl aren't easy as removing packages through a package manager. Because a typical package manager will check for dependencies to make sure that removing 1 package won't break another package.

Removing perl modules typically means manually remove the .pm files (or directory) in perl system wide directory. And this means there is a risk that removing a module might break some program or modules that depends on it.

Problem 2 - no root privileges :
In a hosting environment, normally one doesn't have root or sudo account, which installing CPAN modules needs it to add or change files in perl system wide directory. How nice if a normal users can have their own perl module directory.

Problem 3 - development environment :
Developers likes to setup individual perl projects to test & work on different environment. Naturally sets of modules that can be install into few different directories to serve as environment, switching between these directories are ideal for this situation. In this case, messing up an environment doesn't means messing up the entire perl modules.

Given the above situation, local::lib fits into the picture by solving the problems. Local::lib creates a directory resides in ~/ and cpan modules are install into here afterwards. Local::lib does that by changing the environment parameters @INC & system's $PATH variable.

The below instruction uses the "bootstrap" method of local::lib installation. This is useful on hosting environment where no access for root or sudo execution to install local::lib perl module.

Perl - How to install CPAN module, the easier way

Following IronMan Challenge posts not only for the sake of seeing What Color of Hair & Shirt ends up with Matt but the posts have been much enlightening. Recently in IronMan Challenge, there's been much talk about one of Miyagawa's project, App::cpanminus. Basically, cpanminus is a tool to install perl module from CPAN, without the verbosity of cpan|cpanplus, which also means, more newbie friendly. :)

The beauty of cpanminus is, it does not ask dozens of question before it installs. Instead, cpanminus will use the most suitable defaults & start downloading dependencies and install it. By default, cpanminus will not tell every action and result on the screen, which "scare" less.

In short, cpanminus improves or makes perfect the experience of installing modules from CPAN. Since the first time trying cpanminus, love it and loving it more seems inescapable :p

*** Some assumptions, perl 5.8 at least exist.

Enough talking, here's how to install cpanminus :

FreeBSD - Make ports not to include X11 or GUI related library

FreeBSD is an excellent operating system, be it on a Desktop (PC-BSD) or Server. Its stability and flexibility was never an issue, nor even software availability (21135 ports currently available to deploy).

Most of the time, ports will install the necessary dependencies to fulfill its requirement. But certain ports installs the X11 (X window system) libraries. Which we don't need it if the installation is meant to run on a server environment.

In this case, we will need to tell ports not to include the X11 libraries. This will certainly save some hard disk space, compilation time and avoid unnecessary security vulnerabilities.

Here is how to specify NOT to include X11 related libraries during ports installation :

FreeBSD - How to manage ports in FreeBSD using portmanager

There are various way of managing ports, using utilities. The usual "cd /usr/ports/<ports directory>; make; make install; make clean", portmaster, portupgrade and others. Today, this post will brief on how to use portmanager to install, upgrade and delete ports.

Using portmanager to manage ports have a few advantage :

1. portmanager scans for what dependencies are needed and then determine what needed to be upgrade or install in order to fulfill the dependencies. At this stage, the dependencies are identified install/upgrade which is good as other utilities tends to fail in the mid of the installation just because some ports are too old or cannot install.
2. portmanager supports output its messages to a log file. This is especially useful for trouble shooting what port / dependencies are giving problem.
3. portmanager is also able to backup the current port/dependency before installing the new port. This useful when the new port cannot be install or interrupted during the installation, resulted a state where new port fail to install and new port have been deleted.
4. portmanager collects its data by scanning the ports tree rather then building an index for the installed ports and depend on it. This is very less likely to cause problem as indexes are tend to be corrupted when abruptly (Ctrl-C) stopping installation in the halfway.
5. portmanager gives a summary when installing dependencies/port. This gives an idea how many ports needs to be install or upgrade. This also means how much caffeine needed :p

OSX - How to arrange Menu Bar icons

When too many icon clogging on the Menu Bar, re-arrange them might look better. The shortcut key to arrange the icons is :

Cmd + mouse left click

on an icon then start arranging them.

Adios !!!

FreeBSD - How to setup ports using wget to download

The default download program for ports is using fetch. fetch is fine but my personal preference is wget. wget uses 1 connection to download, unlike my previous post that mentioned "Download ports simultaneously with multiple connections".

But wget gives extra bell and whistle like date/time of the download, log to file, continue from a partially downloaded file, rate limit of the download & others.

The below is how to setup ports to use wget to download :

FreeBSD - Tips to prepare large ports installation

Ports gets it's sources from internet. If the sources is big or there are multiple tens of sources, the tendency to fail in the mid of the ports installation increases.

Scenario 1:
Bumpy internet broadband internet connection, like mine, will disconnect halfway when downloading big files (> 10 MB)

Scenario 2:
Large ports that depends on tens if not hundreds of other ports. e.g. KDE4 or other window manager.

Scenario 3:
Limited online time frame and will need to take the machine off line after a period of internet usage.

Scenario 4:
Need to download the sources so that it can be moved to another box for compilation. A box that does not have reasonable or slow internet bandwidth to download.

Often, the above scenario are stuck with me, unfortunately. But the fortunate part is, while meddling with ports, a few steps can be taken to reduce the chances of these hiccups.

For illustration purpose, we'll use installation of KDE4 (the excellent Desktop environment) as an example. KDE4 depends on hundreds of other ports. The download of the sources alone will take some time and this serve as a good example. Here are the steps :

FreeBSD - Ports is fetching from slow servers

Ports randomly gets it's source server list from /usr/ports/Mk/bsd.sites.mk, as defined by the port maintainer. But the top 1st choice of the list may not be the one physically near to us and thus the download speed is much desirable. Most of the time compiling ports doesn't necessary takes a long time (like my 3 years AMD athlon 1.8Ghz perform fairly good). In fact, downloading of the sources is the one that takes a long time.

In this case, we'll need to tune the ports so that it fetches the sources from the nearest (physically) server. A nearer server normally means less latency and less latency means faster connection (this may not be the whole truth but truth is in it. :p)

The port "fastest_sites" will test the latency of the server listed in /usr/ports/Mk/bsd.sites.mk and output to a file. The file is suitable to be included in /etc/make.conf which tell ports where to fetch the sources from.

Here it is how to:

Gimp - How to convert background of logo or picture into transparency

Gimp stands for "GNU Image Manipulation Program", which is an open source image editing problem. It's capability often compare to professional software but fortunately, Gimp acceptance as Pro image editing software is getting better. Even so, Gimp is still popular among beginner up to hobbyist user.

Gimp's available on multiple platform and since it is open source, you can bet it's available on BSDs.

This post will brief through how to transform a logo or picture to transparent background, suitable for dark or colored web page background.

FreeBSD - Compilation errors on port KDE4 in FreeBSD 8.0

While adding GUI Desktop environment into my development box, an error encountered when compiling KDE4, the stable and nice looking Desktop environment. Compiling KDE4 can take quite some time as it depends on few hundred other ports. Alternatively, you can install KDE4 from the packages by using the command :

pkg_add -r kde4

But installing programs using ports is always my preference. Although ports takes additional time to compile but the flexibility to configure what options to enable or disable is preferred. By using ports, it also allows us to search (make quicksearch name=port-name) the local repository (/usr/ports) on what other related programs we can have.

Back to the topic. Halfway ports compiling KDE4, it halted with an error compiling the port "kdebase-runtime-4.3.4". Instead of searching the errors in a one big whole of logs, manually compile /usr/ports/x11/kdebase4-runtime to zoom in the error. Below is what the ports complained about while installing /usr/ports/x11/kdebase4-runtime (which is part of the dependencies of KDE4) :

[  2%] Built target kcm_mailtransport_automoc
[  2%] Built target mailtransport_automoc
[  2%] Built target microblog_automoc
[  2%] Built target kpimtextedit_automoc
[  2%] Built target syndication_automoc
Scanning dependencies of target gpgmepp
[  2%] Building CXX object gpgme++/CMakeFiles/gpgmepp.dir/gpgmepp_automoc.o
Linking CXX shared library ../lib/libgpgme++.so
[  7%] Built target gpgmepp
Scanning dependencies of target gpgmepp-pthread
[  7%] Building CXX object gpgme++/CMakeFiles/gpgmepp-pthread.dir/gpgmepp-pthread_automoc.o
Linking CXX shared library ../lib/libgpgme++-pthread.so
[ 11%] Built target gpgmepp-pthread
[ 11%] Building CXX object akonadi/CMakeFiles/akonadi-kde.dir/itemserializer.o
In file included from /usr/local/include/boost/parameter/aux_/arg_list.hpp:14,
from /usr/local/include/boost/parameter/aux_/tagged_argument.hpp:10,
from /usr/local/include/boost/parameter/aux_/tag.hpp:8,
from /usr/local/include/boost/parameter/keyword.hpp:10,
from /usr/local/include/boost/parameter/name.hpp:8,
from /usr/local/include/boost/graph/named_function_params.hpp:15,
from /usr/local/include/boost/graph/depth_first_search.hpp:21,
from /usr/local/include/boost/graph/topological_sort.hpp:16,
from /usr/ports/deskutils/kdepimlibs4/work/kdepimlibs-4.3.4/akonadi/itemserializer.cpp:40:
/usr/local/include/boost/parameter/aux_/maybe.hpp:13:53: error: boost/python/detail/referent_storage.hpp: No such file or directory
In file included from /usr/local/include/boost/parameter/aux_/arg_list.hpp:14,
from /usr/local/include/boost/parameter/aux_/tagged_argument.hpp:10,
from /usr/local/include/boost/parameter/aux_/tag.hpp:8,
from /usr/local/include/boost/parameter/keyword.hpp:10,
from /usr/local/include/boost/parameter/name.hpp:8,
from /usr/local/include/boost/graph/named_function_params.hpp:15,
from /usr/local/include/boost/graph/depth_first_search.hpp:21,
from /usr/local/include/boost/graph/topological_sort.hpp:16,
from /usr/ports/deskutils/kdepimlibs4/work/kdepimlibs-4.3.4/akonadi/itemserializer.cpp:40:
/usr/local/include/boost/parameter/aux_/maybe.hpp:90: error: 'boost::python' has not been declared
/usr/local/include/boost/parameter/aux_/maybe.hpp:90: error: expected unqualified-id before '<' token
/usr/local/include/boost/parameter/aux_/maybe.hpp: In member function 'typename boost::add_reference::type>::type boost::parameter::aux::maybe::construct2(const U&) const':
/usr/local/include/boost/parameter/aux_/maybe.hpp:59: error: 'm_storage' was not declared in this scope
/usr/local/include/boost/parameter/aux_/maybe.hpp: In member function 'void boost::parameter::aux::maybe::destroy()':
/usr/local/include/boost/parameter/aux_/maybe.hpp:72: error: 'm_storage' was not declared in this scope
*** Error code 1
1 error
*** Error code 2
1 error
*** Error code 2
1 error
*** Error code 1

Stop in /usr/ports/deskutils/kdepimlibs4.

From the errors, it seems kdepimlibs4 is missing of a dependencies related to "boost::python".

FreeBSD - How to check swap space or partition info

Use the command :

swapinfo

Specified the parameter "-h" to convert the bytes to "h"uman readable form :

swapinfo -h

Sayonara !!!

FreeBSD - ports install Exim email server with patch error

Learned another thing today. The ports tree is just a repository of available software, it can be deleted & regenerated. Why would some one wants to delete the ports tree? It needed to be delete and regenerated because some patch was downloaded incorrectly and the problem stubbornly persists.

Sample of the error :

===>  Applying FreeBSD patches for exim-4.71

Ignoring previously applied (or reversed) patch.
2 out of 2 hunks ignored--saving rejects to src/daemon.c.rej
=> Patch patch-src-daemon.c failed to apply cleanly.
=> Patch(es) patch-OS::Makefile-FreeBSD patch-scripts::exim_install applied cleanly.
*** Error code 1

Stop in /usr/ports/mail/exim.
*** Error code 1

Stop in /usr/ports/mail/exim.

The error complained about a patch file which the patch file did exists.

What have been tried :

• portsnap fetch update;make

  NO, it didn't work

• make clean;make rmconfig;make

  NO, it didn't work

• make config, deselect all options, make

  NO, it didn't work

• rm -rf /usr/ports/distfiles/exim then make

  NO, it didn't work

After meddling around for a few minutes, still no clue. Is time to check out :

IRC server : irc.freenode.net

IRC channel : #freebsd

FreeBSD - Backup and restore FreeBSD using Fixit CD

Since FreeSBIE have not been updated since year 2007, its kernel recognizing new hardware starts to worry me as it is based on FreeBSD 6.2 and it is going to reach EoL (end of life) by end of this year, 2010. Is time to experiment on new way restoring backup.

The FreeBSD installation process does mentioned about "Fixit" CD booting. After meddling around with it, it's actually referring to another bootable CD which its label name consits of "livefs" (Live File System). It's also known as "disk 2" prior to FreeBSD 7.
e.g.

Installation disc -- > 8.0-RELEASE-i386-disc1.iso
Live File systems disc --> 8.0-RELEASE-i386-livefs.iso

Do take note that the dump/restore instruction from the previous post "Freebsd – Backup & restore for disaster recovery" is still valid. This post is served as a "update" as it won't be using FreeSBIE, rather, it will using the livefs to start the restoration of the partition(s).

Without further ado, here's the instruction :