Scratch of the day - ISC dhcpd is complaining about subnet declaration

Just now, restarting the ISC DHCP server, after adding new subnet, spill out some error messages :

No subnet declaration for gem0 (192.168.168.168).
** Ignoring requests on gem0.  If this is not what
   you want, please write a subnet declaration
   in your dhcpd.conf file for the network segment
   to which interface gem0 is attached. **

Sending on   Socket/fallback/fallback-net

This is because the DHCP server doesn't find any "subnet" declaration on the IP used on interface gem0. Tell DHCP server to listen on the intended interface by putting the below line in /etc/rc.conf :

dhcpd_ifaces="gem1"

This will make DHCP server listen on "gem1" and ignore listening on "gem0".

Adios !!!

Scratch of the day - arpresolve: can't allocate llinfo

Today, one of the server was uncontactable after a network switch flood incident. All other servers are working fine except this one. Log into this server through the console, reveals :
- ping to gateway is fine
- ping to a host outside this network failed
- ssh connection to same segment is fine but not beyond gateway





/var/log/messages has a dozens of dozens of these :

Apr  9 00:30:01 hostname kernel: arpresolve: can't allocate llinfo for 10.0.0.1
Apr  9 00:31:00 hostname kernel: arpresolve: can't allocate llinfo for 10.0.0.1
Apr  9 00:32:00 hostname last message repeated 15 times 

It looks like connection from this server to machines within it's own segment is fine but other then that, all are unavailable. Another thing, machines connected before initiated from this server to it, is fine. Connections from other server within own segment to this server (if NOT initiated by this server before) still failed.

After some googling, it seems like a patch is needed to fix this error :
http://lists.freebsd.org/pipermail/freebsd-net/2011-August/029687.html

Since the arp cache might be leaking and causes the table to be corrupted, a reboot is needed for the network stack to work. After reboot, the usual patching applies.

The moral of the story, updating FreeBSD is as important upgrading ports. :p

Ciao !!!

Scratch of the day - Real time network statistic, like top

There's a sudden surge of network traffic in one of the servers and I need to find out what causes it. Roaming around ports show up this :

/usr/ports/net-mgmt/iftop

iftop does for network usage what top(1) does for CPU usage ...

(via)

To start monitoring :

FreeBSD - How to check what services or port opening in tcp/udp

Platform :
FreeBSD

How to :
List/show what services or port is listening/opening.

Installation :
The command comes with base system.

FreeBSD - How to reduce TIME_WAIT connections

Routinely, I did a "netstat -an" on a FreeBSD box, a DNS server. The screen then shower with hundreds of "TIME_WAIT" connections. Seems like some malware infected clients are querying the server and causes the terminated TCP socket waiting to be shutdown, but not fast enough, to be efficient. Fortunately, the numbers of TIME_WAIT sockets accumulated are insignificant.

In order to reduce the number of socket waiting, tune the system value :

N900 - A phone running open source OS & programs

N900 is the latest offering from Nokia that answers the open source crave for software freedom, it runs Maemo, an OS based on linux, specifically Debian. Maemo also uses "apt" as package manager. Its GUI is fantastic, with accurate touch screen experience. Command line comes with it by default, to access the very core of the OS. And not like some other OS that hides hard to NOT let user touch it (even that, users managed to "jailbreak" it so to access the command line :p).

The N900 hardware runs a fast ARM cortex A8 processor (overclockable), 32GB internal storage with expandable micro MMC external storage card, touch screen with 800 × 480 pixel resolution, QWERTY slide hardware keyboard, 3.5 mm audio jack, A-GPS, WiFi, bluetooth, 3G internet connectivity and the long list goes on.

A big note here, the Maemo community is active & generous to its user that wants to have software freedom. Freedom to choose software from a community that offers alternatives, community that patches and write software just so that we all can use freely and not bow our heads to companies that charges software, even minor function. The Maemo community is active in providing patches both to the OS & 3rd party programs.

We all know that Debian is a hacker friendly linux distribution that seems more and more linux distribution derived from. e.g. Ubuntu family, Knoppix, Damn Small Linux, here & here. Maemo is based on Debian.

The price is unbeatable consider that it comes with such fast & furious hardware, OS and most importantly, the active & friendly open source community.

This post will list out my other blog post that's written based on N900. Here are some of it :

1. N900 - How to ssh into N900 using OpenSSH
2. Perl - A happy note, on N900 
3. N900 - How to overclock the processor speed of N900
4. N900 - How to clear all the phone data & settings
5. Stay tuned, this list will be update periodically.

Adios !!!

N900 - How to ssh into N900 using OpenSSH

Since the Nokia N900 runs on Maemo, Debian based Linux, it also offers open source programs. The secure shell is one of open source gems. Maemo offers the OpenSSH variety of secure shell.

OpenSSH on Maemo not only offers encrypted secure remote shell, it also comes with functions that runs like the desktop version. Such as ssh socks 5 tunneling, that can be used to encrypt traffic transmit of the programs that supports socks 5 proxy.

Here's how to do it :

Network IP address calculator - for subnet and range of host

One of the task installing a new FreeBSD box was dealing with IP addresses. Usually, google a web based calculator would do the trick. But today's task is a bit different.

IPv6. A whole row of hexadecimal spinning in front of my eyes, just to figure out what is the range of IP that is usable. Web based IP calculator (based on java script) are not accessible using console based internet browser (links & lynx).

Spontaneously, searching the apt repository return some very interesting stuff.

Console based IP calculator. sipcalc.

Not only it is intelligent enough to figure out the input but also it is intelligent enough to generate useful output. Surprisingly, the output is scripting friendly (using grep and sed will parse the output).

These are the steps to install it under FreeBSD & Debian based distribution (e.g. Ubuntu) :

BIND - name server error "the working directory is not writable"

Recently, there has been a few incident on vulnerabilities of DNS server being exploited. Fortunately, Bind have release patches soon enough for the rest of us to start covering up the loop holes.

After an upgrade or installations of ISC Bind DNS server 9.6 on a FreeBSD box, we will need to stop and start "named" service. As usual, running the below command will restart it :

/etc/rc.d/named stop
/etc/rc.d/named start

During the restart of the service, a few lines of text pops up saying that the permission of some directories in "/var/named" was altered. By default, the named files will need to be owner and group of "bind" in order for it to run properly. But the script in "/etc/rc.d/named" seems to change some directories ownership to "root" and group ownership to "wheel". This actually causes some errors reported in the log file :

Sep  3 02:44:26 nameserver named[9999]: the working directory is not writable

Naturally, we would issued the command :

chown -R bind:bind /var/name/etc

to fix the permission problem. But after another restart of "named" service using the command :

/etc/rc.d/named restart

the permission revert back to "root" and "wheel". This causes problem when the DNS server transfer zone files from the Master DNS server. The "named" service would not be able to write the latest zone info into the file in the directories "/var/named", then it would create some temporary files.

After some investigation, it seems like the ownership define in "/etc/mtree/BIND.chroot.dist" was set in this way. We will need to redefine the ownership of the directories so that it stop changing the directories ownership back to "root" whenever the "named" service is started so that "named" service owner, "bind" user, which runs the DNS server will be able to write the zones files in "/var/named".

ssh - How to encrypt/tunnel internet traffic through ssh

When using an insecure internet connection (e.g. wireless), it is a good practice to tunnel internet traffic through ssh. Tunneling through ssh provide encryption of the traffic starting from local computer to the point ends where ssh server is. It is especially important to encrypt the traffic out of the local PC when using un-secure connection as other malicious intent users might be able to sniff packet off the current connection.

How does ssh tunneling works?

FreeBSD - Quick install guide (aka How to install FreeBSD)

Much great have been said on FreeBSD;

• the license has all the freedom you need
• server apps setup are mostly manual. no wizard of oz, no gui. Which makes the administrator have control over everything.
• it boots faster
• its stability is like rock, hard to crash :)
• default installation are lean and mean. Is meant to be built from scratch. Ain't no rubbish you want on the system to trade for vulnerabilities.

Even though some of the above might scare off a normal user, no doubt it is still a great OS that no others can beat. Don't believe me? Try it yourself with the below few pictures (to guide) :

Linux - Shorewall firewall with PPTP VPN dialup client

PPTP VPN uses protocol GRE on two way, both inbound and outbound. Setting up the firewall to allow PPTP VPN dialup client (e.g. Ubuntu's NetworkManager) to use protocol GRE is just a few steps in adding rules and interfaces. This post is based on the previous post "How to set up Shorewall firewall (alternate to FireStarter)", which is a quick tutorial on setting up Shorewall firewall on a single network interface. Here it is :

Linux - How to set up Shorewall firewall (alternate to FireStarter)

There are a few option of firewall interface available on Linux for iptables, both GUI (Graphical User Interface) and CLI (Command Line Interface). FireStarter user interface on GUI is easy to use and configure. The only problem is it does not work properly with PPTP VPN dialup, it blocks protocol GRE. It does provide a workaround (dont ask, it is too painful to remember) but the steps are too tedious and involve the mixture of setup in command line and GUI, and this kind of setup always spells trouble.

Shorewall is another firewall interface that operate on CLI. This is taken from the Shorewall's introduction :

"The Shoreline Firewall, more commonly known as “Shorewall”, is high-level tool for configuring Netfilter. You describe your firewall/gateway requirements using entries in a set of configuration files. Shorewall reads those configuration files and with the help of the iptables, iptables-restore, ip and tc utilities, Shorewall configures Netfilter and the Linux networking subsystem to match your requirements. Shorewall can be used on a dedicated firewall system, a multi-function gateway/router/server or on a standalone GNU/Linux system. Shorewall does not use Netfilter's ipchains compatibility mode and can thus take advantage of Netfilter's connection state tracking capabilities."

Shorewall's way of configuring iptables firewall made easy by just describing the rules and interfaces into a few files and the firewall is ready to go. Shorewall's website also provides extensive documentation on how it works and how to setup.

This post's intended to setup Shorewall in a few simple steps and thus, it is not really suitable to implement it on medium to large enterprise. Here it is :

Ubuntu - How to resume NetworkManager managing network interface

After manually editing /etc/network/interfaces for dynamically assigned IP or static IP, Network Manager (the icon sits on the upper panel with 2 monitor) stops working. Specifically it stop managing interface, but manually assigning IP through "ifconfig" or editing config files (/etc/network/interfaces) still works. Another reason NetworkManager stop working is after applying some updates between Intrepid (8.10) to Jaunty (9.04) it just stop working. Here is the solution :

Freebsd - How to install FreeBSD as your Desktop

UPDATE : After testing PC-BSD for a while (more then 6 months), PC-BSD indeed is a FreeBSD variant that is superior in Desktop environment. Its user friendly installation and configuration have make FreeBSD so much more suitable running as an Desktop. Unless you're interested in figuring out how to tweak and turn FreeBSD to work in a Desktop, give PC-BSD a try, you'll love it!
Check it out at PC-BSD website !!!

PsyberMonkey is turning his desktop to FreeBSD and this is the work he has done :-

1. FreeBSD quick install guide (aka How to install FreeBSD).

2. make sure hardware are detected properly

• How to install or add sound card driver module
• wireless (how to still in progress)
• ACPI (still having trouble making it work)
• remote control through SSH (how to still in progress)
• all others e.g. external drive formatted in FAT32, display card, track pad, external mouse, gigabit network card & etc are detected and function properly.

3. Using ports to install software - go to Section III.

4. Getting KDE, GNOME and XFCE up and running (how to still in progress)
Other miscellaneous stuff :

• How to disable or turn off the keyboard beep sound
• How to assign or remove fixed IP
• IP address automatically assign aka DHCP
• How to install flash plugin or player to work with FireFox
• Backup & restore for disaster recovery
• How to upgrade the kernel or base system
• Error installing KDE4 or Xine

This post will be updated periodically to reflect the progress of using FreeBSD as a Desktop.

Adios !!!

FreeBSD - Combine 2 (or more) NIC using network link aggregation and load balance (aka round robin)

Hard disk are getting bigger and bigger. Network interface card (NIC) and switch are getting cheaper and cheaper. It usually means more services can squeeze into a server. Combining multiple network interface into 1 big pipe would be a big plus for services that is bandwidth hungry. e.g. file server, email server & etc

Combining 2 or more network interface have a few advantages. One of them is load balancing. Load balancing enables the network interface card to share the load among the 2 or more network interface card defined. It serves as redundancy as well. But load balancing does not makes full use of the 2 or more network interface, it is meant to share network traffic load.

To fully utilize 2 or more network interface links, link aggregation should be used. Link aggregation is a method to combine 2 or more network interface to become a bigger pipe. It is also a way to combine 2 or more interface to use the same IP address. Using link aggregation can also create redundancy, if one of the link fails, the other links will take over.

This post is using FreeBSD as a server because FreeBSD have all the ports (software packages) you will need and server administration on it is made to ease systems administration daily task.Without further ado, below are the steps to configure 2 (or more) network interfaces to work as 1 big trunk or round robin :

Nagios - Error "Error: Could not open command file '/var/spool/nagios/rw/nagios.cmd' for update"

After an upgrade of Nagios, Nagios re-schedule function for next service check is not working. The Nagios web interface complained "Error: Could not open command file '/var/spool/nagios/rw/nagios.cmd' for update". Its problem lies on the webserver was unable to access the file to update the schedule.

How to display route table or default gateway

Route table are part of the area to troubleshoot when a connection have problem. Route table can the the most frequent problem to look at after assigning an IP address. Other times, it is mainly for adding additional routes when wireless & wired connection are both in use. This post will attempt to list out a few ways to display route table and also how to assign default route aka default gateway in Linux & FreeBSD.

FreeBSD networking - How to assign or remove fixed IP

Previously, we have talk about how to assigh dynamic IP to an interface on Freebsd. This post is about how to assign fixed or static IP to an interface on Freebsd. Below are the steps :

FreeBSD networking - IP address automatically assign aka DHCP

This is another follow up post on DHCP aka Dynamic Host Configuration Protocol, which posted first in here and then here. FreeBSD are commonly used as a server. One might ask why does it needs to auto assign IP using DHCP ? Should not it running static IP so that the server are contacted by the host always on that particular IP? To answer the question, the server could be in testing mode so any IP will do. Or the DHCP server have statically map the server's MAC address to a particular IP in its table or pool. The DHCP server will only assign that particular IP to that MAC address when the DHCP client (which in this case is the FreeBSD server) sends out a broadcast requesting info to setup the IP assignment.

In short, just follow the below and IP will be assigned, if there is any DHCP server already configure within the network :