FreeBSD, csup/cvsup, portsnap, and svn

The subject of FreeBSD and it’s migration to SVN has come up on the mailing lists repeatedly over the past 6 months. The biggest problem seems to be the lack of communication and transparency of this change, combined with the recent security breach incident being used as a stepping stone to justify the situation (despite that incident having absolutely nothing to do with the transition of CVS->SVN). Peter Wemm has at least shed some light on the need for the migration itself; but it still leaves end users out in the cold (so-to-speak).

The most common question I’ve seen posted by end users is “how do I migrate?” and “is csup/cvsup broken?” The answers which have been provided by both the developer and user community have been abysmal at best, mainly because nobody actually bothers to take the time to explain the complexities and nuances involved in the migration. The official FreeBSD documentation is overwhelming and doesn’t take into account some of these nuances (nor should it, IMO). The migration itself is actually easy, but there are many catches along the way which are certain to bite even the most educated of administrators. I’ll also use this opportunity to explain why use of portsnap should be avoided if at all possible.

Continue reading


Recent “cluster server” security breach

For the past 4-5 days, users of FreeBSD have been unable to get any updates from servers worldwide. Specifically, this affected the following services:

  • csup and cvsup (for src, ports, doc, etc.)
  • portsnap (for ports only)
  • freebsd-update (for binary updates/security updates, if not using csup/cvsup)
  • Native CVS (note: intentionally using cached copy from Google)
  • GNATS (official FreeBSD problem reporting/bug tracking system)

All that was being stated publicly by FreeBSD Project members was that there was “an issue”, and then later this turned into “maintenance”:

Some others also began to wonder about how the outage was being handled, since nobody had updated the home page (until the aforementioned), updated the official RSS feed, posted anything on Twitter, or (most important of all) sent anything to the official freebsd-announce mailing list.

At the 4-day mark, I began wondering what kind of “maintenance” was unannounced and not done without some degree of prep-work. And it turns out my system administrator gut feeling was right — today the root cause was actually disclosed:

Continue reading

Using FreeBSD graid (GEOM RAID)

Quite some time ago I moved my home system away from ZFS and over to gmirror (GEOM mirror). There were many reasons for this, all of which are outside of the scope of this post thus I will not go into them. Things were good with gmirror, until after rebuilding world somewhat recently I began seeing this message:

GEOM: ada1: the secondary GPT header is not in the last LBA.
GEOM: ada2: the secondary GPT header is not in the last LBA.
GEOM_MIRROR: Device mirror/gm0 launched (2/2).

Continue reading

unssh: what purpose does it serve?

Today I was reviewing CVS commits to ports, and I saw this flow across my screen:

 Edit ports/security/unssh/Makefile
  Add delta 1.4 2011. pav
 Edit ports/security/unssh/distinfo
  Add delta 1.3 2011. pav
 Delete ports/security/unssh/files/

I immediately wondered what unssh was. Apparently it’s a shell script that modifies (removes lines from) your ~/.ssh/known_hosts file when run with the same arguments as ssh.

This immediately made me think “why is this even necessary when OpenSSH has framework to already accomplish this task”? Just edit your ~/.ssh/config and enter the following:

Host gw
StrictHostKeyChecking no

The only difference here is that this will never remove the outdated (“offending”) entry in ~/.ssh/known_hosts and you will continue to see the nasty man-in-the-middle warning every time you SSH to the host “gw”. However, you’ll be able to connect regardless of the warning.

There’s really no harm in ignoring the nastygram when connecting to a machine you know is going to change its SSH identity keys all the time. Really — the person SSH’ing there will already be aware of that (the fact they would go and find/use unssh is proof of that).

Sadly there is no -q equivalent in ssh_config(5), otherwise StrictHostKeyChecking no and Quiet yes would be a great combination for this exact situation.

Dennis Ritchie has passed away

The statement comes from Rob Pike, so I’m inclined to believe it.

If you’re a programmer and don’t know who Dennis Ritchie is, then you should have your programming license revoked permanently.

RIP, dmr. You will never be forgotten.

Systems monitoring nightmares on FreeBSD

For quite some time (years) I’ve been dealing with monitoring of FreeBSD systems. “Monitoring” in this case does not mean service availability, it means data/statistics acquisition of key parts of the system — things like memory and VM usage, CPU load, pf (firewall) statistics, NIC statistics, disk usage, disk I/O, and so on.

You still have to store the acquired data somewhere/somehow. And that’s where RRDTool, unfortunately and unjustifiably, comes into play. If you’re an administrator (or developer) that has to deal with systems monitoring and statistics data acquisition in an open-source world, there’s a very good chance (~90%) that you’ve had to painfully deal with this software.

Continue reading

FreeBSD and ZFS — hot-swapping SATA disks with AHCI

There’s been a large discussion on freebsd-stable@ as of late regarding how to hot-swap SATA hard disks on FreeBSD which are part of a ZFS pool. I wanted to take the opportunity to demonstrate how this works on real server-class hardware.


We have a system that contains 3 SATA drives — one SSD (for the OS, using standard UFS2 and isn’t part of the picture), and two 1TB drives (in a ZFS mirror). The system contains only 4 hot-swap drive bays, 3 of which are populated.

Without powering the system off, we want to replace the two existing 1TB drives with newer 1TB drives (which have more cache and offer SATA 3.x support, although the controller we’re using only supports up to SATA 2.x).

Continue reading