The subject of FreeBSD and it’s migration to SVN has come up on the mailing lists repeatedly over the past 6 months. The biggest problem seems to be the lack of communication and transparency of this change, combined with the recent security breach incident being used as a stepping stone to justify the situation (despite that incident having absolutely nothing to do with the transition of CVS->SVN). Peter Wemm has at least shed some light on the need for the migration itself; but it still leaves end users out in the cold (so-to-speak).
The most common question I’ve seen posted by end users is “how do I migrate?” and “is csup/cvsup broken?” The answers which have been provided by both the developer and user community have been abysmal at best, mainly because nobody actually bothers to take the time to explain the complexities and nuances involved in the migration. The official FreeBSD documentation is overwhelming and doesn’t take into account some of these nuances (nor should it, IMO). The migration itself is actually easy, but there are many catches along the way which are certain to bite even the most educated of administrators. I’ll also use this opportunity to explain why use of portsnap should be avoided if at all possible.
To try and keep it simple, I’ll outline each of the common setups people have, and how exactly to migrate from each one. I should be very clear with my intentions here:
- By “migrate” I do not mean “converting your existing data on the filesystem to svn” — instead, all procedures are based upon the “start fresh, delete old cruft, and go from there” concept. It is not worth fiddling around with something so key/important to FreeBSD; think about it.
- I strongly advocate management of both src and ports entirely via SVN. If you follow -RELEASE branches and do not care about src at all, I still recommend you use SVN to handle ports rather than portsnap (more on that later).
- The SVN checkout URL I use for src is for what has historically been called RELENG_9 (i.e. 9.x-STABLE, 9.x-PRERELEASE, 9.x-RCx, etc.), not -RELEASE branches. I do not follow -RELEASE branches nor do I care about them; -RELEASE branches only contain fixes for security issues, and every single -RELEASE since the 5.x days has had too many general bugs that warrant immediate fixing.
Below is how I migrated my own systems, starting with a fresh OS installation. The following should be performed as root:
rm -fr /usr/ports rm -fr /var/db/sup/* /usr/sup portsnap fetch portsnap extract cd /usr/ports/devel/subversion make install cd / rm -fr /var/db/portsnap/* rm -fr /usr/ports mkdir /usr/ports svn checkout svn://svn.freebsd.org/ports/head /usr/ports rm -fr /usr/src mkdir /usr/src svn checkout svn://svn.freebsd.org/base/stable/9 /usr/src chmod 700 /root/.subversion ...Manually remove any SUP-related variables from /etc/make.conf...
Now to explain this utter madness.
- Line 1 deletes the existing copy of /usr/ports from the filesystem
- Line 2 deletes the CVS repository databases associated with src and ports; these are (were) managed by csup and cvsup themselves
- Expanding on line 2: lots of people don’t know about these databases. Quite simply what’s in /usr/{src,ports} does not necessarily reflect what CVS thinks is in /usr/{src,ports}, and this is how/why there can be mismatches when going from one branch to another. Surprise! This is what bites a lot of people when switching between tags (i.e. going from RELENG_9_0 to RELENG_9 or vice-versa) or versions in general. So to make it clear: csup uses /var/db/sup for its database directory, while cvsup uses /usr/sup
- Lines 3-4 use portsnap to fetch a copy of the /usr/ports tree. Why do we do this? Because we need to install subversion from ports
- You may be asking: why can’t you just use pkg_add -r subversion? The reason is explained here by the port maintainer (read slowly, it’s easy to miss). As of this writing, the subversion package on the official FTP/package servers is old (October 13th) and pulls in an absurd number of dependencies when it shouldn’t. So we can’t use the package, thus we must build it from source ourselves. If/when this gets addressed by the maintainer and/or the package build cluster admins, yes, this procedure will become significantly simpler
- Lines 5-6 build and install the subversion port itself, along with all of its dependencies. Please do not change any of the subversion options! The defaults as of this writing are as minimal as possible
- At this point we have a working subversion (svn) binary that we can use
- Line 7 changes the cwd to /, since we’ll soon be nuking /usr/ports (yes you read that correctly)
- Line 8 deletes the portsnap database/tarball. This is the one of many reasons I cannot stand portsnap; it quite literally keeps an entire copy (tarball) of the /usr/ports tree laying around, by design. I wonder how many people with small /var partitions have gotten bit by this. portsnap also pulls down INDEX-x files for releases which aren’t relevant to your version of FreeBSD (i.e. it will pull down INDEX-7 on a 9.x system). portsnap’s ports tarball also lacks .svn/ directories (starting to get the picture?). All depressingly moronic
- Lines 9-10 delete /usr/ports and recreate the directory
- Line 11 checks out a copy of /usr/ports from the official FreeBSD SVN servers
- Lines 12-14 are the same as 9-11 but for /usr/src and src
- Please use svn:// and not http:// or https:// URI schemes — see below for why!
- Line 15 fixes (what I consider to be) a security concern I have over /root/.subversion being globally readable. Not that I change anything in /root/.suvbersion/{config,servers} but some people might. This is the result of root’s default umask (0022), but I’m surprised svn doesn’t issue a chmod 0700 on the directory itself creates
- Line 16 should be self-explanatory
It’s like I said — a simple procedure which is greatly complicated by many nuances that aren’t documented anywhere or explained by most.
For upgrading a system which already has a copy of /usr/ports, run what’s shown on line 2, then continue onwards starting at line 5. You can skip line 8 too, assuming portsnap has never been used prior.
Some other related topics or answers to questions I’ve seen asked (or asked myself!):
- To update your ports tree, you can do any of the following commands — they are all effectively the same:
cd /usr/ports && make update cd /usr/ports && svn update cd /usr/ports && svn checkout svn update svn://svn.freebsd.org/ports/head /usr/ports svn checkout svn://svn.freebsd.org/ports/head /usr/ports
- To update your src tree, you can do any of the following commands — they are all effectively the same (note that “make update” can no longer be used):
cd /usr/src && svn update cd /usr/src && svn checkout svn update svn://svn.freebsd.org/base/stable/9 /usr/src svn checkout svn://svn.freebsd.org/base/stable/9 /usr/src
- For the ports INDEX-x file itself, just use the not-so-well-documented “fetchindex” target. However, since the security breach, the generation of this file on the official FreeBSD servers has broken many times (3 or 4 so far). This has been complained about repeatedly by users; in that case, the only choice is to generate it yourself, which takes a very long time and often emits lovely warnings. Anyway, using fetchindex is simple:
cd /usr/ports && make fetchindex
