A follow-up to the following two posts of mine:
The problem I described has not recurred since enabling prefetch. So it seems whatever performance-related problems we had with prefetch when ZFS was first committed to FreeBSD “back in the day” have since been addressed. I wish I could pinpoint where/when/how this was fixed, but the beast is complex…
I’ve since re-enabled prefetch on my co-located production servers (Intel ICH7-based) and I’m seeing great improvements there too. Those are single-disk systems (e.g. no raidz1 in use) too.
I’d recommend that users who have previously disabled the ZFS prefetch mechanism on FreeBSD should re-enable it and reboot. :-)