FreeBSD and ZFS — horrible raidz1 speed — part 2

While debugging the aforementioned problem, I decided to try re-enabling ZFS’s prefetch capability as a test; “maybe things have greatly improved between v6 and v13”, I thought.

Based on what I can tell so far (it’s only been 10-15 minutes, and I’ve transferred hundreds of megabytes of data), that appears to have fixed the problem:

$ rm ~/Maildir/header_cache.db
$ time mutt -f ~/Maildir/system

real    0m0.151s
user    0m0.030s
sys     0m0.014s
$ time mutt -f ~/Maildir/system

real    0m0.038s
user    0m0.004s
sys     0m0.012s
$ time mutt -f ~/Maildir/system

real    0m0.493s
user    0m0.012s
sys     0m0.004s

There are occasional times where ZFS “lags”, e.g. the above times will jump up to 0.5 seconds or so, but I believe that’s an issue which has been mentioned many times in the past by users — occasionally ZFS appearing “bursty” in I/O. Not sure what the cause of that is, but I know I’m not the only one seeing that behaviour.

Anyway, I’ll keep an eye on things for a few days and see if things remain speedy or if they get worse. ZFS prefetch being enabled could still be a red herring; for all I know memory fragmentation could be the root of the problem. Who knows — too soon to tell…