Category: Committed Code
Matthew Dillon has made some changes to AHCI support; if you have an Intel motherboard with an SSD drive that occasionally doesn’t want to co-operate on a cold boot, this recent update may fix it.
The i386 architecture now supports LAPIC and I/O APIC. If you had weird interrupt problems when installing DragonFly before, now might be a good time to try the latest bleeding-edge version of DragonFly and see if the problem vanished.
It looks like Sepherosa Ziehau is working on getting multiprocessor kernels able to boot on single-processor systems. This makes life a bit easier, since there’s only one kernel needed for any given processor. I don’t know if it’s in a finished state yet.
If you have a really old DragonFly system, meaning you’ve been upgrading it since version… 1.8 (I think?), you may have libpthread linked to libc_r instead of libxu. This means that if you have a system that old, you will now need to set THREAD_LIB or just recompile your pkgsrc programs on your next upgrade to something after DragonFly 2.10. I don’t think this is going to apply to a lot of people.
(I hope I got the lib details right…)
ipfilter has now been removed from DragonFly, by Sascha Wildner. We now have “only” ipfw2 and pf for software firewalls.
John Marino has updated the GNU Debugger (GDB) from version 7.0 to version 7.2. The lengthy commit message describes how surprisingly complex the upgrade proved to be.
John Marino’s gone on a tear and updated GCC to version 4.4.6, diffutils from 2.87 to 3.0, and texinfo from 4.8 to 4.13. Each commit message that I linked to has plenty of notes on what’s different, so they’re worth following. This is the first update for texinfo in 6 years, so the quantity of updates is not surprising.
Then use the new LINT64 config added by Sascha Wildner. LINT kernels have every option turned on, so it’s pretty easy to have problems due to conflicts or untested parts and so on. You probably won’t get a kernel out of it, but now there’s a comprehensive list of your 64-bit kernel options for when you’re building a kernel that works.
GNU grep on DragonFly has been updated from version 2.4d to 2.7. Other BSDs have switched/will switch to bsdgrep, but as John Marino points out in his commit message, GNU grep’s still faster. He’s also brought in NetBSD’s version of sort, to replace the GNU flavor. I don’t know why on that one.
Peter Avalos also updated file to 5.06.
Sepherosa Ziehau has made some changes to default SCI settings in ACPI. This may make it possible to boot a computer, or to boot a computer with ACPI, that did not boot before. If it causes problems, he lists some various tunables to set. Just don’t ask me what SCI does.
Sascha Wildner has moved gcc in DragonFly to a slightly newer version: 4.4.5. It mostly seems to make things easier to compile, going by the reports I’ve heard. This is the version that will be in DragonFly 2.10.
Enabling the vfs.hammer.double_buffer=1 sysctl will greatly improve Hammer performance when you’ve exceeded your memory cache (at a possible slight penalty when you have not) and also speed things up when using live deduplication.
Update: Venkatesh Srinivas says:
“double_buffer makes sense when: 1) you want all CRCs to be checked on reads. 2) you’re running live dedup and care about dedup performance rather than say read-heavy performance; 3) you have swapcache but are often running into the vnode limit in what you can cache.”
So, not always useful.
Also, Matthew Dillon has made version 6 the default version of Hammer in DragonFly 2.9. Version 6 has improved handling of directory names in some circumstances. Just don’t ask me which, cause I lost track. It’s been a hard day!
If you’re using pf to control how your bandwidth is used, you may want to look at the recent fairq updates from Matthew Dillon. It should perform better now in situations where one traffic group is saturating its available bandwidth. Here’s a handy link that explains this sort of problem, yoinked from IRC.
John Marino’s work on getting support for DragonFly ‘natively’ into binutils, upstream, has been successful. Thanks, John!
There’s two recent changes for pkgsrc and DragonFly:
- DragonFly-current (2.9) now pulls the most recent pkgsrc quarterly release (2010Q4) by default, instead of pkgsrc-current. This means more packages will be working with the default setup, plus pkg_radd and other tools will be pulling the same ‘generation’ of software.
- The DragonFly/git version of pkgsrc can now be created as a shallow clone. This means less file history, but also means a much faster download.
A full buildworld/buildkernel is probably the best strategy. I’ll be rebuilding all the pkgsrc packages for 2.9 using gcc 4.4… This will take at least a week.
Matthew Dillon’s improved bridging to the point where you can now modify the MAC address of the bridge and most everything, including ARP, will come from it correctly. It’s even possible to bond 2 or more interfaces together, with the side effect of dragonflybsd.org having a lot more bandwidth.
Update 2: More notes here.
Matthew Dillon has continued his bridge work, with another commit adding various features. Go, read.
Matthew Dillon has added transparent bridging, mostly to overcome issues with the AT&T DSL modem he’s using. With this non-default feature, IP packets retain the original MAC address when retransmitted through a new interface.
As Matthew Dillon notes in a recent post, procedures are now assumed to be MPSAFE (i.e. without the Giant Lock) by default. Any new work should follow this idea, and it doesn’t have to be documented specially. The inverse used to be true, where the code that happened to work without the Lock was rare, and therefore needed to be pointed out. Now, the good result is the norm.
Matthew Dillon’s made some scheduler changes, which blogbench tests are showing give the default scheduler better performance under heavy load. It’s a pretty technical writeup, so I’ll just point you at it rather than attempt to summarize.
Matthew Dillon’s made some changes that will speed up the booting process for people with a ridiculous amount of memory, like 64G. This is x86_64 only, but that should not be a surprise if you think about it.
Global tokens are now running without the giant lock in DragonFly. Neat! There’s still plenty more to remove, but this is a big step.
The default Hammer version in DragonFly is now version 5, which is the one that includes deduplication. Enjoy, bleeding-edge users! Otherwise, wait for the next release.
Version 6 is there, but don’t upgrade to it yet; there aren’t significant user-visible changes, and the usual disclaimers for new versions apply.
Ilya Dryomov has added live deduplication, or as he titles it, “efficient cp”. It’s experimental and turned on with a sysctl, so approach with caution.
Xerox Network Services is gone from DragonFly. Does anyone, anywhere, use this protocol? Ironically, I don’t recall this even being visible on the Xerox hardware products I have at work.
Another bus bites the dust: EISA is no more on DragonFly. I don’t know if there’s even any system that DragonFly could boot on and would use this. Still, remove your hats and enjoy a moment of silence.
Matthew Dillon fixed a rare and difficult-to-find bug on x86-64 Dragonfly. This means much more of the system can be run ‘MPSAFE’, or without the Giant Lock. Watch for this soon if you’re running 2.9.
Ed Smith was thinking of working on sysctl documentation, but as it turns out, a lot of it has already been done via Google Code-In; Samuel Greear recently committed a lot of it. (Though there’s more sysctl work possible.)
While on that topic, Samuel Greear also posted a lengthy summary of all the Code-In work done so far. We need more code-related tasks! The existing ones have been so popular that they’re all getting done, quickly.
Matthew Dillon has moved CPU support to 63 processors and 512G of RAM. This may cause issues, he warns. It’s also just barely working, so don’t expect to go into production with half a terabyte of RAM in the next few days.
There’s now descriptions for a number of the net.inet.* sysctls, thanks to Taras Klaskovsky as part of Google Code-In.
APIC_IO is back as a kernel config option, though it just toggles the sysctl loader tunable default. This is so a kernel config file with that option still set won’t cause an error.
Ilya Dryomov’s work on deduplication for Hammer has been committed to the tree in an early test form. I guess I need to pay up as part of the code bounty. If you’re wondering how much space it will save, but don’t want to try non-production code yet, there’s a ‘hammer dedup-simulate’ command that will estimate the saving ratio.
This is great news – deduplication is so valuable it adds an extra zero onto the price of any storage device that can do it.
DragonFly has shipped with a uniprocessor kernel by default forever. Shipping with a SMP kernel may not work as well for all possible combinations. With some recent changes by Matthew Dillon, both types of kernel are present and can be picked from at boot time – with the LiveCD!
Venkatesh Srinivas has a large writeup describing just how the memory allocator in DragonFly changed from 2.6 to the upcoming 2.8 version. In all that text, you may notice the cheering statistic that it gave a 20% improvement in sysbench results.
Matt Dillon and Venkatesh Srinivas conspired to fix another nmalloc issue, which should resolve any remaining problems people were having with Firefox, and possibly other applications as well. Due to an oversight of sorts, all locking operations on nmalloc’s depot were ineffective, as if there were no locking at all. Curiously, it worked remarkably well considering such a large race condition was present.
I’m linking to this commit message from Matthias Schmidt simply because it has the correct invocation for installing a vkernel, and I know this will come in handy, someday.
A bump in shared library version for libssl/libcrypto means that any programs dependent on it will require a rebuild – including any pkgsrc programs.
This only affects you if you are running 2.7, for now. It means that on upgrading from 2.6 to 2.8, any libssl-using programs will need to be updated. This may not be a big thing, since pkgsrc-2010Q3 will also be out and people will want to upgrade anyway.