John Marino has made it possible to build world and kernel on DragonFly using GCC 4.6. (We’re currently on version 4.4) Note that version 4.6 isn’t included with DragonFly, so you would need to download and compile GCC 4.6, and set CCVER=gcc46 before building world and kernel to try this out.
Category: DragonFly
Google Summer of Code 2012 announced
It’s on, again! Not that there was any doubt. I need to collect potential mentor names before DragonFly can be involved, so you can guess what I’ll say next…
Odd DVD drive issue
Edward Berger found that using a LG/Hitachi DVD drive kept him from successfully booting a DragonFly install CD. Using other manufacturers worked out fine. What causes the problem? I don’t know, but it’s worth mentioning it out loud in case someone else gets bit by it.
NetBSD Hackathon, February 10-12
There’s a NetBSD Hackathon going on February 10th through 12th, mostly online. I mention this because it may have some effect on pkgsrc, used by both NetBSD and DragonFly. Hackathons for pkgsrc usually happen separately, but no harm in keeping an eye out for any positive benefits.
ISDN really gone
ISDN support has been removed from DragonFly. It was not useful at this point, because it’s rarely used any more. It does make me feel a little sad; this was the technology everyone said was the future before cable modems and DSL were figured out.
3.0 Release Candidate images
They are located in the normal place, in .img (USB) and .iso (CD/DVD) formats. I haven’t made the desktop DVD yet; let’s see how these untested versions do…
http://avalon.dragonflybsd.org/iso-images/
Old ATA also out
A bit of symmetry in that title, there. Old ATA, which was replaced years ago, is finally gone. This should affect nobody…
Do you use ISDN?
If you need to use ISDN with DragonFly, speak up now. I think it may get tossed otherwise.
DragonFly 3.0 branched
Note that it’s branched, not released. I’m building and uploading binary pkgsrc packages for it now, and hope to have a ‘release candidate’ very soon. This is the prep work before the release, really. There’s a catchall ticket for tracking remaining work.
Want to support newer Intel GPUs?
There’s a whopping 250 euro bounty up now on the DragonFly Code Bounties page. It’s for supporting the newer Intel video chipsets, and there’s already examples in FreeBSD to start with.
(David Shao, where are you? If you’re reading this, hop into #dragonflybsd and tell us how things are going with your GEM/KMS work)
Live deduplication marked experimental
‘Live dedup’, where a DragonFly system makes a deduplicative reference to copied data instead of actually copying the data, is now off by default. There’s no definite issue linked to it yet that I know of, but it never hurts to be careful just before a release.
RELRO in a BSD
John Marino has added support for RELRO in DragonFly, which makes it the first BSD to have it. That’s great news! What is it? Apparently a guard against memory corruption or overflow in the linker. His commit message gives better details.
Security problem and a fix
Matthias Schmidt found a discussion about DragonFly’s password encryption. The result, if I am reading it correctly, is that brute-forcing the password from available hashes is quicker than it should be. Matthias also found a contributed fix. Samuel Greear updated to match the reference SHA implementation also in Linux, with this very pertinent warning.
If you liked KDE3, you’ll like this
If you liked KDE3, you may like Trinity. Matthias Drochner would like you to help get it in pkgsrc.
How low can you go? (with RAM and Hammer)
Matthew Dillon has a very detailed commit message with changes to make sure Hammer will run overnight cleanups in situations as low as 256M of RAM. I think you can find that much RAM in breakfast cereal boxes these days.
How long until DragonFly 3.0?
The answer is “not very”. As I wrote in a post to kernel@, DragonFly 3.0 will be tagged soon, and released when there’s pkgsrc-2011Q4 packages to go with it. Probably a week if everything goes to plan.
Building JDK 1.6, maybe 1.7
Chris Turner reports success building JDK 1.6 on DragonFly x86_64, though it requires a bit of fiddling. Building 1.7 on x86_64 is getting closer but not yet, as far as I can tell.
Getting rid of lpr
If you install CUPS, or know that you will never print using lpr(1), you can make sure thatyour DragonFly system never builds lpr again by putting NO_LPR=true in /etc/make.conf.
Setting up a DragonFly wireless access point
What if you have a DragonFly system that you want to use for an wireless access point? Andrey N. Oktyabrski did, and he helpfully listed his solution.
Have problems, become immortalized
What happens when you break enough things in DragonFly that you become a source of test cases? As Antonio Huete Jimenez (AKA “tuxillo” on IRC) found out, you get a stress test named after you.
Things that have been updated
I need to catch up on some older stuff, so here is a longer list of recent updates: libarchive to 3.0.2, xz to 5.0.3, mfi(4) and mfiutil(8) (LSI MegaRAID driver) updated, ATI SB7x0/SB8x0/SB9x0 AHCI devices (on motherboards I assume) updated, and the PHY ID for the Atheros F1 added. Thanks to everyone who did the work! I bet I missed something.
Gnat-aux is the way to go
John Marino has pointed out, with a number of examples, that gnat-aux is the best pkgsrc-based compiler for DragonFly right now, in terms of compatibility and support. It’s certainly good news if you are an Ada programmer. He lists some interesting numbers to demonstrate this superiority, though you can’t buildworld with it yet. (gcc 4.4, on DragonFly as part of the system, will do this normally.)
Mailarchive working again, but not NNTP
Matthew Dillon has the mailarchive working again. It pulled from the NNTP version of the DragonFly mailing lists, and when NNTP broke, so did the archive. NNTP isn’t working, but at least the mailing list archive is functional.
I’m hoping to try out Mailman (with NNTP) as a replacement soon…
Virtual kernels on video
There’s a Youtube video showing how to set up a virtual kernel on DragonFly. I don’t think I linked to this before. (via)
Netgraph update
Nuno Antunes has committed a large quantity of work on updating netgraph to version 7. His goal is to be able to run mpd5, though it’s not there yet. If you want to look at it, go to the monthly page and look around the 10th; there’s too much to link to individually.
OpenSSL update
Peter Avalos has updated OpenSSL to 1.0.0f; this is to fix 6 security issues identified in the OpenSSL changelog.
How to get DNSSEC going
I just mentioned DNSSEC in last week’s Lazy Reading, and here’s a “How to get DNSSEC with BIND 9.8.1 working” article from Michael Lucas. It’s pretty simple… Conveniently, BIND 9.8.1 is available in pkgsrc as net/bind98.
Something for re(4) users
You may want to update for this – a lockup bug with the re(4) (RealTek 8xxx series) driver has been fixed.
dma(8) update
dma, which originated on DragonFly, is now at version 0.7, and so is the version in DragonFly.
How much RAM is too little?
If you’re running DragonFly on a very low-end system, you may be wondering about memory requirements for Hammer. Hammer is much less RAM-hungry than ZFS, so it looks like you can get away with 128M, as long as you don’t mind the occasional error message. You can manually tweak settings for it if you like. 256M is plenty.
It still strikes me as odd to consider systems with less than 1G of RAM as “low-memory”. What rich times we live in!
Libm updates, plus a free security fix
John Marino updated libm, bringing a large quantity of functions. This may be a stopgap measure for now. As a positive side effect, buffer overflow attacks are a bit harder now.
OpenJDK 1.7 on DragonFly
Chris Turner got it working on i386, at least, and his post will help you do the same. I don’t know if these changes have made it through to pkgsrc or for x86_64 yet.
Better MSI support
That’s Managed System Interrupts, for when your hardware is passing a lot of data and generating a lot of corresponding hardware interrupts. MSI is what deals with all that traffic. High-bandwidth (10G) network cards, for instance. Anyway, Sepherosa Ziehau’s made more commits than what I’m linking to here, for support with various devices.
There’s many other MSIs out there, oddly enough.
Going to 28C3?
Are you going to Chaos Communication Congress 28? There’s going to be a number of DragonFly developers there, so it’s a good time to meet up. They’re in EFNet #dragonflybsd IRC, so speak up there if you want to find them.
Telnetd update, and history
The recently discovered telnetd vulnerability has been fixed in DragonFly, thanks to Peter Avalos. Apparently it’s been around everywhere forever. (last link via xhr) Hopefully there aren’t that many people that still need a telnet daemon; SSH has run it out of town for the most part.
NFS fixes, too
Since I’m already talking about imports, several changes from FreeBSD and OpenBSD for NFS, plus more original material, have been brought in by Venkatesh Srinivas. Those changes from FreeBSD apparently improve NFS write performance, though I don’t have numbers to show.
Merry Christmas, here’s an incredibly involved bugfix
There’s been a rare segfault present in DragonFly for quite some time. It’s been difficult to reproduce, and the 2.12 release due some months ago was held up specifically to fix it. Matthew Dillon was, after many days (months?) of work, able to replicate it reliably and eventually find a way around what appears to be a new AMD-specific bug. Read his very detailed explanation of what he did to get to this point.
VFS accounting benchmarks
Francois Tigeot benchmarked his accounting work with blogbench, and posted a PDF with the results. Dmitrij D. Czarkoff made a simpler graph, which can be used to draw the conclusion: blogbench didn’t work well for estimating the impact of VFS accounting. If you want to try accounting yourself, put vfs.accounting_enabled="1" in your /boot/loader.conf.
(The normal DragonFly mailarchive isn’t updating because it feeds from DragonFly NNTP, and that’s not updating, so I’m using Gmane for post links.)
Keeping binutils out of the build
There is now a NO_BINUTILS221 option, added by Sascha Wildner, that will keep your system from building binutils 2.21 during a buildworld. The system will still build binutils 2.22, so there will still be a functioning ld on the system. Use this along with NO_GCC41 (so only gcc 4.4 gets built) to speed up your buildworlds, if you like.
New DragonFly mirror
There’s a new DragonFly mirror, in Colorado, USA – at dragonflybsd.mirrorcatalogs.com. It’s on the mirrors page on the DragonFly site, too.
Loader changes for IPMI
If you’re looking to use IPMI and remotely watch the console of another system, Matthew Dillon has made some changes to help with that.
New ECC device
Sepherosa Ziehau has added updated the ‘ecc’ device, for Intel E3-1200 series systems. What’s it do? It will report on memory errors, and potentially fix them.
You should have ECC memory in your server already. If not, you oughta.
Update: as Sascha Wildner pointed out, ecc(4) already existed, but didn’t support Intel controllers. Also, the Xeon X3400 series is supported now too.
Lazy Reading for 12/11/11
Last week was low on links, but this week is great! I hope you have some time set aside.
- This article “The Strange Birth and Long Life of UNIX” has a picture of a PDP-11. I don’t know if I ever actually saw one and knew it before. (via)
- Also from the same place: Window Managers Bloodlines.
- Anecdotal, but probably true. (via luxh on EFNet #dragonfly)
- nginx is the new cool and unpronounceable web server these days, apparently. Michael Lucas covers how to transition static Apache sites over to it.
- This PDF showing slides from the recent NYCBUG presentation by Ike Levy, titled “Inappropriate Cloud Use”, is entertaining, and makes a good point. Cloud computing is cheap on a per month basis, but since it’s a reoccurring cost, it can cost a surprisingly large amount in the long run. (via)
- Hey, a patch for DragonFly (and other BSD) support in Google’s leveldb.
- “Don’t Be a Free User” (via) The last paragraph is the best.
- An expanded grep and diff. ‘grep’ and ‘diff’ have been present for so long, and people understand what they do, generally, that new tools get named after them just because the concept is ingrained in people’s minds. Note that I said “generally”, as regular expressions can be difficult. (via)
- A lot of people don’t realize how they infringe on copyright. This writeup describes something I’ve seen for years: people think a disclaimer that effectively says “I’m infringing but I’m doing it with the best of intentions” makes a difference. It doesn’t.
- So this is what that Xerox Star GUI interface looked like. You know, the ‘first’ desktop GUI. (via) Also, there was some advanced stuff in 1968.
- I like this indicator light setup. (also via luxh on EFNet #dragonflybsd) There’s some other interesting old computer stuff at that site too. I wish there still were computers like these.
- While we’re talking about old things with a certain feel to them, why not Battersea Power Station? Here’s some pictures. (via)
Your unrelated link of the day: Since we’re talking about old things and environments, why not look at some pictures of my workplace?
Two pkgsrc work directory tips
Two tips for working with pkgsrc, derived in part from this mailing list post on users@ (follow the thread) and from my own experience. If you put WRKOBJDIR=/usr/obj/pkgsrc into /usr/pkg/etc/mk.conf :
- You can clean up any leftover package building files by deleting the files in that directory and leave your pkgsrc files untouched.
- You can have a read-only /usr/pkgsrc, which means it can be shared over NFS (or SMB?) between multiple machines, DragonFly or otherwise.
More high-speed optimizations
I’ve had a bunch of posts like this so far, but that’s not a bad thing. Sepherosa Ziehau has a pair of optimizations that appear to make performance with big pipes (1G) and tiny packets (18b, if I read correctly) reach near the physical maximum for 1000-base-T Ethernet.
Remember rcrun?
I think a lot of people don’t realize rcrun exists, and run files in /etc/rc.d directly. If you’re one of those people, read the rcrun(8) manpage. Then, notice that Sascha Wildner has extended rcrun to work with the etc/rc.d in /usr/local and /usr/pkg too.
vquota(8) now available
Francois Tigeot has been working for quite a while on a VFS accounting system. It doesn’t restrict to a quota (yet), but it will give you byte totals for each mounted filesystem. It has been committed, so it looks like a good way to tell which PFS is eating your disk.
Update: Francois pointed out he’s still adding parts for this. So it’s not quite done yet, but soon.
Parallelized buildworld now possible
Buildworlds are now much faster, because they can run themselves in parallel. Invoke it using the -j option to make. Matthew Dillon saw a 25% reduction in time when using ‘make -j 12 buildworld’ on a 4-core system. You may need to manually update xinstall and mkdir:
cd /usr/src/usr.bin/xinstall make clean; make obj; make all install cd /usr/src/bin/mkdir make clean; make obj; make all install
It’ll also use more memory than a non-parallel build, but heck, that’s cheap these days.
A minor debugging change worth noting
Venkatesh Srinivas made a minor change to a ddb backtrace – it now prints the raw instruction pointers. On x86_64, a backtrace would not print the correct objects out, so this is better. It’s a minor change, but I’m pointing it out because it totally helped solve a problem for me on a package-building machine.
An unexpected way to do strlen()
The general rule of thumb is that if you have a function written in an interpreted language (Perl, Python, etc.), it’ll be faster in C. If you need it faster than that, you go to assembly. Prepare to have your world rocked: Venkatesh Srinivas found that strlen() in libc was actually slower written in assembly than in C. His commit message has numbers to back that up.
Being a good BSD neighbor
Adrian Chadd showed up on the DragonFly kernel@ mailing list, offering some help in keeping things compatible with FreeBSD and 802.11 networking. That’s quite neighborly of him, especially since his hands are already pretty full.
Fast soaccept added
It’s another throughput tweak from Sepherosa Ziehau: soaccept is run differently when pulling in network data from a socket. The commit message once again shows the results of the change using httperf.
Disk serial numbers and how to mess with them
Siju George asked about how he could figure out which serial number (in /dev/serno) maps to which disk. Tim Darby posted a script he used for it, or you can just use devattr(8). There’s also a linking trick described by Chris Turner to remember how the names map.
Binutils update to 2.22
Binutils in DragonFly is now up to version 2.22 – the commit linked is one of several.
fastbulk now added
Some time ago, Matthew Dillon worked on a bulk build system that built as much of pkgsrc in parallel as possible. It’s in the tree now as ‘fastbulk‘, for anyone wanting to try it out. I used it a bit; I didn’t measure the degree of speed increase, but was able to get about 70% of the packages built.
More network speed improvements as reported by netperf
Sepherosa Ziehau has implemented another networking speedup. Read the commit message for details on what he changed, since it’s rather in-depth. He shows an 18% improvement in netperf results.
Video and USB fix
Matthew Dillon has written a contiguous memory mapper, which is designed to fix problems with video cards and USB drives that need a big chunk of memory to keep. This can affect booting or later on, when disconnecting/reconnecting a USB drive. If this still doesn’t fix the problem for you, try adjusting the sysctl ‘vm.dma_reserved’ to something bigger, like 64M. It defaults to 16M.
(Normal mailarchive isn’t updating because of an ongoing upgrade to crater.dragonflybsd.org – sorry!)
What to do with /usr/obj
When building world and kernel on DragonFly, /usr/obj is where the work files get placed. This can eat a bit of space, but it can be safely deleted. If you keep the files around, subsequent rebuilds can be done faster with a quickwork/quickkernel, but this may not matter to you.
(This was answered on the mailing lists by Max Herrgaard, but I don’t have a link to his reply – sorry!)
Linux results for that Postgres benchmarking
Remember the Postgres benchmark I described here a few days ago? Francois Tigeot has updated it with numbers from Scientific Linux running the same pgbench procedure. (see page 2) If you’re too lazy to look at the PDF, his summary is this: Linux is fastest of all, and also crashes the most.
Sendfile speedup
Sepherosa Ziehau has implemented an asynchronous pru_send in sendfile. The results are a 70-90% increase in performance, as shown in his netperf localhost test.
New bug tracker
DragonFly now uses Redmine for bugs.dragonflybsd.org. This means that the bugs@ and submit@ lists have can still be read by anyone, but to post a new bug or patch, or reply, you need to be registered on the bug tracker itself. You don’t have to be subscribed to the mailing list to use the web interface. See the bugs@ and submit@ announcements for other details.
How to make those regression tests
The man page for dfregress has been put together, and you can read it and find out how to contribute, right now.
(That man page should be up by the time this is posted…)
New clang 3.0 and DragonFly work
Juan Francisco Cantero Hurtado has been working with clang and DragonFly, along with Sascha Wildner. DragonFly mostly compiles using clang, with lib/citrus being (the only? one of?) the last holdouts. Juan Francisco Cantero Hurtado detailed how to test it out using clang 3.0 in case someone else wants to help solve this.
route(8) changes require buildworld
If you’re tracking DragonFly current, you will need to do a full buildworld on your next update. Sepherosa Ziehau made some changes in route(8) that a quickworld will not catch.
Huge speed improvements, plus graphs
The two things that make my day! The work on DragonFly-current has led to some significant speed improvements. So good, that Samuel Greear’s post on OSNews.org links to graphed results from him and from Francois Tigeot (multi-page PDF) showing the results from pgbench.
The results show a jump in multi-core/processor numbers that vastly exceeds DragonFly 2.10′s performance, and is comparable to FreeBSD 9/10. Here’s some of what did it.
dfregress, a new test framework
Alex Hornung has created ‘dfregress’, a test framework designed to be as simple as possible for adding tests to DragonFly. This would make it easier to verify an upcoming release is correct, for instance. See his commit note for extensive details, and add a trivial test for anything you value.
Appletalk goes away
This is another one of those features that I bet goes away, and nobody would notice because nobody uses it any more. Sascha Wildner has removed AppleTalk from DragonFly.
Turning over a new leaf
The host leaf.dragonflybsd.org has been upgraded to new hardware. This is the machine used for anyone who wants to develop on DragonFly, so there’s a good performance boost there for developers. It also hosts bugs.dragonflybsd.org, which should be working again soon.
A new memory allocator
DragonFly has a new memory allocator, called (not surprisingly) “dmalloc“. It’s only present on x86_64, not i386, because it could eat up more VSZ (virtual memory) than an i386 kernel may have available.
HEADS UP: package recompilation needed
The presence of /usr/include/crypt.h in DragonFly (starting in December 2010) meant that some programs compiled during that time will expect that file to always be there. It was recently removed, so any programs compiled in that timeframe will also need to be recompiled. Right now, this affects you only if you are running DragonFly 2.13 , since that’s the only place crypt.h was removed. This may be an issue for the release, but we’ll worry about that when we get there… I’m kicking off new 2.13 bulk builds now.
New rpkgmanager page
There’s a new page up on the DragonFly website, about using rpkgmanager to manage your pkgsrc-installed packages.
Ways to eliminate C++
In DragonFly, there’s only a few places C++ is used. If you wanted to make sure DragonFly was pure C, Samuel Greear lists those remaining nooks and crannies.
Do you have a lot of RAM? I mean, a LOT!?
You can now have, in theory, up to 32 terabytes of RAM on your 64-bit DragonFly system, from a change made by Matthew Dillon. I’m curious to see if anyone has even 1 terabyte, as that’s at least feasible.