Rados?aw Szymczyszyn has manged to get support for DragonFly’s bootloader into GRUB. This is part of his Master’s project to make DragonFly multiboot capable, at least for i386.
(I love having new things show up from new people, out of the blue.)
Rados?aw Szymczyszyn has manged to get support for DragonFly’s bootloader into GRUB. This is part of his Master’s project to make DragonFly multiboot capable, at least for i386.
(I love having new things show up from new people, out of the blue.)
Loïc BLOT posted about his benchmark of several operating systems using KVM and Postgres 9.1. Happily, DragonFly is the fastest, with one exception. Linux/ext4 comes out faster – if you run it with barrier=0, which can be dangerous in a non-battery-backed-up volume.
John Marino managed to update GCC from 4.7.2 to 4.7.3 (4.7 changelog), zlib from 1.2.7 to 1.2.8 (changelog), and awk from 20110810 to 20121220 (can’t find a changelog).
In other update news, Matt Dillon has been working on HAMMER2′s flush sequencing.
Update: tcsh too.
I’ve put the 3.4 release images up on terasaur, a Bittorrent seeding site. Please try pulling them and let me know how it goes. I haven’t torrented many things, so I am unsure how to even verbify “torrent’. Hopefully that sentence and those links work out.
If you’re looking to install DragonFly on a Kimsufi server, and you can read French, this explanation may help you. (via Enjolras on EFNet #dragonflybsd)
Have you ever wondered about how the booting process works on DragonFly? Well, Ivan Uemlianin did, out loud. Several different recommendations followed, so now you can learn too.
‘william opensource4you’ posted a summary of the steps he took for setting up a DragonFly system with XFCE4, using dports. It’s pretty straightforward, and thanks to dport’s binary nature, should be exactly reproducible.
As I described in a post to the kernel@ mailing list, the DragonFly 3.4 images are getting uploaded for mirroring and downloaded for testing. Assuming no surprises happen, we will be able to release very soon.
If you administer one of the DragonFly mirrors, there’s a new /dports directory that can be mirrored. See that second link for details.
John Marino has a concise explanation of why dports mostly uses gcc 4.4 still to compile, even if you’re building DragonFly itself with the default 4.7. It’s a reason to not use NO_GCC44 – yet.
Eric Radman sent along a plug for a utility he is working on called entr(1). The desciption is “Run arbitrary commands when files change.” The site for it has several nifty examples – run make when *.c files change, or convert Markdown files to HTML as soon as they are modified. The really nice thing about it is that it’s perfectly BSD-friendly, and uses kqueue, but will also work on Linux. This beats the “This runs on the one flavor of Linux I use, in one particular shell!” approach I’ve seen from some other developers. See the reddit discussion of it for comparisons to inotify. No, it’s not in pkgsrc/ports yet.
Update: And thanks to Thomas Klausner, it’s in pkgsrc as sysutils/entr, and in ports as sysutils/entr thanks to Eitan Adler. You have no reason not to try it now.
NASA’s International Space Apps Challenge is this weekend, 4/20/2013. Fancy as it sounds, it’s really a single-day hackathon around open software and hardware, with the problems to fix coming from NASA and therefore probably very unique. It’s happening in a bunch of places around the world, but there’s one right here in my town.
Peter Hansteen has an extensive writeup of how he has managed the bsdly.net spam blacklists. Normally I’d stick this article in the Lazy Reading links, but the article is good enough to call out separately. It’s excellent not just for the mechanical aspects of how the blacklists were maintained, but for his strict description on how the process is simple, verifiable, and transparent. That last item, transparency, is how many anti-spam groups fall down.
Here’s a status report on the 3.4 release, pulled right from my mailing list post:
I’d like to have the release available with binary packages for dports immediately, because I anticipate a number of people wanting to try it out. So, the release will be delayed a few days while the packages upload.
The very first copy of Absolute OpenBSD (2nd edition), signed by Michael W. Lucas, is being auctioned off in a charity event for OpenBSD. There’s 5 days left to bid, though the price is already somewhere north of $2 per page.
The DragonFly page on the Summer of Code site is set up. If you are a potential mentor that I’ve talked to before, I’ve already sent you an email with details. If you are a potential mentor I haven’t talked to, you can email me or send a request via the DragonFly page. (Google has a new ‘connections’ method for signup this year.)
If you’re an interested student, take a look at the DragonFly Projects Page. Keep in mind that your proposal does not have to be one of those ideas – new projects are always welcome, and often have the advantage of being unique instead of being one of several similar proposals. (hint, hint)
Constantine Aleksandrovich Murenin has put together a new site, bxr.su. His announcement to users@ goes into a lot of detail, but here’s a preview: it’s an OpenGrok site that has a forked version of OpenGrok that’s both speedy and takes BSD into account, along with other nice features.
Here’s the catch: it’s currently IPv6 only. IPv4 will be on as a test just today, and on for good shortly after. Read that announcement I mentioned for details.
If you have a DragonFly 3.2 system and you want to try the 3.4 release candidate, you can delete your local source, edit the Makefile to pull down 3.4 instead of 3.2, and run it.
cd /usr
rm -rf src
vi /usr/Makefile;
(in vi) :%s/DragonFly_RELEASE_3_2/DragonFly_RELEASE_3_4/g
(save, quit vi)
make src-create-shallow
… then proceed to make buildworld and so on, as normal.
The caveats: I haven’t tested this yet, and this assumes you don’t have any local changes in /usr/src that you want to save. The usual warnings about lighting your computer on fire, etc., apply.
The DragonFly Git repository of pkgsrc now has the 2013Q1 branch. You can switch to it by editing your /usr/Makefile (look for existing references to either pkgsrc master or pkgsrc-2012Q3) and using the normal commands.
DragonFly 3.4 is branched – as a release candidate, with the current target for 3.4.0 release as the weekend of April 13-14. See the tagging commit note for a list of all the commit messages.
Note that in previous releases, we tagged “x.y.0″ on branch, and “x.y.1″ on release. I’m now tagging “x.y.0rc” for the release candidate at branch time, and we’ll tag with a more normal (to my ears) “x.y.0″ for the release.
If you build a 3.4.0rc image right now, you’ll get an older quarterly release of pkgsrc. That’ll be changed tomorrow as the DragonFly pkgsrc git source is updated and I change where 3.4′s /usr/Makefile points.
The 2013Q1 branch of pkgsrc has been announced. Along with the normal quarterly material, there’s several notes: preliminary Cygwin support is present, ruby 1.8 will be retired in favor of 1.9 after this release, and the pkgsrc.org web page now has a very nice new look and logo.
I plan to branch DragonFly 3.4 very soon, and that version will have 2013Q1 as default.
Update: The 2013Q1 branch should be available by tomorrow on DragonFly’s git; the repository needs to update and convert from NetBSD’s CVS and that takes a little time. I’ll post when it’s ready.
If you were thinking, “Hey, I’d like to try an early version of DragonFly 3.4 before it’s released”, I’ll just point you at the recent daily snapshots of 3.3. These are close enough to a release candidate, I think.
The next release of DragonFly will be 3.4, and it’s probably going to be mid-April.
Hey, look, DragonFly BSD showing in tweetspam! Don’t bother following the tweeted links; they don’t have anything useful. It’s entertaining to see the structure and coding of these bots; they’re no horse_ebooks, of course.
As the title says, if you install MySQL from pkgsrc-current, you’ll now get version 5.5.
Michael W. Lucas is looking for someone to improve the Extended DNSSEC Validator. Specifically, add BSD support. It’s an idea worth supporting, because the standard it works with makes self-signed certificated perfectly feasible.
If you want to take advantage of the binary packages of DPorts, and have a x86_64 system with a recent DragonFly 3.3 on it: Francois Tigeot has you covered. There’s no i386 packages yet, which are the ones I could use right now, darnit.
If you want to try DPorts, see my earlier article.
The new vm.read_shortcut option has been turned on by default by Matthew Dillon, which should lead to some performance improvements. That improvement has been measured for tmpfs, at least. There’s also some buffer cache improvments that help on x86_64 systems, too.
Update: As Venkatesh Srinivas pointed out, tmpfs also no longer uses the mplock, so it’ll take better advantage of multiple processors.
Thanks to Antonio Huete Jimenez, it’s now possible to set the MAC address for each interface and specify the disk serial number in the command line for a vkernel.
The fine folks at the New York City BSD User Group have created a mailing list specifically for using The Onion Router on BSD. Please join if you are interested in TOR, and especially if you are using something other than FreeBSD, since that’s the only ‘supported’ BSD TOR runs on right now.
There’s two changes in pkgsrc recently that might affect you: graphics/png was updated, so many dependent packages will require recompilation. Also, editors/emacs was moved to a general package instead of being specifically named by version, so now you can install ‘emacs’ instead of ‘emacs24′ or whichever version.
I have a pf question for anyone who is interested. I have this setup in my /etc/pf.conf, to prioritize my VoIP link. (this system also does NAT.)
extif="em0" intif="nfe0" ipphone = "192.168.0.101"
altq on $extif cbq bandwidth 768Kb queue { std, voip }
queue voip bandwidth 168Kb priority 7 cbq(borrow)
queue std bandwidth 600Kb priority 1 cbq(default)
nat on $extif from $intif:network to any -> ($extif)
pass in quick on $intif proto udp from $ipphone to any tag VOIP_OUT keep state
pass in on $intif from $intif:network to any keep state
pass out on $intif from any to $intif:network keep state
pass out on $extif tagged VOIP_OUT keep state queue(voip)
pass out on $extif inet proto tcp all modulate state flags S/SA queue(std)
pass out on $extif inet proto { udp, icmp, gre } all keep state
When I run this, ‘pfctl -s queue’ shows most of the data getting run through the ‘voip’ queue. I unplug the ATA, I still see the number of packets going up. It seems packets are getting tagged that shouldn’t be, but I’m not sure why. Anyone else have a similar – but working – setup?
Update: it was the underscore character in the tag. Everything matched it, it seems. Removing that made it work as expected.
If you’re near Germany, or like IPv6, the Schlund Technologies mirror for DragonFly is for you – it supports HTTP, FTP, and rsync.
The machines at dragonflybsd.org are now on a different part of the Internet, so if you were having problems connecting over the past few days, it should be better now. Matthew Dillon wrote up details of what he changed and why he changed it, including a note about future blade server plans.
Matthew Dillon is moving dragonflybsd.org’s network link to a new VPN today. (It may have already happened; I only just read the email.) This may help the people that have reported their network path to dragonflybsd.org seems to die somewhere in the Cogent network…
Or is it ‘statii’? English is wonderfully inconsistent. Anyway, Michael W. Lucas has posted an update on his two upcoming publications: the second edition of Absolute OpenBSD and DNSSEC Mastery. Both are in progress, and you can download the ‘pre-release’ version of DNSSEC Mastery now.
It was planned some time ago, but versions of Samba older than 3.5 are now out of pkgsrc, and version 3.5 will hopefully be replaced by 4.0 soon. Ruby 3.0 and 3.1 will also be going soon.
Hubert Feyrer wrote a review of Ansible 0.9, a management tool for multiple systems, similar to Puppet or maybe Chef. Just after doing that, Ansible 1.0 came out, with support for pkgsrc via pkgin-installed packages. This is the first solution (that I know of) that supports pkgsrc package management for multiple systems.
Markus Pfeiffer reports success using Xen HVM to run DragonFly, which may be useful for any of you Xen users. He reports not being able to use more than 2 virtual CPUs, though Scott Tincman reports successfully using 4 (with qemu), so your mileage may vary.
Updated: noting qemu usage as Markus pointed out in comments.
If you’ve been feeling the need for reading about filesystems, Daniel Phillips has posted more notes about his Tux3 filesystem design, which can be contrasted with HAMMER. (thanks, Venkatesh Srinivas)
I meant to post this a while ago; it’s a few days old but still useful. John Marino gave some stats on DPorts progress, plus he and Francois Tigeot also had some tips on xorg setup. The successful build count is higher by now, and I think KDE3 is done, though I haven’t tried it.
If you have a BSD Certification, and it’s nearing the end of its 5-year term, the BSD Certification Group has published the guidelines for re-certification. Has it really been 5 years since the first certifications happened? Geez.
I found this off of the NYCBUG mailing list, so hat tip to them.
It’s a very short week this week. I was on the road for work, so I didn’t see anywhere as much of the Internet as I may have liked. Count my dports writeup yesterday as part of this and it averages out to a good amount of reading.
Your unrelated link of the week: New Tokyo Ondo. via Jesse Moynihan, whose Forming comic on that site is an epic read. Epic, as in it’s actually telling a NSFW world creation story.
John Marino’s DPorts project, mentioned here briefly before, is interesting. I had two separate people ask me how it works, so a better explanation is in order. I’ve tried it out on a test machine over the past few weeks.
Background:
Dports is an effort to use FreeBSD’s ports system as a base for DragonFly, and the pkg tool as a way to manage binary packages built from DPorts. This is complicated, so I’ll explain each part in order.
Every port seen in DPorts is known to build on DragonFly. John Marino adds a port only after it builds successfully, using poudriere as a bulk software tool. Ports are only updated to a newer version when that newer version builds, too, so once something arrives in DPorts, it should never break from being updated at some point in the future.
Installing:
To use DPorts, you need two things:
(Don’t confuse pkg, the management tool, with /usr/pkg, the normal installation directory for pkgsrc. ) For the installation of the base port files:
cd /usr make dports-create-shallow
If you’ve already renamed your /usr/pkg directory, git won’t be in your path any more. You can instead download a tarball and unpack it, which also happens to be possible automatically via that same Makefile.
cd /usr make dports-download
Downloading via git is fastest, so if you do need to use the tarball via make dports-download, build devel/git, delete /usr/dports, and then pull it again with make dports-create-shallow. This all comes from John Marino’s Github site for DPorts.
Managing DPorts
DPorts doesn’t use pkg_info, pkg_add, and the other tools traditionally seen on DragonFly for pkgsrc. Instead, package management is done with pkg. Use pkg info, pkg install, pkg remove, and pkg update to list, install, delete, and upgrade various packages on your system. Packages built from source or downloaded as prebuilt binaries are managed the same way, using these tools.
See some of the other writing about pkg for FreeBSD for details on how it works.
Since DPorts doesn’t update a package until it gets a successful build, and installations are of successfully built binary packages, upgrades with prebuilt packages should always succeed. Since they’re binary, they should be fast. There’s a lot of ‘shoulds’ in this sentence, but these are reasonable suppositions.
What about pkgsrc?
Pkgsrc and DPorts shouldn’t be used at the same time, since one system’s packages may be at different versions but still get picked up during building for the other system. That’s about it for restrictions.
I intend to try building an experimental release of DragonFly with DPorts, to see if all the right packages can be added, but no guarantees. DPorts is brand new and does not yet have a repository for downloading packages, so the normal caveats apply; don’t install it on a mission-critical machine, and be ready to deal with any surprises from using it if you do try it out.
What packages are available?
Browsing the Github repo will show you all listed packages. More complex packages like xorg, openjdk7, and libreoffice install, as does xfce. Parts of KDE 3 and KDE 4 are in there. (I haven’t tried either.) I’m not sure about Gnome, but I don’t think anyone ever is. There’s no vim, but there is emacs.
That’s just what I see at this exact minute. It changes daily as more packages are built. Changes from DragonFly builds are sometimes relevant to the original FreeBSD port, so there’s benefits for everyone here.
What next?
Try it now if it has all the packages you need, or wait for a binary repository to be created to speed things up. Remember, this is a new project, so a willingness to deal with problems and contribute to fixes is necessary.
It’s actually been out since the start of January, but the release announcement is available now.
Stéphane Russell, on the users@ mailing list, pointed out an in-depth article about DragonFly’s 3.2 release, on linuxfr.org. It’s in French, which means I’m just going to have to trust his word about the contents.
Ishan Thilina asked for some project ideas, and Samuel Greear gave a list of links that may be useful for anyone looking for a project of their own. I offered strategy. It didn’t work out, but this information’s still useful.
The Open Graphics Project, which is building a completely open video card, needs a wiki maintainer. It’s a volunteer effort. If you were perhaps thinking you wanted to step up to a more complex project but didn’t want to just be writing code, here is a perfect opportunity.
(Not too different from maintaining a project work blog, after all, and I know that’s rewarding.)
There’s a short thread running on the DragonFly users@ list about disk encryption; there’s some descriptions of encryption work there for the curious.
I could have sworn I noted it before, but as Venkatesh Srinivas points out, there’s a port of cpdup to Linux. Also, if you’re using cpdup to copy material out of a Hammer volume’s history, use the -VV switch.
DragonFly 3.2.2 has been tagged. The tag commit has a list of the fixes; this is a bugfix release, but it’s a good one. Download an ISO (they should be at the mirrors by now) or update your system.
There’s been a large number of fixes and improvements to DragonFly 3.2 lately, so I’m planning to roll DragonFly 3.2.2 this weekend so there’s an image with them all.
If you were thinking you wanted to try gcc 4.7 with pkgsrc, John Marino’s described the option you need to set. It only works in pkgsrc-master right now (because of changes John made), and not every package in pkgsrc will build.
The advantage is that it’s also possible, with the same syntax, to set pkgsrc to build with gcc 4.4. This means the default compiler in DragonFly can be changed to gcc 4.7 and pkgsrc packages that aren’t compatible can still be built.
Update: Check this minor change: ‘?=’ instead of ‘=’.
Whomever submitted this story to Slashdot really doesn’t like FreeBSD; they’re describing FreeBSD’s annual end-of-year fund drive as failed. The month-long drive is only about a week old and has already picked up donations at a faster rate than any previous year’s donation drive, but apparently the poster – and Slashdot’s editors – can’t be bothered to do math. While we’re on the topic, donate to the FreeBSD Foundation; they do good things.
(There’s DragonFly too, though we’re not as ambitious or officially 501(c)(3) non-profit.)
Matthew Dillon has written up another update on his progress with HAMMER2. (I need to be consistent in how I write that.) He has disks being exported and mounted on other systems, and adds an explanation of some of the issues around creating reliable multi-master setups. Before you get too excited, no, multi-master isn’t working yet, and this is not production ready.
There’s more benchmarks for DragonFly vs. other systems on Phoronix. It has the same problem as previous benchmarks; some of the benchmarks may have no connection to reality (what does the “Himeno Poisson Pressure Solver” actually test?), and almost every system has a different version of the gcc compiler. So it’s meaningless in terms of comparative or absolute performance. On the other hand, DragonFly doesn’t do badly.
You can also look at the comments to see someone absolutely freak out over the very existence of things that aren’t Linux. I’m not sure if it’s actually trolling, since the comments are so exactly wrong.
Shopping! This is the big holiday shopping weekend in the US, and I usually put together something here.
If you have suggestions, please comment!
Because of the recent good results for pgbench on DragonFly 3.2, Phoronix has a new benchmark of DragonFly using other (possibly unrelated) tests. There’s not a lot of information to glean from them; they are testing operations different than what was optimized for pgbench in 3.2. I’d like to see DragonFly 3.0 tested the same way to see how much improvement there was between versions.
We (as in DragonFly) are not participating in Google Code-In this year, but I’m happy to see there’s another BSD in there - NetBSD. (There’s only 10 participating organizations, so it’s not easy.) Look at their page if you’re in the right age range to do projects.
MaheshaDragonFlyBSD, a ‘liveUSB’ distribution of DragonFly with software preinstalled, has been updated to run using DragonFly 3.2.1 as a base. The linked page contains screenshots and a description of what comes out-of-the-box. (mentioned previously here.)
Today is the day that FreeBSD moves to using clang by default. This is not necessarily a surprise, but I like the finality of calling it “Clang-Day”. I think Clang will probably be the next compiler brought into DragonFly’s base system, instead of the next release of gcc. Don’t make any bets on my statement, though, cause I certainly won’t be the one doing it. (It’s hard.)
There was one more file to change for the bmake import, so if you are running DragonFly 3.3 and updated between the 28th and 30th of October, do a full rebuild.
I mentioned this before in the Lazy Reading from last Sunday, but it’s worth a second look: Apple’s new Fusion Drive product appears to be very much like DragonFly’s swapcache. DragonFly doesn’t have exclusive right to the idea of caching on a faster disk, clearly, so I’m not complaining that it’s “ours”. It’s frustrating to see product announcement/press releases stumbling all over this like it’s a new thing.
Then again, having new ideas about technology ideas and making sure they spread is one of the points of the BSD license, so perhaps there’s no good reason to complain at all.
(Before anyone reads too much into this: No, I don’t know of any direct relationship between swapcache and Fusion Drive; they may have no common background other than structure.)
A thread on pkgsrc-users@ reminds me: adding a specific line for bin-install will save time when rebuilding packages; pkgsrc will use existing binary packages instead of rebuilding from source when possible, when this is set. At least, I’m pretty sure that’s what it does.
Sandip Jadhav asked if anyone was working on an I/O scheduler. Chris Turner replied with a “no”, but also with a list of places to look for details on writing one, which I’m linking here for posterity.
John Marino is working on a very good idea: bringing bmake into DragonFly as a replacement for the current ‘make’. bmake is going through more active development and apparently also in use/will be used? on FreeBSD, so syncing up with the same make flavor as FreeBSD and NetBSD will help everyone. It’ll also remove the problem where you ‘make’ everything in DragonFly, except pkgsrc packages which you ‘bmake’. It’s not changed over yet.
(What does OpenBSD use for make?)
A conversation about compilers in the DragonFly base system led peeter (must) to describe his group’s use of OpenMPI on DragonFly for physics calculations. Apparently he’s had a significant performance improvement on DragonFly.
Along similar lines, John Marino helped out by bringing in libssp and libgomp for gcc 4.7 for use with OpenMP. (This is in DragonFly 3.3, not 3.2).
John Marino did a bulk build of pkgsrc using gcc 4.7.2, and posted the results. The result? About 1% of packages that built with gcc 4.4 did not build with 4.7.2. Whether that’s a problem with gcc or a problem with how each of those software packages were created by the original authors, I don’t know.
Google is hosting a ‘Doc Camp’, where people get together and write documentation for open source projects. There’s a page that talks about it. Last year’s inaugural event was apparently quite successful. I haven’t been to it, but I think a day just for documentation is a good idea.
I’m planning for DragonFly 3.2 to come with pkgsrc-2012Q3, the most recent release. I’m building binary packages to match, and the build should complete by the time we release on the 22nd…
Notice I said “should” – sometimes the universe conspires against bulk builds.
I branched 3.2 tonight. That means 2 weeks until release, so sharpen your bug-poking sticks!
(I’m very tired and unable to think of good analogies, sorry.)
Cause it could be added. The new algorithm could replace SHA-2, in use now in DragonFly. SHA-2 has not been ‘broken’ yet, so it’s not an emergency… yet.
I recreated the by-month thread and date listing from the old mailing lists, but for Mailman. It’s at lists.dragonflybsd.org.
Since the most recent branch of pkgsrc has been released, perl5 in pkgsrc has been updated to 5.16.1, and (ancient) python 2.5 has been removed.
Debian squished with DragonFly, sorta like Debian/kFreeBSD? Don’t know if it will work, but what the heck.
As I typed elsewhere, my general plan is to branch DragonFly 3.2 on the 8th, and release on the 22nd. That should give the recent scheduler and gcc work a chance to settle, and perhaps get a new version of USB support in too. It will probably be using pkgsrc-2012Q3, also, though we may not have binary i386 packages. 3.2 is shaping up to be a much more significant release than I expected.
The machine that runs www.dragonflybsd.org and bugs.dragonflybsd.org is currently down. While it gets figured out, Alex Hornung has a static copy of the dragonflybsd.org main website available.
Sepherosa Ziehau has some suggestions for anyone looking for some kernel hacking. They’re mostly based around busdma(9).
dragonflybsd.org appears to be down right now so I’m linking to the MARC kernel@ post.