This latest commit for the new scheduler means that on your next update, you will want to build a new kernel, and probably a new world too. This only applies if you’re running DragonFly 3.1, of course.
Category: Goings-on
Mailing list archives updated back to 2003
I got the old mailing list archives converted to Mailman. As I wrote in a post to users@, please let me know about problems. There’s some garbled messages from the old archive that were placed into the 2012-Sept. section for each message; I’ll be cleaning those up manually.
Posting but not reading mailing lists
The old mailing list software for @dragonflybsd.org mailing lists, bestserv, apparently allowed people not subscribed to a list to post to it, after answering a confirmation message for each message posted.
The closest way to duplicate that for Mailman is to sign up for the list you want, and then turn off mail delivery for your email address in the config page for that mailing list. This won’t affect a lot of people, since most people want list output in their mailbox, but there’s at least a few I’ve fixed that way.
A bikeshed and a code change
A discussion of why root automatically lists dotfiles with ls and all other users do not led to a long thread that includes some UNIX history. There’s some useful and some not-so-useful parts in the thread, but it did indirectly produce a way to reverse the listing effect itself.
SYSV shared memory vs. mmap
Francois Tigeot benchmarked the recent Postgres 9.3 release. Postgres apparently switched to using mmap instead of SYSV shared memory, and Francois has done this to show the performance differences. (view the PDF in his post.) Of course, work has continued since this was posted, so there should be new numbers soon, and new changes I’ll document in a future post.
I haven’t found a reference to the exact decision Postgres made on how to handle memory; please post a link in comments if you know a good source.
Pkgsrc freeze has started
See the note on pkgsrc-users@. The next quarterly release, pkgsrc-2012Q3, should be fully baked by the end of the month, if all goes well.
pkgsrc freeze for 2012Q3 starts tomorrow
As seen in this pkgsrc-users@ post from Thomas Klausner, the freeze for pkgsrc-2012Q3 starts on Sunday and continues for (probably) two weeks before the release.
NYCBUG, RSS, and SMPng
NYCBUG, the NY BSD user’s group, has an RSS feed for their speaker events, found via Dru Lavigne’s always useful BSD Events twitter. The next event at the start of October is a talk about SMPng in FreeBSD. Given that it was the project that in part led to the creation of DragonFly, I’d like to hear about it. (and even better, have someone more qualified than I compare and contrast that approach with what’s in DragonFly.)
A potential new pkgsrc site
If you look at new.pkgsrc.org, you will see what may become a new site. This is apparently a test, so don’t react as if this was the actual site.
A lot of scheduler talk
If you ever wanted to read an extensive discussion about the scheduler, today’s your day. Mihai Carabas, who posted the details of a long discussion he had with Matthew Dillon about how the scheduler works. You may recall Mihai’s name from the very successful GSoC scheduler project that recently finished.
(look, a link to the new Mailman archive!)
Mailman conversion for DragonFly mailing lists complete
All the mailing lists at @dragonflybsd.org have been converted over to Mailman. The old archives are still functioning, and will continue to update until I can find enough old material to retroactively complete the Mailman archives.
Mailman conversion for dragonflybsd.org mailing lists
If you’re on any of the dragonflybsd.org mailing lists, I’m converting them over from bestserv to Mailman. I’ve done bugs@, commits@, hammer@, and test@ so far, and I’ll move the old archives over to the same format as soon as I find an actual mbox file with the old messages in it. The remaining lists should be tomorrow.
(If you got a note tonight from a list you were sure you were unsubscribed from, that was my fault; sorry! I didn’t understand the format of the bestserv user lists.)
A whole bunch of installation notes
DragonFly user varialus has created a page on the DragonFly website (it’s a wiki, after all) with all the notes taken from trying installation, etc. There’s far more notes than I expected there, so it’s worth a read.
3.0.3 images up
I’ve uploaded DragonFly 3.0.3 disk images, both ISO and IMG. They should start appearing on a mirror site near you in the next 24 hours. This took a while after the tagging, I know, but I wanted to make sure every one of them booted. I didn’t on a previous release, and regretted it.
Postgres benchmarking again
Francois Tigeot benchmarked several different operating systems using Postgres 9.2b3, including DragonFly, and published the results. I have a local copy of the PDF since the attachment didn’t really survive the archiving. Follow the thread for discussion. The Linux results look abnormally high, so it is possible that something different is happening on that platform…
DragonFly and GSoC 2012 wrapup
DragonFly had a successful Google Summer of Code even this year. It marks our 5th time participating, 7th if you count Google Code-In events.
Mihai Carabas worked on adding SMT/HT awareness to the DragonFly scheduler. This project was very successful. The original goal was just to take advantage of threading with the scheduler, but the benchmarks published by Mihai show in general a 5% speedup from these scheduler changes. His work has already been committed.
Vishesh Yadav implemented an inotify interface in DragonFly. inotify is an originally Linux-based system for monitoring files and directories for changes. A specific use for this is an inotify-aware locate program, so that a list of file locations can be kept ‘live’. His code for the inotify interface should be committed to DragonFly very soon.
(This was written in part for Google to use on their Open Source Blog.)
3.0.3 tagged
I’m working on building new images, but: DragonFly 3.0.3 has been tagged. If you’re running 3.0, you can update and get some of the recent bug fixes.
Planning for the next release
3.2 is the next major release of DragonFly, which will be relatively soon by the every-6-months release schedule. John Marino’s put together another catch-all bug report for that release.
Another LiveDVD image
Sascha Wildner’s been working on his own DragonFly live images, in DVD or USB form. It uses XFCE along with a number of other packages listed in his post. They are .xz compressed, so they are nice and small for download, but make sure you have something that knows that format.
GSoC week 9 reports
Mihai Carabas has posted his weekly results, showing a 5% improvement in pgbench resultswhen using his scheduler. Vishesh Yadav is working on IN_MOVED_TO/IN_MOVED_FROM flags (part of inotify, I assume). Ivan Sichmann Freitas I haven’t heard from yet. (Ivan, where are you?)
MaheshaDragonFlyBSD now available
Juraj Sipos wrote me to describe MaheshaDragonFlyBSD, a live DragonFly image that has additional software preinstalled, and can easily be set to understand Sanskrit. It’s available in DVD and USB versions.
Summer of Code status, week 7
Here’s the regular status updates for Mihai Carabas (scheduler) and Vishesh Yadav (inotify). I don’t have the update from Ivan Sichmann Freitas yet. Here’s Ivan Sichmann Freitas.
Here’s a way to donate
If you want to put something towards DragonFly, and you don’t have time or hardware, cash is now an option. (It’s not tax-deductible.)
GSoC updates, week 6
The usual weekly updates from Mihai Carabas, Vishesh Yadav, and Ivan Sichmann Freitas. Mihai has some interesting bugs found this past week by running his code on Matt Dillon’s 48-core system.
pkgsrc-2012Q2 released
The release announcement for pkgsrc-2012Q2 is out. New in this quarterly release: statistics about clang and pkgsrc. A surprisingly large number of packages build just fine with clang instead of gcc.
Midterms coming up for GSoC
Attention students and mentors: the Summer of Code midterms open up on July 9th. This means students fill out an evaluation, and mentors also fill out an evaluation. Don’t forget, because completed evals from mentor and student both are necessary for a project to continue being funded.
More benchmarking
More benchmarks, in this case a comparison of Debian GNU/kFreeBSD and DragonFly. I’m not even sure how to derive meaning from it.
Lazy Reading for 2012/06/24
It’s almost an all-Vim week.
- Unix as IDE, a 6 part completed series. (via)
- VimOrganizer, Emac’s Org-mode for Vim. (via)
- The Vim Clutch. This is hilarious. (via) Also see “Chindogu“, though this might be too useful.
- Those who know UNIX are doomed to reinvent it. (via same place)
- Falsehoods programmers believe about time. Taken from the same thing about names. (via) Also, when someone says, “Oh, I can just look for email addresses using a regular expression”, you’re stepping over a similar cliff.
- Welcome image in your terminal. (via) Will this work in an xterm? rxvt? I don’t know…
- CanI make this Lazy Reading post more nerdy? Yes! Arbitrage and Equilibrium in the Team Fortress 2 Economy. I already enjoy economics writing, and this one is about virtual economies and games. The only thing I know of close to this is the economist that works on EVE Online.
Your unrelated link of the week: Muppet Bohemian Rhapsody. Related: What kind of Muppet are you?
Scheduler changes you can try yourself
If you have an Intel processor with multiple cores and hyperthreading support, you can compile a new kernel and try out Mihia Carabas’s GSoC work already; he’s created a test using the OpenSSL test case to time scheduling performance vs. number of threads.
pkgsrc now frozen, with announcement
I know I already posted that this was on the way, but this time, the quarterly pkgsrc freeze is starting with a detailed announcement. 2 weeks until the next release, if everything goes well.
PHP 5.4 in, PHP 5.2 out
If you’re using some PHP application that requires the old behavior of PHP 5.2, you will need to specify that version of PHP – pkgsrc is moving to version 5.4 5.3 as default, with version 5.4 available. (thanks, Takahiro Kambe for the update.)
pkgsrc freeze for 2012Q2 starts on the 16th
The freeze for pkgsrc-2012Q2 starts on the 16th of June, as recently announced. Freezes are usually 2 weeks, so that means 2012Q2 should be tagged at the end of June.
Multi-architecture pkgsrc packages
Pkgsrc already runs on a large number of different platforms, but that’s not what I’m talking about. In this case, Joyent, which uses pkgsrc internally, has a suggested change that makes binaries usable on both 32 and 64 bit systems. I don’t know if this will go into pkgsrc proper, but it’s interesting to see.
Xorg updates in pkgsrc
Apparently a lot of modular-xorg packages in pkgsrc received updates. I think I found some of the changes, but probably not all, so I don’t have a good way to sum up the actual effect.
Update: see the end of this cvsweb pkgsrc CHANGES-2012 page for all the changed parts.
GNU utilities, correctly named
There’s a number of packages out there that assume you are using the GNU versions of ls, wc, and so on. However, you aren’t when using a BSD system. Pkgsrc has historically dealt with this when GNU tools are needed for a package by prefixing them with a ‘g’. ’ls’ becomes ‘gls’, and so on. Aleksey Cheusov proposed a fix to keep these utilities under their original names, which I think will go into the next quarterly pkgsrc release.
Pkgsrc removal policy set out
Pkgsrc packages that have source files that can’t be redistributed, and go missing for the length of an entire quarterly release, will get removed. They are effectively broken at that point anyway.
That policy is now formally in place; I don’t think there was a clear prescription before.
tmpfs and how fast it can go
Venkatesh Srinivas, currently on his colossal bike ride, introduced a different way of creating a tmpfs. This was test code, and Johannes Hofmann benchmarked it (see same page). It’s interesting cause there are numbers, and nice to see one person jumping in to test someone else’s results/idea.
Building pkgsrc with clang
I think I’ve mentioned building DragonFly with clang before, but not pkgsrc. There’s two variables to set, plus some special handling for libf2c. Thomas Klausner has details. This is not tested on DragonFly.
Phoronix benchmarks revisited
There were some benchmarks of DragonFly 3.0 some time ago on Phoronix. (You may recall it being mentioned here previously.) The disk numbers always seemed weird to me, so I repeated that part of the test, and here’s my writeup.
PHP 5.4 status in pkgsrc
Takahiro Kambe is bringing PHP 5.4 into pkgsrc, probably as lang/php54. Follow the whole thread for a discussion of version numbering. As a side effect of this, PHP 5.2 will leave pkgsrc by the next quarterly pkgsrc release. If you’re using that older flavor, you’ll want to upgrade.
TUI mode added to kgdb
TUI mode is available now for kgdb on DragonFly, thanks to John Marino. It’s apparently a Text User Interface for debugging core files. I haven’t used it, so I’m relying on the testimony of others.
Upgrading to pkgsrc-2012Q1
Here’s a post by yours truly, on how to move to pkgsrc-2012Q1 though building from source. This is for anyone sick of waiting for me to finish the binary build of pkgsrc.
Followup on clustering
Matthew Dillon posted a followup on that fix for clustering I noted yesterday. It describes the exact problems better than I could, though the result is the same: you should update if you’re running bleeding-edge DragonFly.
Update for a clusterfix
A fix for cluster_write() issues reported by multiple people is now available, so if you’re running a version of DragonFly newer than 3.0.2, you’ll want to update.
OpenSSL 1.0.1b updated
Peter Avalos has updated OpenSSL, though this version is apparently a bugfix, not a security fix. Still need it anyway, since it disabled TLS 1.1 in an unexpected way. See the OpenSSL changelog entry at “[26 Apr 2012]” for details.
Google Summer of Code: the projects
Each of the 4 DragonFly participants for Summer of Code have posted an introductory email and details of their projects. Here’s direct links to their posts for your reading convenience:
- Vishesh Yadav - Implement inotify interface and Indexing Service for Filesystem
- Mihai Carabas - Add SMT/HT awareness to DragonFlyBSD scheduler
- Loganaden Velvindron - Privilege Separation in DragonflyBSD
- Ivan Sichmann Freitas - 32 bit API for 64 bit kernels
(Yes, same format as my last post, but now the links are to their posts, not the sparse Google info pages.)
Packages that will go, and packages that might go
There’s a few pkgsrc packages that might be going the way of the dodo, soon. There’s a few more that need love, so speak up if you use them. Maybe you can be the Somebody™ that fixes them?
New committer: Markus Pfeiffer
Welcome our newest committer: Markus Pfeiffer. He’s ‘profmakx’ on EFNet #dragonfly, and has been working on a port of FreeBSD’s USB infrastructure – which I am looking forward to, tremendously.
Mosh for DragonFly developers
Mosh, mentioned on this Digest a few weeks back, is now installed on leaf.dragonflybsd.org. If you’re doing any development work there but dealing with a relatively high latency, this should help. (Thanks Venkatesh Srinivas.)
Where are the pkgsrc-2012Q1 binary packages for DragonFly?
I’m still working on building them. I kept getting panics, which seem to be fixed by this commit, so I should have something soon. Sorry!
Cheap SSH Mastery
Michael Lucas’s worthwhile book, SSH Mastery, is currently having one of those sudden price cuts on Amazon – for the paperback version, about 25%. Now it a good time to nab it before the price bounces back up.
DragonFly time
DragonFly now has its own ntp.org zone. What’s this mean? Nothing material, but it’s nice to do.
pkginteractive: graphical pkgin
Julian Fagir has put together a graphical – meaning it works under curses in a terminal, or under X - interface to pkgin, the binary package manager. Can someone try it and describe how well it works?
Some more pkgsrc expunging
There’s several packages that will be removed from pkgsrc after the 2012Q2 branch, since they haven’t worked in a long time. Also, Python 2.4 has been removed from pkgsrc-current and 2.5 will go the same way before the end of the year.
pkgsrc-2012Q1 is branched
The next quarterly release of pkgsrc, pkgsrc-2012Q1, has been branched. I’ll start building binary packages momentarily.
The branch should show up in DragonFly git later today. Once available, you can change any references to ‘pkgsrc-2011Q4′ in /usr/Makefile to ‘pkgsrc-2012Q1′, and then to switch to it:
- cd /usr/pkgsrc
- git branch pkgsrc-2012Q1 origin/pkgsrc-2012Q1
- git checkout pkgsrc-2012Q1
- git pull
At that point, you can start building and installing newer applications. For more details on that, check the pkgsrc guide on the DragonFly website.
Note that you don’t have to do that; you can stick with the 2011Q4 (or earlier) packages you have installed now, if you don’t want to deal with software changes right now, or if you want to wait for the binary packages to become available. Upgrades/security fixes only happen for the latest quarterly release, though.
Note: don’t assume I tested this before advising you to do it, or anything like that. I mean, come on.
Longest DragonFly review ever?
Steven Rosenberg is writing the longest DragonFly review ever. Here’s parts one, two, and three. There’s 3 more parts to come, 1 per day, so check back for the end of the story.
Packages that might go away, or not?
There’s been some discussion of packages that have been broken for a long time in pkgsrc, over on the tech-pkg@netbsd.org mailing list. It’s interesting to see just what breaks these packages, though it still seems up in the air whether any will be removed or not. (Follow the thread if you have time.) I don’t think the discussion has ended yet.
A minor DragonFly construction project
I’ve been working on a small house project over the past few days. My house has a basement workroom, which I use for whatever I need to do involving pliers or a saw. I’ve been slowly outfitting it over the past few years, and one thing I wanted to do was to wire it for music.
Not just a radio, but a computer that I could play sound file from, and stream audio. You can buy hardware for just that, but I’m cheap. I also wanted to keep it from looking like a computer desk; I have enough of that in my life already. This is a minor project; nothing like what you’d find on Instructables, but entertaining because it let me use DragonFly.
I purchased a set of cheap speakers from Newegg. You’ll notice that the speakers have a metal frame that forms a loop at the bottom – that’s important later. I bought the speakers and hooked to a tiny netbook, running DragonFly 3.0.2. It works fine for playing music, though the case speaker doesn’t shut off when external ones are attached. That’s not a problem here, though, since it’s not loud enough to be audible over the separate speaker output.
Those metal loops on the bottoms of the speakers turned out to be handy. I found some scrap wood, and built a small armature to fit inside the loop and hold it offset from the ceiling joist. Both of these wooden blocks could have the speaker slide over it, upside down.
I stained both of the blocks so that they wouldn’t stand out against the dark wood of the workroom ceiling.
I affixed the wooden hangers as far out as the cord on the speakers would let me, and slid the upside-down speakers onto them. There’s enough length in the cords to place the separate volume control dial on the workbench, and I’m done.
You can see the ceiling speaker in the upper corner. How’s the sound? Okayish. You aren’t going to get much out of a set of speakers this cheap, but at least I don’t have wires over my work area, and I don’t have to worry about puncturing a speaker with a screwdriver by accident, or something similar. I can close the laptop to keep it at least somewhat protected.
This is not a terribly complex project, but it makes me happy to have a DragonFly-based jukebox when I’m home. (This laptop usually travels with me.) I’m playing the music with mpg123, which is a surprisingly capable command-line player for files and for streaming audio.
(Yes, that is a large black velvet painting of a bullfight in the background. It was a wedding present. I also have black velvet paintings of Kenny Rogers as the Gambler, Fat Elvis, and Jesus blessing a tractor-trailer. I don’t know why.)
Plans for pkgsrc
I just removed old pkgsrc binary packages for DragonFly 2.6/2.7 from avalon, so if somehow you are running a version of DragonFly that old, and still using binary packages, you’ll want to upgrade. I’m pretty confident that describes nobody.
Also, I have plans for coordinating the next pkgsrc release of 2012Q1, due April 6th, with the probably next minor upgrade of DragonFly, 3.0.3. I wrote out my plans already, so go read. (plus followup)
ldconfig search path change
John Marino has changed the default search path for ldconfig; it no longer looks along /usr/lib/gcc* since that’s already included via rpath. The end result: you will need to do ‘make upgrade’ after your next buildworld build/installworld.
A 3.0.2 torrent
A torrent for DragonFly 3.0.2, found via Google search. Which ISO or img files does it include? I don’t know. Which architectures? I don’t know. Is it legit? I don’t know. Click at your own risk, just like any other link.
DragonFly 3.0.2 out
DragonFly 3.0.2 is out, and you can update (see /usr/src/UPDATING) an existing install or download a new one. This release turns off I/O APIC when booting in a VM because it caused issues for some users.
AMD processor bug: the followup
Matthew Dillon has posted a link to the errata for the AMD CPU bug that he found. Venkatesh Srinivas has followed with a test case for the bug.
Matthew Dillon also pointed out there’s a workaround to fix it, with no performance impact, it’s only found on revision 10h CPUs (not Bulldozer), and it’s extremely hard to duplicate. Why draw such a heavy line under that? The news of this bug rippled out through various news sites and was almost universally misreported, in a way that made it look bad for AMD without actually realistically quantifying the problem. Remember, it took 6 months just to find it – and he was looking for it!
Freeze for pkgsrc-2012Q1 has started
It runs from now to April 6th, so nothing but bug fixes in pkgsrc until then. If you have any package fixes you needed, now’s the time to ask someone.
Apache in jail: a tip
Konrad Neuwirth is running Apache inside a jail, and getting some weird errors. Obviously I don’t know the fix, but Chris Turner knows what the settings need to be.
Things for, and not for, the next pkgsrc branch
That’s pkgsrc-2012Q1 I’m talking about. It appears KDE will jump from 4.5 (what’s there now) to 4.8, and Zope/Plone will be removed. This will make you happy or sad depending on whether you have these things installed.
Freeze is coming to pkgsrc
The freeze for the next version of pkgsrc, 2012Q1, will start March 22nd and end with the quarterly release being released on April 6th.
(I hope someone gets the joke.)
GCC updated in base
John Marino’s updated DragonFly’s version of GCC 4.4 to 4.4.7, apparently the final version of GCC 4.4. What’s next? I imagine GCC 4.6 at some point. It’s always a fun (maybe bikeshed-ish) conversation on which compiler to install, and which to have in base.
Build report for pkgsrc
For the curious, I recently sent a bulk build report for pkgsrc-2011Q4 to the lists. Other than ruby-193 (which is fixed in pkgsrc HEAD thanks to John Marino), we’re looking pretty good! I’m curious if KDE or Gnome could actually get installed via binary; that’s sort of an ultimate goal due to the number of packages involved.
Speaking of Ruby, the default in pkgsrc may change soon, along with some of the involved Rails packages.
Benchmarks for DragonFly 3.0
As several people have told me, there’s benchmarks of DragonFly 3.0 vs. 2.10, available on Phoronix. CPU performance shows a significant improvement, in tests that actually test it. (I’d think a file compression test would be disk-limited, for instance.) Disk performance isn’t as great, but that may be in part because Hammer no longer will starve reading to benefit writing; that makes benchmarks look worse but improves real-world interactivity. I’m sure there’s more quibbling to do, since it’s lies, damn lies benchmarks.
Old DragonFly images
If for some reason you needed DragonFly 2.6 ISOs, or older, there’s a mirror.
Building FireFox 10
If you want to build Firefox 10 out of pkgsrc, make sure your DragonFly system is up to date; there’s a recent fix needed to make that happen.
Missed notes: JDK1.6, isp(4), PCI ids
I tagged these when they happened in previous months, but I forgot to post them:
“peeter” got wip/jdk16 to build normally on DragonFly, and listed how to do it. I don’t know if it still applies.
Sascha Wildner updated the isp(4) driver from FreeBSD, adding new supported chipsets and making it able to load as a module.
Also from Sascha Wildner, we’re now using one source only for PCI IDs. Think of that next time you are looking at dmesg, and it makes sense.
2 new mirrors
If you’re in New York City or the UK, there’s two new DragonFly mirrors for your downloading pleasure. Check the mirrors page for details.
More on that CPU bug
Matthew Dillon’s CPU bug hunt has scattered its way across various news sites, some more accurate than others. He’s posted a followup that is probably a more valuable read than any of the news reports.
Older Python not dead yet
A few days ago, I posted about Python 2.4 and Python 2.5 leaving pkgsrc – it looks like it’ll be a little bit longer, at least for the 2.5 version. This means the Zope packages will be gone too, since they depend on Python 2.4. This won’t affect you if you aren’t using these packages, of course.
