Month: April 2011

Chromium for DragonFly: please test


Rui-Xiang Guo is looking for testers for wip/chromium.  That’s the zippy Google browser.  He especially wants DragonFly users – this would be useful, especially since I think Firefox 4 does not build on DragonFly right now.

Posted by     Categories: DragonFly, Goings-on, pkgsrc     2 Comments

GSoC: checkpointing vkernels and PUFFS


Two more of the DragonFly and Google Summer of Code projects: Irina Presa’s checkpointing vkernels, where you can save a running virtual kernel and start it again later, and Nick Prokharau’s port of PUFFS.

(Anyone know the HTML character code for ‘s’ with an inverted grave mark so I can spell Irina’s name correctly?)

Posted by     Categories: DragonFly, Goings-on, Google Summer of Code     1 Comment

GSoC: dsched BFQ, virtio, LVM mirror


Yay acronyms! Brills Peng was accepted for the Summer of Code project “Improve dsched interfaces and implement BFQ disk scheduling policy” – and now there’s a nice writeup describing what’s planned. Also, Stéphanie Ouillon did the same thing for the virtio drivers project.  Adam Hoka also joined in with a summary for his LVM mirror project.  Please keep this up, students.

dragonflybsd.it available


If you are a European resident, Federico Biancuzzi has the DRAGONFLYBSD.IT domain name available to donate. He doesn’t want to let it go and have it taken by a domain squatter. Talk to him at sale@securitydaemon.com if you want to hold on to it for some unspecified time.

Posted by     Categories: DragonFly, Goings-on     1 Comment

RAID and what to buy


Francois Tigeot did some testing of various hardware RAID adapters (Areca, LSI, 3ware, and Adaptec) in DragonFly, and reported thoroughly on each.  This may come as no surprise, but it sounds like Areca adapters are worth the money.

Update: There’s an updated mpt(4) driver, and the performance issues are fixed by enabling write caching.

Posted by     Categories: Device support, DragonFly     0 Comments

Grep still GNU, sort is not. Plus, file.


GNU grep on DragonFly has been updated from version 2.4d to 2.7. Other BSDs have switched/will switch to bsdgrep, but as John Marino points out in his commit message, GNU grep’s still faster.  He’s also brought in NetBSD’s version of sort, to replace the GNU flavor.  I don’t know why on that one.

Peter Avalos also updated file to 5.06.

Posted by     Categories: Committed Code, DragonFly     4 Comments

DragonFly 2.10 released


It’s out!  See the 2.10 release page for the startlingly extensive list of updates in this version.   Download images from the mirrors, or follow these steps (using a 2.10 version number) to build from source.

Posted by     Categories: DragonFly, Heads Up!     0 Comments

Summer of Code DragonFly projects announced


Google’s announced the accepted projects for 2011.  DragonFly has 6 slots!

We had a large number of interesting project proposals; far more than than the slots available.  If you’re one of the students who did not get in, please consider working on your project as time allows.  I know it won’t be lucrative, but I’d still like to see them happen.

Here’s the list of accepted projects:

  1. Implementing a mirror target for device mapper: Adam Hoka, mentored by Joe Talbott
  2. Improve dsched interfaces and implement BFQ disk scheduling policy: Brills Peng, mentored by Alex Hornung
  3. Make vkernels checkpointable: Irina Presa, mentored by Venkatesh Srinivas
  4. Port PUFFS from NetBSD/FreeBSD: nickprok, mentored by Nathaniel Filardo
  5. Bring kernel event notification in DragonFly BSD to its logical conclusion: Samuel J. Greear, mentored by Sascha Wildner
  6. Porting Virtio Drivers from NetBSD to DragonFly BSD to speed up DragonFly BSD as a KVM guest: Stéphanie Ouillon, mentored by Pratyush Kshirsagar

 

Posted by     Categories: DragonFly, FreeBSD, Google Summer of Code     0 Comments

Sysbench and DragonFly releases


I did some comparative benchmarking between the 2.6, 2.8, and upcoming 2.10 release for DragonFly.   As several people have guessed, performance has improved significantly, and the difference would probably be even more pronounced if I was using more modern hardware,  e.g. swapcache or a system with AHCI.  I have a mailing list post with details, and here’s the graph that sums it up:

Shorter bars are better(Sorry, no Lazy Reading this week.  Life didn’t co-operate.  At least there’s a pretty graph!)

 

Posted by     Categories: DragonFly, Goings-on     0 Comments

pkgsrc-2011Q1 build reports


We’ll have a full set of pkgsrc packages for the upcoming DragonFly 2.10 release, built from the most recent quarterly release of pkgsrc: 2011Q1.  For the curious, here’s the build reports for i386 and for x86_64 architectures.

Posted by     Categories: DragonFly, pkgsrc     0 Comments

Hardware suggestions:Supermicro


To go along with the recently-added suggested hardware page on the DragonFly website, Francois Tigeot puts in a good word for SuperMicro boards and DragonFly, and links to some good hardware combinations.

Posted by     Categories: Device support, DragonFly     0 Comments

Zombie shirts


Not shirts with zombies on them, but DragonFly shirts that don’t have a seller.  I had a random Google search turn up a store selling DragonFly T-shirts, among other things.  It was essentially a spam store.  The seller wasn’t producing anything but instead reselling other people’s material for a commission, similar to the splogs out there that recopy material from other blogs or Wikipedia and slap ads on it.  (I’ve seen Digest material pop up that way.)

Following the link back shows that the shirt is sold through a Cafepress store called ossgear.  It looks like the original store owner asked permission to use the logo back in 2006.  ossgear.org is no longer a functioning domain, and I can’t find any other reference to this seller; they appear to have stopped doing business 5 years ago.

The moral of this story: Sites like Cafepress will try to profit from your work long after you’ve stopped using them.  The frustrating part is that the logo isn’t even right!

Posted by     Categories: DragonFly, Goings-on     0 Comments

RAM vs. deduplication


Tomas Bodzar asked about RAM usage with Hammer and deduplication, pointing at this example that shows ZFS requiring…  I’m not sure.  Lots?  Anyway, Matthew Dillon noted that offline deduplication in Hammer would use available RAM/swap for CRCs on all files, but only a limited subset for ‘live’ dedup.  For a real-world example, Venkatesh Srinivas described deduplicating about 600G down to 400G, with a machine having only 256M of RAM. Yes, only 256M.

Posted by     Categories: DragonFly, Goings-on, Hammer     1 Comment

Monday the 25th: 2.10 release, GSoC projects


This upcoming Monday should be exciting!  It’s the planned date for the release of DragonFly 2.10.  Also, the accepted projects for Google Summer of Code (including for DragonFly) will be announced.

2.10 branched


The DragonFly 2.10 branch was a bit later than predicted, but that was mostly to avoid merging work.   Release should still be in a few days.

Posted by     Categories: DragonFly, Goings-on     0 Comments

More Hammer documentation


Thomas Nikolajsen has put together more information on Hammer, including formatting and the new deduplication features, conveniently located in the man pages and some other spots.

Posted by     Categories: Committed Code, DragonFly, Hammer     0 Comments

Printing and USB a little bit more possible


If you have a USB printer, you may not have been able to print since the kqueue changes came in for… DragonFly 2.8?  Anyway, Matthew Dillon’s made some changes to ulpt(4) that means USB and kqueue play nicely together.

Posted by     Categories: Device support, DragonFly     0 Comments

Add package-clean to your list


The usual way for building pkgsrc packages from source is ‘bmake install clean’, to build and install the package, and then clean the work files from building it.  Since the recent change to DESTDIR, where a binary package is built before installation, you may want to add ‘package-clean’ to the list, so that the binary package is also removed after installation.

Posted by     Categories: DragonFly, pkgsrc     0 Comments

Lazy Reading for 2011/04/17


I hope I can get this together.

  • This article asks “Does anyone in Silicon Valley care about Windows anymore?”   It’s an inflammatory title, to get you to read it, and it’s based on anecdotal ideas, but I think there’s some truth to it.
  • Something similar, in hardware: I see people who care about what they run either getting a Macbook or a Thinkpad these days.  (I’ve owned both, and they are nice laptops…)  Let’s run with that idea, in fact: Macbook is to Thinkpad running BSD as is… iPhone is to Android phone running custom ROM?  This is turning into a “levels of nerditry” sort of comparison.
  • Community is your best feature, a talk about how to encourage the growth of an open source group.  I link to it because it’s useful and well done, but also because it lets me feel a bit self-congratulatory; we already use many of the listed concepts in DragonFly.
  • Zero knowledge user identification is interesting, though it’s not something you could apply to a lot of users.  (via)
  • Things found via Google: A DragonFly 2.8.2 x86_64 VMWare image on Sourceforge.  Don’t know who put it there.
  • This article about passwords says multiple common words make more secure passwords than adding upper/lower case and numbers to passwords.  An interesting contention, though I don’t think it works as well as it’s described.  (Adding ” ” into the list of possible characters isn’t as effective as having to double the list for case, for instance.)
  • It’s been a while since I posted a roguelike link.  Well, how about “How Rogue Ended Up On The Sofa“?  (via)  It very nicely draws a line connecting rogue and a whole lot of modern games.
Posted by     Categories: DragonFly, Lazy Reading, roguelike     1 Comment

Try newest ACPI, see what changes


If you’re running DragonFly 2.9, now is a good time to update and try various ACPI-related things, like power button shutdowns and battery monitoring.  Sepherosa Ziehau has been updating vigorously.

Posted by     Categories: DragonFly, Goings-on     0 Comments

Supported hardware list appears


There’s a new Supported Hardware page on dragonflybsd.org.  I think the idea is not to be comprehensive, since that’s a nigh-impossible task.  Instead, it’s to note the combinations of hardware that work really well.

Posted by     Categories: DragonFly, Goings-on     0 Comments

ACPI SCI changes


Sepherosa Ziehau has made some changes to default SCI settings in ACPI.  This may make it possible to boot a computer, or to boot a computer with ACPI, that did not boot before.  If it causes problems, he lists some various tunables to set.  Just don’t ask me what SCI does.

Posted by     Categories: Committed Code, DragonFly     0 Comments

Quad port network card support


Hey, it rhymes!  Matthew Dillon’s added support for 4-port Gigabit Ethernet PCI-E cards from Intel.  I wish I had one.

Posted by     Categories: Device support, DragonFly     0 Comments

First pkgsrc-2011Q1 results for 2.10


No, 2.10 is not out.  I built packages for pkgsrc-2011Q1 on 2.9, and set it to think 2.10 so that the pkgsrc tools wouldn’t complain.  We’re close enough to release that this shouldn’t be a problem.  The packages are available for x86_64; i386 packages coming “soon”.   See my note to users@dragonflybsd.org for details on accessing these packages.

Posted by     Categories: DragonFly, pkgsrc     0 Comments

How much swapcache can help (with graphs)


I love graphs.  Jan Lentfer made some! Both of these show recent speed improvements in DragonFly – especially some spectacular results from swapcache(8) and the recent NCQ tagging improvements.  (Note that only the third graph represents the NCQ improvements; the first two graphs were done before.)

The first one is a comparison of pgbench running on the same hardware twice – once with the 2.8 release of DragonFly, and once with a recent 2.9 version.  2.9 is definitely looking to be faster than 2.8.

Next up is a 2.9 system run with and without swapcache, showing an astounding difference between the two.  It’s pretty clear just how much performance improvement you can get from swapcache…  (see Jan’s notes on the setup after the graphic.)

 

Jan’s notes, from EFNet #dragonflybsd on IRC:

15:08 < lentferj> these are SELECT-Only tests
15:09 < lentferj> JustinS: it’S important to note, that the database is 2,5x
bigger than RAM on the swapchache test
15:11 < lentferj> JustinS: I did a Select-Only ramp-up of 30 minutes to get
caches and swapcache filled
15:12 < lentferj> JustinS: and then I ran
15:13 < lentferj> for i in 1 2 3 4 6 8 12 16 24 32; do
/usr/lib/postgresql/8.4/bin/pgbench -U pgsql -h atom -s 400
-S -c “$i” -T 600 pgbench; done
15:13 < lentferj> so, select only pgbench for 10 minutes each
15:13 < lentferj> with increasing numbers of client
15:14 < lentferj> pgbench on another box, 100MBit switched network
15:15 < lentferj> JustinS: the first graph (2.8.2 vs current) is the same w/ a
database that fits in RAM entirely
15:15 < lentferj> so measuring concurrency performance (w/o I/O)
15:17 < lentferj> the swapcache comparison was on a 2GB box with a 5GB database
and 16GB swapcache (INTEL) attached to a sili card
15:17 < lentferj> on a atom 330 :)

Now, here’s testing with the recent NCQ tagging update for AHCI:

These results are astonishing.  Please, someone compare with other operating systems!

Here’s the stats for this last test:

 

  • 5.6GB database, system w/ 2GB RAM –> io benchmark
  • pgbench with increasing no of client 1->32, SELECT-Only Mode
  • sili controller Dawicontrol DC-3410 SATA PCI controller which is using a Silicon Image 3124-2 chip
  • 2 Seagate Barracuda ES.2 250GB SATA II disks
  • lvm stripe over those disks
  • postgresql.conf is default, except shared_buffers set to 512MB and effective_cache_size to 1024MB
  • atom330 on a Foxconn mobo
  • SSD is SATA INTEL SSDSA2M040 2CV1

 

Posted by     Categories: DragonFly, Goings-on     1 Comment

DESTDIR details


I mentioned this before, but there’s now an official announcement that pkgsrc is (now, after the 2011Q1 release) going to DESTDIR support and what that means.

Posted by     Categories: Goings-on     0 Comments

Significant AHCI SATA, CAM changes


This alphabet soup of acronyms is what Matthew Dillon updated, leading to some significant performance improvements for drives that use NCQ tagging.  Try it out cause it’s fast, and try it out because it needs testers; this is a change that affects some basic parts of how data is written to disk, and you don’t want to get that wrong…

Posted by     Categories: Goings-on     0 Comments

Lazy Reading for 2011/04/13


Get out your wallet!  I encourage purchasing here.

  • You should buy a SSD.  Not necessarily news to you, but that article does a good job of summarizing why.
  • On the other hand, SSD prices are already on their way up/availability is way down.  Japan’s disasters are having a ripple effect through the high-tech supply chain.  Either buy immediately or get ready to wait for a while…
  • Introduction to Architecting Systems for Scale – you either don’t care, or find scaling questions immediately engaging.  I am one of the latter, so here’s the link.
  • I’ve been watching pkgsrc-changes@netbsd.org for a little while.  One thing I’ve discovered: there’s a lot of updates going on!  Another thing that’s nice to see: DragonFlyupdates, including ones that help with our move to gcc 4.4.
  • Aw, no more Kermit.  (via)  Not that I have a use for it at this point, but still: aww.  I bet in about 10 years I’ll say the same thing about… gopher?  Remember that?  It’s not even supported in Firefox 4 now, which kinda makes me feel sad.  And old.
  • Server plans: Facebook vs. Google.  (warning: Facebook article is somewhat giddy.)
  • The infinite hard drive.  (via I lost it, sorry)

Here’s an extra little thing: next time you’re dealing with dusty computer equipment, remember this picture:

That is what happens to an exposed RJ45 port after a few years in a salt mine (my employer).  This was inside an enclosed, mostly-sealed  structure, too.

Posted by     Categories: DragonFly, Lazy Reading, pkgsrc     1 Comment

One extra thing: OpenSSH


Peter Avalos squeezed in one more thing before the DragonFly2.10 branch: an update of OpenSSH to version 5.8-p1.  This is mostly a security fix upgrade to 5.7 – see the OpenSSH release notes for details.

Posted by     Categories: Committed Code, DragonFly     0 Comments

Thoughts about RAID1


There’s been plenty of discussion about Summer of Code projects on the mailing lists.  One conversation about “Implementing a mirror target for the device mapper” led to a longer description from Venkatesh Srinivas about mirroring and how he’s looked at implementation.

Next release April 23rd


Branch tomorrow, release in 2 weeks.  There’s a ton of new features for 2.10, so this will be a fun release.  I’m trying to get pkgsrc-2011Q1 packages built for 2.10 ahead of time, too.

Posted by     Categories: DragonFly, Goings-on, pkgsrc     3 Comments

Our newest committer: John Marino


Welcome John Marino, who has been working on some rather difficult updates for gcc and other toolchain items.

Posted by     Categories: Committed Code, DragonFly     2 Comments

April OSBR: Collectives


The Open Source Business Resource issue for April is out, themed on Collectives.   A lot of it covers the Keystone Off the Shelf (KOTS) project, which is a collection of open source software designed to work as a starting package for a technology business.  Read the first article for more details.

Posted by     Categories: BSD, Periodicals     0 Comments

Swapcache updates, long-term status


Matthew Dillon made some changes to swapcache(8).  Swapcache is now able to cache a lot more data, and the result is that general disk performance for _all_ disks is accelerated by an included SSD using swapcache.  Performance previously restricted to all-SSD systems or serious RAID setups is now possible with much less investment.

In addition to that, the long-term wear on the SSD appears to be less of a problem than expected.

Posted by     Categories: Device support, DragonFly     0 Comments

Some Summer of Code projects


If you’d like to read some of the Summer of Code student proposals for DragonFly, there’s a number of them available.  I haven’t found a way to get a comprehensive list without being logged in, yet.

pkgsrc-2011Q1 details


I already noted that the quarterly release is out, but the pkgsrc-2011Q1 release announcement is available now.  There’s good reasons to link to it – the list of updated packages, new packages, and credits for the work people have been doing.  Here’s the part I really want to pick out:

We’re aiming to make this the last branch to support non-DESTDIR packages. We have almost finished the transition to DESTDIR installation, where a staging directory is used to make a binary package, which is then managed by the pkg_install tools.

The reason I’m highlighting this is: it’s good news!  One of the long-term complaints with pkgsrc is that the upgrade process is painful.  If you try to build an upgrade and the build processfails after uninstalling the existing package, not only are you not getting the upgrade, but you’ve lost the existing package.  Binary packages for download helps with this (and generally is faster), but only so many packages can be built separately and made available for download.

Building a package separately and then installing from there removes these issues.  No binary redistribution issues, actual downtime is minimal, and the package is known to work when an upgrade happens.  This removes most of the problems I’ve heard raised about pkgsrc over the years.

Posted by     Categories: DragonFly, pkgsrc     0 Comments

GCC 4.4.5 now


Sascha Wildner has moved gcc in DragonFly to a slightly newer version: 4.4.5.  It mostly seems to make things easier to compile, going by the reports I’ve heard.  This is the version that will be in DragonFly 2.10.

Posted by     Categories: Committed Code, DragonFly, Goings-on     1 Comment

BSD Magazine for April out


The April issue of BSD Magazine is out!   It’s a very full issue, including another news roundup by yours truly.

Posted by     Categories: BSD, Periodicals     1 Comment

pkgsrc-2011Q1 tagged


It’s tagged, though there’s no release announcement yet.  Working on building binaries starting tonight…

Posted by     Categories: DragonFly, pkgsrc     0 Comments

Marvell looks over our shoulder


It’s Dragonfly, and it sounds very similar to swapcache(8).  Coincidence?  Maybe.  (via bodie on EFNet #dragonfly IRC)

Posted by     Categories: DragonFly, Goings-on     1 Comment

Lazy Reading for 2011/04/03


Getting into the swing of this link collection thing…

Posted by     Categories: BSD, Goings-on, Lazy Reading     5 Comments

Double buffering in Hammer usually useful


Enabling the vfs.hammer.double_buffer=1 sysctl will greatly improve Hammer performance when you’ve exceeded your memory cache (at a possible slight penalty when you have not) and also speed things up when using live deduplication.

Update: Venkatesh Srinivas says:

“double_buffer makes sense when: 1) you want all CRCs to be checked on reads. 2) you’re running live dedup and care about dedup performance rather than say read-heavy performance; 3) you have swapcache but are often running into the  vnode limit in what you can cache.”

So, not always useful.

Posted by     Categories: Committed Code, DragonFly, Hammer     0 Comments

binutils, Hammer updates


Sascha Wildner has updated the default version of binutils in DragonFly from 2.17 to 2.21.  You’ll want to do a full buildworld on your next upgrade, if you’re running DragonFly 2.9.

Also, Matthew Dillon has made version 6 the default version of Hammer in DragonFly 2.9.   Version 6 has improved handling of directory names in some circumstances.  Just don’t ask me which, cause I lost track.  It’s been a hard day!

Posted by     Categories: Committed Code, DragonFly, Heads Up!     0 Comments

Pkgsrc packages possibly pruned


There’s a number of pkgsrc packages that have a combination of security vulnerabilites and lack of updates for more than a year which is placing them on the chopping block.  (Follow the discussion to see which ones make it off the list.)  The removals will happen after the next branch, pkgsrc-2011Q1, which is itself due in two days.

Posted by     Categories: Goings-on, I like alliteration, pkgsrc     0 Comments