Matt Dillon explained what he wants the port-creation experience to be for a port maintainer, using VFS, quoted below:
More…
Month: October 2003
VFS and ports for maintainers
FXR created
Hiten Pandya mentioned that Robert Watson has set up the FXR website (FreeBSD/Linux Cross Reference) for BSD code. It’s a personal domain, so please don’t tax it at present.
The SMP path not taken
Greg Lehey put up a paper and slides from his Singapore presentation, describing the FreeBSD-5 SMPng work, both in terms of events and code. It’s an interesting read, since this is the path avoided in DragonFly. (Greg Lehey has a 2001 USENIX paper on SMPng, too, which mentions much of the same content.)
Flames bad, info good
On the freebsd-hackers mailing list, a slight flamewar erupted over discussion of checkpointing code from DragonFly. Amidst the dumbness that normally ensues in a flamewar, there’s some interesting descriptions on what work has been done/will be done on DragonFly. Check out the archive, mostly in the “FreeBSD mail list etiquette” thread. Matt Dillon’s posts here, here, and here are all info-packed.
Berkley talk slides up
Matt Dillon’s posted his slides from his talk at UCB Berkley, in OpenOffice format.
FreeBSD 5.1 and DragonFly disk sharing
As part of another discussion, it’s been noted that trying to boot FreeBSD 5.1 and DragonFly from the same disk currently may not work if it’s UFS1, and definitely won’t if it’s UFS2.
Checkpointing links
Jeroen Ruigrok posted some links to papers about checkpointing and threads:
A User-level Checkpointing Library for POSIX Threads Programs:
http://citeseer.nj.nec.com/james99userlevel.html
Checkpointing and its applications:
http://citeseer.nj.nec.com/wang95checkpointing.html
Migratory applications:
http://citeseer.nj.nec.com/bharat95migratory.html
New site layout
The main DragonFly site has been fixed up, in part with some changes I’ve submitted. Whee! I contributed!
ATAng discussion
Kip Macy brought up the idea of moving FreeBSD’s ataNG into DragonFly; the consensus so far is to bring it in separately from the existing ata support, since ataNG’s not yet completely stable.
Matt Dillon at UCBerkley
Thursday October 23, 306 Soda Hall 6 – 8 PM. Matt Dillon will be talking to whomever shows up to the CSUA meeting.
Futexes in the future
Hiten Pandya brought up futexes for possible inclusion in DragonFly, pointing at this page for reference.
Bill Huey brought up http://people.redhat.com/drepper/futex.pdf, with the caveat that futexes are used for 1:1 threading models, not M:N models such as DragonFly is using/will be using.
Routing reading
Jeroen Ruigrok posted a whole pile of links to routing information for anyone wanting to work on it:
Basics:
TCP Vegas: End to End Congestion Avoidance on a Global Internet
http://citeseer.nj.nec.com/brakmo95tcp.html
Evaluation of TCP Vegas: Emulation and Experiment
http://citeseer.nj.nec.com/ahn95evaluation.html
TCP and Successive Fast Retransmits
http://citeseer.nj.nec.com/floyd95tcp.html
Congestion Avoidance and Control
http://citeseer.nj.nec.com/jacobson88congestion.html
Performance Problems in BSD4.4 TCP
http://citeseer.nj.nec.com/brakmo95performance.html
Comparison of Tahoe, Reno, and SACK
http://citeseer.nj.nec.com/fall95comparisons.html
TCP Vegas: New Techniques for Congestion Detection and Avoidance
http://citeseer.nj.nec.com/brakmo94tcp.html
A Tree-based Packet Routing Table for Berkeley Unix
http://www.cs.berkeley.edu/~sklower/routing.ps
Newer:
TCP Vegas Revisited
http://citeseer.nj.nec.com/hengartner00tcp.html
Analysis on TCP Vegas and TCP Reno
http://citeseer.nj.nec.com/hellal00analysis.html
Scalable Timers for Soft State Protocols
http://citeseer.nj.nec.com/48810.html
End-to-End Internet Packet Dynamics
http://citeseer.nj.nec.com/paxson97endtoend.html
End-to-End Routing Behavior in the Internet
http://citeseer.nj.nec.com/3573.html
Routing with a Clue
http://citeseer.nj.nec.com/afek99routing.html
IP Lookups using Multiway and Multicolumn Search
http://citeseer.nj.nec.com/lampson98ip.html
Fast and Scalable Layer Four Switching
http://citeseer.nj.nec.com/srinivasan98fast.html
High-Speed Policy-based Packet Forwarding Using Efficient
Multi-dimensional Range Matching
http://citeseer.nj.nec.com/lakshman98highspeed.html
Fast Address Lookups using Controlled Prefix Expansion
http://citeseer.nj.nec.com/srinivasan99fast.html
Slab, multicast, sysinstall, SysV
The memory slab allocator is now on by default, and the old system is gone.
Man pages for pim(4) and multicast(4) for “Protocol Independent Multicasting”, are in.
Sysinstall has been modified with changes for NFS; you can pick NFSv3 (default) and TCP protocol (not default), now, along with some internal changes.
The SysV IPC regression test suite has been added, in src/tools/regression/
Compatibility cleanup stats
David P. Reese posted his 9th step into syscall separation, and noted that there remains:
- “37 more uses of stackgap allocations in the linux emulation code”
- “26 more 43bsd syscalls that require relocation”
Checkpointing is in
Kip Macy’s checkpointing code has been committed; I’m pasting Matt Dillon’s post about it as there’s a lot of issues to consider.
For those of you late to the party, checkpointing allows you to “freeze” a copy of an application so that, in theory, you can restore the program to that running state at a later point in time. Useful, for instance, if you have a program that takes a long time to complete and you don’t want to have to restart from the beginning if there’s an interruption.
More…
Doxygen don’t
David Cuthbert brought up the idea of Doxygen headers for the source; Matt Dillon didn’t care for the idea, pointing at this for an example. Hiten Pandya noted that he’d like to have a separate by-hand handbook, for which Eirik Nygaard posted a possible example. Matthew Fuller added that he is working on a SGML application for library documentation as part of another project.
wmake
While noting that he has made a number of changes to the scheduler, Matt Dillon described a new tool called ‘wmake’. wmake allows you to run make in a subdirectory simulating a buildworld environment, without actually having to build world.
(quoted from his post, using libkvm for an example:)
cd /usr/lib/libkvm
wmake obj
wmake
make install
Routing engine description
Joshua Coombs has an extensive writeup of his ideas for routing, located at http://www.outofspec.com/routing/
How to profile
While talking about something else, Hiten Pandya mentioned the steps necessary to turn on profiling, to help catch bugs. I’ve saved the steps here, in case they are useful…
More…
Bootsplash loader and gzip
Emiel Kollof noted that it would be nice if the splash screen loaders could read gzipped files, so that any splash screens could be stored in a compressed format and still used. Hiten Pandya pointed at kern/imgact_gzip.c and boot/i386/kgzldr/boot.c for examples, if anyone wants to tackle this project.
/bin/sh fixed
/bin/sh had a big introduced recently that would make it crash when booting with a new world. It’s been fixed, so recompile /bin/sh with updated sources when possible.
Plan 9 namespaces
David Leimbach noted that Ron Minnich was porting Plan 9 namespaces to FreeBSD, which duplicates some of the security features to be covered by VFS. Ron Minnich’s web page has more data on this and other technologies.
Emiel’s Eye Candy
usr/share/examples/splash has been committed. This is the logo screen Emiel Kollof made from the DragonFly logo.
Emiel Kollof’s also got the NVIDIA binary video driver working; it should show up in dfports.
amr(4) driver patch
Hiten Pandya (hmp@backplane.com) is looking for anyone using a AMI MegaRAID card, who is willing to test a patch for FreeBSD-4/DragonFly.
vmware2 in dfports
dfports/emulators/vmware2 is in, though Matt Dillon warns that it is not yet functional.
Added fsx, fdisk option
File System Exerciser, in tools/regression/fsx, has been added, by David Rhodus.
fdisk now has a -p option, to let it operate on disk images.
NFS fixes
David Rhodus has fixed the problems reported by Richard Sharpe where Linux NFS clients couldn’t achieve above 5 MBps throughput.
Installing from source
I wrote down the explicit steps I used to get a DragonFly machine running and up-to-date; I’m including them here for the benefit of others. This was working as of mid-September.
** Updated December 2003; silly typos.
** Update February 2004: This guide was written using FreeBSD 4.8 as a guide; it may break. There is a guide page on dragonflybsd.org that talks about upgrading from FreeBSD 4.9.
More…
Remaining Checkpoint work
Kip Macy listed a number of tasks left for his checkpoint/restart work, not all of which he will be covering. If this sounds interesting to you, jump in!
More…
How to do a diff
In part of a conversation about checkpoint work, Matt Dillon described the preferred process for code changes. I’m pasting it in, wholesale.
More…
More checkpoint work
Kip Macy’s very spiffy checkpoint work, allowing freezing/thawing of running applications, has resulted in a kernel module. It’s not yet in, but the most recent version he mentioned is available.
Other port systems
Paul Jarc brought up “>Slashpackage as a possible solution to the ports system on DragonFly; Mike Porter pointed out that it covers most everything except shared library handling.
NVIDIA driver work
Emiel Kollof is working on the NVIDIA binary video driver; so far it loads correctly, but doesn’t work in X11.
FFS/softupdates history
In other discussion about a microkernel, Gary Thorpe noted these historical links about softupdates:
http://www.netbsd.org/Documentation/tune/5.html#a3
http://www.netbsd.org/Documentation/misc/#softdeps
http://www.mckusick.com/softdep/index.html
http://www.usenix.org/publications/library/proceedings/usenix99/mckusick.html
And Pedro Guffuni followed up with this:
http://www.ece.cmu.edu/~ganger/smallfiles.html
Firewall Fun
Joshua Coombs noted he would like to work on a new firewall strategy for DragonFly, and pasted in some of his notes. They are complex enough that it’s better to paste than to sum up. (I sure could use the multiple routes he talks about.)
Pasted material follows:
More…
Bugs, lost and found
It looks like the PC Card problems mentioned earlier are really a problem with the DHCP client. David Rhodus is looking for someone to try a new import of the ISC DHCP code. (Updated – an import from FreeBSD-5 is happening instead, since that fixes some problems with the most recent ISC code.)
Galen Sampson found that using NFS without -maproot will cause file truncation; Matt Dillon has a temporary hack that will fix it, which will be committed by the end of the week if it works without trouble.
Matt Dillon disabled background bitmap writes, as it can apparently cause certain race conditions. He notes: “Eventually the problem background bitmap writes solved will be solved more generally by implementing page COWing durign [sic] device I/O to avoid stalls on pages undergoing write I/O.”
Matt Dillon IRC Interview
There’s a Slashdot article noting that Matt Dillon will be available for a Q&A session. It’s on Thursday, Oct. 9th, 9 PM EDT in SlashNET’s #forum. If you miss it, it should be later available in http://www.slashnet.org/forums/.
PCMCIA Problems
Emiel Kollof found that to get a PCMCIA network card going, you need to add these lines to /etc/rc.conf: (replace x’s with your static address, if applicable.)
removable_interfaces="<interface name>"
pccard_ifconfig="inet xxx.xxx.xxx.xxx netmask xxx.xxx.xxx.xxx"
or
pccard_ifconfig="DHCP"
Varsyms in FreeBSD
John De Boskey posted he had variant symlinks working under FreeBSD 4-Stable, and notes these patches:
http://people.freebsd.org/~jwd/mlinks/freebsd-magic-patches
http://people.freebsd.org/~jwd/mlinks/env-magic-patches
And this post to freebsd-stable.
Microkernel link
During a separate thread about microkernels (short summary: it ain’t happening), Pedro Guffuni pointed out Exokernel, a OS project where most everything was in libraries.
Bootsplash
Emiel Kollof made a bootsplash screen at 800x600x8, available here. Download, ungzip to /boot, and put these lines in your loader.conf:
vesa_load="YES"
splash_bmp_load="YES"
bitmap_load="YES"
bitmap_name="dfly.bmp"
The screen, incidentally, looks just like the main site logo.
Last Syscall Separation Step
David P. Reese said his ‘step 5a’ patch is set for syscall separation. He stated this is his last step before starting the stackgap allocation send-off. Super!
Variant Symlink explanations
Mike Porter wrote an extended entry about how variant symlinks could handle multiple versions of ports being installed, including a number of special cases like Perl and the modules installed for it.
More…
More on layouts
César Duque posted his own take on how the DragonFly page should look:
http://www.notfound.org/df (it’s a large jpeg.)
Promise Supertrack in
The Promise Supertrack controller (pst) is now in src/sys/i386/conf/LINT.
A New Look
Hiten Pandya has apparently been working on a new version of the main site. Those changes are not (yet?) in CVS.