Benches being marked


Jean-Marc Zucconi benchmarked FreeBSD 4.9-R, 5.1-R, 5.2-R and DragonFly on a multiprocessor system, and posted the large quantity of data this generated. It’s mashed into a table here:

(updated with better results for 5.2; ACPI was generating so many interrupts it was slowing his system.)

(see end of entry for his notes)

TR>TR>
FreeBSD-4.9 FreeBSD-5.1 FreeBSD-5.2 DragonFly
Dhrystone 2 without register variables660050.1 lps 650148.4 lps 661362.9 lps 667712.8 lps
Dhrystone 2 using register variables 660062.3 lps 650258.6 lps 661430.8 lps 665312.0 lps
Arithmetic Test (type = arithoh) 1518178.8 lps 1527450.3 lps 1055234.6 lps 1513511.1 lps
Arithmetic Test (type = register) 89009.9 lps 91551.8 lps 91672.9 lps 87657.5 lps
Arithmetic Test (type = short) 81709.0 lps 91413.9 lps 88086.6 lps 81487.4 lps
Arithmetic Test (type = int) 89030.8 lps 91550.6 lps 91662.9 lps 88331.6 lps
Arithmetic Test (type = long) 88999.0 lps 91544.6 lps 91682.2 lps 89063.4 lps
Arithmetic Test (type = float) 88148.0 lps 105020.8 lps 105154.2 lps 87531.7 lps
Arithmetic Test (type = double) 88148.2 lps 104999.4 lps 105242.2 lps 87891.6 lps
System Call Overhead Test 107461.9 lps 82630.7 lps 82300.9 lps 89537.9 lps
Pipe Throughput Test 133588.2 lps 86297.9 lps 78012.8 lps 106999.4 lps
Pipe-based Context Switching Test 38436.5 lps 23821.9 lps 23268.0 lps 18203.4 lps
Process Creation Test 2736.3 lps 1574.6 lps 1517.7 lps 1569.6 lps
Execl Throughput Test 1772.9 lps 1521.2 lps 1371.7 lps 1497.6 lps
File Read (10 seconds) 179477.0 KBps 167848.0 KBps 181930.0 KBps 179972.0 KBps
File Write (10 seconds) 18633.0 KBps 12800.0 KBps 7800.0 KBps 18433.0 KBps
File Copy (10 seconds) 16298.0 KBps 11944.0 KBps 11905.0 KBps 16271.0 KBps
File Read (30 seconds) 179598.0 KBps 169150.0 KBps 182251.0 KBps 177467.0 KBps
File Write (30 seconds) 18399.0 KBps 12589.0 KBps 7688.0 KBps 18277.0 KBps
File Copy (30 seconds) 15297.0 KBps 11468.0 KBps 4375.0 KBps 15308.0 KBps
C Compiler Test 428.1 lpm 258.4 lpm 241.2 lpm 407.1 lpm
Shell scripts (1 concurrent) 756.0 lpm 567.9 lpm 572.2 lpm 628.6 lpm
Shell scripts (2 concurrent) 507.3 lpm 346.6 lpm 324.9 lpm 385.6 lpm
Shell scripts (4 concurrent) 271.0 lpm 183.0 lpm 163.9 lpm 208.3 lpm
Shell scripts (8 concurrent) 137.7 lpm 93.0 lpm 82.0 lpm 107.0 lpm
Dc: sqrt(2) to 99 decimal places 83900.0 lpm 52378.5 lpm 54949.6 lpm 25753.6 lpm
Recursion Test–Tower of Hanoi 7920.6 lps 8162.1 lps 8382.6 lps 7825.9 lps

INDEX VALUES ——— FreeBSD-4.9 ——– ——— FreeBSD-5.1 ——– ——— FreeBSD-5.2 ——– ———- DragonFly ———
TEST BASELINE RESULT INDEX BASELINE RESULT INDEX BASELINE RESULT INDEX BASELINE RESULT INDEX
Arithmetic Test (type = double) 2541.7 88148.2 34.7 2541.7 104999.4 41.3 2541.7105242.241.4 2541.7 87891.6 34.6
Dhrystone 2 without register variables 22366.3 660050.1 29.5 22366.3 650148.4 29.1 22366.3 536903.0 24.0 22366.3 667712.8 29.9
Execl Throughput Test 16.5 1772.9 107.4 16.5 1521.2 92.2 16.51371.783.1 16.5 1497.6 90.8
File Copy (30 seconds) 179.0 15297.0 85.5 179.0 11468.0 64.1 179.011811.066.0 179.0 15308.0 85.5
Pipe-based Context Switching Test 1318.5 38436.5 29.2 1318.5 23821.9 18.1 1318.523268.017.6 1318.5 18203.4 13.8
Shell scripts (8 concurrent) 4.0 137.7 34.4 4.0 93.0 23.2 4.082.020.5 4.0 107.0 26.8
========= ========= ========= =========
SUM of 6 items 320.7 268.0 258.2 281.3
AVERAGE 53.4 44.7 43.0 46.9
“Finally to test if there is a problem with the compiler I used the lorenz benchmark (it simply does raw math computations, and does not use much memory)”:
Lorenz on 5.1
4.9binary:15.77 real 15.73 user 0.01 sys
5.1binary:16.52 real 16.51 user 0.00 sys
5.2binary:16.36 real 16.32 user 0.02 sys
Lorenz on 5.2
4.9binary:19.35 real 18.12 user 1.14 sys
5.1binary:27.65 real 26.05 user 1.48 sys
5.2binary:29.61 real 27.60 user 1.87 sys
“The 5.1 results indicate that there is no problem with the compiler generated code, but I don’t know how to interpret the 5.2 results: I would expect also a bad behavior with the 4.9 binary (all are static) Then I booted a uniprocessor kernel and reran the latest test:”
Lorenz on 5.2 UP
4.9binary:15.80 real 15.71 user 0.02 sys
5.1binary:22.57 real 22.45 user 0.03 sys
5.2binary:24.16 real 24.05 user 0.03 sys

Jean-Marc Zucconi wrote:


  • I used the same kernel configuration file for the 4 systems. In particular I removed ‘cpu 486′, maxusers was set to 0, and I don’t have INVARIANTS or WITNESS options.

  • the kernels and the worlds were compiled with CFLAGS=-O -pipe.

  • /etc/malloc.conf is always linked to aj

  • I/O tests were done on the same filesystem (IDE UDMA33 in all cases)

  • bytebench binaries were compiled static. I used the same binaries
    on FreeBSD 4.9-R and DragonFly; they were recompiled on
    FreeBSD-5.[12] as the compiler is not the same.

Posted by     Categories: Goings-on     1 Comment
1 Comment on Benches being marked

Closed

  1. Mezz says:

    Umm, 5.2 has changed from static to dynamic by default now. Do you think, it’s because of dynamic that hurt the performance big time?

    It’s possible already answered in the mailing list/newsgroup. I am on the different computer right now, so this computer doesn’t has any mail/newsgroup client.