Stephan Paukner :: syslog

I complained about my old notebook being too slow at image processing. As I now run a new machine, it’s time to measure the performance increase:

Old system: 1×1.6 GHz CPU, 1 GB RAM
New system: 8×2.67 GHz CPU (by threads), 6 GB RAM
Test file sizes: 11.2–12.5 MB

Task #1: Render 8 RAW images to PPM in multi-parallel
Command line:

Old system time: 1 min 41.1 sec
Old system mem usage: 1 GB (100%)

New system time: 0 min 04.6 sec
New system mem usage: 2 GB (33%)

→ 22× faster

Task #2: Render 8 RAW images to JPEG in multi-parallel
Command line:

Old system time: 5 min 0.0 sec
Old system mem usage: 1 GB (100%)

New system time: 0 min 18.2 sec
New system mem usage: 2.5 GB (42%)

→ 16.5× faster

Hell yeah!

Debian’s standard kernel 2.6.26 has a little drawback: The coretemp module doesn’t recognize Intel’s Core i7 processor. The sensor chip W83667HG of my Asus P6T Deluxe (LGA 1366 socket) is not yet supported as well. This is a typical symptom when using Linux on too recent hardware. However, if it’s enough for you to read an aggregated CPU temperature instead of eight individual core temperatures, you can force loading the w83627ehf module with

and tune the /etc/sensors3.conf to get rid of false alarms.

Luckily, for the current prerelease 2.6.29-rc6 there’s a very recent bunch of patches available that brings support for W83667HG into the w83627ehf module, whereas coretemp already finds the CPU. Follow this guide on how to compile and install a new kernel the Debian way. A drawback is that source modules from the official Debian repositories might not compile anymore, e.g. the MadWifi modules. You have to get them from the project directly (via SVN). You also need to build a current version of lm-sensors (via SVN) to correctly gather the values from w83627ehf. You could then visualize the values e.g. with gkrellm.

Now have fun stress-testing your system with MPrime.

Update 03/24: It appears that these patches didn’t make it into the final 2.6.29 release.

Update 04/09: The patches went into 2.6.30-rc1.

Two days ago I finished assembling my new workstation and set up a 64-bit Debian ‘lenny’ on it. When I decided to go for Intel Core i7 I wasn’t aware that due to Intel’s Hyperthreading technology each of the four cores is able to handle two threads at once, making it effectively an eight(!)-core system to the OS! Dig this:

I thought that the onboard Intel SATA-RAID controller provided a hardware RAID, but it’s just a software RAID in disguise, so I can well let Linux do the job. Then I had problems setting up WLAN with the Atheros card I built in, but thanks to the MadWifi project I got it running. What else? Yes, Memtest86 v3.5 didn’t boot on Core i7 and/or 6 GB RAM, but Memtest86+ v2.11 did.

I named the new system ‘davinci’, honoring the technological genius Leonardo da Vinci, what is consistent with the weapon-like appearance of the Cooler Master Storm Sniper case.

I proudly list the installed components here:

• CPU: Intel Core i7-920 4×2.67GHz
• RAM: OCZ Platinum 3×2GB DDR3-1600 SDRAM, CL7-7-7-24
• Mainboard: Asus P6T Deluxe, 6×DDR3-1333 slots
• GPU: Zotac GeForce GTX 260 AMP²! Edition, 896MB GDDR3, PCIe 2.0 ×16, 2×DVI
• Harddisks: 2× Western Digital RE3 1TB SATA II, RAID-1
• Optical drive: LG Electronics GH22LS30 SATA, 22×/8×/12× DVD±R/+RW/-RAM
• WLAN: D-Link AirPlusXtremeG DWL-G520, 108Mbit, PCI
• Case: Cooler Master CM Storm Sniper
• Power supply: Cooler Master Silent Pro 700W
• Monitor: Samsung SyncMaster 2693HM 26″ TFT-LCD, 1920×1200, DVI, HDMI, speakers, USB
• Input devices: Logitech Cordless Desktop Wave

Currently I just use Intel’s boxed CPU cooler, so I haven’t done any overclocking yet. I’m having an eye on the Cooler Master V10 that’ll be available by April. With this I aim to overclock the cores to 3.2 GHz for the first time, what I might increase to a maximum of 3.8 GHz one day.

Processing images with my current notebook (1.6 GHz CPU, 1 GB RAM) finally is a real pain in the ass. Having the RAW converter render some JPEGs while the HDR software performs tonemapping, it becomes impossible to do image editing or even browse the web at the same time, as the mouse pointer only moves stutteringly like in those “good old days”. In addition, my 160 GB disk is almost full. My computer usage history documents the years where I got a PC (for the use as a workstation, not counting those few boxes for my server and firewalling experiments): 1995, 1998 and 2001; the notebook’s from 2004. So, it’s about time for an upgrade.

When I first thought about whether it should be a notebook again, I noticed that I don’t use my current notebook “on the road” anyway. And notebooks don’t have the computing power of dedicated desktop machines.

Nerds who buy a new PC every year are probably surprised that a Linux geek like me gets along with that few hardware upgrades. Well, my strategy is more like: Buy rarely, but wisely. I don’t feel well when messing around on a productive machine that is supposed to work; just a matter of experience. Thus, I select the components for my new PC carefully and according to my demands: Fast, multi-core, RAM expandable, good disk size, RAID-1, average good GPU, and—very important—silent operation. The price for the box should traditionally be around EUR 1,500 (formerly ATS 20,000). These are now my considerations:

On April 1st, 2008, it’s 10 years that I’m a netizen. I already had an e-mail address a few months before, but it was April 1998 when I seriously started to use the internet. This gives me some motivation to round it all up a bit.

I took my first programming steps around 1986 on a Texas Instruments TI-99/4A. It was a home computer similar to the later following Commodore 64: It had a black/silver keyboard with a slot for cartridges. It could be connected to an ordinary TV, and own software was saved on ordinary MCs by a special tape drive—It was quite funny to listen to the noisy sounds when playing the MCs on an ordinary player. We had two game cartridges: TI Invaders (screenshots) and Hunt The Wumpus (screenshots). The computer had a TI BASIC interpreter built in. The first program my stepdad taught me was:

I finally had to buy a 160GB notebook disk and already transferred all data from the old 60GB disk to the new one by using an IDE-to-USB converter. After having transferred the remaining data from my aging backup PC, I’ll have 52GB free space.

As that backup host dates back to 2001, has various quirks (Gentoo Linux, /usr on LVM on RAID-5 ) and isn’t really reliable anymore for data backup (the array sometimes dissolves), I’ll have to change some things, but won’t buy new hardware. I’ll replace Gentoo with Debian and arrange the three 80GB disks as a linear NRAID array. The system will take 5GB on RAID-1 over three disks, and the remaining three 75GB partitions become a linear 225GB device. This means that if one disk fails, only the data on that disk is lost. But that doesn’t matter, as I still have the things on my notebook anyway. Various personal and multimedia files will be backed up on DVD unregularly, and my photos are already getting backed up to CDs and stored out of house.

I elaborated the following strategy, mainly leering towards collecting a huge number of photos with my future DSLR camera:

• Short-term storage: Notebook. Here I’ll keep the most recent pictures only, for digital manipulation or simply for quick access.
• Mid-term storage: Backup host (without RAID), DVDs or plain external USB harddisks. Everything that’s on my notebook is also on that disks. But the disks will contain all pictures ever made. If a disk breaks, I’ll either have the data on the notebook or on the DVDs anyway. Only photos undergo the long-term storage.
• Long-term storage of photos: CDs. Only CDs are designed for long-term storage, DVDs aren’t. And to be safe from environmental influences, they are transferred to a house 160km away from here. Sure, CDs only take 700MB, but they’re cheap; for duplicating a possible 500GB disk full of photos, I’ll need 732 CDs. Gee! Maybe I should nevertheless take DVD±Rs for that, where I’ll only need 125 pieces. But they might not be readable anymore in a few years.

The reason for considering an external 500GB disk is that portable data tanks for photographers nowadays have a capacity of 40-80GB. I’ll reuse my 60GB notebook disk by placing it into a corresponding device. If I really manage to fill it on a holiday (>3000 RAWs?? Well, 210 RAWs per day on an exciting 2-week holiday possible, why not?), this already takes 88 CDs for long-term storage. That takes a lot of time to burn, and a lot of room to store. We’ll see how my photo rate of yield really changes.

Addendum: It seems that DVD-RAM is the appropriate media, as it lasts for 30 years according to Wikipedia, whereas CDs only last for a few years. I’d need 15 DVD-RAMs for a filled 60GB data tank, what is a more reasonable number.

In my notebook, I have a 60GB disk, 27+14=41GB considered for the whole Linux system and 11.5GB left for various personal files. In my backup PC, I have a 112GB /home partition (LVM on RAID-5) and 12GB left, another 12GB are available on the /usr partition and 8.6GB in /opt, and as everything is on LVM, it might be resizable. So there’s currently no real need for additional disk space. The only drawback is that I can’t hold all music files on my notebook and that the available space is splitted into two partitions. But I can live with that, I just have to consider that some things will be in a mounted subdirectory.

When space will get small, I’ll buy a 160GB notebook-disk. And my backup PC will be a new one with two 500GB SATA-disks on RAID-1. I won’t take a kind of commercial network storage array, as these aren’t capable of rsync or the like.

I thought about taking RAID-5 again instead of RAID-1. Three 320GB disks are cheaper than two with 500GB, but data redundancy decreases from 2 to 1.5. Only one disk may fail in both cases, no matter if you’ve got two disks at RAID-1 or three disks at RAID-5. In addition, the probability that two disks fail is three(!) times higher when there are three disks as if there were only two. For me, RAID-5 is therefore just a strategy for expansion, not for starting freshly.

Free space is getting small on my workstation as well as on my web-&-mail server, and I want to set up a separate firewall computer anyway. My first idea had been to use my current workstation as server, turn the current server to a firewall and get myself a bleeding-edge shiny new workstation.

But I found a cheaper solution: As the only real reason is missing space - my workstation CPU and RAM are still big enough, and I don’t do any gaming anyway -, I will buy a SATA-controller with 2-3 new 200GB SATA harddisks for my workstation. Then I will purchase an average PC by auction, put my current workstation harddisks (2x 80GB ATA) into it and set up a Debian Testing system on it - this will be the new server then. When everything is tested, the old server with the old, old 2x 4GB SCSI harddisks will become an OpenBSD firewall.

With three 200GB disks on RAID-5, I would then have 400GB space to waste for music, movies, video editing and other things.

Update: As my mainboard doesn’t have 64bit PCI, it’s getting a bit trickier than I thought. I don’t really want to buy 3 new harddisks of a dying technology, so a simple additional ATA controller and a third harddisk of the given type could be the solution for my workstation, but what about the server and firewall then? I still have to think about it.

Update 2: I will keep an even cheaper solution: A simple PCI SATA-controller with 2 new harddisks on (SW-)RAID-1 will do it for a while on my workstation, RAID-5 has to wait for another day. The solution for server and firewall will stay as mentioned above.