GNU/Linux - 3

CUDA is a technology by NVIDIA to accelerate scientific computations by the help of GPUs. Unfortunately, Debian isn’t supported officially yet. The toolkit for Ubuntu should come closest to it. First, the toolkit has to be installed, e.g. to $HOME/share/cuda, and then the SDK, e.g. to $HOME/share/NVIDIA_CUDA_SDK. These two paths shall be referred to as $CUDA_PATH and $SDK_PATH in the following. GCC and g++ have to be downgraded to version 4.1 (from ‘etch’), as 4.2 (from ‘lenny’) doesn’t work with CUDA yet.

Like described in $SDK_PATH/ReleaseNotes.html, $CUDA_PATH/bin has to be in the $PATH and $CUDA_PATH/lib in the $LD_LIBRARY_PATH. Unfortunately, this isn’t enough for Debian to successfully compile the examples (via cd $SDK_PATH && make): The $CUDA_INSTALL_PATH in line 38 of the Makefile $SDK_PATH/common/common.mk has to be corrected to the $CUDA_PATH.

To compile the GLUT code, the development package freeglut3-dev has to be installed. With that it’s now possible to compile the examples; the binaries go into $SDK_PATH/bin/linux/release.

If your amavisd-new suddenly doesn’t want to start anymore, reporting that some required Perl modules aren’t installed allegedly, then maybe the cause is something like

Took me 2 hours. I need that CPAN stuff apart from Debian at least for Geo::IPfree to map IPs to country names for my web statistics.

It’s a really good tip to use the -x flag to debug bash scripts. I had a problem that GDM wasn’t starting anymore after the regular aptitude upgrade. I didn’t find a hint in the log files and had a look at its init-script /etc/init.d/gdm, which appeared quite complex to me: It sources various config files and calls macros that are defined in there. How the heck should I trace the problem down? The best way was to change the initial line to

what starts the debug mode when the script is invoked. Here’s what it does: It prints every single command that is executed line by line in a fully expanded form, no matter if macros were defined or several other shell scripts are invoked from within. The lines are indented by a sequence of plus symbols to represent the invocation depth of the other scripts. By this, I saw that the last calls referred to splashy, a bootsplash package that I had installed for test purposes but never had configured. The solution was to simply remove that package.

Another tip: Don’t do experiments on a system that is supposed to work! This also means to not install the complete Debian ‘testing’ version on it. If you need a selected number of some more recent software, upgrade it individually, but keep the basic packages ‘stable’. On the other hand, you wouldn’t learn much if you weren’t forced to fix things once in a while.

In short: Debian lenny’s current libc6 version 2.7 doesn’t work on kernel 2.6.9 with SMP enabled. Downgrade to libc6 2.6 if you can’t change the kernel. You might have to do this on a non-2.6.9 (or >2.6.9) system.

Here’s the story: I use to maintain a duplicate system (on real hardware) with the same package structure as my virtual private server to be able to run through the upgrade procedures in advance. Yesterday, the regular upgrade included a new libc6 package going from 2.6.1 to 2.7. As everything worked well on the duplicate system, I started the apt-get upgrade on the virtual machine. But then dpkg crashed with an “Unknown error 530”. I could still navigate through the file system, but directory listings returned e.g.

and the system slowly began to “fade”. I tried a reboot, but the system didn’t come up again, showing a problem with the quota in the log (“Running vzquota off failed... vzquota: (error) Quota off syscall for id 12345: Device or resource busy”).

I opened a ticket at my provider, and the guy was telling me that the system were completely broken and had to be reinstalled! I was shocked and didn’t believe him. The virtualization engine provides a repair mode, but as I chroot’ed into the mounted root directory of my system, those errors came up again.

I asked the support guy to have a look at that article, but he didn’t seem to find it useful. As it became late afternoon and their support went off duty, I synced the mounted repair directory to my duplicate system overnight (about 5.6GB; I had to install rsync into the temporary system). This should enable me to chroot there and make experiments.

As I finally chroot’ed into that directory this morning, the system was working perfectly! So it couldn’t be completely broken. Another inquiry on Google unveiled this new posting. So the problems are really due to the new libc6 version! It worked on the duplicate system as it runs on kernel 2.6.22, and the virtual server depends on a custom 2.6.9! And that’s also why I couldn’t repair it with their repair system, as it uses the same kernel! On the duplicate system I looked into /var/log/dpkg.log what the previous version of libc6 was, and downloaded the libc6_2.6.1-1+b1_i386.deb from a Debian mirror. I did the downgrade and synced the directory back to the virtual server’s repair directory. I went out of repair mode and—lo and behold!—the system came up again! I also had to downgrade the locale package to match with the old libc6 version.

The support guy was glad that I provided them with a solution for a problem that could emerge for other customers as well. I’ll better stick to Debian ‘stable’ as soon as it is out.

Addendum 12/08: The correct way to fetch previous package versions is via the Debian snapshot archive, as they’re removed from the official mirrors within a short time.

I wanted to do a regular upgrade of xserver-xorg-core in Debian ‘testing’. Unfortunately, dpkg complained about a file ./usr/lib/xorg/modules/extensions/libglx.so not being present, although Nvidia’s installer had placed it right there and it was still present. Because of that broken upgrade my X server couldn’t start anymore. Removal of Nvidia or xorg didn’t change anything on this situation. With Google I saw that someone found a solution by installing nvidia-glx-legacy temporarily. Here’s what to do:

1. Remove Nvidia by calling sh /path/to/NVIDIA-$SOMETHING --uninstall.
2. Install nvidia-glx-legacy. This package depends on a linux-image-2.6.18-n-486, what will therefore be installed and will replace your current symlinks /vmlinuz and /initrd.img.
3. Install/upgrade xserver-xorg-core. (The top-dependency would be xorg).
4. Remove nvidia-glx-legacy and the above mentioned linux-image-$SOMETHING again and purge their configuration files (Key “_” in aptitude).
5. Restore your original symlinks /vmlinuz[.old] and /initrd.img[.old]. You’ll have to run lilo. Run lilo. Did I mention to run lilo?
6. Reinstall Nvidia and restart gdm.

Recently, I had another issue with the Nvidia installer: It didn’t work anymore due to the activated paravirtualization feature in Debian’s default kernel what conflicts with the GPL-incompatible module nvidia.ko. Here’s a solution I found in the web:

1. Install linux-source-2.6.xx-n-686.
2. Uncompress /usr/src/linux-source-2.6.xx-n-686.tar.bz2.
3. Delete the symlink /lib/modules/2.6.xx-n-686/build and make it new by ln -s /usr/src/linux-source-2.6.xx /lib/modules/2.6.xx-n-686/build.
4. Copy .config from headers to sources by cp /usr/src/linux-headers-2.6.xx-n-686/.config /usr/src/linux-source-2.6.xx.
5. In the sources dir make menuconfig and disable paravirtualization in section ‘processor features’.
6. make prepare.
7. make scripts.
8. Now compile and install Nvidia with its own installer.

For the sake of documentation, I list the enhancements I configured for SpamAssassin since February:

• I use SA’s internal sa-update script nightly to update the standard rules that are changed between the releases of new versions. The standard channel is updates.spamassassin.org. The files automatically go into /var/lib/spamassassin/$VERSION and Debian automatically finds it, as can be seen from a call of spamassassin -D --lint.
• In addition to the previous, I call sa-compile to compile the body rulesets into binary form. This makes checking of body rules more efficient. To enable SA using these binary rules in /var/lib/spamassassin/compiled/$VERSION, I have to activate the Rule2XSBody plugin in /etc/spamassassin/v320.pre.
• I don’t update the SARE rules against stock spam manually anymore, but also use the provided sa-update channel for it.

Recently, PDF spam has become “popular”. Therefore I enabled some more things to accomplish this:

• The ClamAV virus scanner provides inofficial databases by Sanesecurity to catch various sorts of spam. So there’s actually some kind of redundancy, as the virus scanner and the anti-spam filter are now sharing some responsibility. As not a single virus occured at my site in the recent months, this has now changed in some way.
• I installed the PDFInfo plugin from SARE and enabled it in /etc/spamassassin/init.pre.

Sure, as soon as we catch enough of that new PDF spam, spammers might change to some other document file format, such as DOC or RTF or even ODF, and we are forced to scan those attached documents for spam text or even for contained images that contain spam text, what we are already considering with FuzzyOCR. There must be some better way, actually.

However, I had to reduce the score of the Botnet plugin, as the default value of 5 points is way too high. Maybe I should add the plugin that checks the operating system such that only botnet clients using that Windows crap get high scores.

The fight continues.

I posted my question to comp.os.linux.security and continued the discussion on the Serendipity mailing list. For the sake of documentation and to provide another spot in the net with a solution, I repeat the posting here:

I wondered about strange HTTP connections from 127.0.0.1 appearing in my access.log at irregular times:

What irritated me was that those requests originate locally, are invalid (400 = Bad Request) and have no User-Agent identification string. [...]

I finally found out that this ought to be Apache-2.2’s internal dummy connections. They had the above form as long as my Apache-SSL config looked like

Now, I use the IP instead of the ‘*’ and—lo and behold—the requests transform into

I didn’t want to spend much time trying to understand what that dummy connections are good for. It seems like Apache2 kills some of its children such that the number of MaxSpareServers isn’t exceeded. And I wasn’t aware that the Apache syntax ‘*:443’ is somehow deprecated.

I want to keep track of the spam scores that show up in my /var/log/mail.log on my Debian Etch machine. My idea was to run a shell script grep’ing the wanted info out of the mail.log right before that file is to be rotated. logrotate provides configuration options for how and when log files should be rotated. This includes a possibility to run a script right before and after the rotation is done. So, I created a configuration /etc/logrotate.d/mail like

sysklogd is Debian’s package containing syslogd and klogd. The syslog itself already cares for rotating the basic log files, and this includes /var/log/mail.{log,info,warn,err}. sysklogd contains the cron-jobs /etc/cron.daily/sysklogd and /etc/cron.weekly/sysklogd, both of which contain logic for rotating the system’s log files. To obtain the list of the log files, the command syslogd-listfiles is issued in both of the scripts—the weekly script uses the option --weekly. When I issue syslogd-listfiles, I only get /var/log/syslog as answer, whereas with --weekly I get:

So I have to use the -s switch to exclude mail.log and mail.info such that logrotate can take care of them. As I also have a separate config for /var/log/messages in /etc/logrotate.d/messages, I modified the call of syslogd-listfiles --weekly in /etc/cron.weekly/sysklogd to