Play a CD, get a rootkit installed

If you play a Sony music CD it installs a rootkit on your windows machine. What a stunning idea.

clisp now builds on alpha and ia64

I used the more conservative flags of -D SAFETY=3 -D NO_MULTIMAP_SHM -D NO_MULTIMAP_FILE -D NO_SINGLEMAP -D NO_TRIVIALMAP for the more exotic architectures and it seems to work for alpha and itanium. The hppa buildd failed in a most strange way and I'm waiting with impatience for the results of the m68k and sparc jury :-).

telenet and proxy errors

My bother had severe problem with telenet (the local cable company) and he actually called the helpdesk while I was with him. I noticed the error they made and so could correct it. Normally they ask you to configure http://pac.telenet.be:8080 as pac file. The pack file is:

/* PAC FILE: VERSION MANAGEMENT WITH CVS
 * USE CVSWEB FOR UPDATING !!!
 *
 * Proxy autoconfig file
 *
 * $Revision: 1.29 $
 * $Date: 2004/04/01 06:58:30 $
 *
 */

function FindProxyForURL(url, host)
{
       return "DIRECT";
}

Notice that there is no proxy configured. When this did not work they asked to configure proxy.telenet.be:8080 as a proxy without disabling the pac stuff. This also did not work. After the long call on the expensive helpdesk I tried configuring the proxy and disabling the pac. That worked.

Of course as he was using an inferior operating system as a first step he had reinstalled everything, and lost some data he had forgotten to backup.

I wonder when they will fix the pac file...

Gains and losses

I've removed the hoary and hoary24 sections of the CL package repository I host on p.d.o.
On the other hand I and David D. Smith will adopt the clisp and libsigsegv packages that Will Newton took good care off up until now. We will coordinate the work through the darcs archives on cl-debian.alioth.debian.org.

Heads up: clc security problem

There is a security problem with common-lisp-controller: please read DSA-811 and upgrade as soon as possible.

The irony is that this is caused by moving to a simpler architecture because of (among other problems) security concerns :-(.

The sparc of fustration

sbcl now seems to compile on alpha and ppc again, on amd64 it already worked. sparc is leaving me in suspense already for 6 days now, on the buildd page you can see it going up and down the build queue. Highest position so far: 3. Now it is at 5 again.

Anyway, as sbcl has to wait on glibc to enter testing, which is waiting for gcc-4.0 (nice overview here), I reckon a new testing sbcl is not going to happen anytime soon.

sbcl progress

For the last few days I've been searching for obscure sbcl crashes. It seems at the end it will not only crash if the locale is not known, but also if you are running a threading-enabled sbcl on a 2.4 kernel.
I narrowed the problem down to sbcl crashing in a pretty standard libc call: nl_langinfo. This seems related to the fact that on 2.6 glibc uses futex'es to lock non-re-entrant functions, but on 2.4 it uses signals but I fail to see how this could create a simple crash. Trying to investigate it is rather difficult because the problems changes if you use ltrace or strace, actually the crash location moves sometimes.
So in the end I decided:

• To make sbcl-with-threading die on a 2.4 kernel or if NPTL is not available.
• make separate sbcl images without threading for people that are still using 2.4

.
So now we have multiple cl-debian repositories:

• deb http://people.debian.org/~pvaneynd/cl-sarge-packages ./

  for sarge 2.6 kernel

• deb http://people.debian.org/~pvaneynd/cl-sarge24-packages ./

  for sarge 2.4 kernel

• deb http://people.debian.org/~pvaneynd/cl-hoary-packages ./

  for hoary 2.6 kernel

• deb http://people.debian.org/~pvaneynd/cl-hoary24-packages ./

  for hoary 2.4 kernel

• deb http://people.debian.org/~pvaneynd/cl-breezy-packages ./

  for breezy (only 2.6 kernel)

• deb http://people.debian.org/~pvaneynd/cl-packages ./

  for sid (only 2.6 kernel)

Regarding my darcs repository (at http://cl-debian.alioth.debian.org/repository/pvaneynd: for now it is rather nice to work with darcs. The only problem I'm having is with large changes like moving lazy-sbcl from version 0.9.0 to 0.9.3 with all the changes involved. But going step by step also seems to work well.

3 up, 2 down

The last few days I've been buzy putting my numerous packages into darcs. So far I quite like this versioning system, there are however a few problems:

• it cannot handle epochs
• a lot of my packages' upstream seems to have gone or moved
• some of my packages are native packages, but they are not directly related to debian. I think this was just because the previous maintainer (like me) was part of the upstream team and disliked the additional problems caused by a non-native package. This is starting to bite however and when I try to reverse the situation I often see that the debian version is higher then the last official release of that project. So I need to use epochs. See point 1 :-(.

So my big plan to have all my packages in darcs and attracting a multitude of helpers seems to have fallen foul. I will still try to get this into some shape and make it available...

sbcl has run into some gcc-4 problem on some platforms and cmucl won't even build using gcc-4. A lot of work needs to be done and there is so little time, mostly because of the evil daen free library that is sucking away hours like a baby milk.

New Common Lisp based repositories

I've expanded my repositories on people.debian.org. Add to /etc/apt/sources.list:

• for sarge (aka stable):

  deb http://people.debian.org/~pvaneynd/cl-sarge-packages ./

• for unstable (for packages still in the NEW queue and pre-release stuff):

  deb http://people.debian.org/~pvaneynd/cl-packages ./

• and finally (upon great request) for ubuntu/hoary:

  deb http://people.debian.org/~pvaneynd/cl-hoary-packages ./

I also subscribed the cl-debian mailing list to most Common Lisp packages in debian, so members will see traffic when people upload new versions etc. Feel free to suggest new packages by subscribing them to the pts@qa.

I'm also planning to upload an experimental lazy allocating sbcl to those repositories, but will not upload it into debian proper unless there is popular demand for it.

19b-pre1 as debian package

in the p.d.o/~pvaneynd repository and directly here.

created a MOTU Common Lisp team for ubuntu

I created a Common Lisp MOTU team for ubuntu. As the entry requirements for membership are quite a bit lower for ubuntu then for debian I hope that more people will want to co-maintain packages.
To this end I should investigate version control systems that can handle 'upstream' in CVS or darcs. Maybe the debian directory in darcs and then use dpatch to patch the original source.
In related news I got the lazy allocation patch for sbcl. In the end it gets pretty nice I only need to use the sigsegv handler for static, read-only and linkage-table space. For the dynamic space the lisp uses the C alloc function to allocate memory, and using this function you can mmap memory on the fly. This patch opens the way for having a bigger on-demand dynamic space or a dynamic space with holes. But reception to the patch has been rather cold. I'll try to create a sbcl-lazy package for debian anyway, but only on the p.d.o repository, not in unstable unless people demand it.

ubuntu too?

A user asked if the p.d.o repository would work with ubuntu. A quick check showed that it would not work (libc version conflict), but I guess this is just a false dependency and a quick rebuild should solve it. But I would rather not keep up with 2 build systems and 2 bts. So googling for 'debian developers faq to ubuntu' got me nothing.
I fear I will have to start listening to Queen and talk to the MOTU :-).

Repository for cl packages

To keep the momentum in the cl packages during the freeze I created a apt-getables repository. Add

deb http://people.debian.org/~pvaneynd/cl-packages ./

to /etc/apt/sources.list to get updates Common Lisp packages.

To slime or not to slime

I have mutilated the slime package of Sean Champ with the current CVS HEAD of slime, changed the version number to something without epoch and I'm actually pretty happy with it.

The moral problem now is to upload it, or not. There is a dire need for slime in debian and the package seems to work correctly. The problem is that the packager seems unresponsive to email already for quite a while and I don't want to hijack his package. Should I just bite the bullet and upload or wait a little more... Sigh.

European Common Lisp Meeting

Yesterday I went to the ECLM, which has out-grown Arthurs living room by a very wide margin indeed: 92 participants!
The talks were interesting and diverse. PrimeTrader looked like a real killer-app, but I must admit that I was confused when people starting laughing at the "simplified architecture" because I agree with the 'simple' part. My day-job must do more damage to my judgment then I feared.
The Slime and sbcl presentations both gave me a strong urge to package up some stuff, while the Linj (a non-DFSG free Lisp to Java and back converter) was impressive. Next time I'm forced to write Java I know what to do. For climacs I see a strong future not (only) as a general editor, but as a framework on which people can build LL editors. When the problem becomes too complex for a gui you need a LL, and if you need a LL, why not use an editor that 'knows' you LL and can make editing code a breeze?
The weak-references talk was good because it gave people clear instructions on how to solve most weak references related traps, while the presentation of the two CL vendors was impressively optimistic from the both of them. I've seen other feelings being transmitted.
Last and most importantly: I won a raffle and I could buy a signed copy of Practical Common Lisp and I must say the quote by Xach is correct: the book feels and reads really nice. Impressively nice in fact. I think it will convert people to CL just by the quality of the work itself! Oh, and it is also a nice CL book :-).
Several people insisted to buy me a drink for my debian work. I'm stunned...
My only regret is that we had to leave before I could chat with more people. The way back was a lot longer and slower then how we came, but still I managed to get home before 23:00.

Lazy allocating sbcl, part II

After working on this in my spare time for a long time, I have something that almost works. The lazy sbcl 0.8.20.5 patches will with the following customize-target-features.lisp: :sb-lazy-linkage-table-space, :sb-lazy-dynamic-space, :sb-lazy-read-only-space, :sb-lazy-static-space, :sb-linkage-table-space. They change the specific allocations for that space so the memory is only allocated when it is needed. The net effect is:

$ diff -u /proc/13546/status /proc/13598/status
--- /proc/13546/status  2005-03-18 07:14:16.085383120 +0100
+++ /proc/13598/status  2005-03-18 07:14:16.085383120 +0100
@@ -1,18 +1,18 @@
 Name:  sbcl
 State: T (stopped)
 SleepAVG:      88%
-Tgid:  13546
-Pid:   13546
+Tgid:  13598
+Pid:   13598
 PPid:  8211
 TracerPid:     0
 Uid:   1000    1000    1000    1000
 Gid:   1000    1000    1000    1000
 FDSize:        256
 Groups:        4 6 20 24 27 29 44 109 111 114 1000
-VmSize:          886848 kB
+VmSize:           33624 kB
 VmLck:        0 kB
-VmRSS:     9160 kB
-VmData:          860116 kB
+VmRSS:     8804 kB
+VmData:            6900 kB
 VmStk:        8 kB
 VmExe:      104 kB
 VmLib:     1416 kB
@@ -22,7 +22,7 @@
 ShdPnd:        0000000000000000
 SigBlk:        0000000000000000
 SigIgn:        0000000000000000
-SigCgt:        0000000b000034fe
+SigCgt:        0000000f000034fe
 CapInh:        0000000000000000
 CapPrm:        0000000000000000
 CapEff:        0000000000000000

There are 2 gotchas:

• The patch is at the wrong place (linux-os.c) because there is no abstraction layer for 'get memory' and 'handle memory fault'.
• There is voodoo going on with the dynamic space: if I just allocate one page on every fault the system gets strange errors like 'file is not there' when renaming. If I allocate 10 pages on every fault (like in the patch) it works. This seems to be that sometimes the system needs the memory without first touching it. With the cmucl lazy patches this was due to system calls pointing to unallocated memory, but this does not seems the case here. I should investigate this further.

The good news is that the system build itself a dozen times without problems, and passes all the tests I've thrown it.

So Alexander I would translate "Beutelratte" as "pack rat".

new sbcl package

I've uploaded a split sbcl package. There is now a sbcl-common package that contains the architecture-independent parts of sbcl. You can grab a copy at the new-sbcl directory on people.

What a difference a few days make

Before

No. Maintainer  Source Binary Active Bugs Bug Rate   Archived Bugs Resolve Rate
263 Peter Van Eynde   9(263) 20(209) 24(387)    2.667(693) 230(209)      90.551%(291)

After

No. Maintainer  Source Binary Active Bugs Bug Rate   Archived Bugs Resolve Rate
6 Peter Van Eynde  78(6)  95(15)  32(309)   0.41(1170) 365(126)    91.94%(254)

In other news, a few people interested in packaging Common Lisp stuff for debian came out of the woodwork. I created the cl-debian mailing list to coordinate any work.

Interesting future packages?

While investigating the packages I adopted. I ran across a few interesting unpackaged ones.

• Common Lisp GNU Scientific Library
• elephant: an object database for Common Lisp
• AspectL is a library that provides aspect-oriented extensions for Common Lisp / CLOS
• CL-PREVALENCE is an implementation of Object Prevalence for Common Lisp
• A high-level Common Lisp framework for parallel and distributed programming.
• A Portable Common Lisp Localization Package
• A System Management Substrate for Common Lisp
• UnCommon Web is a Common Lisp web application development framework.

But before I package yet another set of Common Lisp libraries I would like to know if there would be users of these packages. If so, speak up.

I'm insane

I agreed to adopt 70 packages from kmr. Most of them are unlikely to ever get updated again, but just rebuilding them will already take quite a while.

I'm still uncertain if I should go to LSM2005 or Debconf-5, I'm leaning towards LSM2005 because it is in car-range and I would like to meet the other lispers. But their website is rather bad... On the other hand: Helsinki is rather nice...

fosdem 2005

I could manage to catch a few debian talks at fosdem today. As traditional Matteo had a bad evening and just after I left there was another little crisis. Still I managed to see 'Porting to Debian GNU variants: Targetting POSIX instead of GNU/Linux' which was interesting to see how people deal with a slightly broken standard that is not ANSI X3J13. Then the composition of the audience changed for 'The Debian Women Project'. It was a nice talk that gave some interesting ideas and opinions about a movement that seems to deserve my full support. Then came 'Free as in DFSG: an exploration of Debian's views on free software'. I actually feared this talk, maybe due to the harshness of debian-legal. But in the end it turned out that in real life debian developers seem less incendiary then in email and I think a fairly balanced view was presented. This gives me some degree of hope that we will exit the current issues without traumatic events.
Then I joined an old colleague for a talk about creating a stable 2.6 kernel from Alan Cox. This was an interesting talk, not only because he is a good speaker and liked the subject, but mainly because it confirmed my general idea that the new 2.6.x kernels are not for general consumption anymore. The kernel developers depend on the fact that most people will use kernels from distributions. This is not good as it will increase the fragmentation of the kernel. It also means that the debian-kernel group is getting more and more important.

How low can we go?

After restoring the backup the portable will not go on the network with the ethernet connection. The wifi works. Ethernet not. Note that the broken portable was fixed by replacing the motherboard.
Now can anyone tell me why I spend hours fixing this problem?
Hint: I use MAC address filtering.
Hint: the ethernet interface is on the motherboard.
Sigh.

Fixed problems day

The thinkpad got repaired. It still does not see the PCMCIA cards I insert, but at least the DVD works again.
Yesterday I had to restore a file for my wife and the firewire drive also br0ke in XP. So I searched and on this page was a discussion of this bug. It seems the PL3507 controller has a history of "issues". In XP you get a delayed write failure. In Linux just tries to reset the drive and fails. At this location there is an updated firmware (it can also be found at the site of the manufacturer, but that is a JavaScript hell).
Anyway I updated the firmware and now it "just works"! :-)
Now to restore the thinkpad from my backups...

Useful iso 8601 tool

When trying to convert people to start using ISO 8601 a very useful tool in the ISO 8601 implementation page that lists code for a myriad of languages.

sync ; sync ; sync

I managed to get a picture of me in the blogger.com profile, so if you want to scare small children you know where to look :-).

I should decide to what conference I try to go this summer. There is debconf 5 in Helsinki (note the title "See you in hel") which is July 10 to 17. Or I could go to LSM 2005 which is in Dijon (much closer) and from July 5 to 9 and is in general a hotbed of Common Lisp activity. I'll wait a little more until the programms pan out, but after the Danish disaster of last year I doubt I can convice my wife to go to Helsinki. She wants Calabria :-).

I made almost no progress with the lazy memory allocation in sbcl. I think I know where the problem is, but I don't have enough time to check it out :-(. Also I wonder why nobody seems to care about this problem. What do you do on a 64 bit machine, do you start by allocating 2**41 bytes? Does the OS permit this? What does root think of this? Or do you want to have to select at startup how large your image might become? If not, you need lazy allocation.

Found the USB problem

I've been having USB problems with my own kernels recently. Now I found why. If you cannot use your USB memory stick, in my case my Iomega stick with 2.6.* and you get the message "uba: device 9 capacity nsec 0 bsize 512". It could be because you enabled BLK_DEV_UB. If you do the ub driver kicks in and it seems to dislike memory sticks with avengence. The debian images have this disabled already.

Now I only need to find out why my configuration causes the transfer to the PL3507-based 1394 bridge doesn't work my my kernel and does with the debian image. Sigh.