Lazy allocating sbcl, part II

After working on this in my spare time for a long time, I have something that almost works. The lazy sbcl 0.8.20.5 patches will with the following customize-target-features.lisp: :sb-lazy-linkage-table-space, :sb-lazy-dynamic-space, :sb-lazy-read-only-space, :sb-lazy-static-space, :sb-linkage-table-space. They change the specific allocations for that space so the memory is only allocated when it is needed. The net effect is:

$ diff -u /proc/13546/status /proc/13598/status
--- /proc/13546/status  2005-03-18 07:14:16.085383120 +0100
+++ /proc/13598/status  2005-03-18 07:14:16.085383120 +0100
@@ -1,18 +1,18 @@
 Name:  sbcl
 State: T (stopped)
 SleepAVG:      88%
-Tgid:  13546
-Pid:   13546
+Tgid:  13598
+Pid:   13598
 PPid:  8211
 TracerPid:     0
 Uid:   1000    1000    1000    1000
 Gid:   1000    1000    1000    1000
 FDSize:        256
 Groups:        4 6 20 24 27 29 44 109 111 114 1000
-VmSize:          886848 kB
+VmSize:           33624 kB
 VmLck:        0 kB
-VmRSS:     9160 kB
-VmData:          860116 kB
+VmRSS:     8804 kB
+VmData:            6900 kB
 VmStk:        8 kB
 VmExe:      104 kB
 VmLib:     1416 kB
@@ -22,7 +22,7 @@
 ShdPnd:        0000000000000000
 SigBlk:        0000000000000000
 SigIgn:        0000000000000000
-SigCgt:        0000000b000034fe
+SigCgt:        0000000f000034fe
 CapInh:        0000000000000000
 CapPrm:        0000000000000000
 CapEff:        0000000000000000

There are 2 gotchas:

• The patch is at the wrong place (linux-os.c) because there is no abstraction layer for 'get memory' and 'handle memory fault'.
• There is voodoo going on with the dynamic space: if I just allocate one page on every fault the system gets strange errors like 'file is not there' when renaming. If I allocate 10 pages on every fault (like in the patch) it works. This seems to be that sometimes the system needs the memory without first touching it. With the cmucl lazy patches this was due to system calls pointing to unallocated memory, but this does not seems the case here. I should investigate this further.

The good news is that the system build itself a dozen times without problems, and passes all the tests I've thrown it.

So Alexander I would translate "Beutelratte" as "pack rat".

new sbcl package

I've uploaded a split sbcl package. There is now a sbcl-common package that contains the architecture-independent parts of sbcl. You can grab a copy at the new-sbcl directory on people.

What a difference a few days make

Before

No. Maintainer  Source Binary Active Bugs Bug Rate   Archived Bugs Resolve Rate
263 Peter Van Eynde   9(263) 20(209) 24(387)    2.667(693) 230(209)      90.551%(291)

After

No. Maintainer  Source Binary Active Bugs Bug Rate   Archived Bugs Resolve Rate
6 Peter Van Eynde  78(6)  95(15)  32(309)   0.41(1170) 365(126)    91.94%(254)

In other news, a few people interested in packaging Common Lisp stuff for debian came out of the woodwork. I created the cl-debian mailing list to coordinate any work.

Interesting future packages?

While investigating the packages I adopted. I ran across a few interesting unpackaged ones.

• Common Lisp GNU Scientific Library
• elephant: an object database for Common Lisp
• AspectL is a library that provides aspect-oriented extensions for Common Lisp / CLOS
• CL-PREVALENCE is an implementation of Object Prevalence for Common Lisp
• A high-level Common Lisp framework for parallel and distributed programming.
• A Portable Common Lisp Localization Package
• A System Management Substrate for Common Lisp
• UnCommon Web is a Common Lisp web application development framework.

But before I package yet another set of Common Lisp libraries I would like to know if there would be users of these packages. If so, speak up.

I'm insane

I agreed to adopt 70 packages from kmr. Most of them are unlikely to ever get updated again, but just rebuilding them will already take quite a while.

I'm still uncertain if I should go to LSM2005 or Debconf-5, I'm leaning towards LSM2005 because it is in car-range and I would like to meet the other lispers. But their website is rather bad... On the other hand: Helsinki is rather nice...