Can't we all just get along?

(These notes were used as the basis of a paper given at the 2003 International Lisp Conference.)

1. The analogy to Greenspun

Any sufficiently large lisp application contains a badly thought-out, poorly implemented version of Common Lisp extensions.

2. History

What we said we needed... in 1989 (the last ever SLUG meeting, i.e. the zeroth ALU meeting).

• defsystem
• ffi
• multithreading
• resources
• network streams (e.g. tcp)

Everybody was doing bits and pieces on their own, as required.

Lack of standard (or, in some cases, any) de-facto extensions really hindered collaboration between early users of diverse lisps.

So, 14 years later, what's changed?

• The internet has drawn us together.
• Lisp has followed other communities into an era of more freely available and discoverable public libraries.
• Collaboration on public libraries is also now possible amongst temporally and geographically dispersed individuals.

It may not be rocket science, but it's possible now with an order of magnitude less effort.

Discoverability was the key.

This lead to individuals and groups implementing their own needs (cross-platform or not).

3. What's happened since then

3.1. ANSI Common Lisp

• standard since 1994, would be procedurally painful to reopen
• problems with standard:
  • minor typos
  • gaps in the description (pathnames are a case in point, logical-pathnames especially so)
  • things just not even mentioned by the standard (defsystem etc etc) - a lot of work required to get them in now
  • what's in it for the vendors?
• repeated suggestions: why can't we add such-and-such?
• KMP's take on this at ILC2002:
  • standards are permanent documents, attempts at consensus;
  • standards enable interchangeable parts and "second sourcing";
  • standards are expensive to produce, there are versioning problems, they presuppose that "two ways of doing foo is bad";
  • something called substandards (hopefully above par...).

3.2. Technology

The arrival of the internet drove the need to interface to a whole bunch of standards.

• html
• http
• smtp
• ftp
• xml
• sql
• numerous other interchange standards (COM, CORBA, ...)

The flip side is: the internet (e.g Google) has solved the first-order discoverability problem. These days, nobody needs to write their own search engine. Or even: nobody needs to "know the url".

(If you don't know the URL for google, please send email to norvig@google.com.)

All this has been accompanied by social changes:

• new verbs: download, google, warchalking;
• cheap computing;
• omnipresent computers;
• the network is everywhere;
• omnipresent data capture / databasing (and hence privacy issues);
• the internet knows everything - it's our global collective memory.

Mobility has improved:

• the consumer can download and move the bytes to the location, very easily;

• the producer can relocate the bytes, leave forwarding pointers behind; and remotely manage the whole process;

• the servers (and hence the bytes) can be located anywhere and tend to be managed under some pretence of professionality.

4. Current state of extensions to Common Lisp

4.1. Bodies of information

• Google (were there any others?)
• Vendor sites
• Cliki
• Franz's "Dynamic Learning Center"
• Common Lisp cookbook

4.2. Bodies of code

• Vendor sites
• CLOCC
• Sourceforge (com-lisp-utils package)
• ACL-COMPAT
• CLIM-LISP
• CL-HTTP
• McCLIM
• various proprietary systems
• a vast and growing set of libraries and other eclectic conglomerations of software, some of it with a specific use, some of it more general - as these emerge they suggest new "de-facto extensions" (e.g. cl-typesetting)

5. Where do we go from here?

Our requirements might be...

• consensus on what the modules are
• existance
• available
• discoverable / announced
• affordable (people have a right to try to get paid for their work; people imagine they have a right to obtain quality software on the cheap)
• coallescing the fine details of overlapping existing packages
• modularity
• actually doing it this time (c.f. 1989)
• [open to the floor...]

Solving these requirements... (some of these are clearly no-brainers)

• Reopen ANSI (please reject this one...)
• A different standard or semi-standard process
• De-facto / community of use
• Noticeboards (cliki / cookbook)
• RFC or equivalent
• ALU
• comp.lang.lisp
• let google look after it
• focus groups (track 2 sometime today?)
• [open to the floor...]

$Id: //info.ravenbrook.com/doc/2003/10/14/get-along/index.html#3 $