Manuel Simoni on CL's control flow primitives

Manuel Simoni dusts his Axis of Eval blog off and writes about Common Lisp's BLOCK / RETURN-FROM and UNWIND-PROTECT. A summary for non-Lispers.

Reddit 1.0

As many Lisp programmers may remember, Reddit was initially written in Common Lisp. Then, in late 2005, it was rewritten in Python. Much discussion ensued. Steve Huffman (aka spez, current CEO of Reddit) wrote about why they switched to Python. The late Aaron Swartz also wrote Rewriting Reddit, focusing on the Python side of things.

Last week, that original, Common Lisp version of Reddit was published on GitHub: reddit-archive/reddit1.0. There's no documentation, but we know from the articles above that it ran on CMUCL. reddit.asd is perhaps a good entry point, where we can see it used TBNL, CL-WHO and CLSQL, a common toolset at the time.

It's missing various bits of static HTML, CSS and JS, so I don't think you can actually run this website without a fair bit of scrapping and stitching from the Wayback Machine. The code is quite readable and clean. It's not terribly interesting, since it's a fairly simple website. Still, it's a website that grew to become the 6th most visited worldwide (with 542 million monthly visitors) and now has 230 employees, says Wikipedia, so it's nice to see what it looked like at the very beginning.

A Lisp REPL in your pocket

Thanks to Polos Ruetz, you can now play with Common Lisp directly on your Android phone. All you need to do is install CL REPL from the Google Play Store. CL REPL is one of the examples part of EQL5-Android which is built on top of EQL5, a project that marries ECL with Qt.

CL REPL

A Common Lisp REPL with command line and history, plus a simple editor with syntax highlighting, simple visual paren-matching, a file dialog for opening/saving files, and a simple debug dialog. It uses the ECL implementation for the Lisp side, and Qt5/QML for the UI. This is an open source project (see EQL5-Android).

(via @dk_jackdaniel)

Re: Querying plists

Zach's Querying plists blog post showcases a neat little querying DSL for plists. I couldn't shake the feeling that it looked an awful lot like pattern matching. I've often been impressed by optima, but I barely get to use it, so I thought I should try and see what querying plists looked like using pattern matching.

Here's what I came up with. Zach's example

(query-plists '(:and (:= :first-name "Zach") (:= :state "ME")
                     (:not (:= :last-name "Beane")))
              *people*)

becomes:

(remove-if-not (lambda-match ((plist :first-name "Zach" :state "ME"
                                     :last-name (not "Beane")) t))
               *people*)

It turned out more succinct than I initially expected! Also, it's trivially adaptable to other kinds of objects. E.g., given the following class:

(defclass person ()
  ((first-name :initarg :first-name)
   (last-name  :initarg :last-name)
   (state      :initarg :state)))

all we have to do is swap plist with the class name person and we're all set:

(remove-if-not (lambda-match ((person :first-name "Zach" :state "ME"
                                      :last-name (not "Beane")) t))
               *people*)

We can't quite define something exactly like Zach's query-plists because, AFAICT, optima's patterns are not first-class objects but perhaps we can cheat a little bit.

;; naming things is hard. :-/
(defmacro matchp (pattern) `(lambda-match (,pattern t)))
(defun filter (predicate list) (remove-if-not predicate list))

(filter (matchp (plist :first-name "Zach" :state "ME"
                       :last-name (not "Beane")))
        *people*)

Making this equally succinct when the query criteria are not constant is a challenge for another day and makes it clear that matchp is a lousy abstraction. ;-)

slime-macrostep

SLIME's just got a new contrib called slime-macrostep, courtesy of Jon Oddie who also wrote the underlying macrostep package.

And what is slime-macrostep? It's an interactive inline macro-expander. Have a look:

In this quick demo, using a CFFI example, I start by expanding the top-level WITH-FOREIGN-OBJECT form, then I expand the WITH-ALIEN form, but regret it and collapse it back. Then I proceed to expand everything else, including compiler macros!

A nice feature of slime-macrostep is that the expansions are annotated to show which forms are further expandable and macrostep-expand will jump automatically to the next expandable form. That's what makes it a stepper: pressing e repeadly will step through the macroexpansion. Plus, it expands macrolets!

If you'd like to try it out, please grab SLIME's bleeding edge (via MELPA or Git). It's enabled by default if you use the slime-fancy meta contrib. Feedback is most welcome.

Common Lisp Recipes: A Problem-Solution Approach

Edi Weitz, author of things like CL-PPCRE and Hunchentoot (amongst many other pieces of ediware), has just published a new book, “Common Lisp Recipes: A Problem-Solution Approach”. I've ordered mine. Get yours from Apress or a European Amazon store.

Common Lisp Recipes is a collection of solutions to problems and answers to questions you are likely to encounter when writing real-world applications in Common Lisp. Written by an author who has used Common Lisp in many successful commercial projects over more than a decade, this book is the first Common Lisp book which covers areas as diverse as web programming, databases, graphical user interfaces, communication with other programming languages, multi-processing, and mobile devices as well as debugging techniques and optimization, to name just a few. It is organized around specific problems or questions each followed by ready-to-use example solutions and clear explanations of the concepts involved, plus pointers to alternatives and more information. Each recipe can be read independently of the others and thus the book will hopefully earn a special place on your bookshelf as a reference work you always want to have within reach.

Common Lisp Recipes is aimed at programmers who are already familiar with Common Lisp to a certain extent but do not yet have the experience you typically only get from years of hacking in a specific computer language. It is written in a style that mixes hands-on no-frills pragmatism with precise information and prudent mentorship.

If you feel attracted to Common Lisp's mix of breathtaking features and down-to-earth utilitarianism, you'll also like this book.

My answer to the Pretty Printer Puzzle

As promised, I'll describe my solution to the Pretty Printer Puzzle I proposed last week. To recap, we wish to pretty print a Lisp form to a string and identify the textual positions of arbitrary subforms therein.

First attempt

A couple of folks proposed a clever solution that goes like this: (1) replace the CAR of each subform with some unique token (a gensym should be close enough), (2) pretty-print that, (3) find the token positions and replace them with the original CARs.

Problem with this solution: changing the form affects pretty-printing. In particular, it will no longer be able to properly indent macros and special forms.

Second attempt

Another approach is to pretty-print then read the form back and track positions by either using a custom reader that keeps track of form positions (such as hu.dwim.reader) or instrumenting the standard readtable by wrapping the #\( reader-macro and doing the reading from a so-called form-tracking-stream.

Problem with this solution: it breaks down if the form contains unreadable objects.

Third attempt, getting closer

The pretty printer is customisable through a pprint-disptach-table. It is analogous to the reader's readtable. So, we try and instrument it like in the previous approach. Each time a list is about to be pretty-printed, we store the current position in the output stream.

Problem: we have been defeated by the pretty printer's intermediate buffer. Turns out the pretty printer only writes to the output stream at the very end of the process. Back to the drawing board.

Fourth and final attempt

But these attempts have not been in vain, and my final solution involves elements from all three. It goes like this:

1. Pretty print the form normally.
2. Pretty print the form again, this time instrumenting the pprint-dispatch-table to wrap lists with some token identifying the subform being printed. (I decided to use the unicode range U+E000..U+F8FF which is reserved for private-use, which seemed neat.) This messes up the pretty-printing a little bit, but not too much, it turns out.
3. Cross-reference the token positions in #2 with #1 by taking advantage of the fact these outputs differ by whitespace (and tokens) only!

And that's it!

With this tool in hand, there are some interesting utilities that can be built in SLIME, but that's another blog post. :-)