Kvardek Du


Re: Querying plists

Zach's Querying plists blog post showcases a neat little querying DSL for plists. I couldn't shake the feeling that it looked an awful lot like pattern matching. I've often been impressed by optima, but I barely get to use it, so I thought I should try and see what querying plists looked like using pattern matching.

Here's what I came up with. Zach's example

(query-plists '(:and (:= :first-name "Zach") (:= :state "ME")
                     (:not (:= :last-name "Beane")))
(remove-if-not (lambda-match ((plist :first-name "Zach" :state "ME"
                                     :last-name (not "Beane")) t))

It turned out more succinct than I initially expected! Also, it's trivially adaptable to other kinds of objects. E.g., given the following class:
(defclass person ()
  ((first-name :initarg :first-name)
   (last-name  :initarg :last-name)
   (state      :initarg :state)))
all we have to do is swap plist with the class name person and we're all set:
(remove-if-not (lambda-match ((person :first-name "Zach" :state "ME"
                                      :last-name (not "Beane")) t))

We can't quite define something exactly like Zach's query-plists because, AFAICT, optima's patterns are not first-class objects but perhaps we can cheat a little bit.
;; naming things is hard. :-/
(defmacro matchp (pattern) `(lambda-match (,pattern t)))
(defun filter (predicate list) (remove-if-not predicate list))

(filter (matchp (plist :first-name "Zach" :state "ME"
                       :last-name (not "Beane")))

Making this equally succinct when the query criteria are not constant is a challenge for another day and makes it clear that matchp is a lousy abstraction. ;-)



SLIME's just got a new contrib called slime-macrostep, courtesy of Jon Oddie who also wrote the underlying macrostep package.

And what is slime-macrostep? It's an interactive inline macro-expander. Have a look:

In this quick demo, using a CFFI example, I start by expanding the top-level WITH-FOREIGN-OBJECT form, then I expand the WITH-ALIEN form, but regret it and collapse it back. Then I proceed to expand everything else, including compiler macros!

A nice feature of slime-macrostep is that the expansions are annotated to show which forms are further expandable and macrostep-expand will jump automatically to the next expandable form. That's what makes it a stepper: pressing e repeadly will step through the macroexpansion. Plus, it expands macrolets!

If you'd like to try it out, please grab SLIME's bleeding edge (via MELPA or Git). It's enabled by default if you use the slime-fancy meta contrib. Feedback is most welcome.


Common Lisp Recipes: A Problem-Solution Approach

Edi Weitz, author of things like CL-PPCRE and Hunchentoot (amongst many other pieces of ediware), has just published a new book, “Common Lisp Recipes: A Problem-Solution Approach”. I've ordered mine. Get yours from Apress or a European Amazon store.

Common Lisp Recipes is a collection of solutions to problems and answers to questions you are likely to encounter when writing real-world applications in Common Lisp. Written by an author who has used Common Lisp in many successful commercial projects over more than a decade, this book is the first Common Lisp book which covers areas as diverse as web programming, databases, graphical user interfaces, communication with other programming languages, multi-processing, and mobile devices as well as debugging techniques and optimization, to name just a few. It is organized around specific problems or questions each followed by ready-to-use example solutions and clear explanations of the concepts involved, plus pointers to alternatives and more information. Each recipe can be read independently of the others and thus the book will hopefully earn a special place on your bookshelf as a reference work you always want to have within reach.

Common Lisp Recipes is aimed at programmers who are already familiar with Common Lisp to a certain extent but do not yet have the experience you typically only get from years of hacking in a specific computer language. It is written in a style that mixes hands-on no-frills pragmatism with precise information and prudent mentorship.

If you feel attracted to Common Lisp's mix of breathtaking features and down-to-earth utilitarianism, you'll also like this book.


My answer to the Pretty Printer Puzzle

As promised, I'll describe my solution to the Pretty Printer Puzzle I proposed last week. To recap, we wish to pretty print a Lisp form to a string and identify the textual positions of arbitrary subforms therein.

First attempt

A couple of folks proposed a clever solution that goes like this: (1) replace the CAR of each subform with some unique token (a gensym should be close enough), (2) pretty-print that, (3) find the token positions and replace them with the original CARs.

Problem with this solution: changing the form affects pretty-printing. In particular, it will no longer be able to properly indent macros and special forms.

Second attempt

Another approach is to pretty-print then read the form back and track positions by either using a custom reader that keeps track of form positions (such as hu.dwim.reader) or instrumenting the standard readtable by wrapping the #\( reader-macro and doing the reading from a so-called form-tracking-stream.

Problem with this solution: it breaks down if the form contains unreadable objects.

Third attempt, getting closer

The pretty printer is customisable through a pprint-disptach-table. It is analogous to the reader's readtable. So, we try and instrument it like in the previous approach. Each time a list is about to be pretty-printed, we store the current position in the output stream.

Problem: we have been defeated by the pretty printer's intermediate buffer. Turns out the pretty printer only writes to the output stream at the very end of the process. Back to the drawing board.

Fourth and final attempt

But these attempts have not been in vain, and my final solution involves elements from all three. It goes like this:
  1. Pretty print the form normally.
  2. Pretty print the form again, this time instrumenting the pprint-dispatch-table to wrap lists with some token identifying the subform being printed. (I decided to use the unicode range U+E000..U+F8FF which is reserved for private-use, which seemed neat.) This messes up the pretty-printing a little bit, but not too much, it turns out.
  3. Cross-reference the token positions in #2 with #1 by taking advantage of the fact these outputs differ by whitespace (and tokens) only!

And that's it!

With this tool in hand, there are some interesting utilities that can be built in SLIME, but that's another blog post. :-)


Pretty printer puzzle

This past week, I came across a Lisp challenge that turned out to be trickier than one might expect at first. I needed to pretty-print a Lisp form to a string and identify the positions of certain subforms. Here's an example:

CL-USER> (with-output-to-string (*standard-output*)
           (pprint '(defun factorial (x) (if (zerop x) 1 (* x (factorial (1- x)))))))
      (* X (FACTORIAL (1- X)))))"
In this, output the bounding indices of the IF form are 24 and 77. In other words:
CL-USER> (subseq * 24 77)
      (* X (FACTORIAL (1- X))))"
The challenge, then, is to write a function that, given a form and list of subforms, returns a string with the pretty-printed output and a list of bounding indices for each subform. E.g.
CL-USER> (pprinted-bounds '(defun factorial (x) #1=(if (zerop x) 1 (* x #2=(factorial (1- x)))))
                          (list '#1# '#2#))
      (* X (FACTORIAL (1- X)))))"
((24 77) (57 75))
I'll post my solution later. Have fun. :-)


LOOP quiz

  1. Does (loop for i below 10 finally (return i)) return 9 or 10?
  2. Does (loop for i upto 10 finally (return i)) return 10 or 11?
  3. What does (loop for i below 10 for j upto 10 finally (return (list i j))) return?
  4. What about (loop for i below 10 and j upto 10 finally (return (list i j)))?

I stumbled upon the semantics of this last example in a recent bugfix and thought it was worth sharing. (Reminded me of the joke about what's hard in CS, too.)

Turns out LOOP's FOR ... AND not only mimics LET (rather than LET*) in terms of binding visibility, it also influences when the loop termination checks take place. That was new to me. I initially expected examples 3 and 4 to return the same values. What about you? Which ones, if any, did you get wrong? :-)

P.S.: LOOP for Black Belts is my favorite LOOP tutorial.



Taylor Campbell's paredit is one of those Emacs extensions that I can't live without. In a nutshell, it forces you to deal with Lisp code exclusively via operations that don't introduce unbalanced parenthesis (or other similar violations of structure). The genius about this approach is that it completely eliminates the step of making sure parentheses are properly balanced after you write or edit a piece of code. After you get used to paredit, performing — or even watching — manual parenthesis balancing becomes painful.

Recently, I've come across these two introductions to paredit:

  1. Emacs Rocks! Episode 14: Paredit
  2. The Animated Guide to Paredit
So, if you're still not using paredit, have a look at those and give it a try. At first you might feel like the karate kid doing frustrating chores — you can always take a break with M-x paredit-mode — but I promise it'll soon pay off!