Setting up CommonQt on OSX

CommonQt can be tricky to setup due to its dependencies from C++ land. Tricky enough to warrant writing down the steps I went through to get it running on Mac OS 10.7:

1. If you don't already have Xcode, install it because you'll need g++.

2. Grab and install the Qt Libraries (not the Qt SDK). I used version 4.7.4.

3. Install CMake. (I used brew install cmake.)

4. Download, compile and install the SMOKE library:

   $ git clone git://anongit.kde.org/smokegen
   $ cd smokegen
   $ cmake .
   $ make install
   
   $ cd ..
   $ git clone git://anongit.kde.org/smokeqt
   $ cd smokeqt
   $ cmake -DSmoke_DIR="$PWD/../smokegen/cmake" .
   $ make install
   

   At this point, smokephonon failed to build so I had to manually install the two modules I actually needed:

   $ make -C qtcore install
   $ make -C qtgui install
   

5. Get an SBCL with threads enabled and Quicklisp.

6. CommonQt needed a couple of tweaks for OSX and recent changes in SMOKE. While said changes aren't reviewed and integrated into the main repository, you can fetch them as follows:

   $ cd ~/quicklisp/local-projects
   $ git clone git://gitorious.org/~luismbo/commonqt/commonqt-luis.git
   $ cd commonqt-luis
   $ git checkout modular-smoke-and-osx-fixes
   

7. Start SBCL and (ql:quickload :qt).

Hopefully that went well. Next we'll try and run an application.

1. Enable the swank-listener-hooks contrib by adding (slime-require 'swank-listener-hooks) to your SLIME configuration.
2. (asdf:load-system :qt-tutorial)
3. (asdf:load-system :qt-repl)
4. (qt-repl:start-gui-thread)
5. And finally, (qt-tutorial-14::test)!

If not running under SLIME, (asdf:load-system :qt-tutorial) followed by (qt-tutorial-14:main) would be sufficient.

If you want to try and skip the C++ compilation steps, grab this tarball with libcommonqt.dylib and libsmoke*.dylib. You should place libcommonqt.dylib in CommonQt's source directory. The SMOKE libs go into /usr/local/lib or similar.

TIME on steroids

Nikodemus's recent post about a yet-to-be-released micro-benchmarking tool prompted me to talk about Perfpiece. I describe it as a tool for measuring the performance of Lisp code, not unlike the standard CL:TIME and I wrote over two years ago to measure the performance of SBCL's garbage collector.

Features

Here are its main features:

• Lispy interface to PAPI (Performance Application Programming Interface). PAPI is a library that enables access modern CPU’s hardware counters. This allows us to measure several events such as processor cycles, cache misses, number of floating-point instructions, and almost two hundred other events. Perfpiece dynamically inspects the current platform’s supported events at runtime and enables the user to inspect this list and measure these events. We also use this library to measure real (wall-clock) time and user (virtual) time.

• Support for other non-PAPI events such as the number of GC runs, CPU usage, and operating system resource usage (via the getrusage() system call).

• Transparent support for multi-threaded programs. It includes a helper C library that when loaded through the POSIX LD_PRELOAD mechanism will preempt pthread creation/termination calls and allow for the individual measurement of events across threads created during a measurement session. This includes both Lisp threads as well as threads created by C code. This is rather limited at the moment; only real/user/cpu time is measured for new threads.

• Segregation of measurements between mutator and GC.

• Support for sampling. Perfpiece can repeat a given a measurement a number of times then calculate and report basic statistic analysis: minimums, maximums, geometric means, and standard deviations for each measured event.

Usage

The simplest way to interact with this library is through the ascertain macro, which works very much like cl:time. The following example shows the default events measured for a very simple arithmetic form:

PERFPIECE> (ascertain (+ 1 1))

                                          non-GC            GC         Total
────────────────────────────────────────────────────────────────────────────
                     Total cycles:         8,343             0         8,343
           Instructions completed:           481             0           481
        Level 2 data cache misses:            78             0            78
 Level 2 instruction cache misses:            24             0            24
                         GC count:             -             -             0
     Involuntary context-switches:             0             0             0
       Voluntary context-switches:             1             0             1
                      Page faults:             0             0             0
                    Page reclaims:             6             0             6
                      System time:             0             0             0
                        CPU usage:       100.00%             -       100.00%
                        User time:       6.00 µs             0       6.00 µs
                        Real time:        613 ns             0        613 ns

0 new threads were spawned

2 ; printed result of (+ 1 1)

Having loaded the helper library using LD_PRELOAD, we can measure multi-threaded code:

PERFPIECE> (ascertain (loop repeat 2 do
                        (sb-thread:join-thread
                         (sb-thread:make-thread (lambda () (sleep 0.5)))))
                      :events '(:real-time :user-time :cpu-usage))

                                          non-GC            GC         Total
────────────────────────────────────────────────────────────────────────────
                        CPU usage:         0.02%             -         0.02%
                        User time:     206.00 µs             0     206.00 µs
                        Real time:       1.335 s             0       1.335 s

2 new threads were spawned
  #0 real: 667.54 ms, user: 101.00 µs, cpu: 0.02%
  #1 real: 667.60 ms, user: 144.00 µs, cpu: 0.02%

The other main function is sample. In the following example, we're measuring FP instructions, invoking some code 10 times, and aggregating measurements in several ways:

PERFPIECE> (sample (lambda () (* 2 pi)) :events '(:papi-fp-ins) :samples 10)

[Floating point instructions]       Min           Max          Mean     Stddev
──────────────────────────────────────────────────────────────────────────────
                   total:            35            36         35.60  ±  1.376%
                  non-gc:            35            36         35.60  ±  1.376%
                 gc-only:             0             0             0  ±  0.000%

sample's got a :report keyword argument that you can use to get machine-readable results:

PERFPIECE> (sample (lambda () (* 2 pi)) :events '(:papi-fp-ins) :samples 10
                   :report nil)

((:PAPI-FP-INS :MIN (35 35 0)
               :MAX (38 38 0)
               :MEAN (184/5 184/5 0)
               :STDDEV (1.0770329 1.0770329 0.0)))

Fork it!

That's pretty much it. There is some SBCL-specific code, but it shouldn't be too hard to port to other implementations. Its use of PAPI could be made optional since that makes this library pretty much Linux-only otherwise. Patches are most welcome!

ECLM videos

Just a quick note: Vsevolod Dyomkin has posted a couple of ECLM 2011 videos with more to come.

ECLM 2011

Contrary to my expectations earlier this year, ECLM 2011 happened and ~90 people turned up. Yay!

Saturday, a bunch of us went on a guided tour across Amsterdam. We had a great guide with an awesome sense of humor and this was a nice way to meet new Lispers and catching up with folks I'd met in past meetings.

Dinner was fun. One very prominent Lisper who shall remain anonymous didn't know what Quicklisp was. Everyone sitting at that particular table was shocked. :-)

Sunday started off with Nick Levine talking about learnt lessons from his failed attempt at writing a Lisp book. Midway through, he took the opportunity to rant a little bit about how there were at least 3 overlapping, incomplete websites (cliki, c-l.net, cl-user.net) and how the solution would be to, huh, create yet another one. The consensus seems to be that cliki is the one that's best suited for a starting point, but Nick complains that its focus on Free Software Lisps is too narrow. Anyway, cliki could indeed use a revamp.

Jack Harper talked about using Lisp on portable fingerprint scanners. He had great things to say about Lispworks which, by the way, had 3 people at this year's meeting.

Luke Gorrie did an extended version of the presentation on Teclo I had seen earlier this year in Zürich. This one included even more interesting diagrams, in particular some illustrating the contrast between 3G network performance before and after enabling their product.

After lunch, Dave Cooper talked about his GDL product, which includes yet another object constraint-ish system framework thingie.

Hans Hübner picked a controversial topic: code style and conventions. (A bit reminiscent of Norvig and Pitman's slides.) He picked on the veritable 80-column rule (blasphemy!) and pushed for project/company style guides. The discussion was entertaining. The general conclusion I drew from that is that while it might be useful to write the more important points down, it's even better to programmatically enforce the more important ones like tabs vs. spaces and trailing whitespace. In my experience, good programmers pick up and follow (or challenge) code style fairly quickly without needing to go through an explicit style guide.

Paul Miller demoed his company's data analysis tool written in Lispworks and talked a little bit about how it interacted and integrated with things like Excel via COM.

Lisp hero extraordinaire Xach Beane did an awesome presentation on Quicklisp detailing some of his implementation strategies, what problems Quicklisp purports to solve, its social impact on the Lisp community, and what his vision for the future is. I was particularly excited about his plan to enable hacking on random project à la clbuild. Definitely the juiciest talk in the meeting in my opinion.

Finally, there were lots of lightning talks this year:

• Marco Antoniotti announced ELS 2012.
• Christophe Rhodes's talked about R's lispiness and demoed his swankr project that brings R to SLIME and has nifty features like graphical presentations.
• Erik Huelsmann announced ABCL 1.0! (There were three quite enthusiastic ABCL developers at the meeting.)
• Pierre-Yves Baccou shared some thoughts on Zen, his X-server in ~5K lines of CL code.
• ... and many others including a valiant attempt at subverting the 5+2 minute rule.

Drinks, dinner, then drinks again. Hanging out with Lispers is the best part of ECLM and there was plenty of that. Looking forward to the next one already!

Figuring out the print size of a photo at 300 DPI

I was looking for an easy way to tell the print size of a photo at 300 DPI. It wasn't as easy to find the answer as I'd expected. For future reference, here's a solution using ImageMagick:

identify -format "%[fx:w/300*2.54] x %[fx:h/300*2.54] cm" photo.jpg

Joey Comeau learns CL

Noticed via Twitter that Joey Comeau of "A Softer World"-fame has been doing some programming in Lisp. I liked the way he put it:

Joey: (4:01:05 PM) maybe i should take up a self abusive hobby
Joey: (4:01:08 PM) like lisp programming
Ryan: (4:01:20 PM) haha
Ryan: (4:01:23 PM) sec phone
                   after the phone call i leave the computer and don't come back
                   the next day
Joey: (9:20:29 AM) I am learning common lisp

With apologies to Joey and Emily:

(And here's one of my favourite strips.)

Workshop com o trio Azul

[este artigo foi originalmente publicado no blogue do Sítio de Sons em 2007; estou a passá-lo para este blogue, para mais tarde recordar. :-)]

O workshop com o trio Azul no passado dia 22 de Junho [de 2007] foi um sucesso. Este é um relato pessoal desse fantástico dia.

Começámos o dia logo pela manhã com uma primeira parte dedicada aos instrumentos. Cada participante optou por assistir a uma das três sessões independentes. Chegada a hora do almoço, iniciaram-se as trocas de impressões entre os participantes visivelmente entusiasmados. Circularam descrições das reharmonizações com Coltrane changes do Frank Möbus; circularam frenéticas histórias sobre os sons de arco com pratos de bateria utilizados pelo Jim Black; circularam relatos da clareza com que Carlos Bica demonstrou o seu gosto pela simplicidade, a importância das pausas, das notas individuais e da colocação rítmica em geral. Circulou música.

À tarde juntámo-nos todos e formaram-se grupos para tocar diversos temas. Dois felizardos tiveram a oportunidade de tocar um pouco de free jazz com o Jim Black; dizem que foi uma excelente experiência. :-) Os vários músicos que tocaram nesta tarde tiveram a oportunidade de ouvir críticas altamente relevantes, construtivas e pedagógicas. Por certo, cada participante terá tido as suas epifanias distintas; estas foram duas das ideias principais que retive:

• é indispensável ter convicção no que se toca. Por exemplo, não é boa ideia tocar um walking no baixo ou comping na guitarra só porque «é suposto» fazê-lo num determinado estilo de música;
• é essencial ter consciência de diversas questões ao nível da orquestração. Isto é, cada músico deve ouvir o som que o grupo está a gerar e, em função disso, escolher a cada instante onde se quer colocar a nível harmónico, rítmico, tímbrico, etc. Nos seus concertos, o Jim Black ilustra frequentemente este processo de uma forma muito óbvia quando opta por simplesmente parar de tocar; o efeito com isto gerado é muito interessante.

À noite, dirigimo-nos para a Pensão Flôr de Coimbra na baixa de Coimbra para jantarmos um excelente Tofu à Espiritual, entre outros pratos com nomes menos interessantes. Seguiu-se uma jam session no Café Santa Cruz que se iniciou com dois temas do trio: «Canção de Embalar» e «Tea for Two».

O convívio musical prosseguiu pela noite fora. Penso que todos os participantes ficaram positivamente impressionados com a simpatia e abertura oferecidas pelo Carlos Bica, Jim Black e Frank Möbus. Ficamos-lhes imensamente agradecidos.

Another ZSLUG meeting report

I seem to have picked up the habit of doing at least one Lisp conference per year. This year I was hoping for an ILC (or ECLM?), but I'll have no such luck, it seems. So, the first ZSLUG-ng meeting was it and here's a summary of my trip to Zürich.

Sunday. Met the Teclo folks for dinner. The stuff they're working on is exciting in a low-level-ish kind of way — more on that later — and their team is composed by some of the best Lispers I know. (Also, as it turns out, they must have the very best intersection between Lisp and cooking skills!) Their startup vibe is intoxicating; if you're into systems programming, low-level networking stuff, et cetera, you should definitely get in touch with them.

Monday. Could see the snowy Alps thanks to the lovely clear sky. Zürich is posh, expensive, clean, and it's got great a public transportation system, mostly trams, which I much prefer to subways. Met Jorge Tavares who was in town for the meeting. Jorge was a TA of mine who pointed me at Paul Graham's ANSI Common Lisp back in my first year of college. (Note: this was before the best Lisp book had been released.) Among many things, we talked about his rather interesting research around genetic programming and evolutionary design of algorithms using Lisp.

Hans Hübner talked about his thwarted plans to replicate the Lisp Machine using FPGAs, and how he ended up implementing a Forth system instead. I was expecting some sort of tutorial or more technical details, perhaps a demo, but, alas, no such luck. Great talk, though.

Luke Gorrie then talked about Teclo. After introducing the team, he proceeded to show us how TCP badly misbehaves in today's mobile networks using insightful time-sequenced diagrams produced by their analysis tools. Some of the TCP sessions exhibited pathological behaviour, with TCP senders getting awfully confused by the odd packet lost in the ether. It was a bit reminiscent of buffer bloat to the untrained eye; perhaps because part of the problem is that radio networks go out of their way to not drop packets. There was some explanation about how Sambal sits as a proxy between radio and wired networks and massages TCP connections to make them more amenable to the lossy radio networks. They have their own TCP/IP stack written in Lisp that bypasses the OS and handles packets in under 100 ns each. Neat stuff.

The evening carried on, to a local pub, as usual. It was a good meeting, if a bit short. I think it could have included lightning talks; those have worked quite well in past conferences.

Tuesday. Had time for some light hiking around Üetliberg, then back to Lisbon.