Kvardek Du

2014-03-02

SLIME 2.4

SLIME 2.4 has been released without any exciting release management, but with extensive release notes! :-)

2014-02-23

Call for SLIME testers

As you may have heard, SLIME's recently moved to GitHub where, for the last 3 months or so, code has been refactored, bugs have been fixed and some new features have been introduced.

SLIME's commit activity over the last 12 months

The last SLIME release has a very long beard, so we'd like to make a new one next weekend. To make it a good release, it'd be very really helpful if fellow SLIME users grabbed the latest code from github (via either git or the ZIP download link), took it for a spin and, if need be, used the issue tracker to report issues that may pop up.

Thanks!

2013-11-21

SISCOG's ECLM 2013 slides (finally) up


It took a while, but our ECLM 2013 slides “SISCOG: a story written in Lisp” have been published. Incidentally, they are also featured in Franz's ACL Success Stories, yay.

On a related note, Vsevolod Dyomkin published his video recordings of the meeting. Sadly, we didn't get permission to publish our video.

Finally, SISCOG is hiring and has open positions for Software Engineers and Operations Research Analysts that require C++ programming, which is rather ironic and anti-climatic for this blog post. :-)

2013-06-06

ECLM 2013

Another odd year, another ECLM. This time around in sunny Madrid at the posh TRYP Ambassador within walking distance from Teatro Real and Plaza Mayor. 63 people from 20 countries turned up.

On Saturday I failed to meet some other lispers for lunch. Apparently, there's two or three Museos del Jamón around Plaza Mayor and I stood for 20 minutes in front of the wrong one. Took the afternoon to tweak my slides. Dinner was nice for catching up with familiar faces from past meetings. Met some new lispers as well. Some of us went for a quick drink at Mercado de San Miguel afterwards.

Sunday started off with a visibly jet lagged Wes Henderson talking about mocl, a Lisp-to-C compiler with two backends tailored for the iPhone and Android environments. Despite the jet lag, he was quite eloquent! mocl makes it trivial to call Lisp from Java/ObjC:

    (declaim (call-in foo)) ; declares FOO can be called from foreign land
    (defun foo (x)
      (format t "~a~%" x))

After compiling this function, it will be available from the ObjC or Java environments. I didn't quite figure out how mocl deals with the mismatch between static and dynamic typing, but apparently that's taken care of. Note that the goal is for mocl to be embedded in Android/Java or iOS/ObjC applications. I.e., you'll be writing your GUIs in Java/ObjC and call out to Lisp for the remaining bits. I don't recall anything being mentioned about calling Java/ObjC from the Lisp side.

I enjoyed his thought out justification of why Common Lisp is a good match for mobile applications. First off, CL has a stable specification and a rich ecosystem with many alternative implementations. That means that, if he follows the spec, our existing code (namely libraries) will work as is. Secondly, CL is fast and that's important for resource-constrained mobile devices. (What does it mean for a language to be fast? I guess it means that there are plenty of implementations around that show how its semantics can be implemented efficiently.) Thirdly, the CL ecosystem has a relatively reduced dependency on foreign libraries which may or may not work on mobile environments. I was convinced. :-)

Wes took the opportunity to announce that mocl will be released at 3581000000 lisp universal time. (That's from memory, I hope I got it right.)

Michael Eisfeld presented ConED, an application to help engineers design the first stages of building structures.


There weren't many lispy details, but the application seemed quite useful and interesting. I'm told this is a hard market to get into, so best of luck to Michael and his team!

Michael Compton talked about Accenture's digital optimisation software, ADO. In a nutshell, ADO will feed various versions of a webpage to users and cleverly measure which ones work best. I didn't understand most of what was discussed, but apparently they're market leaders.

Then, Tiago Maduro Dias and I talked about the company where we work, SISCOG. Tiago presented SISCOG's history, problem domain and products and I tried to illustrate some interesting ways how we use Lisp. Thanks to Murphy's law, projector woes took up a fair chunk of our allotted time and I had to compress 25 minutes of talk into 15 or so. In the end, I think it went OK and the feedback was pretty positive. SISCOG is a 27 year old company with a rich Lisp history (going back to the Lisp Machines!) and it's quite successful in its market: resource planning, management and optimisation for railway companies. Oh, and we have about 70 full-time lispers.

Tapas for lunch.

After lunch, R. Matthew Emerson (rme) talked about Clozure CL's Objective-C bridge. I enjoyed his presentation style, and there were plenty of demos, which is nice. I had heard about CCL's IDE before, but this was the first time I got to see it.

Janusz Podrazik then demoed Opusmodus, written using CCL and the aforementioned ObjC bridge, which looked quite slick! It felt a bit like an Emacs-inspired IDE for writing music with lots of pretty graphs and visualizations. Plus, it lets you write music using Lisp! IIUC, the coding was done by rme who did a very fine job! Looking forward to its release this coming September.


Erik Huelsmann (ehu) talked about ABCL, whose version 1.2.0 RC had been released that morning from the venue. He basically talked about the latest developments and how ABCL is being used out there (including an application of his). I liked how the ABCL developers are using cl-test-grid to assess ABCL's compatibility with major libraries and how Quicklisp's download statistics helps them prioritise bug fixing.

Sven Emtell demoed ScoreCleaner. (I don't recall exactly which bits were written in Common Lisp. It was a very polyglot product, IIRC.) He showed nicely done marketing videos, such as this one.

At dinner, Faré tried to explain to me what Interface Passing Style was all about and how he used it in his Lisp Interface Library. He almost succeeded!

There were lightning talks throughout the day. Here are the ones I remember. Mark Evenson presented a very clear explanation of what went wrong with the latest common-lisp.net migration and how he and other folks plan to fix outstanding issues and generally improve how the logistics of this service are managed. Nick Levine talked about Ravenbrook. Christophe Rhodes released SBCL 1.1.8 live.

All in all, I think this was the best ECLM yet. Lots of cool applications and a vibrant community. Looking forward to the next one!

2013-03-18

Kálmán Réti on Lisp Machines

An interesting video popped up on the Lisp Reddit last week: Kálmán Réti, the Last Symbolics Developer, Speaks of Lisp Machines. The first half goes into satisfyingly low-level detail about the Lisp Machine's history and architecture, and the second half contains the most in-depth demo I recall watching. Enjoy.

2013-03-09

CFFI 0.11.0 released

CFFI 0.11.0 has been released on Feb 25th, 2013, but I only finished writing the release notes today:

  • new feature: fsbv has been integrated into CFFI, courtesy of Liam Healy. This means we now support passing structs by value via [libffi](http://sourceware.org/libffi/)! NB: in order to differentiate between structure passing by value from structure by reference, (:struct foo) and :pointer should be used. The old way of referring to struct types by bare name is now deprecated.
  • new feature: a new MKCL backend has been added courtesy of Jean-Claude Beaudoin.
  • enhancement: the ABCL backend has improved considerably thanks to Mark Evenson and Stas Boukarev.
  • enhancement: the ECL backend now supports multiple linking strategies thanks to Juanjo Garcia-Ripoll.
  • optimization: foreign-free is non-consing on SBCL. (Thanks to Stas Boukarev.)
  • bugfix: in some situations we were not signalling a warning upon defining a foreign type using a symbol from the :cl or :keyword packages. Because of this, some projects will see new compilation warnings that will require fixing.
  • and various other bugfixes.
This version will be included in the next Quicklisp dist.

2012-06-06

Augmenting bordeaux-threads with atomic operations

Bordeaux-threads is a portability layer that defines a low-level API for programming shared-state concurrency: basic thread management, locks, timeouts, and condition variables. It has been ported to plenty of CL implementations: ABCL, Allegro CL, CLISP, Clozure CL, CMUCL, Corman Lisp, ECL, Lispworks, MCL, SBCL, and SCL. As such, it's an ubiquitous building block for useful higher-level libraries such as lparallel, PCall, CL-STM, and many others.

One important piece missing from bordeaux-threads is the support for atomic operations such as compare-and-swap, fetch-and-add, atomic-swap, etc. Among other things, these operations are useful for implementing scalable, efficient lock-free or wait-free algorithms and data-structures. The availability of such data-structures in a portable fashion would benefit many of the higher-level libraries that use bordeaux-threads. (E.g.: lparallel, when running on SBCL, takes advantage of the lock-free queues provided by that implementation.)

Back when bordeaux-threads was created, virtually no CL implementation supported these operations.1 Nowadays, though, more and more implementations support at least a subset thereof. What follows is a survey of that support in modern CL implementations, a necessary first step to add an atomic operation API to bordeaux-threads.


compare-and-swap

I'll start with compare-and-swap (CAS), which is the most powerful of the atomic operators we'll be looking into, and consequently the most widely supported in CL implementations. Typically, this operation is syntactically similar to SETF but is restricted to a handful of CL places:

  • SLOT-VALUE.
  • SYMBOL-VALUE, i.e. the value of a dynamic variable.
  • The global value of a dynamic variable, a common extension to standard CL.
  • Structure accessors.
  • CAR and CDR.
  • SYMBOL-PLIST.
  • SVREF.
  • MOP's STANDARD-INSTANCE-ACCESS and FUNCALLABLE-STANDARD-INSTANCE-ACCESS.

Table 1. CAS support per place.
slotval symval gsymval structslot car/cdr symplist svref mop
ACL 9.0b
CCL 1.7
LW 6.0
SBCL 1.0.54
SCL 1.3

(2012-06-07 update: fixed table to correctly reflect the fact that SCL does support SYMBOL-GLOBAL-VALUE as a CAS place.)

(2012-06-10 update: Nikodemus wrote in to point out that (cas (symbol-value '*foo*) ...) on SBCL modifies the special variable's global value rather than its dynamic binding. Fixed table accordingly.)

As far as I can tell, ABCL, CMUCL and ECL don't yet have any support for atomic operations. I didn't check MCL or Corman Lisp. Also, note that CCL's CAS — ccl::conditional-store — is in fact an unexported interface. All in all, the feature overlap amongst the Lisps that better support CAS (ACL, LW, SBCL, SCL) is not too bad.

Conclusion: bordeaux-threads should support CAS for SLOT-VALUE, CAR/CDR, structure accessors, and SVREF on ACL, LW, SBCL, and SCL. The other partially supported places could perhaps be supported where possible but with a compile-time style warning about their unportability; haven't decided yet whether this would be worth the trouble.


fetch-and-add

A concrete implementation of fetch-and-add is the x86 LOCK-prefixed XADD instruction. It atomically increments an integer using word-sized modular arithmetic, i.e. the addition can overflow and the result wraps.

SBCL has ATOMIC-INCF whose name is slightly misleading because it is pretty much a direct interface to the aforementioned XADD instruction (on x86, of course), and thus only works on word-sized places: structure slots with type SB-EXT:WORD or simple arrays with element-type SB-EXT:WORD.

Lispworks on the other hand, provides ATOMIC-FIXNUM-INCF. Although it also boils down to XADD and the addition will overflow, it works on all the same places as CAS since it handles plain fixnums.

Conclusion: bordeaux-threads can define ATOMIC-WORD-INCF and ATOMIC-FIXNUM-INCF, support those operations efficiently on SBCL and LW respectively, and emulate them on other Lisps using CAS.


atomic-swap

Only Lispworks provides this functionality (via ATOMIC-EXCHANGE). Not yet sure whether it's sensible to emulate this using CAS on other Lisps.


Coming next

Having surveyed the support for atomic operations, we'll proceed to define a useful interface and some slightly higher-level utilities around CAS. There's plenty of inspiration to be found in the various implementations and we should be able to draw various ideas from each.

(P.S.: hopefully this doesn't overlap too much with Nikodemus's MADEIRA efforts.)


1 (2012-06-07 update) I hadn't realized this until Douglas Crosher wrote in to set some facts straight: turns out SCL has supported SMP (including compare-and-swap) for nearly a decade!

Kategorioj