Oh Lisp

Published 2013-08-15

If you ever tell someone you like Lisp and that Lisp is a more suitable language for the work you are doing most of the time you’ll get weird looks and comments like:

“Isn’t that… old?”

“All I know is that it has parenthesis everywhere.”

“Is that like COBOL?”

“Why would you use an underpowered language like Lisp?”

Which is unfortunate because I believe Lisp to be one of the most important programming languages we currently have. I’ve yet to find an article that concisely, clearly and effectively describes the advantages of Lisp to the Lisp skeptic, so here goes my attempt at it.

I’m sure you’re familiar with XML so let’s consider that for a moment.

You can think of Lisp as being like XML. There really isn’t anything to XML besides balanced <>’s yet it is tremendously useful. Because of how open ended the XML syntax is, you can use it to represent nearly any kind of data or program. Because of how simple the XML syntax is, it is easily read and manipulated by humans and programs.

Some examples where XML is in use:

  1. Ant uses XML to define build files.
  2. WSDL uses XML to describe web services
  3. XML databases
  4. XSLT is an XML language for translating XML to other formats (Lisp’s analog would be macros)
  5. AIML — Artificial Intelligence Markup Language

As you can see from those examples and likely your own experience with XML, XML allows you to come up with your own tags and thus your own sub-language. Ant uses XML as a build language, WSDL uses it as a service language, ALICE uses it as an AI language and XSLT uses it as a macro language.

An external program or parser (e.g., Ant, XSLT, Maven, JAXB) is what confers meaning upon those tags by using them. When you’re defining a new XML schema, you’re defining a new language for your problem domain.

To make the comparison of XML and Lisp concrete, here is a comparison of an XML document and a representation of it in Lisp:

<!--xml-->
<println>
    <even>
      <list><elem>1</elem><elem>2</elem><elem>3</elem></list>
    </even>
</println>

; Lisp
(println (even (list 1 2 3))) ; => 2

In Lisp, every “tag” is a function or a macro. So in our above example, println, even and list are functions or macros that you would define in Lisp.

Since each “tag” is a function or macro, instead of having one program to confer meaning upon (i.e., to interpret) a set of tags like in XML, Lisp allows every “tag” to consume and interpret the expressions given to it. It’s like allowing Ant as a built in function to handle a section of your code and then a ALICE to handle another section.

Every “tag” in Lisp is another opportunity to define a perfectly tailored way of doing things. Just like Ant and Maven are two uses of the XML format for their own goals, the Lisp functions and macros you define are tailored uses of S-expressions (the nested lists or “parenthetical expressions” in Lisp) for their own goals.

The minimal syntax of Lisp and XML allows your programs to talk solely in terms of the problem domain (XML is certainly verbose but it is minimal in the number of components that make up its syntax).

Let’s look at a comparison of Lisp, XML and Javascript.

Lisp code to construct a deck of slides:

(deck (slides
        (slide
          (components
              (text-box "some text" (position 0 0))
              (image "img.png" (position 0 200)))
        (slide ...)))

XML:

<deck>
  <slides>
    <slide>
      <components>
        <text-box text="some text" position="0,0"/>
        <image src"img.png" position="0,200"/>
      </component>
    </slide>
  </slides>
</deck>

Javascript to construct a deck of slides:

var deck = new Deck();
var slide1 = new Slide([new TextBox("some text", new Position(0, 0)), new Image("img.png", new Poisiton(0, 200)]);
deck.add(slide1);
...

You can see that while the Lisp and XML examples only have items relevant to the problem domain, the Javascript one is afflicted with extra ceremony (just imagine what it’d be like in Java…). Of course we could make the Javascript one more lisp-like (and take out some of the ceremonial syntax of the language) by using factories:

Deck([Slide([TextBox("some text", Pos(0,0)), Image("img.png", Pos(0,200))])]);

but it is still lacking. It is lacking because the Javascript code is stuck and rigid in that state. Javascript code isn’t data. In Lisp, the code can be treated as and manipulated as a list of lists. The code is data.

Lisp code is written as lists. In the Lisp code above, (deck starts the first list of symbols, (slides is a sub-list of (deck and (slide a sub-list of (slides and so on. In Lisp, you can create code and macros that take in and manipulate these code-as-list forms. E.g., calling first on the Lisp code above would return the symbol ‘deck’. Calling _second _(to get the second element of code) would return the code (as a list) that defines the list of slides. You can even add and remove from these lists to generate new code. This is similar to XML where you can use XSLT translate the XML into a new form.

This notion of code as data that can be manipulated by Lisp allows Lisp programs to be extremely flexible and expressive. With Lisp you can create whatever forms you need that aren’t yet present in the language.

Essentially, Lisp can manipulate Lisp programs just as easily as XML can be manipulated with the DOM or as easily as HTML can be manipulated via jQuery.

Recap,

Here is an interesting comparison of Lisp & Lisp Macros to XML & XSLT (courtesy of http://c2.com/cgi/wiki?LispVsXml) to further illustrate the nature of Lisp.

Notice how the code passed to the Lisp macro is accessed and used as a list of lists (which is a tree).

Lisp Macro:

; words is the code passed to the macro
(defmacro dictionary (&rest words)
`(progn
(format t "<dictionary>~%")

        ; for each word in words
(dolist (word ',words)
  (format t " <word>~%")

          ; get the first item (e.g., xml,java,lisp)
  (format t "	<id>~A</id>~%" (first word))

          ; get the second item (i.e., the definition)
  (format t "	<def>~S</def>~%" (second word))

  (format t " </word>~%"))
  (format t "</dictionary>~%"))
)

Now, when I enter the following sexpr...

(dictionary
  (xml "data language")
  (java "programming language")
  (lisp "programming and data language"))

...I get the following result:

<dictionary>
  <word>
    <id>xml</id>
    <def>"data language"</def>
  </word>
  <word>
    <id>java</id>
    <def>"programming language"</def>
  </word>
  <word>
    <id>lisp</id>
    <def>"programming and data language"</def>
  </word>
</dictionary>

XSLT:

<xsl:template match="dictionary">
  <xsl:text>(dictionary</xsl:text>
  <xsl:apply-templates />
  <xsl:text>)&#10;</xsl:text>
</xsl:template>

<xsl:template match="word">
  <xsl:text>&#10; (</xsl:text>
  <xsl:value-of select="id" />
  <xsl:text> &quot;</xsl:text>
  <xsl:value-of select="def" />
  <xsl:text>&quot;)</xsl:text>
</xsl:template>

Now, when I enter the following XML,

<dictionary>
  <word>
    <id>xml</id>
    <def>data language</def>
  </word>
  <word>
    <id>java</id>
    <def>programming language</def>
  </word>
  <word>
    <id>lisp</id>
    <def>programming and data language</def>
  </word>
</dictionary>

I get the following result:

 (dictionary
	(xml "data language")
	(java "programming language")
	(lisp "programming and data language"))

I hope this will have at least ignited some interest in Lisp in you.

It would be interesting to extend this analogy of Lisp and XML to something like “JSON as a programming language…”, maybe another time.

Lisp resources:

XML based programming languages (not that I think these are a terribly good idea due to the verbosity of XML but it illustrates the flexibility of XML, and thus Lisp, effectively):