Common Lisp nits/tips/libraries: Iteration


Every once in a while, I see a mention of how Common Lisp really does support “non-functional” styles, and then cl:loop is trotted in, with attendant examples that make your eyes glaze over.

As you might expect, this isn’t the only way to do it; there’s a “Lispier” way, if you will (none of this is new, the original manual/memo dates from 1989!).


CL-USER> (ql:quickload "iterate")
To load "iterate":
Load 1 ASDF system:
Install 1 Quicklisp release:
; Loading "iterate"
[package iterate]...........................

(while I’m here, a plug for sly instead of slime; you know you’re successfully connected when you see the [sly] Connected. Take this REPL, brother, and may it serve you well. message 😀)

Simple uses

For a really simple example, iter is not too different from loop, but still:

Basic for loop

CL-USER> (loop for i from 1 to 10 collect i)
(1 2 3 4 5 6 7 8 9 10)
CL-USER> (iter:iter
(for i from 1 to 10)
(collect i))
(1 2 3 4 5 6 7 8 9 10)

Collecting tuples

CL-USER> (loop
for x in '(a b c d)
for y in '(d e f g)
collect (list x y))
((A D) (B E) (C F) (D G))
CL-USER> (iter:iter
(for x in '(a b c d))
(for y in '(d e f g))
(collect (list x y)))
((A D) (B E) (C F) (D G))

Intermediate example

Here is an example (from the CL cookbook) of looping, with an auxiliary variable on which we have a terminating condition, with a combination of “doing something” and collecting something else, at the same time:

CL-USER> (loop for x from 1
for y = (* x 10)
while (< y 100) do (print (* x 5)) collect y) 5 10 15 20 25 30 35 40 45 (10 20 30 40 50 60 70 80 90)
CL-USER> (iter:iter
(for x upfrom 1)
(for y = (* x 10))
(while (< y 100)) (print (* x 5)) (collect y)) 5 10 15 20 25 30 35 40 45 (10 20 30 40 50 60 70 80 90)

Another example, though a bit contrived (there’s a one-liner to do this without using either of these two, but …)

CL-USER> (let ((s "alpha45"))
(loop for i from 0 below (length s)
for ch = (char s i)
when (find ch "0123456789" :test #'eql)
return ch) )
CL-USER> (let ((s "alpha45"))
(for ch in-string s)
(finding ch such-that
(find ch "0123456789" :test #'eql))))

Misc cool stuff

Making modifications

I find it easier to “splice in” new changes to iter. This is another contrived example, but sort of shows what I mean:

CL-USER> (iter:iter
(for i from 1 to 10)
(collect i into nums)
(finally (return nums)))
(1 2 3 4 5 6 7 8 9 10)
CL-USER> (iter:iter
(for i from 1 to 10)
(collect i into nums)
(collect (* i i) into nums)
(finally (return nums)))
(1 1 2 4 3 9 4 16 5 25 6 36 7 49 8 64 9 81 10 100)

Natural iteration for different types

The (for ... in ...) gathering clause applies quite naturally to a great many types of structures.

CL-USER> (iter:iter
(for x in '(1 5 6))
(when (oddp x)
(collect x)))
(1 5)
CL-USER> (iter:iter
(for x in-vector #(1 5 6))
(when (oddp x)
(collect x)))
(1 5)
CL-USER> (iter:iter
(for (k v) in-hashtable (alexandria:alist-hash-table '((foo bar) (baz quux))))
(collect v))
((BAR) (QUUX))

Accessing previous values

CL-USER> (iter:iter
(for x from 1 to 10)
(for p-x previous x initially 0)
(collect (+ x p-x)))
(1 3 5 7 9 11 13 15 17 19)

Collecting all vs. unique values

CL-USER> (iter:iter
(for x in '(7 1 4 3 2 1 7 1 0))
(collect x))
(7 1 4 3 2 1 7 1 0)
CL-USER> (iter:iter
(for x in '(7 1 4 3 2 1 7 1 0))
(adjoining x))
(7 1 4 3 2 0)


You can splice in a reduction step (counting, summing, maximizing, minimizing, etc.) in ad-hoc way.

This extremely contrived example is essentially equivalent to (reduce #'* '(1 2 3 4 5 6 7 8 9 10)), but hopefully you get the point:

CL-USER> (iter:iter
(for x from 1 to 10)
(reducing x by #'* ))
3628800 (22 bits, #x375F00)


YMMV, but iter seems to have (for me) a more uniform syntax, a few extra features, better comparability of clauses, and I personally prefer it to loop. If you’ve never used either, I’d recommend just sticking with the former.

General interesting links: February 2018

Wangenheim’s drawings of the aurora borealis.

Some random links from last month:



  • This month’s “art pick” are sketches by Alexey Feodosievich Wangenheim, who was the first head of the Soviet Union’s weather bureau in the 1930s, and drew these while spending the rest of his life in the Gulag.



Some amazing quotes:

Maya civilisation, at its peak some 1,500 years ago, covered an area about twice the size of medieval England, with an estimated population of around five million.

”With this new data it’s no longer unreasonable to think that there were 10 to 15 million people there,” said Mr Estrada-Belli, “including many living in low-lying, swampy areas that many of us had thought uninhabitable.”

The archaeologists were struck by the “incredible defensive features”, which included walls, fortresses and moats.

They showed that the Maya invested more resources into defending themselves than previously thought, Mr Garrison said.

One of the hidden finds is a seven-storey pyramid so covered in vegetation that it practically melts into the jungle.

The game-changing technology here appears to be Lidar.



… and the pursuit of Googleyness

The “I left Google” genre is quite over-populated so I won’t add much to it. Suffice it to say that I met a whole bunch of smart, talented, interesting, eccentric, helpful and just all-around wonderful people, there, learned more than I thought I would, and generally had a very good time!

I had spent about seven years there, was coming in on an eighth, and didn’t want to spend my whole life there. Leaving was certainly nerve-racking, made possible only by continuous self-prods of “if not now, when?” It was really a sort of gradual process, where I began by looking at other teams internally, then switched my default response to recruiters from “no” to “maybe”, then to “yes”, and so on.

The idea of going for an interview at all, after seven years (yes, really!) was somewhat daunting, and I now feel silly about it, in the sense that I should have done it sooner. But after the first, the second became easier, and after the third, it was more of a joyride than something to unnecessarily worry about.

I had this feeling of wanting to “be able to work in a different environment”, and felt drawn to the idea of working at a smaller place. One of the places I interviewed at fit this bill and was really an incredible sweet spot in terms of location, work, size, people, everything, I had made a new years’ resolution of starting and finishing this process and then moving on quickly, and I get a good feeling about Pure Storage 😀

A month in, I’m still soaking in a lot of very different stuff, technically, but I’m extremely happy with my decision and feel quite grateful to have found a fun group of people to work with, from whom I have a LOT to learn!

There are a great many things I will miss about my time at Google, but I think I will take with me a certain “sense of mission”. I have been very fortunate to have been acquainted with or worked alongside, some real wizards, to whom I owe a sort of love of learning, and a deeper appreciation of the science and art of computing than I ever thought possible.

I now recognize that while I felt quite confident ten years ago that “I knew everything”, I now feel I know very little indeed, and everything recedes forever into the distance, as some kind of idea to be ever-more-closely approximated.

Still, it doesn’t mean I have to stretch myself very thin. I’ve also learned that there are some areas I like better than others, and can happily spend years and decades learning more and more there. I don’t think I know nearly as much as I can in Operating Systems, Databases, Compilers, Language design, Filesystems (and much more!), and I’ve always been a polyglot and loved “all kinds of systems”, so I don’t think I’ll ever run out of ideas to think about, or projects to tinker with.

Anyway, here’s hoping I have a great year ahead! 🙂

Programming/Math/Science roundup: February 2018

Interesting stuff I came across last month:

Oliver No. 1, 1896 
  • The giant shipworm (three foot long, lives on hydrogen sulfide, within a calcium carbonate tube that it secretes) has finally been located (by scientists, that is … local people in certain parts of Thailand have been eating it as a delicacy)
  • Say what you want about Stephen Wolfram, I’m beginning to love the long-form posts I read at his blog, such as this one on civilizational artifacts (enjoyable apart from the plug for Wolfram Language)
  • Apparently, plants are quite active at long time scales, and can be sedated, just like animals! (there’s a good GIF on that page, of pea tendrils, that I wasn’t able to embed here)

    When the dope wore off, the plants returned to life, as if something had hit pause — almost like they were regaining consciousness, something we typically don’t think they possess. It’s all so animal-like.

  • If you like typewriters …
  • An example of a new species (the mutant crawfish) emerging in just a couple of decades
  • Blast from the past: in 1995, an article about disruption in the supercomputer industry (never happened)
  • Continuing on the sea creatures theme for a bit: starfish have eyes. Below their arms. Scallops have eyes too, on their tentacles (with two retinas each) … I suppose the real puzzle being why?
  • Excellent summary of the importance of the role of “the log” in distributed systems and real-time data processing
  • Part-entertaining and part-insightful, an opinionated guide to Haskell for 2018
  • Oracle has finally made dtrace available for Linux. Personally this seems too little too late given that we have BPF now, but it’s still a good thing
  • Yes, there is a turing-complete Powerpoint
  • Couple of viewpoints on “unthinkable thoughts” using programming languages.
  • A certain unicode character became an iOS-crashing bug.
  • Finally, something for the very young with lots of time on their hands: learning Physics using Haskell!

Monthly recap (February 2018)

Sunset at La Jolla

Major updates:

  • Started at a new place, Pure Storage … having a blast!
  • Visited my brother in San Diego (and saw the two-month baby!)

Minor updates:

  • Tara getting a bit better at swimming, less scared, having more fun
  • Got me a new Faber Castell pen
  • Trying out YogaGlo (impressive close to a real studio experience)


  • Birdman (yes I know, four years too late, but … still awesome)