On Kafka, Confluent, and … Clojure

As I’m absorbing more of the general flavor of event-streaming and everything around it (… in the beginning was the log …), I keep gettin a bit of deja vu and realized that there’s an uncanny resemblance to something else, a different place I’ve encountered similar relationships between things in a different concrete form … in Clojure.

I feel Clojure keeps getting compared to other “languages for talking to machines”, and discussions tend to end up rehashing either the static-vs-dynamic-typing cliche, the lisp cliche(s), the scripting-Java cliche, and so on.

What is missed (I feel) is the symbolic nature of the language, which allows it to really represent concepts at an arbitrary level of abstraction … something that will be incomprehensible if all you ever compare it to is examples of computing factorials in other languages.

In particular, it is a sort of “language of the system”. The same patterns I see Apache Kafka getting involved in, for distributed data-processing systems, are the patterns I imagine with symbolic objects in Clojure code.

So instead of a log with brokers and producers and consumers and streams, I see … LazySeq, with transducers; instead of Schema Registry, spec … and so on.

This is not a fully-formed thought yet, but … dunno, I’m enjoying this parallel so far.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s