Site Color

Text Color

Ad Color

Text Color

Evergreen

Duotone

Mysterious

Classic

Sign Up to Save Your Colors

or

Clojure Macros — Lessons from unspoken symbols by@aravindbaskaran

Clojure Macros — Lessons from unspoken symbols

Aravind HackerNoon profile picture

Aravind

Engineering @ Swym and other stuff. Opinions are only mine, when not useful.

At Swym, we use Clojure for a lot of things. And invariably, we have ended up with use cases where macros are deemed necessary. In this post, I try to take through the lessons I learned (read mistakes) along the way.

Source — https://www.videoblocks.com/video/hacker-text-terminal-fake-data-scroll-computer-terminal-display-computing-random-text-generations-scrolling-down-the-page-vxglayqceimdkqwzw

I started off by creating a simple macro using the macro defmacro.

Note — The code output is commented out so anyone can copy paste to a repl without issues, but the syntax highlight doesn’t show it as commented out.

Not really a macro

(defmacro not-really-macro [a] (do (println a) a))
;#'user/not-really-macro

(macroexpand `(not-really-macro "test"))
;test
;"test"

(not-really-macro "test")
;test
;"test"

not-really-macro is just a println code that will execute as soon you call it. So not really a macro. Moving on

Lesson 0 — Not everything defined with defmacro qualifies as a macro.

Maybe a macro

;; without list, using syntax quoting
(defmacro wrong-macro [a] `(do (println a) a))

(macroexpand `(wrong-macro "test"))
;(do (clojure.core/println user/a) user/a)

(wrong-macro "test")
;CompilerException java.lang.RuntimeException: No such var: user/a, ...

May have been a macro if it worked. The error is clear enough, there is no a defined in the user namespace. Which is weird as I expect that to come from the input arg to the macro, right? Hmm, what if I switch the namespace and try expanding

(ns outerspace)
;nil
;in outerspace now

(macroexpand `(user/wrong-macro "test"))
;(do (clojure.core/println user/a) user/a)

Now, that is interesting. The namespace of a didn’t change. So the macro expansion is not picking up the arg passed.

Enter Unquote — Variable capture

Since I quoted the code, I need to unquote to access the symbols outside the quote.

(in-ns 'user)
(defmacro ok-macro [a] `(do (println ~a) ~a))
;#'user/ok-macro

(macroexpand `(ok-macro "test"))
;(do (clojure.core/println "test") "test")

(ok-macro "test")
;"test"
;test

A double WooHoo! moment, both the macro expansion and the actual output was correct.

Lesson 1 — When in quoted code, access outside references using unquote.

More un/quoting — figuratively and “literal”ly

So, there are clojure functions quote, unquote and unquote-splicing. Trying to use them in the macro,

(defmacro forcequote-macro [a] (quote (do (println ~a) ~a)))
;#'user/forcequote-macro

(macroexpand `(forcequote-macro "test"))
;(do (println (clojure.core/unquote a)) (clojure.core/unquote a))

(forcequote-macro "test")
;CompilerException java.lang.RuntimeException: Unable to resolve symbol: a in this context...

Huh? Why not?

Lesson 2 — ` is not the literal shortcut to quote.

To confirm that lesson 2 is actually true,

(= 'a (quote a))
;true

(= `a (quote a))
;false

Success! Sort of.

Lesson 3 — ‘ is the literal shortcut to quote.

Now, doing the equivalent for unquote.

(defmacro forceunquote-macro [a] `(do (println (unquote a)) (unquote a)))
;#'user/forceunquote-macro

(macroexpand `(forceunquote-macro "test"))
;(do (clojure.core/println (clojure.core/unquote user/a)) (clojure.core/unquote user/a))

(forceunquote-macro "test")
;CompilerException java.lang.RuntimeException: No such var: user/a, compiling...

Hmm, there seems to be a pattern to this “madness” (or so I called it).

Lesson 4 — ~ is not the literal shortcut to unquote.

So, the unquoting and quoting were not working when called by reference, meaning it was too late to identify the symbols that are used inside the macro. Literals had to be used, no two ways about it.

Symbol generation

If the problem is the symbol, why not just generate a symbol I needed on demand inside that macro and referenced that instead. Here it goes

;; gensym
(defmacro sym-gen-macro [a]
(let [dyn-a (gensym a)]
`(let [~dyn-a ~a]
(println ~dyn-a)
~dyn-a)))
;#'user/sym-gen-macro

(macroexpand `(sym-gen-macro "test"))
;(let* [test1663 "test"] (clojure.core/println test1663) test1663)

(sym-gen-macro "test")
;test
;"test"

gensym to the rescue, the macro worked! But that didn’t seem right. It works, but at what cost. Every time the macro was invoked, there is a new symbol created and the reference is updated to a inside the let anyway. So nope, definitely not it.

Lesson 5 — gensym cannot solve your problem of unquoting.

Use the source

After confirming there is no getting away from those mystery literals, moved to unquote-splicing. Very powerful in using entire body of args to be passed

(defmacro expand-body [& body]
`(println [email protected]))
;#'user/expand-body

(macroexpand `(expand-body "test1" "test2"))
;(clojure.core/println "test1" "test2")

(expand-body "test1" "test2")
;test1 test2
;nil

Worked well, getting the hang of it now. Now I tried using the definition given here. It didn’t work without literals as expected, but I was in for a rude but interesting shock.

(source unquote)
;(def unquote)
;nil

(source unquote-splicing)
;(def unquote-splicing)
;nil

(source macroexpand))
;(defn macroexpand
; "Repeatedly calls macroexpand-1 on form until it no longer
; represents a macro form, then returns it. Note neither
; macroexpand-1 nor macroexpand expand macros in subforms."
; {:added "1.0"
; :static true}
; [form]
; (let [ex (macroexpand-1 form)]
; (if (identical? ex form)
; form
; (macroexpand ex))))
;nil

(source quote)
;Source not found
;nil

As you can see, unquote and unquote-splicing are defs just symbols, unlike the other defn‘s. So, of course I can’t use them instead of the literals, duh.

Lesson 6 — Not all literals have an equivalent defn.

Lesson 7 — Use [email protected] to take a list of args expand inside the macro

Extra — Try (source defn), it is an interesting read.

Getting into the inner circle — Creating inner args

I tried adding a new symbol which would prepend to the input string.

(defmacro innersym-macro [a]
`(let [dyn-a# (str "Prepend-" ~a)]
(println dyn-a#)
dyn-a#))
;#'user/innersym-macro

(macroexpand `(innersym-macro "test"))
;(let* [dyn-a__1749__auto__ (clojure.core/str "Prepend-" "test")] (clojure.core/println dyn-a__1749__auto__) dyn-a__1749__auto__)

(innersym-macro "test")
;Prepend-test
;"Prepend-test"

That is getting close to being awesome!

Lesson 8 — Use # — to create symbols inside the quote-d code block, also known as autogensym.

I will have some Destructuring, please? Please?

Pushing my luck, I tried destructuring the inner level args.

(defmacro innerdestructure-macro [a]
`(let [{:keys [prepend#] :as aprepender#} {:prepend "Prependtext" :append "Appendtext"}
dyn-a# (str prepend# ~a (:append aprepender#))]
(println dyn-a#)
dyn-a#))
;#'user/innerdestructure-macro

(macroexpand `(innerdestructure-macro "test"))
;(let* [map__1853 {:prepend "Prependtext", :append "Appendtext"} map__1853 (if (clojure.core/seq? map__1853) (clojure.lang.PersistentHashMap/create (clojure.core/seq map__1853)) map__1853) aprepender__1844__auto__ map__1853 prepend__1843__auto__ (clojure.core/get map__1853 :prepend__1843__auto__) dyn-a__1845__auto__ (clojure.core/str prepend__1843__auto__ "test" (:append aprepender__1844__auto__))] (clojure.core/println dyn-a__1845__auto__) dyn-a__1845__auto__)

(innerdestructure-macro "test")
;testAppendtext
;"testAppendtext"

The destructuring didn’t work, as prepend# got treated as a nil, but the direct get key worked well.

Lesson 9 — Destructuring doesn’t work in the first level of quoted code block

Using intern-ness

Now, using the awesome macro skills acquired so far, I ventured into creating dynamic symbols inside namespaces whenever a macro is executed.

(defmacro interning-macro [a]
`(let [{:keys [prepend#] :as aprepender#} {:prepend "Prependtext" :append "Appendtext"}
dyn-a# (str prepend# ~a (:append aprepender#))]
(intern
*ns*
'~'ooh-fn
(fn [oohargs#]
(println oohargs# dyn-a#)
oohargs#))))
;#'user/interning-macro

(macroexpand `(interning-macro "test"))
;(let* [map__1995 {:prepend "Prependtext", :append "Appendtext"} map__1995 (if (clojure.core/seq? map__1995) (clojure.lang.PersistentHashMap/create (clojure.core/seq map__1995)) map__1995) aprepender__1985__auto__ map__1995 prepend__1984__auto__ (clojure.core/get map__1995 :prepend__1984__auto__) dyn-a__1986__auto__ (clojure.core/str prepend__1984__auto__ "test" (:append aprepender__1985__auto__))] (clojure.core/intern clojure.core/*ns* (quote ooh-fn) (clojure.core/fn [oohargs__1987__auto__] (clojure.core/println oohargs__1987__auto__ dyn-a__1986__auto__) oohargs__1987__auto__)))

(ns outerspacestar)
(user/interning-macro "star")
;#'outerspacestar/ooh-fn

(ns outerspaceplanet)
(user/interning-macro "planet")
;#'outerspaceplanet/ooh-fn

(in-ns 'user)
(outerspacestar/ooh-fn {:a 10})
;{:a 10} starAppendtext
;{:a 10}

(outerspaceplanet/ooh-fn {:b 20})
;{:b 20} planetAppendtext
;{:b 20}

That was awesome! Having some internal references from when the macro was instantiated. This comes handy in creating repeatable modules with configuration changes. Many lessons in this one

Lesson 10 — *ns* — refers to current namespace where the code is executing.

Lesson 11 — (intern somens '~'symname <<symdefinition>>) is equivalent to adding (def symname symdefinition) in that somens namespace.

Lesson 12 — (def x (fn [])) = (defn x []).

Extra — Checkout Protocols sometime, if you haven’t already i.e.

Namespaces inside interns inside macros inside namespace

Now, going for the limit-breaker of my understanding — How about loading a namespace inside the intern of current namespace generated from a macro

(defn resolvable-fn1 []
(println "resolved1"))
;#'user/resolvable-fn1

(defn resolvable-fn2 []
(println "resolved2"))
;#'user/resolvable-fn2

(defmacro interning-resolve-macro [a]
`(let [{:keys [prepend#] :as aprepender#} {:prepend "Prependtext" :append "Appendtext"}
dyn-a# (str prepend# ~a (:append aprepender#))]
(intern
*ns*
'~'resolvens-fn
(fn [rargs#]
((ns-resolve (symbol "user") (symbol "resolvable-fn1")))
((ns-resolve '~'user '~'resolvable-fn2))
(println rargs# dyn-a#)
rargs#))))
#'user/interning-resolve-macro

(macroexpand `(interning-resolve-macro "test"))
;(let* [map__2276 {:prepend "Prependtext", :append "Appendtext"} map__2276 (if (clojure.core/seq? map__2276) (clojure.lang.PersistentHashMap/create (clojure.core/seq map__2276)) map__2276) aprepender__2266__auto__ map__2276 prepend__2265__auto__ (clojure.core/get map__2276 :prepend__2265__auto__) dyn-a__2267__auto__ (clojure.core/str prepend__2265__auto__ "test" (:append aprepender__2266__auto__))] (clojure.core/intern clojure.core/*ns* (quote resolvens-fn) (clojure.core/fn [rargs__2268__auto__] ((clojure.core/ns-resolve (clojure.core/symbol "user") (clojure.core/symbol "resolvable-fn1"))) ((clojure.core/ns-resolve (quote user) (quote resolvable-fn2))) (clojure.core/println rargs__2268__auto__ dyn-a__2267__auto__) rargs__2268__auto__)))

(ns outerspacestar)
(user/interning-resolve-macro "star")
#'outerspacestar/resolvens-fn

(ns outerspaceplanet)
(user/interning-resolve-macro "planet")
#'outerspaceplanet/resolvens-fn

(in-ns 'user)
(outerspacestar/resolvens-fn {:a 100})
;resolved1
;resolved2
;{:a 100} starAppendtext
;{:a 100}

(outerspaceplanet/resolvens-fn {:b 200})
;resolved1
;resolved2
;{:b 200} planetAppendtext
;{:b 200}

Adding on to aforementioned awesomeness, resolving symbols can be done in interesting ways to successful results.

Lesson 13 — (symbol "xyz") = 'xyz.

Lesson 14 — In macro world — (symbol "xyz") = '~'xyz.

Lesson 15 — Before a ns-resolve is called, the ns needs to have been loaded. So better do a (require 'nssymbol) before using ns-resolve.

It was getting a little messy to see all those dynamic symbols in one go. Got me to try out macroexpand-1.

(macroexpand-1 `(interning-resolve-macro "test"))

;(clojure.core/let [{:as aprepender__2266__auto__, :keys [prepend__2265__auto__]} {:prepend "Prependtext", :append "Appendtext"} dyn-a__2267__auto__ (clojure.core/str prepend__2265__auto__ "test" (:append aprepender__2266__auto__))] (clojure.core/intern clojure.core/*ns* (quote resolvens-fn) (clojure.core/fn [rargs__2268__auto__] ((clojure.core/ns-resolve (clojure.core/symbol "user") (clojure.core/symbol "resolvable-fn1"))) ((clojure.core/ns-resolve (quote user) (quote resolvable-fn2))) (clojure.core/println rargs__2268__auto__ dyn-a__2267__auto__) rargs__2268__auto__)))

Neater.

Lesson 16 — macroexpand-1 goes 1 level of expansion and macroexpand goes to all levels and expands every. single. macro.

Extrapolations

Some good practices that evolved out of the lessons learnt

  1. Keep destructuring to a limit inside the quoted block to avoid confusion.
  2. Keep core functions that can be outside the macro, outside it. There is no need for those functions to be created every time the macro is loaded
  3. For the heavy weight macros that initialize a bunch of interns on namespaces, keep the number of interns and actual instantiation to a judicious limit. The higher this number, the time to startup could get that much slower.
  4. On the flip side, if there is an increase in startup time, take a look at the require‘s that force load of macros and help cut down to the only mandatory startup namespaces.
  5. If you are doing a lot of intern ns generations, it is probably time to give Protocols a look and see if that works better. There is a small chance you might like it :)

Macros sure are powerful in many ways, allowing for data to become the code, executable and everything. But (of course there is a “but”) the documentation around it is kind of shrouded in mystery and (for a lack of better word) not simple. Hopefully the lessons from this post help some of those mysteries reveal to the uninitiated.

I am sure I have missed a point or two, so please feel free to correct me wherever necessary in the comments section below. Also, do share your experiences with Clojure macros below, would love to know them!

References and useful links

  1. https://www.braveclojure.com/writing-macros/ — Deep and useful when facing the macro music i.e. errors from macros
  2. https://stackoverflow.com/questions/3667403/what-is-the-difference-between-defn-and-defmacro
  3. http://stackoverflow.com/questions/4571042/can-someone-explain-clojures-unquote-splice-in-simple-terms
  4. https://clojure.org/reference/special_forms
  5. https://clojure.org/reference/protocols
  6. https://www.braveclojure.com/multimethods-records-protocols/ — Another one from braveclojure, it is filled with Clojure goodness!

Tags