Metaprogramming

Quote

The first step to metaprogramming is understanding how expressions are represented. In Elixir the abstract syntax tree (AST), the internal representation of our code, is composed of tuples. These tuples contain three parts: function name, metadata, and function arguments.

In order to see these internal structures, Elixir supplies us with the quote/2 function. Using quote/2 we can convert Elixir code into its underlying representation:

iex> quote do: 42
42
iex> quote do: "Hello"
"Hello"
iex> quote do: :world
:world
iex> quote do: 1 + 2
{:+, [context: Elixir, import: Kernel], [1, 2]}
iex> quote do: if value, do: "True", else: "False"
{:if, [context: Elixir, import: Kernel],
 [{:value, [], Elixir}, [do: "True", else: "False"]]}

Notice the first three don’t return tuples? There are five literals that return themselves when quoted:

iex> :atom
:atom
iex> "string"
"string"
iex> 1 # All numbers
1
iex> [1, 2] # Lists
[1, 2]
iex> {"hello", :world} # 2 element tuples
{"hello", :world}

Unquote

Now that we can retrieve the internal structure of our code, how do we modify it? To inject new code or values we use unquote/1. When we unquote an expression it will be evaluated and injected into the AST. To demonstrate unquote/1 let’s look at some examples:

iex> denominator = 2
2
iex> quote do: divide(42, denominator)
{:divide, [], [42, {:denominator, [], Elixir}]}
iex> quote do: divide(42, unquote(denominator))
{:divide, [], [42, 2]}

In the first example our variable denominator is quoted so the resulting AST includes a tuple for accessing the variable. In the unquote/1 example the resulting code includes the value of denominator instead.

Macros

Once we understand quote/2 and unquote/1 we’re ready to dive into macros. It is important to remember that macros, like all metaprogramming, should be used sparingly.

In the simplest of terms macros are special functions designed to return a quoted expression that will be inserted into our application code. Imagine the macro being replaced with the quoted expression rather than called like a function. With macros we have everything necessary to extend Elixir and dynamically add code to our applications.

We begin by defining a macro using defmacro/2 which, like much of Elixir, is itself a macro (let that sink in). As an example we’ll implement unless as a macro. Remember that our macro needs to return a quoted expression:

defmodule OurMacro do
  defmacro unless(expr, do: block) do
    quote do
      if !unquote(expr), do: unquote(block)
    end
  end
end

Let’s require our module and give our macro a whirl:

iex> require OurMacro
nil
iex> OurMacro.unless true, do: "Hi"
nil
iex> OurMacro.unless false, do: "Hi"
"Hi"

Because macros replace code in our application, we can control when and what is compiled. An example of this can be found in the Logger module. When logging is disabled no code is injected and the resulting application contains no references or function calls to logging. This is different from other languages where there is still the overhead of a function call even when the implementation is NOP.

To demonstrate this we’ll make a simple logger that can either be enabled or disabled:

defmodule Logger do
  defmacro log(msg) do
    if Application.get_env(:logger, :enabled) do
      quote do
        IO.puts("Logged message: #{unquote(msg)}")
      end
    end
  end
end

defmodule Example do
  require Logger

  def test do
    Logger.log("This is a log message")
  end
end

With logging enabled our test function would result in code looking something like this:

def test do
  IO.puts("Logged message: #{"This is a log message"}")
end

If we disable logging the resulting code would be:

def test do
end

Debugging

Okay, right now we know how to use quote/2, unquote/1 and write macros. But what if you have a huge chunk of quoted code and want to understand it? In this case, you can use Macro.to_string/2. Take a look at this example:

iex> Macro.to_string(quote(do: foo.bar(1, 2, 3)))
"foo.bar(1, 2, 3)"

And when you want to look at the code generated by macros you can combine them with Macro.expand/2 and Macro.expand_once/2, these functions expand macros into their given quoted code. The first may expand it several times, while the latter - only once. For example, let’s modify unless example from the previous section:

defmodule OurMacro do
  defmacro unless(expr, do: block) do
    quote do
      if !unquote(expr), do: unquote(block)
    end
  end
end

require OurMacro

quoted =
  quote do
    OurMacro.unless(true, do: "Hi")
  end

iex> quoted |> Macro.expand_once(__ENV__) |> Macro.to_string |> IO.puts
if(!true) do
  "Hi"
end

If we run the same code with Macro.expand/2, it’s intriguing:

iex> quoted |> Macro.expand(__ENV__) |> Macro.to_string |> IO.puts
case(!true) do
  x when x in [false, nil] ->
    nil
  _ ->
    "Hi"
end

You may recall that we’ve mentioned if is a macro in Elixir, here we see it expanded into the underlying case statement.

Private Macros

Though not as common, Elixir does support private macros. A private macro is defined with defmacrop and can only be called from the module in which it was defined. Private macros must be defined before the code that invokes them.

Macro Hygiene

How macros interact with the caller’s context when expanded is known as macro hygiene. By default macros in Elixir are hygienic and will not conflict with our context:

defmodule Example do
  defmacro hygienic do
    quote do: val = -1
  end
end

iex> require Example
nil
iex> val = 42
42
iex> Example.hygienic
-1
iex> val
42

What if we wanted to manipulate the value of val? To mark a variable as being unhygienic we can use var!/2. Let’s update our example to include another macro utilizing var!/2:

defmodule Example do
  defmacro hygienic do
    quote do: val = -1
  end

  defmacro unhygienic do
    quote do: var!(val) = -1
  end
end

Let’s compare how they interact with our context:

iex> require Example
nil
iex> val = 42
42
iex> Example.hygienic
-1
iex> val
42
iex> Example.unhygienic
-1
iex> val
-1

By including var!/2 in our macro we manipulated the value of val without passing it into our macro. The use of non-hygienic macros should be kept to a minimum. By including var!/2 we increase the risk of a variable resolution conflict.

Binding

We already covered the usefulness of unquote/1, but there’s another way to inject values into our code: binding. With variable binding we are able to include multiple variables in our macro and ensure they’re only unquoted once, avoiding accidental re-evaluations. To use bound variables we need to pass a keyword list to the bind_quoted option in quote/2.

To see the benefit of bind_quoted and to demonstrate the revaluation issue let’s use an example. We can start by creating a macro that simply outputs the expression twice:

defmodule Example do
  defmacro double_puts(expr) do
    quote do
      IO.puts(unquote(expr))
      IO.puts(unquote(expr))
    end
  end
end

We’ll try out our new macro by passing it the current system time. We should expect to see it output twice:

iex> Example.double_puts(:os.system_time)
1450475941851668000
1450475941851733000

The times are different! What happened? Using unquote/1 on the same expression multiple times results in re-evaluation and that can have unintended consequences. Let’s update the example to use bind_quoted and see what we get:

defmodule Example do
  defmacro double_puts(expr) do
    quote bind_quoted: [expr: expr] do
      IO.puts(expr)
      IO.puts(expr)
    end
  end
end

iex> require Example
nil
iex> Example.double_puts(:os.system_time)
1450476083466500000
1450476083466500000

With bind_quoted we get our expected outcome: the same time printed twice.

Now that we’ve covered quote/2, unquote/1, and defmacro/2 we have all the tools necessary to extend Elixir to suit our needs.

Caught a mistake or want to contribute to the lesson? Edit this lesson on GitHub!