Chemanalysis: Dialyzing Elixir

Sean Cribbs

What is Dialyzer?

Did you know…

Elixir Has Types!

They're in the Elixir guides!

@spec days_since_epoch(year :: integer, month :: integer, day :: integer) :: integer
@type color :: {red :: integer, green :: integer, blue :: integer}

Types Help You Find Bugs!

Unknown type: Elixir.Map:t/0

lib/myapp/event_logger.ex:38: Expression produces a value of type 'ok' | {'error',_},
but this value is unmatched

lib/collectable.ex:1: The specification for 'Elixir.Collectable':'__protocol__'/1
states that the function might also return 'true' but the inferred return is
'Elixir.Collectable' | 'false' | [{'into',1},...]

Type Annotations are Optional!

👍 No compilable programs are rejected

😭 Bugs slip through!

Types can be inferred!

defmodule Inference do
  def ignore_return do
    two_types()
    :ok
  end

  def two_types do
    if rem(System.monotonic_time(), 2) == 0 do
      {:ok, 2}
    else
      :odd
    end
  end
end

lib/inference.ex:3: Expression produces a value of type 'odd' | {'ok',2},
but this value is unmatched

Dialyzer automates checking types

Figures out what your code calls
Computes the "success typing" of your program
Checks for known classes of type errors

Error Classes

Defaults (`-Wno_`)	Optional (`-W`)
behaviours	error_handling
contracts	race_conditions
fail_call	✅ underspecs
fun_app	✅ unknown
improper_lists	✅ unmatched_returns
match	overspecs
missing_calls	specdiffs
opaque
return
undefined_callbacks
unused

`underspecs`

Have you ever seen or written a spec like this?

@spec myfun(any()) :: any()

`unknown`

Unknown type: Elixir.Map:t/0

`unmatched_returns`

lib/inference.ex:3: Expression produces a value of type 'odd' | {'ok',2},
but this value is unmatched

Logger.debug("result of call was #{inspect(result)}")

Dialyzer + Elixir

How do I run dialyzer against my Elixir code?

$ mix dialyzer

The Process

Compile the project
Check or build PLT(s)
Analyze your code
Print warnings (possibly filtered)

Persistent Lookup Tables (PLTs)

MFA	Type
Kernel.round/1	((integer() \| float()) -> integer())
Map.key/0	any()
:queue.queue/0	:queue.queue(term())

🗂 Function + Type Index

🚀 Greatly speeds up analysis

`dialyxir`

`dialyzex`

Strictest set of default warnings

PLTs include all of OTP, Elixir, deps

Filter warnings using match-patterns

😉 I wrote it

📦 Big PLTs

🙈 Not very popular

dialyzex is a smaller and slightly more opinionated integration than dialyxir. As such, it uses the strictest set of warnings by default, basically everything in the right column of that table I showed earlier. While in some cases annoying, this surfaces the maximum number of possible bugs.

When it builds PLTs, rather than narrowing to the applications used directly or indirectly by your project, it will collect all of the apps in OTP, Elixir, and transitive dependencies. This takes longer than dialyxir's method, but the built OTP and Elixir PLTs can be reused across all projects on your machine that use the same respective versions, and won't be as likely to have unknown function/type errors.

dialyzex filters warnings based on match-patterns, which you'll be familiar with if you've ever used them with ets tables. While this requires you to understand the data that dialyzer returns, you can get really precise on what warnings to tolerate. If you have any warnings that aren't ignored, the task will exit with a failed status-code.

Downsides: Well, I wrote it, if that's a concern for you.

It does create large PLTs, which can be an issue if you're concerned about disk space usage. That said, we automated creation of these PLTs in our base docker images for CI builds and they add < 10MB to the image.

Lastly, it's not very popular in comparison to dialyxir, and because it's pretty stable, I don't push new things to it that often.

`mix_dialyzer`

🚨 Not production-ready, GSoC 2018

Elixir-friendly warnings (from dialyxir)

.dialyzer.exs config file

dialyzer.info task 💁

`.dialyzer.exs`

[
  apps: [remove: [], include: []],
  warnings: [
    ignore: [
      {"non_existing", :*, :unknown_type},
      {"lib/dialyzer/config/config.ex", 73, :invalid_contract},
      {"lib/dialyzer/config/config.ex", 129, :invalid_contract},
    ],
    active: [
      :unmatched_returns,
      :error_handling,
      :unknown
    ]
  ],
  extra_build_dir: []
]

`mix dialyzer.info`

Chemanalysis

or… bugs I found in Elixir with Dialyzer

Bug 1

Fix dialyzer underspec warning with __protocol__/1: PR #5679

Fixed January 19, 2017 (released in Elixir 1.5)

Protocol code generation: `def`

quote do
  name  = unquote(name)
  arity = unquote(arity)

  @functions [{name, arity} | @functions]

  # Generate a fake definition with the user
  # signature that will be used by docs
  Kernel.def unquote(name)(unquote_splicing(args))

  # Generate the actual implementation
  Kernel.def unquote(name)(unquote_splicing(call_args)) do
    impl_for!(t).unquote(name)(unquote_splicing(call_args))
  end

  # ...
end

Protocol code generation: dispatch

Kernel.def impl_for!(data) do
  impl_for(data) || raise(Protocol.UndefinedError, protocol: __MODULE__, value: data)
end

Kernel.def impl_for(%{__struct__: struct}) when :erlang.is_atom(struct) do
  struct_impl_for(struct)
end

Kernel.defp struct_impl_for(struct) do
  target = Module.concat(__MODULE__, struct)
  case impl_for?(target) do
    true  -> target.__impl__(:target)
    false -> any_impl_for()
  end
end

Kernel.defp impl_for?(target) do
  Code.ensure_compiled?(target) and
    function_exported?(target, :__impl__, 1)
end

Protocol code generation: boilerplate

@doc false
@spec __protocol__(:module) :: __MODULE__
@spec __protocol__(:functions) :: unquote(Protocol.__functions_spec__(@functions))
@spec __protocol__(:consolidated?) :: boolean
Kernel.def __protocol__(:module), do: __MODULE__
Kernel.def __protocol__(:functions), do: unquote(:lists.sort(@functions))
Kernel.def __protocol__(:consolidated?), do: false

lib/collectable.ex:1: The specification for
  'Elixir.Collectable':'__protocol__'/1 states that the function might also
  return 'true' but the inferred return is 'Elixir.Collectable' | 'false' |
  [{'into',1},...]

Fixing underspecified `protocol/1`

-  @spec __protocol__(:consolidated?) :: boolean
+  @spec __protocol__(:consolidated?) :: false

defp change_impl_for([{:attribute, line, :spec, {{:__protocol__, 1}, funspecs}} | t], 
                     protocol, types, structs, is_protocol, acc) do
  newspecs = for spec <- funspecs do
    case spec do
      {:type, line, :fun, [{:type, _, :product, [{:atom, _, :consolidated?}]}, _]} ->
        {:type, line, :fun,
         [{:type, line, :product, [{:atom, 0, :consolidated?}]},
          {:atom, 0, true}]} # CHANGE HERE
      other -> other
    end
  end
  change_impl_for(t, protocol, types, structs, is_protocol, 
                  [{:attribute, line, :spec, {{:__protocol__, 1}, newspecs}}|acc])
end

Bug 2

Remove underspecification warning for consolidated protocols: PR #6627

Fixed October 2, 2017 (released in Elixir 1.6)

Updated Protocol boilerplate

@spec __protocol__(:module) :: __MODULE__
@spec __protocol__(:functions) :: unquote(Protocol.__functions_spec__(@functions))
@spec __protocol__(:consolidated?) :: false
@spec __protocol__(:impls) :: :not_consolidated
Kernel.def __protocol__(:module), do: __MODULE__
Kernel.def __protocol__(:functions), do: unquote(:lists.sort(@functions))
Kernel.def __protocol__(:consolidated?), do: false
Kernel.def __protocol__(:impls), do: :not_consolidated

Changing the spec during consolidation

defp change_impl_for([{:attribute, line, :spec, {{:__protocol__, 1}, funspecs}} | tail], 
                     protocol, types, structs, protocol?, acc) do
   new_specs = for spec <- funspecs do
     case spec do
       # ...
       # ADDED:
       {:type, line, :fun, [{:type, _, :product, [{:atom, _, :impls}]}, _]} ->
         {:type, line, :fun,
          [{:type, line, :product, [{:atom, 0, :impls}]},
           {:type, 0, :tuple,
            [{:atom, 0, :consolidated},
             {:type, 0, :list, [{:type, 0, :module, []}]}]}]}
       # ...

@spec __protocol__(:impls) :: {:consolidated, [module()]}

Underspecified list of modules

'Elixir.Collectable':'__protocol__'('impls') -> {'consolidated',[module()]}
  ; ('consolidated?') -> 'true'
  ; ('functions') -> [{'into',1},...]
  ; ('module') -> 'Elixir.Collectable' 
is a supertype of the success typing: 
  'Elixir.Collectable':'__protocol__'('consolidated?' | 'functions' | 
                                      'impls' | 'module') -> 
    'Elixir.Collectable' | 
    'true' | 
    [{'into',1},...] | 
    {'consolidated',['Elixir.BitString' | 'Elixir.File.Stream' | 
                     'Elixir.HashDict' | 'Elixir.HashSet' | 
                     'Elixir.IO.Stream' | 'Elixir.List' |
                     'Elixir.Map' | 'Elixir.MapSet',...]}

Fixing underspecification AGAIN!

            [{:type, line, :product, [{:atom, 0, :consolidated?}]},
             {:atom, 0, true}]}
         {:type, line, :fun, [{:type, _, :product, [{:atom, _, :impls}]}, _]} ->
+          impls = for mod <- types, do: {:atom, 0, mod}
           {:type, line, :fun,
            [{:type, line, :product, [{:atom, 0, :impls}]},
             {:type, 0, :tuple,
              [{:atom, 0, :consolidated},
-              {:type, 0, :list, [{:type, 0, :module, []}]}]}]}
+              {:type, 0, :list, [{:type, 0, :union, impls}]}]}]}
         other -> other
       end
     end

Bug 3

Guard test tuple_size(…) can never succeed warning: Issue #8157

Fixed by PR #8196 on September 14, 2018 (Released in Elixir 1.8)

The warning

Guard test tuple_size(__@5::#{'__exception__':='true', '__struct__':=_, _=>_}) 
  can never succeed

Prometheus.ex delegation macro

defmacro delegate(fun, opts \\ []) do
  fun = Macro.escape(fun, unquote: true)

  quote bind_quoted: [fun: fun, opts: opts] do
    target = Keyword.get(opts, :to, @erlang_module)

    {name, args, as, as_args} = Kernel.Utils.defdelegate(fun, opts)

    def unquote(name)(unquote_splicing(args)) do
      Prometheus.Error.with_prometheus_error(
        unquote(target).unquote(as)(unquote_splicing(as_args))
      )
    end
  end
end

Prometheus.ex error-handling macro

defmacro with_prometheus_error(block) do
  quote do
    try do
      unquote(block)
    rescue
      e in ErlangError ->
        reraise(
          Prometheus.Error.normalize(e),
          unquote(
            if macro_exported?(Kernel.SpecialForms, :__STACKTRACE__, 0) do
              quote(do: __STACKTRACE__)
            else
              quote(do: System.stacktrace())
            end
          )
        )
    end
  end
end

Elixir exception normalization

error:#{'__struct__' := __@4,'__exception__' := true} = __@3:___STACKTRACE__@1
    when __@4 == 'Elixir.ErlangError' ->
    _e@1 =
        'Elixir.Exception':normalize(
           error, __@3,
           case __@3 of
               __@5 when tuple_size(__@5) == 2 andalso 
                         element(1, __@5) == badkey;
                       __@5 == undef;
                       __@5 == function_clause ->
                   ___STACKTRACE__@1;
               _ ->
                   []
           end),

I had to resort to the decompile library (by Michał Muskała) to spit out Erlang code. I highly recommend picking up this library so you can understand (for now) what the Elixir compiler generates.

Anyway, it turns out that Elixir does normalization of Erlang exceptions as well! If you rescue ErlangError, it will generate two catch clauses in the Erlang try, one for the non-normalized exception, and one for the normalized exception. The already-normalized exception case is above. If we trace this out, we can see that the struct is captured in the underscore-3 variable, which is matched on inside a case expression, looking for some known Erlang errors. It calls tuple_size to check the length and then checks the first element of the tuple.

But it could never be a tuple, because we already determined above via the exception pattern-match that it was a map! Hence "Guard test tuple_size can never succeed". Incidentally, these other guards that check for undef or function_clause will never succeed either!

Fixing the guard test

 dynamic_normalize(Meta, Var, [H | T]) ->
+  Generated = ?generated(Meta),
+
   Guards =
     lists:foldl(fun(Alias, Acc) ->
-      {'when', Meta, [erl_rescue_stacktrace_for(Meta, Var, Alias), Acc]}
-    end, erl_rescue_stacktrace_for(Meta, Var, H), T),
+      {'when', Generated, [erl_rescue_stacktrace_for(Generated, Var, Alias), Acc]}
+    end, erl_rescue_stacktrace_for(Generated, Var, H), T),

-  {'case', Meta, [
+  {'case', Generated, [
     Var,
     [{do, [
-      {'->', Meta, [[{'when', Meta, [Var, Guards]}], {'__STACKTRACE__', Meta, nil}]},
-      {'->', Meta, [[{'_', Meta, nil}], []]}
+      {'->', Generated, [[{'when', Generated, [Var, Guards]}], {'__STACKTRACE__', Generated, nil}]},
+      {'->', Generated, [[{'_', Generated, nil}], []]}
     ]}]
   ]}.

Conclusion

Dialyzer is a useful tool for improving your code
Elixir integrates well with Dialyzer
You can find bugs in Elixir!

Thanks!

https://seancribbs.com/presentations/chemanalysis-dialyzing-elixir.html

Chemanalysis: Dialyzing Elixir

Sean Cribbs

What is Dialyzer?

Did you know…

Elixir Has Types!

Types Help You Find Bugs!

Type Annotations are Optional!

Types can be inferred!

Dialyzer automates checking types

Error Classes

underspecs

unknown

unmatched_returns

Dialyzer + Elixir

The Process

Persistent Lookup Tables (PLTs)

dialyxir

dialyzex

mix_dialyzer

.dialyzer.exs

mix dialyzer.info

mix dialyzer.info

Chemanalysis

Bug 1

Protocol code generation: def

Protocol code generation: dispatch

Protocol code generation: boilerplate

Fixing underspecified __protocol__/1

Bug 2

Updated Protocol boilerplate

Changing the spec during consolidation

Underspecified list of modules

Fixing underspecification AGAIN!

Bug 3

The warning

Prometheus.ex delegation macro

Prometheus.ex error-handling macro

Elixir exception normalization

Fixing the guard test

Conclusion

Thanks!

`underspecs`

`unknown`

`unmatched_returns`

`dialyxir`

`dialyzex`

`mix_dialyzer`

`.dialyzer.exs`

`mix dialyzer.info`

`mix dialyzer.info`

Protocol code generation: `def`

Fixing underspecified `protocol/1`