Webmachine in Elixir Tutorial, Part 3

Jul 18, 2016 by Sean Cribbs

Last time we started serving static files from the priv directory, supporting validation caching, conditional requests, and compression. But since static files are pretty boring, let’s populate our application’s page with some dynamic content.

Dynamic content

Instead of the static “tweets” in the homepage, let’s replace them with some content injected via Ajax.

First, remove all of the <li> elements from the <ul> in priv/www/index.html (shown below):

<div id="content">
  <ul id="tweet-list" class="tweet-list">
  <!-- Remove all the "li" elements here -->
  </ul>
</div>

To generate the dynamic content, we need to keep the base data somewhere that we can fetch at runtime, we need to generate some JSON for the browser, and we need to render that JSON into HTML.

Let’s start by storing our tweets somewhere. Normally one would use a real database and probably pull in Ecto to handle it. Since this is just a tutorial, we’ll use ets (Erlang Term Storage). Create the ets table and populate it when the application starts up in the start/2 function in tweeter.ex.

  # Monotonic time gives us unique, increasing integers to use as
  # identifiers.
  defp now, do: :erlang.monotonic_time
  # The timestamp is a standard Erlang 3-tuple format expected by
  # Webmachine
  defp timestamp, do: :erlang.timestamp

## --- inside start/2

    # Create a public ets table to store our tweets
    :ets.new(:tweets, [:public, :ordered_set, :named_table,
                       read_concurrency: true, write_concurrency: true])

    # Insert a bunch of tweets
    :ets.insert(:tweets,
                [{now, [avatar: "http://upload.wikimedia.org/wikipedia/en/thumb/f/f4/The_Wire_Jimmy_McNulty.jpg/250px-The_Wire_Jimmy_McNulty.jpg",
                         message: "Pawns.",
                         time: timestamp]},
                 {now, [avatar: "http://upload.wikimedia.org/wikipedia/en/thumb/1/15/The_Wire_Bunk.jpg/250px-The_Wire_Bunk.jpg",
                         message: "A man's gotta have a code.",
                         time: timestamp]},
                 {now, [avatar: "http://upload.wikimedia.org/wikipedia/en/thumb/f/f4/The_Wire_Jimmy_McNulty.jpg/250px-The_Wire_Jimmy_McNulty.jpg",
                         message: "You boys have a taste?",
                         time: timestamp]}
                ])

If you refresh your browser now, you won’t see anything because we deleted the content without repopulating it. Instead, start up iex and view the table using :ets.i(:tweets):

$ iex -S mix
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Compiled lib/tweeter.ex
Interactive Elixir (1.2.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> :ets.i(:tweets)
<1   > {-576460751446910149,
 [{avatar,<<"http://upload.wikimedia.org/wiki  ...
<2   > {-576460751446908987,
 [{avatar,<<"http://upload.wikimedia.org/wiki  ...
<3   > {-576460751446908195,
 [{avatar,<<"http://upload.wikimedia.org/wiki  ...
EOT  (q)uit (p)Digits (k)ill /Regexp -->q
:ok

For some reason, the monotonic time returns a negative number on my machine, but it should be sufficient to ensure uniqueness and ordering. Now we should get that content into the browser, so it’s time to make a new resource! Open up lib/tweeter/resources/tweet_list.ex and paste in the boilerplate:

defmodule Tweeter.Resources.TweetList do
  # Basic initialization
  def init(_), do: {:ok, []}

  # Boilerplate function, which we should inject later
  def ping(req_data, state), do: {:pong, req_data, state}
end

I’ve chosen a list this time for the resource state, because we have a list of tweets to send to the browser. Let’s tell Webmachine we want to serve JSON and how to render it:

def content_types_provided(req_data, state) do
  # Provide JSON!
  {[{'application/json', :to_json}], req_data, state}
end

def to_json(req_data, state) do
  # We assume we have already fetched the tweets from ets into the state:
  tweet_list = for {_id, attributes} <- state, do: {:struct, attributes}
  # We could use Poison or jsx here, but mochijson2 is included with
  # mochiweb.
  {:mochijson2.encode({:struct, [tweets: tweet_list]}), req_data, state}
end

Before we actually fetch the data from ETS, let’s hook up our new resource and see that it’s working.

# -- lib/tweeter.ex
    # Some configuration that Webmachine needs
    web_config = [ip: {127, 0, 0, 1},
                  port: 8080,
                  dispatch: [
                    {['tweets'], Tweeter.Resources.TweetList, []},
                    {[], Tweeter.Resources.Assets, []},
                    {[:'*'], Tweeter.Resources.Assets, []}
                  ]]

Now we should be able to bounce our server and see the JSON via curl:

$ curl -i http://localhost:8080/tweets
HTTP/1.1 200 OK
Server: MochiWeb/1.1 WebMachine/1.10.9 (cafe not found)
Date: Wed, 16 Mar 2016 15:49:36 GMT
Content-Type: application/json
Content-Length: 13

{"tweets":[]}

Now let’s go back and fetch the tweets from ETS properly:

# Fetches the data from ets, even though our resource "always"
# exists.
def resource_exists(req_data, _state) do
  {true, req_data, :ets.tab2list(:tweets)}
end

After bouncing our server, I get this response:

$ curl -i http://localhost:8080/tweets
HTTP/1.1 500 Internal Server Error
Server: MochiWeb/1.1 WebMachine/1.10.9 (cafe not found)
Date: Wed, 16 Mar 2016 15:51:11 GMT
Content-Type: text/html
Content-Length: 1166

<html><head><title>500 Internal Server Error</title></head><body><h1>Internal Server Error</h1>The server encountered an error while processing this request:<br><pre>{error,{exit,{json_encode,{bad_term,{1458,141859,956053}}},
             [{mochijson2,json_encode,2,
                          [{file,"src/mochijson2.erl"},{line,181}]},
              {mochijson2,'-json_encode_proplist/2-fun-0-',3,
                          [{file,"src/mochijson2.erl"},{line,199}]},
              {lists,foldl,3,[{file,"lists.erl"},{line,1262}]},
              {mochijson2,json_encode_proplist,2,
                          [{file,"src/mochijson2.erl"},{line,202}]},
              {mochijson2,'-json_encode_array/2-fun-0-',3,
                          [{file,"src/mochijson2.erl"},{line,189}]},
              {lists,foldl,3,[{file,"lists.erl"},{line,1262}]},
              {mochijson2,json_encode_array,2,
                          [{file,"src/mochijson2.erl"},{line,191}]},
              {mochijson2,'-json_encode_proplist/2-fun-0-',3,
                          [{file,"src/mochijson2.erl"},{line,199}]}]}}</pre><P><HR><ADDRESS>mochiweb+webmachine web server</ADDRESS></body></html>

Woops! We got that timestamp but didn’t make it something that makes sense in JSON. Let’s turn it into a numerical timestamp for the client (microseconds since the epoch, basically).

def to_json(req_data, state) do
  # We assume we have already fetched the tweets from ets into the state:
  tweet_list = for {_id, attributes} <- state do
    {:struct, put_in(attributes, [:time], convert_timestamp(attributes[:time]))}
  end
  # We could use Poison or jsx here, but mochijson2 is included with
  # mochiweb.
  {:mochijson2.encode(tweet_list), req_data, state}
end

defp convert_timestamp({mega, sec, micro}) do
  mega * 1000000 * 1000000 + sec * 1000000 + micro
end

Let’s try fetching our resource again:

$ curl -i http://localhost:8080/tweets
HTTP/1.1 200 OK
Server: MochiWeb/1.1 WebMachine/1.10.9 (cafe not found)
Date: Wed, 16 Mar 2016 15:51:57 GMT
Content-Type: application/json
Content-Length: 534

{"tweets":[{"avatar":"http://upload.wikimedia.org/wikipedia/en/thumb/f/f4/The_Wire_Jimmy_McNulty.jpg/250px-The_Wire_Jimmy_McNulty.jpg","message":"Pawns.","time":1458141859956053},{"avatar":"http://upload.wikimedia.org/wikipedia/en/thumb/1/15/The_Wire_Bunk.jpg/250px-The_Wire_Bunk.jpg","message":"A man's gotta have a code.","time":1458141859956054},{"avatar":"http://upload.wikimedia.org/wikipedia/en/thumb/f/f4/The_Wire_Jimmy_McNulty.jpg/250px-The_Wire_Jimmy_McNulty.jpg","message":"You boys have a taste?","time":1458141859956056}]}

Great! Now we can hook this up to our front-end with a little JavaScript. We’re going to keep it simple by using jQuery instead of a framework.

// priv/www/js/app.js
$(document).ready(function() {
  var generateTweet = function(tweet) {
    return "<li><div class='avatar' style='background: url(" +
           tweet.avatar +
           "); background-size: auto 50px; background-position: center center;'></div><div class='message'>"
           + tweet.message + "</div></li>";
  };

  $.ajax({
    url: '/tweets',
    success: function(d) {
      if(d.tweets) {
        var tweetList = $('#tweet-list');

        d.tweets.reverse().forEach(function(i) {
          tweetList.append(generateTweet(i));
        });
      }
    }
  });
});

And hook up our JavaScript code at the bottom of index.html:

    </article>
    <!-- This is probably bad form, but it's a demo. -->
    <script type="application/javascript" src="http://code.jquery.com/jquery-1.12.1.min.js"></script>
    <script type="application/javascript" src="js/app.js"></script>
  </body>
</html>

Refresh your browser and see the Ajax populate the tweet-list!

Up next

Now that we’ve got some dynamic content being displayed, in the next installment we’ll allow the browser to post new tweets and insert them into our ets table.

Webmachine in Elixir Tutorial, Part 3

Dynamic content

Up next

Comments