I wanted to make a simple key-value server in Elixir - json in, json out, GET, POST, with an in-memory map. The point is to reinvent the wheel, and learn me some Elixir. My questions were: a) how do I build this without Phoenix and b) how do I persist state between requests in a functional language?

Learning new stuff is always painful, so this was frustrating at points and harder than I expected. But I want to emphasize that I did get it working, and do understand a lot more about how Elixir does things - the community posts and extensive documentation were great, and I didn't have to bug anyone on StackOverflow or IRC or anything to figure all this out.

Here's what the learning & development process sounded like from inside my head.

  1. First, how do I make a simple JSON API without Phoenix? I tried several tutorials using Plug alone. Several of them were out of date / didn't work. Finally found this one, which was up-to-date and got me going. Ferret was born!

  2. How do I reload code when I change things without manually restarting? Poked around and found the remix app.

  3. Now I can take JSON in, but how do I persist across requests? I think we need a subprocess or something? That's what CodeShip says, anyhow.

  4. Okay, I've got an Agent. So, where do I keep the agent PID so it's reusable across requests?

  5. Well, where the heck does Plug keep session data? [3] That should be in-memory by default, right? Quickly, to the source code!

  6. Hrm, well, that doesn't tell me a lot. Guess it's abstracted out, and in a language I'm still learning.

  7. Maybe I'll make a separate plug to initialize the agent, then dump it into the request bag-of-data?

    Pretty sure plug MyPlug, agent_thing: MyAgent.start_link will work. Can store that in my Plug's options, then add it to Conn so it's accessible inside requests

  8. Does a plug's init/1 get called on every request, or just once? What about my Router's init/1 ? Are things there memoized?

  9. Guess I'll assume the results are stored and passed in as the 2nd arg to call/2 in my plug.

  10. Wait, what does start_link return?

    14:15:15.422 [error] Ranch listener Ferret.Router.HTTP had connection process started with :cowboy_protocol:start_link/4 at #PID<0.335.0> exit with reason: \{\{\%MatchError{term: [iix: {:ok, #PID<0.328.0>}]}

    ** (MatchError) no match of right hand side value: {:ok, #PID<0.521.0>}
    (ferret) lib/plug/load_index.ex:10: Ferret.Plug.LoadIndex.init/1
  12. figures out how to assign arguments

    turns out [:ok, pid] and {:ok, pid} and %{"ok" => pid} are different things

  13. futzes about trying various things to make that work

  14. How do I log stuff, anyway? Time to learn Logger.


    14:29:45.127 [info]  GET /put
    14:29:45.129 [error] Ranch listener Ferret.Router.HTTP had connection process started with :cowboy_protocol:start_link/4 at #PID<0.716.0> exit with reason: \{\{\%FunctionClauseError{arity: 4,
  16. half an hour later - Oh, I'm doing a GET request when I routed it as POST. I'm good at programmering! I swear! I'm smrt!

  17. Turns out Conn.assign/3 and conn.assigns are how you put things in a request - not Conn.put_private/3 like plug/session uses.

  18. Okay, I've got my module in the request, and the pid going into my KV calls

  19. WTF does this mean?!?!

    Ranch listener Ferret.Router.HTTP had connection process started with :cowboy_protocol:start_link/4 at #PID<0.298.0> exit with reason: {\{:noproc, {GenServer, :call, [#PID<0.292.0>,
  20. bloody hours pass

  21. The pid is right bloody there! Logger.debug shows it's passing in the same pid for every request!

  22. Maybe it's keeping the pid around, but the process is actually dead? How do I figure that out? tries various things

  23. Know what'd be cool? Agent.is_alive? . Things that definitely don't work:

    1. Process.get(pid_of_dead_process)
    2. Process.get(__MODULE__)
    3. Process.alive?(__MODULE__)

    Which is weird, since an Agent is a GenServer is a Process (as far as I can tell). This article on "Process ping-pong" was helpful.

  24. Finally figured out to use GenServer.whereis/1 , passing in __MODULE__ , and that will return nil if the proc is dead, and info if it's alive.

  25. Turns out I don't need my own plug at all: just init the Agent with the __MODULE__ name, and I can reference it by that, just like a GenServer.

  26. IT'S STILL SAYING :noproc ! JEEBUS!

  27. Okay, I guess remix doesn't re-run Ferret.Router.init/1 when it reloads the code for the server. So when my Agent dies due to an ArgumentError or whatever, it never restarts and I get this :noproc crap.

  28. I'll just manually restart the server - I don't want to figure out supervisors right now.

  29. This seems like it should work, why doesn't it work? Agent.get_and_update __MODULE__, &Map.merge(&1, dict)

  30. Is it doing a javascript async thing? Do I need to tell Plug to wait on the response to get_and_update ?

  31. Would using Agent.update and then Agent.get work? Frick, I dunno, how async are we getting here? All. the. examples. use a pid instead of a module name to reference the agent.

  32. How would I even tell plug to wait on an async call?

  33. Oh, frickin'! get_and_update/3 has to return a tuple , and there's no function that does single-return-value-equals-new-state.

    I need a function that takes the new map, merges it with the existing state, then duplicates the new map to return, but get_and_update/3 's function argument only receives the current state and doesn't get the arguments.

    get_and_update/4 supposedly passes args, but you have to pass a Module & atom instead of a function. I couldn't make that work, either.

  34. Does Elixir have closures? I mean, that wouldn't make a lot of sense from a "pure functions only" perspective, but in Ruby it'd be like

    new_params = conn.body_params
    Agent.get_and_update do |state|
      new_state = Map.merge(state, new_params)
      [new_state, new_state]

    ...errr, whelp, no, that doesn't work.

  35. The Elixir crash-course guide doesn't mention closures, and I'm not getting how to do this from the examples.

  36. hours of fiddling

  37. uuuuugggghhhhhhhh functional currying monad closure pipe recursions are breaking my effing brain. You have to make your own curry, or use a library. This seems unnecessary for such a simple dang thing.

  38. Is there a difference between Tuple.duplicate(Map.merge(&1, dict), 2) and Map.merge(&1, dict) |> Tuple.duplicate(2) ? I dunno, neither one of those are working.

  39. What's the difference between?????

    1. def myfunc do ... end ; &myfunc
    2. f = fn args -> stuff end ; &f
    3. &(do_stuff)
  40. Okay, this is what I want: ​ &(Map.merge(&1, dict) |> Tuple.duplicate 2)

    Why is dict available inside this captured function definition? I dunno.

  41. BOOM OMG IT'S WORKING! Programming is so cool and I'm awesome at it and this is the best!

  42. Let's git commit!

  43. Jeebus, I better write this crap down so I don't forget it. Maybe someone else will find it useful. Wish I coulda Google'd this while I was futzing around.

  44. I'm gonna go murder lots of monsters with my necromancer while my brain cools off. Then hopefully come back and figure out:

    1. functions and captures
    2. pipe operator's inner workings
    3. closures???
    4. supervisors

Links I used:

Elixir Getting-Started Guide

Maps: elixir-lang

Logging with Logger

Processes & State

Statefulness in a Stateful Language (CodeShip)

Processes to Hold State

When to use Processes in Elixir

Elixir Process Ping-Pong

Using Agents in Elixir

Agent - elixir-lang

Concurrency Abstractions in Elixir (CodeShip)

GenServer name registration (hexdocs)

GenServer.whereis - for named processes

Agent.get_and_update (hexdocs) - hope you are good with currying: no way to pass args into the update function unless you can pass a module & atom (and that didn't work for me)


How to build a lightweight webhook endpoint with Elixir

Plug (Elixir School) - intro / overview

Plug body_params - StackOverflow

plug/session.ex - how do they get / store session state?

Plug.Conn.assign/3 (hexdocs)

Plug repo on Github

Function Composition

Currying and Partial Application in Elixir

Composing Elixir Functions

Breaking Up is Hard To Do

Function Currying in Elixir

Elixir Crash Course - partial function applications

Partial Function Application in Elixir

Elixir vs Ruby vs JS: closures