---
title: Erlang + CZMQ
transition: fade
---
{% from "deckjs.html" import section with context %}
{% from "deckjs.html" import slide with context %}
{% call section('-------------------------------------------------------') %}
## Erlang + CZMQ
### Erlang Factory - March 6, 2014
### Garrett Smith @gar1t
{% endcall %}
{% call section('-------------------------------------------------------') %}
# CZMQ = C Binding for ZeroMQ
{% endcall %}
{% call section('-------------------------------------------------------') %}
# ZeroMQ?
{% endcall %}
{% call section('-------------------------------------------------------') %}
## ZeroMQ
- Fast Messaging
- Simple "Socket Like" API
- Heterogeneous - 40+ Languages
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Yes, 40+ Languages!
Ada
Bash
Basic
C
Chicken Scheme
Common Lisp Binding
C#
C++
Clojure
D
delphi
Eiffel
Erlang
F#
Felix
|
Flex
Go
Guile
Haskell
Haxe
Java
JavaScript
Julia
LabVIEW
Lua
Nimrod
Node.js
Objective-C
Objective Caml
ooc
|
Perl
PHP
Python
Q
Racket
R
REBOL 2
REBOL 3
Red
Ruby
Scala
Smalltalk
Tcl
Twisted
XPCOM
|
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Why ZeroMQ Matters
- Shift from applications to systems
- Language diversity
- Messages are very, very, VERY valuable!
{% endcall %}
{% call section('-------------------------------------------------------') %}
# Code Me An Example
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Problem
- Need to build something
- Option 1: Modify an existing application
- Option 2: Add a service
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Option 1: Modify existing application
- Become proficient with a language AND tools, libraries, frameworks, idioms
- Risk breakage
- Contribute to a monolith
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Option 2: Add a service
- Pick a language you're proficient in
- Avoid changes to other applications
- Focus on small and independent
- Contribute to an ecosystem
{% endcall %}
{% call section('-------------------------------------------------------') %}
## JSON Formatting Service
- Convert raw JSON into formatted JSON
- Use Python - because it's already done
- Ship as a service!
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Python Code
:::python
import zmq
def start():
ctx = zmq.Context()
socket = init_socket(ctx)
while True:
handle_msg(socket.recv(), socket)
if __name__ == "__main__":
start()
{% endcall %}
{% call section('-------------------------------------------------------') %}
## More Python Code
:::python
def init_socket(ctx):
socket = ctx.socket(zmq.REP)
socket.bind("tcp://*:5555")
print "Listening on port 5555"
return socket
def handle_msg(msg, socket):
print "Got %s" % msg
try:
socket.send("+%s" % format_json(msg))
except Exception, e:
socket.send("-%s" % e)
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Yet More Python Code
:::python
import json
def format_json(raw):
json_obj = json.loads(raw)
formatting_opts = {
'sort_keys': True,
'indent': 4,
'separators': (',', ': ')
}
return json.dumps(json_obj, **formatting_opts)
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Erlang Code
:::erlang
main([Raw]) ->
State = init_state(),
request(Raw, State),
cleanup(State).
request(Raw, {_, Socket}) ->
ok = czmq:zstr_send(Socket, Raw),
handle_recv(czmq:zstr_recv(Socket, [{timeout, 1000}])).
{% endcall %}
{% call section('-------------------------------------------------------') %}
## More Erlang Code
:::erlang
init_state() ->
{ok, Ctx} = czmq:start_link(),
Socket = czmq:zsocket_new(Ctx, req),
ok = czmq:zsocket_connect(Socket, "tcp://localhost:5555"),
{Ctx, Socket}.
request(Raw, {_, Socket}) ->
ok = czmq:zstr_send(Socket, Raw),
handle_recv(czmq:zstr_recv(Socket, [{timeout, 1000}])).
handle_recv({ok, "+"++JSON}) -> print_reply(JSON);
handle_recv({ok, "-"++Err}) -> print_error(Err);
handle_recv({error, Err}) -> print_error(Err).
{% endcall %}
{% call section('-------------------------------------------------------') %}
# Demo Me Some ZeroMQ!
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Service Using ZeroMQ
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Service Using ZeroMQ
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Service Using ZeroMQ
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Service Using ZeroMQ
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Service Using ZeroMQ
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Seriously?
- All this to format JSON?
- What about deployment? Operations?
- Libraries just scale
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Seriously
- JSON is just an example, der
- Operations is a requirement anyway
- Libraries don't scale wrt programming expertise
- Services are discrete
{% endcall %}
{% call section('-------------------------------------------------------') %}
## ZeroMQ Approach To Software
- Micro service oriented
- Follow patterns from The Guide
- Patterns based on socket types
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Router / Dealer
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Req / Rep
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Pull / Push
{% endcall %}
{% call section('-------------------------------------------------------') %}
# erlang-czmq
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Highlights
{% endcall %}
{% call section('-------------------------------------------------------') %}
## czmq-port
{% endcall %}
{% call section('-------------------------------------------------------') %}
## czmq-port
{% endcall %}
{% call section('-------------------------------------------------------') %}
## CZMQ
- Super clean!
- Wraps ZeroMQ (more than just a wire protocol)
- Closely attended to
- Security features
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Sending Messages
{% call slide('---------------------------') %}
### Send a string
:::erlang
zstr_send(Socket, String)
{% endcall %}
{% call slide('---------------------------') %}
### Send a frame with more to follow
:::erlang
zsocket_sendmem(Socket, Binary, more)
{% endcall %}
{% call slide('---------------------------') %}
### Send the last frame
:::erlang
zsocket_sendmem(Socket, Binary)
{% endcall %}
{% call slide('---------------------------') %}
### Send all at once
:::erlang
zsocket_sendall(Socket, Binaries)
{% endcall %}
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Receiving Messages
{% call slide('---------------------------') %}
### Receive a string
:::erlang
zstr_recv_nowait(Socket) -> {ok, String} | error
{% endcall %}
{% call slide('---------------------------') %}
### Receive a frame
:::erlang
zframe_recv_nowait(Socket) -> {ok, {Binary, More}} | error
{% endcall %}
{% call slide('---------------------------') %}
### Receive all at once
:::erlang
zframe_recv_all(Socket) -> {ok, Binaries} | error
{% endcall %}
{% endcall %}
{% call section('-------------------------------------------------------') %}
# All operations are non blocking
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Polling
### Subscribing
:::erlang
{ok, Poller} = czmq:subscribe_link(),
recv_loop(Poller),
czmq:unsubscribe(Poller).
### Handling Messages
:::erlang
recv_loop(Poller) ->
receive
{Poller, Msg} ->
handle_msg(Msg),
recv_loop(Poller);
stop -> ok
end.
{% endcall %}
{% call section('-------------------------------------------------------') %}
# WIP: Simulated blocking recv functions using polling
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Observations
- Big payoff with a small library
- Slow (relative) but solid
- Stable API
- Easy to enhance / maintain
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Other ZeroMQ Libraries
- erlzmq2 - NIF
- ezmq - pure Erlang
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Performance
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Performance
Recv / Send |
MPS |
C / C |
1,277,001 |
C / erlzmq |
688,429 |
C / ezmq |
574 |
C / erlang-czmq |
42,837 |
erlzmq / C |
581,162 |
ezmq / C |
72,734 |
erlang-czmq / C |
12,130 |
{% endcall %}
{% call section('-------------------------------------------------------') %}
## The Future
- Complete CZMQ API + tests
- Stabilize API + type specs + edocs
- Community feedback + improvements
- Performance optimization
{% endcall %}
{% call section('-------------------------------------------------------') %}
## Where To Start?
- Get erlang-czmq from github
- Run
make check
to compile and run tests
- Read the tests (fun!)
- Read The Guide (life changing!)
- Start evolving!
{% endcall %}
{% call section('-------------------------------------------------------') %}
## References
{% call slide('--------------') %}
### erlang-czmq source/project
http://github.com/gar1t/erlang-czmq
{% endcall %}
{% call slide('--------------') %}
### The Guide
http://zguide.zeromq.org
{% endcall %}
{% call slide('--------------') %}
### CZMQ API
http://czmq.zeromq.org
{% endcall %}
{% endcall %}
{% call section('-------------------------------------------------------') %}
# Musings
{% endcall %}
{% call section('-------------------------------------------------------') %}
# Questions, Tomatoes?
{% endcall %}