How to make blocking, callback, and promises based APIs

A presentation I gave at Nordic Perl Workshop and Mojoconf.

Remarkjs source

class: center, middle, inverse

# How to make blocking, callback, and promises based APIs

Jan Henning Thorsen

[jan.henning@thorsen.pm](mailto:jan.henning@thorsen.pm)

???

t:30

---
class: middle

.center[
# TOC
]

* What is the differences between blocking, callback and promises based APIs?
* Why would you be interested in the different versions?
* What is an IOloop?
* How would I write a module using the different versions?
* How would I use that new module?
* What else should I be looking into?
* Bonus slides

???

Hope you will interrupt me if you disagree or if you have any questions. I
have time, so don't hesitate if you like to interrupt me.

---
class: center, middle

# What is a blocking API?

.left[
```perl
# Return data
my @users = $db->get_users;

# Void context
$db->update_user({id => 42, nick_name => "Supergirl"});

# Catch an exception
eval { $db->update_user({id => 42, nick_name => "Supergirl"}) };
```
]

???

I guess everybody have used a blocking API before. The idea here is that you
call a method or function, and the program will stop at that line until the
method or function has completed. The time for the application to halt on this
line is not defined, so the program might "stop" for just a couple of
microseconds, seconds or many hours.

This might be perfectly fine if you have a script that only does one thing,
but it's not so good if you have something like a web server that is designed
to handle many requests in parallel.

t:30

---
class: center, middle

# What is a callback based API?

.left[
```perl
$db->get_users(sub {
  my ($db, $err, $users) = @_;
  return warn $err if $err;
  return print $_->{name} for @$users;
});
```
]

???

A callback based API on the other hand can be used in a cooperative
environment. Each method call have to complete in milli- or microseconds,
so they don't block other parts of the application from running, leaving the
heavy work to a remote server or to the operating system. That each method
call is done in a short periode of time is what makes this "cooperative". Each
part of the application need to make time for other functions to be called.

In the example shown here, the method "get_users" could send a request to a a
database, and then later the sub block, also called a callback, will be called
after the database server send back the result from the query, or if some sort
of failure happens.

This means again that the time that your application "stops" while calling
"get_users" it s pretty much constant, while the time between the method call
and the callback is completely unknown.

The return value from the "get_users" method is not important here. What you
normal call a "return value" will be the arguments passed on to callback, and
often with an indication of the error as the first argument after the
invocant.

t:80

---
class: center, middle

# What is a promise based API?

.left[
```perl
my $promise = $db->get_users_p;

$promise->then(sub {
  my $res = shift;
  print $res;
});

$promise->catch(sub {
  my $err = shift;
  warn $err;
});
```
]

???

A promise based API provides the same benefits as a callback based API,
meaning it's suitable for a cooperative environment, but instead of passing
on a callback to the method, you get a promise object in return that can
either be "fulfilled" or "rejected". When the promise is fulfilled, it will
call the callback passed on to "then()" and if rejected the callback passed
on to "catch()" will be called.

The convention in the Mojolicious world is to add "_p" to the end of the
method to indicate that this method returns a promise.

t:40

---
class: center, middle

# Why use promises instead of a blocking API?

???

When trying to explain the difference between a blocking and promise based
API, I like to use the analogy from a work place: Let's say you ask a
"blocking coworker" to do a task for you, then you need to sit down beside
that coworker for the time she or he spends on the task, and then afterwards
when you got the result you wanted, you can go back to your desk and go on
with your work.

But on the other side, if you have a coworker that gives you a promise that
they will get back to you when they are done, then you can do whatever other
things you need to do in parallel with that coworker. And instead of just
getting one promise, you can ask for promises from multiple coworkers, or
let's say get a promise from a timer that can race the other promises you
received.

t:50

.left[
```perl
Promise->race(
  Promise->all(
    $coworker->[0]->bring_coffee_p(milk => 1, sugar => 0),
    $coworker->[1]->write_report_p(subject => "NPW2018"),
  ),
  MyPromise->timer_p(3600),
)->then(sub {
  my $coworkers_or_timer = shift;
  …
});
```
]

???

In the example on this slide, we see that one coworker is set to bring you
coffee, another one is set to write a report, and they both need to finish
before the one hour timer expires, or they will loose the race. All the three
tasks are performed in parallel, and the "fulfillment" callback will be called
when either the timer expires or your coworkers have completed their tasks.

DISCLAIMER: Your coworkers might not like to bring you coffee, nor race a
timer...

t:20

---
class: center, middle

# Why use promises instead of a callback based API?

???

So why would you use promises instead of callbacks?

---
class: center, middle

## Avoid spaghetti code

.left[
```perl
$db->get_users(sub {
  my ($db, $err, $users) = @_;

unless ($err) {
    $db->update_user({id => $users->[0]{id}, nick_name => "Supergirl"}, sub {
      my ($db, $err) = @_;

unless ($err) {
        $db->get_users(sub {
          my ($db, $err, $users) = @_;
          ...
        });
      }
    });
  }
});
```
]

???

A callback based API can easily result in "spaghetti code" when you need to do
multiple things in sequence, since each method call is placed inside another
callback. Keeping track of variable scoping can also be hard, and it might be
easy to "leak" variables from one callback to another.

And the last and maybe the most important thing is that it simply looks like a
mess.

t:20

---
class: center, middle

## Let your code look sequential

.left[
```perl
$db->get_users_p->then(sub {
  my $users = shift;
  return $db->update_user_p({id => $users->[0]{id}, nick_name => "Supergirl"});
})->then(sub {
  return $db->get_users_p;
})->then(sub {
  my $users = shift;
  ...
})->catch(sub {
  my $err = shift;
});
```
]

???

A promise based API on the other hand allow you to chain each promise on to
each other, making your code look sequential even though it is all
non-blocking in the background. The trick here is that each of the methods
inside the callbacks can return a promise which again need to be fulfilled for
the next "then()" callback to run. Also, if one of the promises in the chain
is rejected, then every promise will be rejected, meaning you only need one
"catch()" callback.

t:30

---
class: center, middle

# What makes all of the non-blocking code work?

## An IOLoop

???

In general an IOLoop is a while-loop that runs some code if a given condition
is fulfilled.

t:10

---
class: center, middle

# The IOLoop

.left[
1. Does a file descriptor, such as a socket or filehandle, have new data?
2. Can a file descriptor be written to?
3. Is there a timer that has expired?
]

???

Things that the IOLoop checks for are:

1. Does a file descriptor, such as a socket or filehandle, have new data?
2. Can a file descriptor be written to?
3. Is there a timer that has expired?

With these three checks you can build applications that perform incredible
complex tasks seemingly in parallel on top. You just need to make sure that
each block of code that run on a file descriptor change or a timer event is
cooperative, meaning it need to finish quickly, so the IOLoop can go back
doing more checks and executing more code.

So how does this all go around? Well, disk access or a request to a remote
server is relatively slow compared to the CPU powering the IOLoop, so that
difference in time makes it all go around.

One example would be if you're fetching a document from a remote web server:
How much time does it take to get the document back? Maybe it takes about a
second? Well, a second for the CPU is a lot of time, allowing many
instructions in your application to run locally, while waiting for the
response from the remote server. Even a database query that completes in 5ms
is a long time for the CPU.

t:75

---
class: center, middle

# Code example

???

The rest of this talk will have example code that use Mojo::IOLoop and
Mojo::Promise, which are non-blocking components in the Mojolicious framework.

Note that I skipped "Web Framework", since the different parts in the
framework can be used for so much more than just web.

t:15

---
class: center, middle

# A very, very primitive UserAgent

.left[
```perl
package VeryBasic::UA;
use Mojo::Base -base;

use Mojo::IOLoop;
use Mojo::Promise;
use Mojo::URL;

sub get   { }
sub get_p { }
```
]

???

In this example, we will create a module that can perform a GET request to a
web server. Just to be clear, it is in no way a replacement for
Mojo::UserAgent, but I wanted to use a protocol that people at least have
heard about and many have seen.

The code in this slide is mostly boilerplate: We simply create a package,
import some modules and define the two methods we will use to run the GET
request. The first method will be used the blocking and callback based calls,
while the second will be used for promises, hence the "_p" at the end.

t:30

---
class: center, middle

# How to write the promise method

.left[
```perl
sub get_p {
  my $self    = shift;
  my $url     = Mojo::URL->new(shift); # Ex: http://metacpan.org/
  my $promise = Mojo::Promise->new;

Mojo::IOLoop->client(
    {address => $url->host, port => $url->port || 80},
    sub {
      my ($loop, $err, $stream) = @_;
      return $promise->reject($err) if $err;

$stream->on(read => sub {
        my ($stream, $bytes) = @_;
        $promise->resolve($bytes);
      });

$stream->write("GET @{[$url->path]} HTTP/1.1\x0d\x0a");
      $stream->write("Host: @{[$url->host]}\x0d\x0a\x0d\x0a");
    }
  );

return $promise;
}

```
]

???

I usually start out by making the promise method, since this is often what
I will use in my Mojolicious based web applications.

In this method we accept an URL, such as "http://metacpan.org/". Next, we will
create the promise object that will later be fulfilled or rejected. This
object is also the return value from the method.

After that, we use Mojo::IOLoop to create a TCP client that can talk to the
host and port specified in the input URL.

When the IOLoop is started later on, it will try to connect to the server and
then when the socket is ready or a connection error occurs, the callback
passed on to the "client()" method will be called.

On an error, we will reject the promise and skip out of the method.

On success on the other hand, we will attach a "read" event to the stream
object that will run a new callback when some bytes are received from the web
server and then fulfill the promise.

At the end, we tell the IOLoop to write the GET request to the server once the
socket is ready for writing.

Note that there's a ton of complexity that is not taken into consideration
here, so a real client need to check for more events, also buffer up the whole
response from the server, and so much more.

t:75

---
class: center, middle

# How to write the callback method

.left[
```perl
sub get {
  my $self = shift;
  my $url  = Mojo::URL->new(shift); # Ex: http://metacpan.org/
  my $cb   = shift;                 # sub { … }

Mojo::IOLoop->client(
    {address => $url->host, port => $url->port || 80},
    sub {
      my ($loop, $err, $stream) = @_;
      return $self->$cb($err, undef) if $err;

$stream->on(read => sub {
        my ($stream, $bytes) = @_;
        $self->$cb("", $bytes);
      });

$stream->write("GET @{[$url->path]} HTTP/1.1\x0d\x0a");
      $stream->write("Host: @{[$url->host]}\x0d\x0a\x0d\x0a");
    }
  );

return $self;
}
```
]

???

The convention for a callback based API is to pass in the callback at the end
of the argument list, and store it in a variable called "$cb". Also at the end
of the callback based method, we often return the invocant, so the method call
can be chained with other method calls.

Other than that, this method looks a lot like "get_p()", but instead of
fulfilling or rejecting the promise, we call the callback with two arguments:
The first is an error, if any, and the second is the actual response, unless
there was an error.

But... If it is very similar to "get_p()", then maybe that means we
can share some code?

t:30

---
class: center, middle

# How to mix callback and promises

.left[
```perl
sub get {
  my $self = shift;
  my $url  = shift; # Ex: http://metacpan.org/
  my $cb   = shift; # sub { … }

$self->get_p($url)
    ->then(sub { $self->$cb("", shift) })
    ->catch(sub { $self->$cb(shift, undef) });

return $self;
}
```
]

???

Well...  Instead of sharing code, we can just call "get_p()" and
then call "$cb" inside the fulfillment and rejection callbacks. That is pretty
neat, since less code means less chance for bugs!

But one thing you need to make sure of, is that you keep passing on the same
arguments to the "$cb" callback, since the caller might be sensitive about the
number and position of arguments. So...

t:20

---
class: center, middle

# Do not do this

.left[
```perl
$promise->then(sub { $self->$cb(shift) });
$promise->catch(sub { $self->$cb(shift) });
```
]

???

Do not do the this, since it will make it hard or even impossible for the
caller to understand if there was an error or success.

t:10

---
class: center, middle

# How to write the blocking method

.left[
```perl
sub get {
  my $self   = shift;
  my $url    = Mojo::URL->new(shift); # Ex: http://metacpan.org/
  my $ioloop = Mojo::IOLoop->new;
  my ($bytes, $err) = ("", "");

$ioloop->client(
    {address => $url->host, port => $url->port || 80},
    sub {
      my ($loop, $e, $stream) = @_;
      return $loop->stop if $err = $e;

$stream->on(read => sub {
        $bytes = pop;
        $loop->stop;
      });

$stream->write("GET @{[$url->path]} HTTP/1.1\x0d\x0a");
      $stream->write("Host: @{[$url->host]}\x0d\x0a\x0d\x0a");
    }
  );

$ioloop->start;
  die $err if $err;
  return $bytes;
}
```
]

???

In this blocking version of the code, we will still use the IOLoop to perform
the request.

Wait? What? Still use the IOLoop?

Yes: The IOLoop is a while-loop that can check for input and output on a
socket, and that's exactly what we need here. But to make sure we don't run
other events in the non-blocking IOLoop, we need to create a new IOLoop
object, with its own while-loop.

At the very end of the method, we will start this new instance and wait until
it is stopped inside either an error event or a "read" event. Until either
happens, the program will stop on the "$ioloop->start" line.

At the very end, you can see that we die on an error and return the result on
success. It is very common to write the methods in this way, since it is so
easy to forget to check for errors, so it's nice throw an exception, since it
will help the user of that method to spot bugs.

t:50

---
class: center, middle

# Can you mix blocking with non-blocking?

## No.

### (Or not always)

???

Writing a blocking based API does not (always) allow us to share code with the
promise based method. The reason for this is that for example a blocking
VeryBasic::UA does not share the same IOLoop object as a non-blocking request.

So why can't we share the code? The reason is that the promises based API will
use one instance of the IOLoop, which is not the same the blocking request
use, meaning that by starting one IOLoop, the other one will never be able to
start.

So what is the alternative then?

The alternative is to factor out the part that generate the request, we can minimize
repetition of code, which makes it easy to read and subject for less bugs.

t:45

---
class: center, middle

# Blocking or non-blocking context?

???

But how to know if the "get()" method is supposed to use the singleton
non-blocking IOLoop or a new instance of the IOloop?

.left[
```perl
sub get {
  my $cb   = ref $_[-1] eq "CODE" ? pop : undef;
  my $self = shift;
  my $url  = Mojo::URL->new(shift); # Ex: http://metacpan.org/

if ($cb) {
    # Non-blocking code
  }
  else {
    # Blocking code
  }
}
```
]

???

We need to figure out if we are in a blocking or non-blocking context. We do
this on the first line inside the method, by checking if the last argument is
a reference to a CODE or not. The if-callback-block then runs the non-blocking
code, while the else-block takes care of the blocking version.

t:20

---
class: center, middle

# How to use the code

.left[
```perl
my $ua = VeryBasic::UA->new;

# Blocking
print $ua->get("http://mojolicous.org/");
print $ua->get("http://metacpan.org/");

# Non-blocking
Mojo::Promise->all(
  $ua->get_p("http://mojolicious.org/"),
  $ua->get_p("http://metacpan.org/"),
)->then(sub {
  my ($metacpan, $mojolicious) = map { $_->[0] } @_;
  print "metacpan = $metacpan";
  print "mojolicous = $mojolicous";
})->catch(sub {
  my $err = shift;
  warn $err;
});
```
]

???

The final slide show how the incredible basic and primitive user agent can be
used in a blocking and non-blocking context. What is interesting to see here
is that the non-blocking version will complete much faster than the blocking
code, since we execute both requests in parallel and then call a fulfillment
method when both servers respond back. This is done by calling the
"Mojo::Promise->all()" method which require all the promise objects given to
the method to be fulfilled, before calling the next fulfilment function.

On the other hand, the blocking requests need to wait for the previous request
to complete, before the next starts.

The "map{}" is used to decompose the results, since each element in the
argument list is an array-ref of all the arguments normally passed on to a
fulfilment function.

t:40

---
class: center, middle

# A bit more about Mojo::Promise

???

So now we've come to the section about what else you might be interested into
knowing about.

.left[
```perl
Mojo::Promise->all(@promises)->then(...);
Mojo::Promise->race(@promises)->then(...);
```
]

.left[
```perl
$db->get_users_p->then(sub {
  …
})->wait;
```
]

---
class: center, middle

# Adding a promise method to a callback API

.left[
```perl
sub get_p {
  my ($self, $url) = @_;
  my $promise = Mojo::Promise->new;

$self->get($url, sub {
    my ($self, $err, $response) = @_;
    return $promise->reject($err) if $err;
    return $promise->resolve($response);
  });

return $promise;
}
```
]

---
class: center, middle

# Mojolicious::Plugin::PromiseActions

.left[
```perl
plugin "PromiseActions";

get "/server/ip" => sub {
  my $c = shift;

return $c->app->ua->get_p("http://ifconfig.me/all.json")->then(sub {
    my $tx = shift;
    $c->render(json => $tx->res->json);
  });
};
```
]

---
class: center, middle

# Mojolicious::Plugin::PromiseActions

.left[
```perl
sub register {
  my ($elf, $app, $config) = @_;
  $app->hook(
    around_action => sub {
      my ($next, $c) = @_;
      my $res = $next->();
      if (blessed($res) && $res->can("then")) {
        my $tx = $c->render_later->tx;
        $res->then(undef, sub { $c->reply->exception(pop) and undef $tx });
      }
      return $res;
    }
  );
}
```
]

---
class: center, middle

# Bonus slide

## OpenAPI::Client

.left[
```perl
$openapi->call_p("getPets")->then(...);
$openapi->getPets_p->then(...);
```
]

---
class: center, middle

# Bonus slide

## Mojo::Redis

---
class: center, middle

# Questions?

Email me at [j@thorsen.pm](mailto:jan.henning@thorsen.pm) or come and talk to me!

???

Hope you enjoyed this presentation and if you have any questions or feedback,
then don't hesitate to contact me.

Thanks for listening!

t:10