Getting started with C++ futures





We are currently writing an async framework for managing remote network devices over SSH. 
This is part of the Magma project led by Facebook. By the end of 2019 we at FRINX have contributed the CLI stack which provides various functions on top of the basic SSH client like keep alive, reconnect, caching etc. This post describes the the evolution of how we are using async functionality provided by the Folly library to meet our scalability requirements.

Architecture

The following text describes the high level architecture accompanied with pseudo code.
We started with a simple abstract class describing the CLI client:

class CLI {
    string executeRead(ReadCommand c);
    string executeWrite(WriteCommand c);
}

For the purpose of this text it is not important how those two methods differ. The basic idea is to create one CLI object per device, run one or more commands and collect output. This design allows us to introduce nesting: we define several classes with single responsibility:

●      KeepAliveCli - send an empty command periodically. execute* methods just call inner CLI
●      ReconnectingCli - both execute* methods call inner CLI, however if exception indicating connection error is returned, schedule reconnect
●      Etc.

We can see that even though the layers are separated, they may require some work to be done asynchronously. We need an Executor where those tasks can run. We might need to separate executors for each layer so that resource starvation on one layer can be isolated in the testing environment. In production we expect that most of CLI layers will share the same executor with a thread count equal to CPU cores.

Unblocking callers

The first version clearly blocks callers, so we came up with a second version:

class CLI {
    Future < string > executeRead(ReadCommand c);
    Future < string > executeWrite(WriteCommand c);
}

This allows us to have a thread safe interface which can be used by many callers simultaneously. Returning Futures in folly is very easy, just pass a lambda to an executor:

auto future = via(getCPUExecutor())
    .thenValue([params](auto) {
        // do some computation
    });

Sometimes we need to create futures without running them immediately. This is possible using Promise class. Promise is the holder of a value that should be computed later, whereas Future is a read only view into its Promise.

shared_ptr < Promise < string >> promise = make_shared < Promise < string >> ();
return promise - > getFuture();
// later, fulfill the promise asynchronously:
promise - > setValue(result);

Future Chaining

To get the actual result of the computation one can call the `get()` method on the Future, but this call blocks the caller until the value is available. As reported in other environments, it is important not to mix async and blocking calls. Luckily, folly Futures allow chaining more work that will be executed when the target Future finishes:

auto future = via(getCPUExecutor())
    .thenValue([](auto) {
        /* 1. */
        MLOG(MDEBUG) << "setReconnecting: sleeping";
    })
    .delayed(1 s, getTimekeeper()) /* 2. */
    .thenValue(
        [](auto) - > Future < string > {
            return doSomething();
        }) /* 3. */
    .thenValue([](string value) {
        MLOG(MDEBUG) << "doSomething finished with: " << value; /* 4. */
    });

First future will only write a log message, after that a delay is executed (non blocking), after which the third future is executed. Note that futures may return not only values but also futures: return type of `doSomething()` is a Future, so folly will execute the last future only after the string value is obtained.

Queueing CLI commands

Moving from one blocking caller to many async callers has brought one major challenge: We need to create a queue of requests so that if a command is executed or we are waiting for output, no other command is sent to the device.
Executors have their own queues, there is even SerialExecutor that guarantees non-concurrent execution of tasks, however our task is just calling inner CLI’s execute methods, obtaining inner Futures and possibly chaining lambdas for post processing. SerialExecutor would only wait until inner Future is obtained. Calling `get()` on the inner Future would solve the issue at the expense of blocking the current thread.
This was the reason we created QueuedCli, which is yet another layer, with artificial Promise/Future model described above and a MPSC queue that is both safe and efficient. When `execute*` method is called, a new Promise/Future pair is created and put into a queue. QueuedCli contains its own SerialExecutor where the queue is processed: we wait until the inner Future finishes asynchronously, then set the value of artificial Promise and then process the head of the queue again. Check out the QueuedCli source code for details.

Common pitfalls

Scheduling chained futures on separate executors

If an API returns Future, the caller is able to chain subsequent futures on the same executor. This can lead to hard to trace bugs, especially if you want to have separate executors per API layer.
Solution is to always return SemiFuture instead of Future in public methods. The continuations (thenValue, thenError) will still be available, but the caller must explicitly specify executor that will run them.

Chaining errors

Contrary to futures in e.g. Java, folly future continuations form flat hierarchy:
In Java, it is possible to write tree-like handlers - success and error listeners can have separate list of listeners.
In folly, the structure forms a list:

auto f1 = getFuture() // f1
    .thenValue(...) // f2
    .within(...)
    .thenValue(...) // f3
    .thenError(...) // f4

If we want to define a listener, we have to move the value, and this prevents us from defining more handlers for the same future. The example above shows the problem: if f2 does not finish within specified timeout, it will throw FutureTimeout. We might wish to stop execution there, and use the f4 handler only if f3 execution fails, but it is not possible to define the relationships this way. Instead, f4 handler needs to take into account all previous failures - timeout as well as failures from f1.

Destructors competing with lambdas that capture `this`

One problem that has bitten us is interaction between destructor and async functions. In our first version, we did not anticipate this and wrote our lambdas in fashion similar to this pseudo code:

void executeRead(cmd) {
    innerCli.executeRead(cmd).thenValue([ = ](...) {
        LOG(INFO) << this.connectionId << “finished”;
    });
}

We were capturing `this`, and since there was no way to synchronize lambda with destructors, our program would segfault when destructors were called before lambdas finished.
Our second version used an enable_shared_from_this pattern. We would stop getting segfaults but our reconnect functionality started acting strange: since dropping Cli would not necessarily call the destructor, we often ended with trying to connect without dropping the old connection first.
To make sure our destructors work as expected, we redesigned our classes to capture a subclass instance instead of `this`. However we would still need to wait until lambdas finish before the destructor could finish. This manifested as various timeouts on every Cli layer, but worse was blocking in possibly async context.
Our last design which is currently in place looks like this:
We introduced a new method, `SemiFuture destroy()`, which is required not to block. It cancels all timeouts, using our custom Timekeeper and propagates the call to innermost cli. This will drop the connection and effectively cancel all inflight futures. Our reconnecting Cli will be notified when the `destroy()` method finishes, and then drops the Cli stack. Destructors do not need to block anymore, since the connection is dropped and lambdas do not hold any resource we might care about.

Summary

In this post we tried to explain basic usage of folly’s Future API together with real world code usage that leverages an async architecture.


Comments

Post a Comment

Popular posts from this blog

VodafoneZiggo deploys network-wide automation with FRINX

Image
FRINX, a leading network services automation vendor, today announced it has partnered with VodafoneZiggo, the market-leading Dutch Service Provider, to deploy FRINX Machine™, a network and services automation solution. FRINX Machine provides VodafoneZiggo with Zero Touch Provisioning (ZTP) for their Vodafone Flex B2B service, delivering a superior customer experience with industry-leading service activation velocity. “We are excited to partner with VodafoneZiggo. Together we have demonstrated that by leveraging open source-based solutions we were able to improve the activation velocity of their network services. Service activation processes used to take hours when done manually, with FRINX Machine this is a matter of minutes”, says Gerhard Wieser from FRINX. ”Using our Workflow Manager workflow automation tool and our network controller UniConfig, we have been able to implement VodafoneZiggo end-to-end ZTP service enablement into a heterogeneous network.” Enterprise and service provide...
Read more

Elisa Polystar acquires FRINX to broaden its network automation portfolio

Image
Elisa Polystar has acquired FRINX s.r.o, a Slovak telecom network automation software supplier. FRINX products and software will complement Elisa Polystar’s zero-touch automation and analytics offering, which helps communications service providers (CSPs) automate their network management processes in a multivendor telecom network environment. The acquisition is consistent with Elisa’s strategy to grow digital services internationally and accelerate development of its telecom software business under Elisa Polystar. With this transaction, Elisa Polystar is taking another step on its strategic journey to make self-driving networks happen. FRINX is a privately owned software provider founded in Bratislava, Slovakia, in 2016. FRINX helps CSPs automate their network configuration management. Its products – proven by installations among leading CSP customers – are cloud-native, microservices-based and network vendor-independent. Today, the company has around 40 experts working in its projects...
Read more

Second time´s a charm

Image
Conventional wisdom says that it is not a good use of time to rewrite existing code with new technologies. Nevertheless, we had an opportunity to do some of that and in our case it offered us the opportunity to reflect on our existing design choices and technology stack. Here are our initial thoughts and experiences during the process.  What we’ve realized early on was that this provides an opportunity for improving on our original design. Some issues we’ve identified manifest themselves as never-to-be-addressed TODOs and FIXMEs sprinkled throughout the codebase. Some were not evident to us as authors of the code looking through the lenses of technologies choices, frameworks and our general assumptions. That’s why a set of new technologies can be helpful in providing a fresh perspective. The other major thing that helped identify some problems was adding new members to the team. There’s nothing like a well aimed “Why?” question to make you revisit and rethink your des...
Read more