Back

Azure Durable Functions - Stateful function orchestrations (part 1)

HttpStart DurableFunction

Durable Functions

Since my Getting started with Serverless Architectures using Azure Functions session at Techdays I've been closely monitoring the latest news about Azure Functions. The most recent addition is called Durable Functions. With this extension long running and stateful function orchestrations can be developed. This is a welcome addition to the Azure serverless product suite since it is now much easier to implement function chaining and fan-in/fan-out messaging scenarios.

How does it work?

Durable Functions are built on top of the Durable Task Framework which enables development of long running persistent workflows by using a pattern called Event Sourcing. This pattern ensures that all actions in the orchestration function are stored and can be replayed. One of the benefits of this approach is that when the orchestration function instance has triggered another function to perform a task, the orchestration function itself can hibernate (and will not cost anything when the consumption plan is used) until the other function returns its result.

Durable Functions use Azure Storage queues, tables and blobs to manage state and messages. It uses the storage account which is created when you create a new Function App through the Azure portal.

Lets look at the two most important classes in the Durable Functions family; Durable​Orchestration​Client & Durable​Orchestration​Context.

Durable​Orchestration​Client

The Durable​Orchestration​Client class manages orchestration function instances. In order to use this client the following input binding needs to be added:

[OrchestrationClient]DurableOrchestrationClient orchestrationClient

The client exposes the following functionalities:

Starting an instance

This will start a new instance of an orchestration function.

string instanceId = await orchestrationClient.StartNewAsync(functionName, eventData);

This Gist shows an HttpTrigger function which can start orchestration functions using the orchestration/{functionName} route of the Function App:

HttpStart.cs

using System.Net.Http;
using System.Threading.Tasks;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Azure.WebJobs.Host;

namespace DurableFunctionsDemo
{
    public static class HttpStart
    {
        [FunctionName("HttpStart")]
        public static async Task<HttpResponseMessage> Run(
            [HttpTrigger(AuthorizationLevel.Function, "post", Route = "orchestration/{functionName}")]HttpRequestMessage req, 
            [OrchestrationClient]DurableOrchestrationClient orchestrationClient,
            string functionName,
            TraceWriter log)
        {
            log.Info("HttpStart triggered.");

            dynamic eventData = await req.Content.ReadAsAsync<object>();
            string instanceId = await orchestrationClient.StartNewAsync(functionName, eventData);

            log.Info($"Started orchestration with ID = '{instanceId}'.");

            return orchestrationClient.CreateCheckStatusResponse(req, instanceId);
        }
    }
}
Stopping an instance

The orchestration function instance can be terminated without waiting for the results from other functions triggered by the orchestration function instance.

await orchestrationClient.TerminateAsync(instanceId, terminationReason);

Retrieving the status of an instance

Since Durable Functions are meant to be long running it is useful to query the status of the orchestration:

var status = await orchestrationClient.GetStatusAsync(instanceId);

The returning type is DurableOrchestrationStatus which contains a RuntimeStatus property indicating if the orchestration function instance is still running, completed or terminated.

Raising events

The orchestration client is also capable of raising events:

await orchestrationClient.RaiseEventAsync(instanceId, eventName, eventData);

These events can be picked up by other functions referenced in the orchestration function by using the Durable​Orchestration​Context.

Durable​Orchestration​Context

The DurableOrchestrationContext class is used to call & schedule other functions, sub-orchestrations or wait for events. The orchestration function requires the following input binding:

[OrchestrationTrigger]DurableOrchestrationContext orchestrationContext

This Gist shows the most basic orchestration function, which actually doesn't do any orchestration, it only retrieves input from the context:

HelloWorld.cs

using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;

namespace DurableFunctionsDemo.Functions
{
    public static class HelloWorld
    {
        [FunctionName("HelloWorld")]
        public static string Run(
            [OrchestrationTrigger]DurableOrchestrationContext context,
            TraceWriter log)
        {
            string name = context.GetInput<string>();
            log.Info($"HelloWorld function triggered with: {name}.");

            return $"Hello {name}";
        }
    }
}

Although the DurableOrchestrationContext class contains about 15 methods I'll only describe a few of them in this initial post.

Get function input

When input is passed to the orchestration function (as JSON) this can be retrieved & deserialized by the GetInput<T> method.

To retrieve a list of names the following can be used:

var names = orchestrationContext.GetInput<List<string>>()

Call another function

The orchestration function can call other functions using the CallActivityAsync<T> method.

The following example calls the CollectNames function. Passes an initial list of names (names) and returns a Task of type List<string>:

var collectNamesResult = await context.CallActivityAsync<List<string>>("CollectNames", names);

Wait for an event

As mentioned earlier, the OrchestrationClient can raise events and other functions can react on these by using the WaitForExternalEvent<T> method.

The following example describes the method to wait on the addname event which has a return type of string:

var addNameResult = await orchestrationContext.WaitForExternalEvent<string>("addname");

When it's required to await several events use the await Task.WhenAll(...) or Task.WhenAny(...) methods.

Restart the orchestration

When an orchestration is required to be running forever and its full history is not of importance the ContinueAsNew(object input) method can be used to restart the orchestration which resets its history. The current state of orchestration can still be kept because it can be passed to this method and used again at the start of the orchestration using GetInput<T>.

This example restarts the orchestration function and passing it a list of strings (names):

orchestrationContext.ContinueAsNew(names);

The Durable Functions documentation shows how this can be used in Eternal Orchestrations.

Developing Durable Functions

I recommend creating compiled orchestration functions using Visual Studio 2017 because currently durable functions can only be developed in C# (support for other languages will follow).

The following tools/packages are required:

  • The Microsoft Azure Storage Emulator is a standalone tool which uses SQL Server LocalDB and the local file storage instead of Azure Storage. The emulator needs to be started before you can run & debug Durable Functions locally.
  • The Azure Functions and Web Jobs Tools Visual Studio extension. This extension adds a project template to Visual Studio to create Function Apps and run/debug them locally.
  • In your Function App you need a reference to this NuGet package: Microsoft.Azure.WebJobs.Extensions.DurableTask (currently 1.0.0-beta).

I had some issues while adding this package since it has a dependency on Microsoft.Azure.WebJobs 2.1.0-beta4 while the Function App project template uses 2.1.0-beta1. Make sure when you create a new Function App using the project template you update Microsoft.NET.Sdk.Functions NuGet package to the most recent one (now 1.0.6) so the Microsoft.Azure.WebJobs versions match up.

Finally make sure you have the following local connection strings in your local.settings.json in your Function App:

local.settings.json

{
    "IsEncrypted": false,
    "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true;DevelopmentStorageProxyUri=http://127.0.0.1:10002/",
    "AzureWebJobsDashboard": "UseDevelopmentStorage=true;DevelopmentStorageProxyUri=http://127.0.0.1:10002/"
  }
}

Next steps

I've now spent a couple of days tinkering with Durable Functions and I have to say that I enjoy this framework a lot. It's more powerful than I imagined and although I was a bit skeptical about a more direct coupling of functions by using these orchestration functions I definitely see their value.

In next posts I'll share more examples about the DurableOrchestrationContext and include a demo about the orchestration function I wrote.

Continue with Part 2

Back to top