Introduction to Azure Batch Toolkit

The Azure Batch Service is an exciting offering from Microsoft allowing you to build systems that can run workloads in the cloud.

An easy way to get started with Azure Batch is to identify command-line programs that are currently run on premise and "lift-and-shift" those programs to run on virtual machines in Azure.

There's a ton of documentation of how to do this because the Batch Service provides a rich selection of options:

You could use the Powershell interface to script programs to do this.
You could write programs using a variety of languages (C#, Python) using the SDK and Client Libraries available.
You could write programs using a language that isn't currently supported by interacting with the REST interface.
You could use this toolkit.

This toolkit enables you to focus on your domain and workflow, and also to reuse and share workload units from other projects.

The toolkit also provides a little language to express workloads with clarity and composability:

For example, a the following program defines a workload which performs a 'parametric sweep' over a provided set of user names; and creates and submits a batch job to execute it.

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
16: 
17: 
18: 
19: 
20: 
21: 
22: 
23: 
24: 
25: 
26: 
27: 
28: 
29: 
30:

    let helloUserCommand = 
        ``:parametric_cmd`` "echo 'Hello %user%'" ``:with_params`` ["user"]
    
    let workloadUnitTemplate = 
        ``:unit_template`` 
            ``:admin`` false 
            ``:main`` 
                [
                    ``:do`` helloUserCommand
                ] 
            ``:finally``
                []
            ``:files`` 
                []

    let workload =
        ``:workload``
            ``:unit_templates`` 
                [
                    workloadUnitTemplate 
                ]
            ``:files`` 
                []
            ``:arguments`` 
                [
                    ``:range`` "user" ``:over`` ["John"; "Ivan"; "Pablo"]
                ]

    runSampleWorkload workload

The best way to use this toolkit would be to write your workflow in F#, because that allows for the use of the little DSL present in the toolkit. The library is, however, interoperable with C#, and it's possible (although not as elegant) to develop these workflows there as well.

An important thing to note is that the workload you are designing can involve executables written in any language - and indeed, which run on any supported platform. The F# (or C#) requirement to use the toolkit is limited only to the workflow definition and submission stages of your development experience

Let's get started by installing the library from NuGet

The Azure.Batch.Toolkit library can be installed from NuGet:

PM> Install-Package Batch.Toolkit

Samples & documentation

Tutorial 0 Get your development environment set up for using Azure Batch, and the toolkit
Tutorial 1 Use the DSL to create your first workload with a simple command and submit it
Tutorial 2 A lap around the Job Object Model

Contributing and copyright

The project is hosted on GitHub where you can report issues, fork the project and submit pull requests. If you're adding a new public API, please also consider adding samples that can be turned into a documentation. You might also want to read the library design notes to understand how it works.

The library is available under Public Domain license, which allows modification and redistribution for both commercial and non-commercial purposes. For more information see the License file in the GitHub repository.

val helloUserCommand : obj

Full name: Index.helloUserCommand

val workloadUnitTemplate : obj

Full name: Index.workloadUnitTemplate

val workload : obj

Full name: Index.workload