Why Unfurl?

With the recent release of Unfurl 1.0, our open-source tool for deploying services and applications, it’s a good time to step back and try to answer the question: Why Unfurl?

Before getting into the big picture let’s focus on its day-to-day usefulness. How is it helpful? Well, have any of these things happened to you?

• You try to deploy your app on a cloud provider and it doesn’t work. After searching through a pile of log files you finally track it down to a silly typo in a config file.

• You try to deploy something and something stops working. But you check your configuration and code and nothing’s changed. Finally you look at a process you’ve never touched and it turns out it auto upgraded to a minor new revision that broke something. or maybe an API provider finally removed a long deprecated API that a 3rd party component was relying on.

• There’s a new service you want to try but your configuration is too fragile and it seems too risky to touch it.

Now if you are a coder, think back to when you last integrated a 3rd party library into a program you were working on? I’d bet that went more smoothly even though the integration with your code was more complicated and deeply intertwined than the integration between your app and the cloud services it relies on.

Why is that? Think about the tools you have available to when working with modern programming language and development environment:

• Most fundamentally, a programming language lets you build abstractions and rely on a type system to guarantee invariants.
• There are integrated IDEs with autocompletion and tooling like linters and unit test frameworks, catching errors a lot earlier.
• It has a well-defined packaging mechanism with semantic versioning and a package manager to resolve and install dependencies.
• Version control is core to your workflow – every change is recorded and you can rollback if something breaks.

All these facilities make it easy to integrate your code with 3rd party components. But if you zoom out and take a holistic view of your cloud application, its more than just what you build from your code, it depends on the the environment it runs in, cloud provider infrastructure it runs on, on the tools used to deploy it, and the services it connects to.

Because these are all very different systems, well-defined semantics, if they exist at all, are lost. Some examples:

• Scripts can make any number of implicit assumptions about the environment they’re running in.
• Environment variables are untyped strings.
• A 3rd party service’s behavior can change out from under you.

Unfurl

Let’s see how Unfurl can help with this. At its core, Unfurl implements the TOSCA (Topology and Orchestration Specification for Cloud Applications) standard – you declare the resources you want to deploy and how to do that (e.g. via shell script, Ansible, or Terraform) and Unfurl builds a plan to deploy them. With TOSCA as its foundation, Unfurl provides development facilities, similar to the ones we listed above, that you can apply in an incremental, opt-in basis:

• TOSCA provides an object-oriented type system that you can apply to those resource declarations. In addition to TOSCA’s native YAML representation, Unfurl provides a Python mapping so you have the full expressiveness of a familiar programming language available.

• IDE and tooling: Using our Python DSL we can leverage its IDE and tooling integrations and take advantage of the rich ecosystem that already exists, for example, for code completion and navigation or for unit testing.

• Package management: You can treat any git repository as a package with semantic versioning and Unfurl tracks upstream changes to those repositories, detecting conflicts and potential breaking changes.

• Version control: Unfurl records both configuration and deployment history and status in git. And through its integration with Unfurl Cloud you get a UI for managing Unfurl that records all configuration changes in git and supports git-based workflow, such as pull requests with CI/CD checks and approvals.

An example

To illustrate, here is a TOSCA service template that deploys a compute instance running a web application. This example uses our Unfurl Cloud standard library which is a library of TOSCA types that provide cloud-provider agnostic abstractions with implementations for the major cloud providers as well as Kubernetes.

import unfurl
import tosca
from tosca import MB, GB  # scalar types to avoid mistaken assumptions
from tosca_repositories import std
# std provides uniform interfaces for the major cloud providers and Kubernetes
from tosca_repositories.std import aws as cloud
from tosca_repositories.std.dns_services import Route53DNSZone

compute = cloud.ComputeInstance(
    num_cpus=2,
    mem_size=2048 * MB,
    disk_size=32 * GB,
    floating_ip = cloud.StaticIPAddress(),
)

dns_zone = Route53DNSZone(name="example.com")  

container_host = std.HttpsProxyContainerComputeHost(
    host=compute,
    dns=dns_zone,
)

class MyAppContainer(std.ContainerService):
    environment: "AppEnvVars"
    container = unfurl_datatypes_DockerContainer(
        image="registry.gitlab.com/gitlab-org/project-templates/express/main:latest",
        ports=["5000:5000"],
    )

# declare a class to provide structure to your app's environment variables:
class AppEnvVars(unfurl.datatypes.EnvironmentVariables):
    # bind the environment value to the container's configuration so they are always in sync:
    PORT: tosca.datatypes.NetworkPortDef = MyAppContainer.container.ports[0].target

# __root__ is a like traditional program's main()
__root__ = std.WebApp(
    container=MyAppContainer(host=container_host, 
                             environment=AppEnvVars()),
    subdomain="app")

When this script is included in an Unfurl ensemble and deployed, it converted to TOSCA and a plan is created that instantiates the compute instance, configures it to run the app behind a HTTPS proxy server, and update DNS, with the implementations of those operations defined by the ComputeInstance, HttpsProxyContainerComputeHost, and Route53DNSZone classes, respectively. If you look at those implementations, you’ll see they invoke Terraform, Ansible jinja2 templates, and a Python library (octodns) – but without having to stray beyond declarative TOSCA – in fact they were originally implemented in YAML. And because well-defined strongly-typed interfaces, you can safely extend the implementations, for example, to customize the Terraform it uses.

The big picture

Unfurl wants to be more than just a great deployment tool – the facilities described above are designed to enable building and sharing what we call service blueprints, as a building block for something like a package manager for the cloud. Building on top of Unfurl, there’s:

• The Unfurl Cloud standard library that we used in the example above to provide cloud-provider agnostic abstractions and interoperability. It supports the major cloud providers as well as Kubernetes.

• Unfurl Cloud is our free, open-source platform for developing blueprints and managing deployments. Unfurl Cloud simplifies deploying blueprints to your own cloud accounts by generating custom UI for each blueprint. All configuration is automatically committed to git as user-editable YAML. This git-centric approach allows Unfurl Cloud to provide familiar, developer-friendly workflows for collaboratively developing blueprints and managing live deployments, such as pull requests and deploying from code via CI/CD pipelines. And when your blueprint is ready, you can publish it to our public catalog of popular open-source applications.

• We want Unfurl Cloud to be a collaboration platform for developing, sharing, combining, and remixing blueprints to help build a free and open cloud. To that end, we’ve created a cloudmap, the very beginnings of a package manager for the cloud. When you publish a public blueprint on Unfurl Cloud we commit it to the cloudmap so that anyone with a compatible TOSCA orchestrator can deploy it – and get added to a neat visualization too :)

The even bigger picture

A package manager for the cloud would be cool for sure but what really excites us about it is that it’s a means to bootstrap a free and open cloud – one that is open-source, decentralized, and transparent.

If that sounds exciting, please consider getting involved in this open-source project. Or maybe you just want an easy way to manage your deployments throughout your development process… Either way check out Unfurl and Unfurl Cloud and let us know what you think!