ReScript on Deno: Declarative Command Line Tools

In our last instance of ReScript on Deno, we learned how to use the Flags module to build command-line tools that were type-safe. To achieve this we used either entirely ad-hoc methods, or we changed our approach and typed the entire set of command-line arguments and flags as a very flexible map.

Today we're going to introduce a small layer to help us build declarative command-line tools. This will give us the most type-safety and provide a great developer experience.

What makes an API declarative?

There used to be a time when you'd download a library, in almost any language, by manually finding the source code and literally downloading it into your own project's folder. If we wrote a program to do this, it could look like this.

let lib = await fetch("https://codez.com/pkgs/myLibrary-0.2.2.zip")
let src = await unzip(lib)
let dst = path.join(projectRoot, "3rdparty/${lib.name}")
await cp(src, dst)

Today we use package managers where we specify what package we want, and which version we want, and they figure out how to do all of the steps that used to be manual. First, we write our tiny description of what we want, then we tell the package manager to figure it out.

// package.json
{
    "dependencies": {
        "rescript": "9.2"
    }
}

// shell
yarn install

Going from a list of manual steps that get something done, to a description of what we want to have, is the move from providing an imperative to a declarative API.

Another example of this is CSS. There are a dozen ways or so of telling the browser what you want it to do: you can write CSS directly, you can use the entirety of the DOM APIs to set specific CSS attributes of a DOM Node, but whatever you do you can't tell the browser how to actually render the CSS. This means the browser vendor can take care of the how, and you can focus on the what.

Yet another example is Pulumi and Terraform, both of which use a programming language to create a description of what you need. Later on, their engines will figure out how to get you what you need.

It does not matter if your API uses external static files to declare what you want, or if it uses a library that builds up descriptions of what you want without executing them. In both cases, we are talking about Declarative APIs.

Building Declarative APIs in ReScript

In ReScript we have amazing tools for building declarative APIs. For example, if we wanted to build an API for incrementing and decrementing numbers, we can do it declaratively by creating data that represents what should happen, not how.

/* Here's a type representing what can be done */
type action =
  | Inc(int)
  | Dec(int)
  
/* Here's a value representing what we want */
let incBy2 = Inc(2)

And in a separate place, we can define how this can be executed.

let run = (initialState, action) => {
  switch action {
  | Inc(x) => initialState + x
  | Dec(y) => initialState - y
  }
}

This separation of what and how is incredibly powerful, because it allows us to swap out the how with alternative versions that can be faster, simpler, easier to understand, better for local development or testing, etc.

However, we have a small problem. How do we run 2 increments one after the other? We would need to call run twice! This seems like it breaks the declarative-ness of our API because now we have to manually interact with it through what could be seen s an imperative command.

If you wanted to run several actions, you would need to write code that calls our run function several times. So let's make one more change we can add to our API here is to allow us to chain actions, so we can represent multiple actions done in a sequence without having to write more code.

/* Our action type now supports arbitrary chains of actions! */
type action =
  | Inc(int)
  | Dec(int)
  | Chain(action, action)

Excellent. Putting this together we can write our example of 2 increments like this.

let incBy2Twice = Chain(Inc(2), Inc(2))

And our run function needs to be amended to support Chain:

let run = (state, action) => {
  switch action {
  | Chain(a, b) => state->run(a)->run(b)
  | Inc(x) => state + x
  | Dec(y) => state - y
  }
}

This function is an implementation detail of the library that we only need to call once, similar to our yarn install above. Then the library can figure out how to do what we asked it to do.

The next step from here is to build declarative APIs that are actually driven by type information.

Type-Driven Declarative APIs in ReScript

Imagine we're building now a library to read data from a file. We know the data is highly structured, but the shape of the data can vary. The kinds of data are simple data types, and the shape of the data is expected to be like an array: one field after the other.

module Data = {
  // We have 3 kinds of data
  type kinds =
    | Int
    | String
    | Bool
  
  // And we describe data shapes as an array of kinds
  type shape = array<kinds>
}

One example of this would be a little file like this one, with each field on one line.

// file: test.data
"hello"
1
true

Which has a shape: [String, Int, Bool].

Excellent. Our library also should provide a read function that will take a path to a file, the shape of the data in the file, and a function that will be called with the data after it has been processed to fit the expected shape. It has the following type-signature.

module Data = {  
  let read :
    (
      ~filepath: string,
      ~shape: shape,
      ~handler: ??? => unit
    ) => unit
}

What should the input type of the handler function be? Let's examine some of our options here:

• We can make the type be a specific thing, like a new type t that includes the data that we read. So the handler function always has the same type, like t => unit.
  
• We can ask for a function that takes as many arguments as elements we have in the shape of the data we're asking for. So the handler function type fits the shape of the data. If our shape is [String, Int, Bool] then our handler type is (string, int, bool) => unit.

Adding a new type for Read Data

In this option, we could imagine a type t that represents data that was read from a specific shape. It could look like this.

module Data = {
  type read_data = 
    | Int(int)
    | Bool(bool)
    | String(string)
    
  type t = array<read_data>
}

This type is very useful because it would mean the implementation of the library just has to map from our kind type to our read_data type as it goes reading the file. In pseudocode:

// pseudocode for mapping the expected kind of data
// to the actual read data type
let parseFile = () => {
   switch expectedKind {
   | Int => Int( currentData.value->int_of_string )
   | Bool => Bool( currentData.value->bool_of_string )           
   | String => String ( currentData.value )
   }
}

A downside of this approach is that when our handler function receives the data, it will receive an array of fields but:

• We don't know what is the kind of each field upfront
• We don't know how many fields we have!
• We don't even know exactly how many fields we should have

So the cost of making this library easier to implement is that its usage becomes harder. Our handler function will have to iterate over the read data, find the right data types, and do its own state management to make sure it doesn't accidentally read data in the wrong order.

Asking for a Function that fits the Shape of the Data

In practical terms, this means that if our data shape was [String, Int, Bool] our handler function type should look like (string, int, bool) => unit.

To do this, we will need to introduce one fancy type-level tool called Applicative Functors.

Applicative Functors help us build up the type of a function, one argument at a time before we actually call the function. This tripped me up real good when I first encountered it, but it roughly means that the following code is type-safe.

module User = {
  // Sample record that represents the file
  type t = { name: string, age: int, isAdmin: bool }

  // a handy helper function to build this record
  let make = (name, age, isAdmin) => {name, age, isAdmin}
}

let fromFile = 
    Data.handler(User.make)
    ->Data.field(0, Data.string) 
    ->Data.field(1, Data.int)
    ->Data.field(2, Data.bool)

What you can see here is that we have a tiny User.t that is the type we'd like to use in our application. We also have a small value called fromFile, which is using our new Type-Driven Declarative API to make 100% sure that we are reading this file's fields in the right order, and that the types match up too.

This is a clearly better developer experience since the library user has to do almost nothing to consume this data in the right order. Just make sure we specify upfront the index of each one of the fields and the type they have. In fact, if you already had a User.make function around (which you should anyway), we didn't need to change anything about your existing application to add this new feature.

Not only that, but if you refactor your code, say we switch up one of the data types in our record, say age from int to string, we see a type error!

Now let's see how the sausage is made. What goes into making a library like this? As usual, we will start with the interface.

module type Data = {
    type t<'a>

    let handler: 'a => t<'a>
    let field : (t<'a => 'b>, int, t<'a>) => t<'b>
    
    let int: t<int>
    let string: t<string>
    let bool: t<bool>
}

The interface to our Data module now provides a few high-level functions that help us construct a Data.t. When we call Data.handler(fn), we create a handler function that will handle whatever type. For example, for our User.make this looks like this.

From this point onwards, our Data.t has all the information we need to know what arguments to parse. Our next function of interest is Data.field, which takes two t values, and the position of the field, and returns a new one. The interesting part is actually inside of the type arguments of t.

(t<'a => 'b>, int, t<'a>) => t<'b>

If this seems familiar to a function application, is because it is! It is function application at the type level 🤯.

What this function lets us do is grab a value of type t that has a function inside, and apply one argument to that function. Remember our handler from the last example, it had type: Data.t<(string, int, bool) => User.t>.

Because all functions in ReScript are curried by default, we can also write this as Data.t< string => int => bool => User.t >. Now we can see that if we try to call field with our handler, then the first parameter we must provide, must be a string.

Once we do that, we can see how the type-level function type starts to get smaller. In this case, we know that we have a string, so the next argument must be an int, and the one after that a bool.

The rest of the functions we have been using are internal helpers that know how to parse a specific piece of data. At this point, we are not concerned with the implementation of this library, but we will look at one soon.

This is just how the consumer side of this library looks: simply amazing.

Declarative Command-Line Applications

Now that we have seen how to build Type-Driven Declarative APIs, we can try to apply some of these ideas to building command-line applications.

Let's start by defining the interface of a library that will help us describe our CLI. Let's call our library Clipper.

module type Clipper = {
  type t<'a>
  
  let command : 'a => t<'a>
  
  module Arg : {
    type t<'a>
    
    let string: t<string>
    let bool: t<bool>
  }
  
  let arg: (t<'a => 'b>, int, Arg.t<'a>) => t<'b>

  let run: t<'a> => 'a
}

Amazing! Still a little bare, but we can now use the same type-level technique to start describing our CLI. This looks like this:

let sayHi = name => `Hello, ${name}!`->Js.log

let hello = Clipper.command(sayHi)->Clipper.arg(0, Clipper.Arg.string)

And the inferred type without arguments is what we would expect:

You'll notice that we also added a separate module for dealing with arguments, called Arg. Let's use it to put together our full command-line app.

let hello =
    Clipper.command(sayHi)
    ->Clipper.arg(0, Clipper.Arg.string)

Done! 🎉

Okay, not quite. We have cheated here by not digging into the implementation of a library like this, but since we also haven't really shown what the actual values look like, our implementation could be doing a lot of things.

So let's look at a super naive implementation that will make use of our last set of general bindings for the Flags module. If you need a refresher, you can find those bindings here:

ReScript on Deno: Command Line Tools and the Flags module

We will learn how to bind to the Flags module in Deno’s standard library to make it easy and type-safe to pass in arguments to our command-line tools.

Practical ReScriptLeandro Ostera

Here's our implementation.

module Clipper : Clipper = {
  module Arg = {
    type t<'a> = string => 'a
    
    let string = x => x
    let bool = bool_of_string
  }

  type t<'a> = array<string> => 'a

  let command = (handler: 'a): t<'a> => {
    (_args) => handler
  }

  let arg = (handler, pos, arg) => {
    args => {
      switch Belt.Array.get(args, pos) {
      | Some(rawArg) =>
        let lastArg = arg(rawArg)
        handler(args)(lastArg)
      | None =>
        Js.log2("ERROR: Expected argument on position ", pos)
        assert false
      }
    }
  }

  let run = handler => {
    Deno.Flags.parse(Deno.args)
    ->Deno.Flags.command
    ->handler
  }
}

It can power small CLIs tools like this one:

let sayHi = (name, yell) =>
  if yell {
    `Hello, ${name}!!!`->Js.log
  } else {
    `Hello, ${name}.`->Js.log
  }

let hello =
    Clipper.command(sayHi)
    ->Clipper.arg(1, Clipper.Arg.string)
    ->Clipper.arg(0, Clipper.Arg.bool)

Clipper.run(hello)

As you can see in the Clipper.run function, we are relying on our Flags bindings to get access to all of the commands specified by the user (not the flags tho!). The order in which we define our arguments in the code doesn't matter, since all the arguments will be parsed any way!

We could take this Clipper library forward in many directions:

• Declaratively specify default values for arguments
• Distinguish between a flag and a valued argument
• Add support for multiple commands
• Add support for custom types, not just basic ones
• Generate help outputs

All of these can be added within the same core API we have built above.

Wrapping Up

Declarative APIs, and in particular the Type-Driven subset of them, are incredibly powerful. They can let us build very complex programs that are much easier to guarantee safety on, while also letting us provide very good developer experiences by helping developers focus on what they want. The library can then focus on how to get it done.

They really put the burden of complexity on the library author, but this means that at the end of the day the developer experience can be great. Nonetheless, remember that the more complex your types get, the harder it will be for people to debug what they're building!

So next time you're about to write some types for a Deno library, consider all of the typing approaches we have seen so far:

• Ad-hoc typings – to quickly get up and running
• More general typings – to be more flexible as a library user
• Type-driven Declarative APIs too – to provide the safest developer experience, and keep the most flexibility as a library author

Now it's your turn to try and build something cool with ReScript on Deno, and don't forget to tell me how it went on Twitter! Just @ me: @leostera

See you on the next issue of Practical ReScript 👋🏽

Thanks to Malcolm Matalka for improvements suggested on the Data API above!