Ebro

Ebro is a task runner. Tasks are defined inside YAML files, scripted with Bash, and configured with a name, requirements, when to skip execution, and other details. Tasks can also be imported from other files or extend other tasks.

Ebro is distributed as a single binary, including the script interpreter (mvdan/sh).

It's heavily inspired in go-task/task, but originally built around a personal need for configuring servers, although it's not tied to this use case and remains agnostic.

Getting started

The format of Ebro.yaml files is defined here. Here is an example:

tasks:
  default:
    requires: [echoer, producer]

  echoer:
    script: |
      cat cache/A.txt
      cat cache/B.txt
    when:
      output_changes: |
        cat cache/A.txt
        cat cache/B.txt

  producer:
    requires: [produce_a, produce_b]

  produce_a:
    requires: [cache_dir]
    required_by: [echoer]
    script: echo 'this is A'>cache/A.txt
    when:
      check_fails: test -f cache/A.txt

  produce_b:
    requires: [cache_dir]
    required_by: [echoer]
    script: echo 'this is B'>cache/B.txt
    when:
      check_fails: test -f cache/B.txt

  cache_dir:
    script: mkdir -p cache
    when:
      check_fails: test -d cache

To give it a try, create a folder in your system, copy the content above in a file inside it called Ebro.yaml, and also download the ebrow script alongside it.

curl --location --output ebrow 'https://github.com/sirikon/ebro/releases/latest/download/ebrow'

Before running it, read the script and understand what it does (because you shouldn't blindly execute scripts from the internet).

ebrow is Ebro's "workspace script". It is a Bash script that contains a reference to an exact Ebro version and is able to download it, verify its integrity, and place it inside the .ebro directory created next to itself. The next time you execute it, it will use the already-downloaded Ebro binary. It ensures that the correct binary is present in the workspace.

Now, let's give it execution permissions and execute it:

chmod +x ebrow
./ebrow

Ebro on start will check for a file called Ebro.yaml in the working directory and parse it if present, constructing what is called the inventory, a collection of every task available with their definitive configuration for running.

You can check the inventory yourself by calling ./ebrow -inventory, you'll notice extra details like the definitive list of extra environment variables that will be included in each task execution, or the working directory.

Next, it will construct a plan, which is an ordered list of all the tasks that will be executed sequentially in order to reach our target task, which by default is default, but can be any other by passing it as an argument (./ebrow echoer).

Again, check it yourself by running ./ebrow -plan. This plan is deterministic, which means that given the same configuration, it will always be the same.

Finally, it will execute the plan, running tasks sequentially until the end.

During the first execution it will execute everything, with no skips, which should output something like this:

███ [:cache_dir] running
███ [:produce_a] running
███ [:produce_b] running
███ [:echoer] running
this is A
this is B
███ [:producer] satisfied
███ [:default] satisfied

But if we execute ./ebrow again, we'll see this output:

███ [:cache_dir] skipping
███ [:produce_a] skipping
███ [:produce_b] skipping
███ [:echoer] skipping
███ [:producer] satisfied
███ [:default] satisfied

Ebro skips tasks whenever possible, and the task definition is what mandates when a task should be skipped. In our example, the task echoer is skipped whenever the output of running cat cache/A.txt and cat cache/B.txt doesn't change. In the case of the task produce_a, it skips whenever the command test -f cache/A.txt succeeds, because the file cache/A.txt already exists.

Now we'll manually edit the file cache/A.txt, run ./ebrow again, and see the result.

echo 'hello world!' > cache/A.txt
./ebrow

███ [:cache_dir] skipping
███ [:produce_a] skipping
███ [:produce_b] skipping
███ [:echoer] running
hello world!
this is B
███ [:producer] satisfied
███ [:default] satisfied

The when.output_changes checker of the echoer task detected that running cat cache/A.txt and cat cache/B.txt produced a different output when compared with the previous execution, hence, the task is executed again.

Importing tasks

The Ebro.yaml file supports importing tasks from other Ebro.yaml files by defining the imports.*.from parameter. It works like this:

# Ebro.yaml
imports:
  something:
    from: ./somewhere # a directory containing an `Ebro.yaml` file.

# somewhere/Ebro.yaml
tasks:
  default:
    script: echo 'something'
  else:
    script: echo 'something else'

Now, running ebro something has this output:

███ [:something:default] running
something

And running ebro something:else has this output:

███ [:something:else] running
something else

It's important to note that the contents of an Ebro.yaml file are considered a module. When we import another Ebro.yaml file, we're creating a new module that hangs from the root module and has an explicitly-given name. In this case, something.

Targeting a module by its name is equivalent to targeting the module's default task. As something is a module, it translates to something:default.

Task inheritance

Ebro has a system of task inheritance. Tasks can extend other tasks, merging the parent properties with their own. Check the merging strategy in the schema documentation.

Here's an example Ebro.yaml file and what happens when running ebro on it:

tasks:
  default:
    script: echo 'Hello World'

  parent:
    abstract: true
    environment:
      FOO: "foo"
    required_by: [default]

  child:
    extends: [parent]
    script: echo $FOO

Now, running ebro default child (targeting both default and child tasks) has this output:

███ [:child] running
foo
███ [:default] running
Hello World

We can see how the child task looks after the merging strategy is applied with ebro -inventory. It inherited parent's required_by and FOO environment variable, but kept its own script.

:child:
  working_directory: /workdir
  environment:
    EBRO_ROOT: /workdir
    FOO: foo
  required_by:
  - :default
  script: echo $FOO
:default:
  working_directory: /workdir
  environment:
    EBRO_ROOT: /workdir
  script: echo 'Hello World'

CLI

Ebro's command line interface is very straightforward, but has a couple of general rules:

• Commands define an specific action. At most, there is one command in a call. Absence of a command means the default command of running. Commands are prefixed with a single hyphen (-command).
• Flags depend on the command being executed. Their mere presence can mean a boolean value (true or false) or be accompanyed with a value. Flags are prefixed with two hyphens (--flag).
• Targets are the names of the tasks that we want to run. When no target is specified, the task default is assumed.

To know Ebro's available commands with their flags and explanations, run ebro -help (or ./ebrow -help if using the workspace script).

ebro [--flags...] [targets...]
  # Run everything
  flags:
    --file value  Specify the file that should be loaded as root module. default: Ebro.yaml
    --force       Ignore when.* conditionals and dont skip any task. default: false
  targets:
    defaults to [default]


ebro -inventory [--flags...]
  or -i
  # Display complete inventory of tasks with their definitive configuration in YAML format
  flags:
    --file value  Specify the file that should be loaded as root module. default: Ebro.yaml


ebro -list [--flags...]
  or -l
  # Display only the names of all the tasks in the inventory
  flags:
    --file value  Specify the file that should be loaded as root module. default: Ebro.yaml


ebro -plan [--flags...] [targets...]
  or -p
  # Display the execution plan
  flags:
    --file value  Specify the file that should be loaded as root module. default: Ebro.yaml
  targets:
    defaults to [default]


ebro -version
  or -v
  # Display ebro's version information in YAML format


ebro -help
  or -h
  # Display this help message

The Ebro.yaml format

This is also available as a JSON Schema.

The contents of the Ebro.yaml represent a module, which has the following properties:

• working_directory (string)

  The default working directory for all the tasks inside the module. Defaults to the parent module working directory or the current working directory if it is the root module. Relative paths are valid and will be added to the default value to compute the final value.

• environment (object)

  Additional environment variables available for all the tasks inside the module. A limited subset of Bash's capabilities for interpolation (like using ${VAR}) are available to interpolate variables that come from the parent environment (parent module or Ebro's execution environment), but not from the same environment object.

• imports (object)

  Other modules can be imported.

  • * (string)

    Each key in the object will be the module's name when imported.

    • from (string)

      The path to a directory containing an Ebro.yaml file. Relative paths will be added to the module's working directory to compute the final path.

      EXPERIMENTAL: Ebro supports importing from Git repositories when this field is given a reference in the format git+https://.... It should work fine, but it's considered experimental for two reasons:

      First, it has only been tested against public Git repositories using the http transport.

      Second: Automatically downloading and executing code from the internet is dangerous, and it shouldn't really be used without some sort of integrity checking in place.

      Use at your own risk.

    • environment (object)

      Additional environment variables available for all the tasks inside the imported module. A limited subset of Bash's capabilities for interpolation (like using ${VAR}) are available to interpolate variables that come from the parent environment (current module or Ebro's execution environment), but not from the same environment object.

• tasks (object)

  Collection of tasks defined in the module.

  • * (string)

    Each key in the object will be the task's name.

    • working_directory (string)

      The working directory for the task. Defaults to the module working directory. Relative paths are valid and will be added to the default value to compute the final value.

    • abstract (boolean)

      When true, flags the task as abstract. An abstract task is a task that cannot be executed directly and its only purpose is to be extended by other tasks by using the extends property.

    • extends (array)

      References other tasks and extends them, which effectively means merging their properties with the properties of the referenced task. Here's the merging strategy for each property:

      • working_directory: Untouched.
      • abstract: Untouched.
      • environment: Merged. Child task values take precedence over parent task values.
      • requires: Merged and deduped.
      • required_by: Merged and deduped.
      • script: If present, child task value takes precedence over parent task value.
      • quiet: If present, child task value takes precedence over parent task value.
      • when.check_fails: If present, child task value takes precedence over parent task value.
      • when.output_changes: If present, child task value takes precedence over parent task value.
    • environment (object)

      Additional environment variables available for the task. A limited subset of Bash's capabilities for interpolation (like using ${VAR}) are available to interpolate variables that come from the parent environment (task referenced in extends, module or Ebro's execution environment), but not from the same environment object.

    • requires (array)

      List of task names that need to execute before this one does.

      It is possible to reference tasks from submodules by writing the full path of the task (module and task name) with each name separated with a colon (:). Example: submodule:task.

      It's also possible to reference tasks by their absolute path by prepending the whole path with another colon (:). Example: :module:submodule:task.

    • required_by (array)

      List of task names that require this task to be executed before those. The opposite of requires.

      It is possible to reference tasks from submodules by writing the full path of the task (module and task name) with each name separated with a colon (:). Example: submodule:task.

      It's also possible to reference tasks by their absolute path by prepending the whole path with another colon (:). Example: :module:submodule:task.

    • script (string)

      Bash script of the task. set -euo pipefail is always prepended to the script. Interpreted by mvdan/sh.

    • quiet (boolean)

      When true, flags the task as quiet. Quiet tasks are task that, visually and logging-wise, look almost the same as skipped tasks. This effectively means that their output will be hidden by default and only shown if the task fails. Also, their log line for "running" will be tinted green instead of yellow.

    • when (object)

      Configure ways in which the task could be skipped. These are computed as an OR, meaning: One of them triggering is enough to trigger the task execution.

      • check_fails (string)

        Bash script. set -euo pipefail is always prepended to the script. Interpreted by mvdan/sh.

        After execution, the exit code will be checked. If it succeeded (exit code 0), the task is skipped. If it fails (exit code different than 0), the task is executed.

      • output_changes (string)

        Bash script. set -euo pipefail is always prepended to the script. Interpreted by mvdan/sh.

        After execution, the output (stdout and stderr, combined) will be compared with the output of the last time it executed and the task succeeded. If the output is the same, the task is skipped. If the output is different, the task is executed. If there is no previous output stored, the task is executed.

• modules (object)

  Modules can have other modules inside

  • * (string)

    Each key in the object will be the module's name

Versioning

Ebro follows SemVer, this means that it's important to clarify what the API is in Ebro, or, "the exposed parts that will only break compatibility on major version releases".

Ebro's API in SemVer terms is composed of, but not limited to:

• The Ebro.yaml specification. The main idea is that an Ebro.yaml file that worked on Ebro version 1.x.x MUST work exactly the same on any future version under the 1.x.x major release. This includes, but is not limited to:
  • How Ebro.yaml is interpreted.
  • Environment variables added by Ebro during the execution.
    • New environment variables prefixed with EBRO_ COULD be added, and this would not be considered a breaking change.
  • How scripts are interpreted and executed.
  • How task names are processed and referred to.
  • In which order are tasks executed.
• The CLI actions, meaning how the machine state changes given an Ebro.yaml file and a set of commands and flags.
  • How the internals of the .ebro directory operate IS NOT part of the API, and this could change at any time without being considered a breaking change.
• The CLI stdout output (NOT stderr) of the -version, -plan, -inventory and -list commands. These commands output a structured representation of data. New data could be added in the future, but existing data will remain the same. This doesn't apply to formatting, as it could change as long as it satisfies the output format specification.
  • The CLI stdout and stderr output of any other command apart from the ones explicitly mentioned above are NOT considered part of the API and could change at any time without it being considered a breaking change.

Unintended functionality that gets included in Ebro but is considered a bug afterwards will NOT be considered part of the API. Bugs won't be kept for the sake of API stability and will be removed. If this was the case, a release including a bugfix that is also a breaking change will have proper instructions for updating any affected Ebro.yaml file.