2.1 Trailblazer Test Documentation

Testing a Trailblazer project is very simple. Your test suite usually consists of two separate layers.

• Integration tests or system tests covering the full stack, and using Capybara to “click through” the happy path and possible edge-cases such as an erroring form. Smoke tests make sure of the integrity of your application, and assert that controllers, views and operations play well together. We will provide more documentation about system tests shortly.
• Operation unit tests guarantee that your operations, data processing and validations do what they’re supposed to. As they’re much faster and easier to write than full stack “smoke tests” they can cover any possible input to your operation and help quickly asserting the created side-effects. The trailblazer-test gem is here to help with that.

There’s no need to test controllers, models, service objects, etc. in isolation - unless you want to do so for a better documentation of your internal APIs. As operations are the single entry-point for your functions, your entire stack is covered with the two test types.

The trailblazer-test gem allows simple, streamlined operation unit tests.

Installation

Add this line to your application’s Gemfile:

gem 'trailblazer-test', ">= 1.0.0"

The provided assertions and helpers work with Minitest. For RSpec support use rspec-trailblazer which provides matchers such as pass_with and fail_with around our assertions.

Operation unit tests

trailblazer-test

Whenever you introduce a new operation class to your application, you have four choices for testing.

1. You could skip testing and program the next feature - then, you shouldn’t be reading this.
2. If the operation is simple enough, cover its behavior in a smoke test and test both the successful invocation and its invalid state in a UI test. Nevertheless, this can be cumbersome and slow.
3. Write your own operation unit tests.
4. Use assert_pass and assert_fail to quickly test all possible inputs and outcomes in a unit test.

Overview

The goal of trailblazer-test is to provide simple, strong, non-verbose testing of operations and edge cases.

it "converts {duration} to seconds" do
  assert_pass( {duration: "2.24"}, {duration: 144} )
end

The assertion helpers focus on minimizing test code and the associated pain with it.

A typical operation test case might look as follows.

# test/operation/song_operation_test.rb
class SongOperationTest < OperationSpec

  # The default ctx passed into the tested operation.
  let(:default_ctx) do
    {
      params: {
        song: { # Note the {song} key here!
          band:  "Rancid",
          title: "Timebomb",
          # duration not present
        }
      }
    }
  end

  # What will the model look like after running the operation?
  let(:expected_attributes) do
    {
      band:   "Rancid",
      title:  "Timebomb",
    }
  end

  let(:operation)     { Song::Operation::Create }
  let(:key_in_params) { :song }

  it "passes with valid input, {duration} is optional" do
    assert_pass( {}, {} )
  end

  it "converts {duration} to seconds" do
    assert_pass( {duration: "2.24"}, {duration: 144} )
  end

  it "converts {duration} to seconds" do
    assert_pass( {duration: "2.24"}, {duration: 144} ) do |result|
      assert_equal true, result[:model].persisted?
    end
  end

  it "converts {duration} to seconds" do
    result = assert_pass( {duration: "2.24"}, {duration: 144} ) # with our without a block

    assert_equal true, result[:model].persisted?
  end

  describe "we have {:song}, not {:model}" do
    let(:operation) {
      Class.new(Song::Operation::Create) do
        step :set_song

        def set_song(ctx, **)
          ctx[:song]  = ctx[:model]
          ctx[:model] = "i don't exist"
        end
      end
    }

    it "what" do
      assert_pass( {duration: "2.24"}, {duration: 144}, model_at: :song )
    end
  end

  it "fails with missing {title} and invalid {duration}" do
    assert_fail( {duration: 1222, title: ""}, [:title, :duration] )
  end
  # ...
end

As - hopefully - visible in this example trailblazer-test focuses on minimalistic, less verbose testing of operations. This in turn makes testing literally a no-brainer where testing all edge cases actually is fun.

Note that only #assert_pass and #assert_fail are specific to Trailblazer. Structural helpers such as it and describe are provided by Minitest, and the test style is completely up to you. See below for conventions of how we test operations.

Base class

It is a good idea to maintain a slim OperationTest or OperationSpec base class in your test_helper.rb.

# test/test_helper.rb
# ...

class OperationSpec < Minitest::Spec
  include Trailblazer::Test::Assertions
  include Trailblazer::Test::Operation::Assertions
end

By including the assertion modules your tests can use our assertions such as assert_pass and assert_exposes.

Configuration

The trailblazer-test gem allows to configure common options on the test level.

Configuration default ctx

You can define a default_ctx using let() (and maintain different versions in describe blocks). default_ctx defines what to pass into the operation per default.

let(:default_ctx) do
  {
    params: {
      song: { # Note the {song} key here!
        band:  "Rancid",
        title: "Timebomb",
        # duration not present
      }
    }
  }
end

Note that default_ctx is always the full context that you’d normally pass into Operation.call().

Configuration Expected attributes

The counterpart of default_ctx defines what the attributes of the operation’s model will look like when the operation was run successfully.

# What will the model look like after running the operation?
let(:expected_attributes) do
  {
    band:   "Rancid",
    title:  "Timebomb",
  }
end

As in 99% of operation usage a model is created or altered, this hash represents the attributes of the model that were modified by the operation.

Configuration operation

Instead of having to pass what operation gets tested to each assertion you can conveniently define it using let().

let(:operation)     { Song::Operation::Create }

Again, you may use any level of describe to fine-tune your tests.

assert_pass

The assertion helpers are bringing together operation, default_ctx and expected_attributes.

Use assert_pass to run an operation and assert it was successful, while checking if the attributes of the operation’s :model are what you’re expecting.

it "passes with valid input, {duration} is optional" do
  assert_pass( {}, {} )
end

assert_pass with empty hashes will invoke the operation configured via let(:operation) by passing the hash from let(:default_ctx) into call. After that, it grabs the operation’s result[:model] and tests all attributes from expected_attributes on it.

If you were to hand-write this code, something along this would happen.

it "passes with valid input, {duration} is optional" do
  result = Song::Operation::Create.(default_ctx)
  assert result.success?
  assert_equal "Rancid",   result[:model].band
  assert_equal "Timebomb", result[:model].title
end

That’s quite a lot of code, right?

assert_pass Manual options

assert_pass makes it really simple to override ingoing form attributes. This is perfect for testing validations of your contracts by simulating differing user input.

it "converts {duration} to seconds" do
  assert_pass( {duration: "2.24"}, {duration: 144} )
end

Per default, the assertion accepts two arguments.

1. The first hash to assert_pass is merged into the default_ctx.
2. The second argument is merged with expected_attributes and allows to assert a different outcome.

However, and now listen up, default_ctx and the first hash are not merged on the top-level! If let(:key_in_params) is set the merging behavior is changed.

let(:key_in_params) { :song }

With this configuration in place, the first argument given to assert_pass is merged with the hash found at default_ctx[:params][:song]. This allows for quickly changing the input that usually represents a form or API document - things to test that you will spend a lot of time on.

Again, if you’re hand-coding, here’s what is going on.

assert_pass( {duration: "2.24"}, {duration: 144} )
# identical to

input  = default_ctx[:params][:song].merge!(duration: "2.24")

result = Song::Operation::Create.(input)
# ... assertions

Changing the form input part in order to test validations and processing is a pretty common task. The assert_pass assertion is built to minimize test code and helps you with just that.

assert_pass Omit key

In case you don’t use a key in your form input you can set :key_in_params to false.

let(:key_in_params) { false }

let(:default_ctx) do
  {
    params: {
      band:  "Rancid",
      title: "Timebomb",
    }
  }
end

Note that this decision is also reflected in your default_ctx, which now doesn’t have the :song key anymore.

When using assert_pass and assert_fail you can still worryless pass form input. The assertion now merges it directly with :params.

it "sets {title}" do
  assert_pass( {title: "Ruby Soho"}, {title: "Ruby Soho"} )
end

assert_pass Model name

assert_pass usually tests the expected attributes on result[:model]. Use :model_at to change the key where to find the asserted object.

In this example, the asserting code will now grab the model from result[:song].

assert_pass Block

If you need more specific assertions, use a block with assert_pass.

it "converts {duration} to seconds" do
  assert_pass( {duration: "2.24"}, {duration: 144} ) do |result|
    assert_equal true, result[:model].persisted?
  end
end

This will run all assertions built into assert_pass. By yielding the operation’s result in case of success, additional assertions can be made manually.

Note that the block is not run when an earlier assertion in assert_pass has failed.

assert_pass Result

Both versions of assert_pass (with or without a block) return the result object of the operation call.

it "converts {duration} to seconds" do
  result = assert_pass( {duration: "2.24"}, {duration: 144} ) # with our without a block

  assert_equal true, result[:model].persisted?
end

Use this for adding assertions outside the block.

assert_fail

To test an unsuccessful outcome of an operation, use assert_fail. This is helpful for testing all kinds of validations. By invoking the operation with insufficient or wrong data it will fail and add errors onto the errors object. Those can be matched against a list of expected errors.

it "fails with missing {title} and invalid {duration}" do
  assert_fail( {duration: 1222, title: ""}, [:title, :duration] )
end

The merging of the form input works exactly as in assert_pass, respecting keys_in_params and default_ctx.

1. First, the operation is run, returning the result object. The first assertion is whether result.failure? is true.
2. After that, the operation’s error object is extracted and its keys are compared to the list that was provided to assert_fail as the second argument. If the keys in the actual errors object and your array match, this assertions passes, and proves that particular validations do work.

This translates to the following manual test case.

it "fails with missing {title} and invalid {duration}" do
  assert_fail( {duration: 1222, title: ""}, [:title, :duration] )
  # is almost identical to:

  input  = {params: {song: {title: "", band: "Rancid", duration: 1222}
  result = Song::Operation::Create(input)

  assert result.failure?
  assert_equal [:title, :duration], result["contract.default"].errors.messages.keys
end

Per default, no assumptions are made on the model. Use assert_fail with a block to do so.

assert_fail Error messages

If testing the presence of validation errors is not enough you can pass a hash to assert_fail to check that a particular error message (or multiple) is set.

it "fails with missing {title} and invalid {duration} with given error messages" do
  assert_fail( {duration: 1222, title: ""},
    {title: ["must be filled"], duration: ["must be String"]} # hash instead of array!
  )
end

When there is only one error message per field, you may omit the array for your convenience.

it "fails with missing {title} and invalid {duration} with given error messages" do
  assert_fail( {title: "", duration: "1222"}, {title: "must be filled"} )
end

assert_fail automatically wraps error messages in an array.

assert_fail Block

You can use a block with assert_fail for more specific assertions.

it "fails with missing {title} and invalid {duration}" do
  assert_fail( {duration: 1222, title: ""}, [:title, :duration] ) do |result|
    assert_equal false, result[:model].persisted?
    assert_equal 2,     result[:"contract.default"].errors.size
  end
end

The block is run in addition to assert_fail’s built-in assertions. Just as in assert_pass it receives the operation’s result object and allows you to add assertions as you desire.

assert_fail Result

Both versions of assert_fail (with or without a block) return the result object of the operation call.

it "fails with missing {title} and invalid {duration}" do
  result = assert_fail( {duration: 1222, title: ""}, [:title, :duration] )

  assert_equal false, result[:model].persisted?
end

Use this for adding assertions outside the block.

Debugging

Both assert_pass and assert_fail allow to run the operation using wtf?. When you’re unsure about what is going on inside the tested operation, the :wtf? option comes to rescue.

it "fails with missing {title} and invalid {duration}" do
  assert_fail( {duration: 1222, title: ""}, [:title, :duration], :wtf? )
  #=>
  # -- Song::Operation::Create
  # |-- Start.default
  # |-- model.build
  # |-- contract.build
  # |-- contract.default.validate
  # |   |-- Start.default
  # |   |-- contract.default.params_extract
  # |   |-- contract.default.call
  # |   `-- End.failure
  # `-- End.failure
end

In the trace on the console, it’s obvious that the contract validation failed. Again, note that this works for both assert_pass and assert_fail.

Options

All let() options can also be passed directly to the assertion, allowing you to override the let() option or not having to define it on the test level at all.

it "Update allows integer {duration}" do
  assert_pass( {duration: 2224}, {duration: 2224}, operation: Song::Operation::Update )
end

Accepted options are :operation, :default_ctx, :expected_attributes and :key_in_params.

Ctx Helper

If you need a customized ctx to pass into the tested operation, use Ctx(). It’s mostly used as the first argument to assert_pass and assert_fail.

it "converts {duration} to seconds" do
  assert_pass( Ctx({current_user: yogi}), {title: "Timebomb"} )
end

The helper brings some nice API especially for testing edge cases and failures.

it "fails with missing key {:title}" do
  assert_fail( Ctx(exclude: [:title]), [:title] ) do |result|
    assert_equal ["must be filled"], result[:"contract.default"].errors[:title]
  end
end

Whenever you use Ctx() the computed hash is passed through directly into the operation, so you have full control over what’s going in. Both assert_pass and assert_fail are not altering your ctx anymore.

Ctx Helper Merge

The standard behavior of Ctx() is to merge the passed hash with default_ctx.

it "provides {Ctx()}" do
  ctx = Ctx({current_user: yogi}  )
  #=> {:params=>{:song=>{:band=>"Rancid", :title=>"Timebomb"}},
  #    :current_user=>#<User name="Yogi">}
  # ...
end

This allows to quickly add injections such as the :current_user to the ctx.

Anything under the :params key is deep-merged with your default_ctx, which is super handy when you need to add non-form variables.

it "provides {Ctx()}" do
  ctx = Ctx(
    {
      current_user: yogi,
      params: {song: {duration: 999}} # this is deep-merged!
    }
  )
  #=> {:params=>{:song=>{:band=>"Rancid", :title=>"Timebomb", duration: 999}},
  #    :current_user=>#<User name="Yogi">}
  # ...
end

Note how :duration is added to the existing :song hash.

Ctx Helper Exclude

If you want to delete a certain form field from the input, you can use :exclude. This is great for testing presence or required validations.

it "provides {Ctx()}" do
  ctx = Ctx(exclude: [:title])
  #=> {:params=>{:song=>{:band=>"Rancid"}}}
  # ...
end

The :title field under :song is now removed from the input.

You may also use :exclude in combination with the deep-merge.

it "provides {Ctx()}" do
  ctx = Ctx({current_user: yogi}, exclude: [:title])
  #=> {:params=>{:song=>{:band=>"Rancid"}},
  #    :current_user=>#<User name="Yogi">}
  # ...
end

assert_exposes

Test attributes of an arbitrary object.

Pass a hash of key/value tuples to assert_exposes to test that all attributes of the asserted object match the provided values.

it do
  assert_exposes model, title: "Timebomb", band: "Rancid"
end

Per default, this will read the values via model.{key} from the asserted object (model) and compare it to the expected values.

This is a short-cut for tests such as the following.

assert_equal "Timebomb", model.title
assert_equal "Rancid",   model.band

Note that assert_exposes accepts any object with a reader interface.

assert_exposes Reader

If the asserted object exposes a hash reader interface, use the :reader option.

it do
  assert_exposes model, {title: "Timebomb", band: "Rancid"}, reader: :[]
end

This will read values with via #[], e.g. model[:title].

If the object has a generic reader, you can pass the name via :reader.

it do
  assert_exposes model, {title: "Timebomb", band: "Rancid"}, reader: :get
end

Now the value is read via model.get(:title).

assert_exposes Lambda

You can also pass a lambda to assert_expose in order to compute a dynamic value for the test, or for more complex comparisons.

it do
  assert_exposes model, title: "Timebomb", band: ->(actual:, **) { actual.size > 3 }
end

The lambda will receive a hash with the :actual value read from the asserted object. It must return a boolean.

factory

You should always use operations as factories in tests. The factory method calls the operation and raises an error should the operation have failed. If successful, it will do the exact same thing call does.

it "calls the operation and raises an error and prints trace when fails" do
  exp = assert_raises do
    factory Create, params: {title: "Shipwreck", band: "The Chats"}
  end

  exp.inspect.include? %(Operation trace)
  exp.inspect.include? "OperationFailedError: factory(Create) has failed due to validation "\
                       "errors: {:band=>['must be Rancid']}"
end

If the factory operation fails, for example due to invalid form input, it raises a OperationFailedError exception.

factory( Song::Create, { title: "" } )["model"]
#=> Trailblazer::Test::OperationFailedError: factory( Song::Create ) failed.

It is absolutely advisable to use factory in combination with let.

let(:song) { factory( Song::Create, { title: "Timebomb", band: "Rancid" } ) }

Also, you can safely use FactoryGirl’s attributes_for to generate input.

factory mock_step

This helper allows you to mock any step within a given or deeply nested activities. For example,

class Show < Trailblazer::Activity::FastTrack
  class Complexity < Trailblazer::Activity::FastTrack
    class ExternalApi < Trailblazer::Activity::FastTrack
      step :make_call
      # ...
    end

    step :some_complex_task
    step Subprocess(ExternalApi)
    # ...
  end

  step :load_user
  step Subprocess(Complexity)
  # ...
end

To skip processing inside :load_user and use a mock instead, use mock_step.

it "mocks loading user" do
  new_activity = mock_step(Show, id: :load_user) do |ctx|
    ctx[:user] = Struct.new(:name).new('Mocky')
  end

  assert_pass new_activity, default_params, {} do |(signal, (ctx, flow_options))|
    assert_equal ctx[:user].name, 'Mocky'

    assert_equal ctx[:seq], [:some_complex_task, :make_call]
  end
end

Internally, it creates and returns a fresh, subclassed activity (via patching) whilst replacing the step for given :id. Be advised that this does not change the original activity class.

You can also mock any nested activity (aka Subprocess) which does any heavy computations or I/O calls.

new_activity = mock_step(Show, id: Show::Complexity) do
  true # no-op to avoid any Complexity
end

In case you want to mock only single step from the nested activity, you can do so by passing it as a subprocess.

new_activity = mock_step(Show, id: :some_complex_task, subprocess: Show::Complexity) do
  # Mock only single step from nested activity to do nothing
  true
end

It’ll search the :id to be mocked within nested activity instead of top-level activity.

In addition, if you want to mock any deeply nested step in subprocess's activity, it can be done via passing subprocess_path.

new_activity = mock_step(Show, id: :make_call, subprocess: Show::Complexity, subprocess_path: [Show::Complexity::ExternalApi]) do
  # Some JSON response
end

subprocess_path should list n-level of nested activities in the order they are nested. Internally, it uses patching API supported by Subprocess helper.

Rspec

rspec-trailblazer

System Test

This document will describe how we test the full stack using Capybara and system tests.

Also described in part II of the new book.