Trailblazer - Blog
- Last updated 27 Feb 21
Official 2.1 Release Notes
(Not that thereâs an unofficial one anywhere, but yeahâŚ)
We have fully documented the migration path to 2.1 while upgrading several complex apps. It worked.â˘
After around 3 years of silence, Trailblazer is back with its 2.1 release. Lots of improvements on the core gems come hand-in-hand with a quite flat migration path, so donât ya worry! Weâre also proud to tell you all about our new upcoming projects in this post, as well as communicating a looot of background about whatâs been going on in those past few years.
Hereâs a quick summary for those who want to get back to coding.
- API CHANGE The
call
API has slightly changed from two arguments to just one. - AUTOLOADING We removed the
trailblazer-loader
gem just like Apple removed the headphone jack from the iPhone 6. This brings you faster startup and consistency with Rails autoloading. - DOCUMENTATION We have a new website.
- TRACING We heard your cries. You can now see what path an operation took and use exception tracing for a beautiful dev experience.
Good to see you, again!
It almost feels unreal finishing up this release post. Itâs been so long! The work put into Trailblazer 2.1 has been tremendous, it could easily have been TRB 3.0, or even TRB III, since Roman version numbering turns out to be quite a fancy thing to do. However, as much as the internals have been improved, as little has changed on the public APIs of Trailblazer, so we decided to go with a minor release.
With the introduction of another new callback after_save_commit
in Rails 6 it becomes obvious that we do have a problem in software engineering: how do we structure business code? We do have a few dozen database abstractions in every language, well written and designed, but, yeah, how do I structure the actual code of my domain, of my applicationâs needs? Whatâs the best way to put together application functions, put them in order, prevent other functions from being executed when itâs not the time, and so on?
Trailblazer is an architectural style coming as a framework to implement our patterns, trying to answer that one question: âHow do I implement business processes?â. It gives you strong conventions along with patterns to structure your code flow better, handle errors, and integrate those components into bigger flows - or processes.
Having said that, I let you decide whether or not you join the ride! There is nothing wrong with building your own âservice layerâ, and many companies have left the Traiblazer track in the past years due to problems they had and that we think we now fixed.
Speaking of problems, the craziness of the past years working on Trailblazer 2.1 can be visualized with the following screenshot.
Very often, in order to fix one function A in a commercial app, I had to rewire a track in the endpoint
gem, which would only work if I add another DSL option in the dsl
gem, which couldnât work straight away because the normalizer in the context
gem needed to be extended, which was only possible if you could finally extract the start event via the activity
gem whose test suite was currently broken because I needed to document feature x
first, which couldnât be documented because before I had to allow dynamic params in the macro
gem and⌠I think you got my point.
Luckily, I got help from a band of great people, I took it easy on the conference front, and for some other reasons I still donât understand all this is now finished and hereâs TRB 2.1.
About the Design
A big change from TRB 1.1 and even 2.0 is that we apply functional principles in most places. We barely keep state in instance variables. Rather, data is passed around from operation to operation, from step to step. We use OOP and inheritance solely for compile-time configuration. You define classes, steps, tracks and flows, inherit those, customize them using Rubyâs built-in mechanics, but this all happens at compile-time. At runtime, no structures are changed anymore, your code is executed dynamically but only the ctx
(formerly options
) and its objects are mutated. This massively improves the code quality and with it, the runtime stability, which Ruby is famous for *cough.
I extracted a lot of smaller libraries from the original trailblazer
gem. Most gems use the activity
gem itself, which is the core of an operation and provides the runtime object that executes your steps in a certain order. Itâs so simple that I sometimes wonder why it took years to develop it! But all those little interfaces, structures and principles that you see as âcommon senseâ, they took a while to figure out.
A lot of work went into the dsl-linear
gem that provides a DSL to create activities (or operations) using your old friend #step
. Itâs not even a DSL since it doesnât create code at run-time, it doesnât even create code: it creates runtime objects, activities, that are executed when your operation is run. You could create those objects manually without the DSL, or even write your own DSL, which weâre hoping for!
So, whenever you hear the medieval argument âTrailblazer is just a nasty DSL!â, forgive your opponent, you now know better. The entire framework is based on small, clean Ruby structures that can be executed programmatically.
Also, the more I use Trailblazer in projects or even in Trailblazer itself, I feel how needed those new abstractions are. Yes, you can write everything with your own code, you donât need abstractions for flow control and automatic error handling, which makes me wonder why youâre not programming in assembler since Ruby is also an âunnecessary abstractionâ on top of a processor. We need abstractions, unless you want to program like we did 30 years ago.
Ok, enough of this - letâs jump into the things that actually help you.
Loader / Startup Speed
Startup speed and loading of operations has been a never-ending problem.
To make it short: we returned to the Rails Wayâ˘, lowering our heads in shame, and adhere to the Rails file and class naming structure for operations. What used to be named Memo::Create
is now Memo::Operation::Create
, which reflects its file location concepts/memo/operation/create.rb
(still the same!) and works with Railsâ autoloading.
The big benefit is: we could remove the trailblazer-loader
gem. The alternative loader was a naive attempt by me to combine âourâ naming style with the Rails file structure. After endless fire-fighting against broken development reloading, inconsistencies, slow startup times in develpment mode, more incompatibilities with Railsâ autoloader and some clarifying discussions with the great Xavier Noria himself (author of Rails loading and zeitwerk), we ditched our loading mechanism.
If you want to take advantage of the fast server startup times and (almost) seamless hot reloading of classes, you need to rename your operations.
This change is not super dramatic at all, given that in the future, you wonât be calling operations yourself all too much anymore. With the upcoming workflow
and endpoint
gems, those abstractions will invoke the business logic for you, so donât worry too much.
New website
In case you havenât noticed: you are reading this release blog post on a newly designed website! Not only does it come with the most beautiful, romantic and inspiring hero backdrop ever (featuring a James-Cameron-worthy parallax effect, too!), it also attempts to provide all the old documentation while bringing you the hottest off-the-press docs for our new gems.
We try to keep the âinformation architectureâ - a word I wouldnât have learned without the inspiring Alex Coles - as simple as possible: so far, we got a handful of pages accessible through the top navigation, and then the documentation behind the DOCS link. Here, the right sidebar helps you to navigate within the chapter.
After all, weâre a nerdy developerâs library and donât need no million hipster landing pages! One is more than enough and took months of collaboration between us, the design and web team in Argentina and our awesome comic artist Josh Bauman who also drew the epic Trailblazer book illustrations in 2016.
The documentation has been designed to make coherent information as quickly accessible as possible. We even added a search feature - something our beloved user base has asked for many times. Just hit the /
key (or Shift+7
) in the docs.
API CHANGES
The new 2.1 version comes with a few necessary but reasonable changes in method signatures. As painful as that might sound to your Rails-spoiled ears, we preferred to fix design mistakes now before dragging them on forever.
In versions before 2.1, the automatic merging of the params
part and the additional options was confusing many new users and an unnecessary step.
# old style
result = Memo::Create.(params, "current_user" => current_user)
The first argument (params
) was merged into the second argument using the key "params"
. You now pass one hash to call
and hence do the merging yourself.
# new style
result = Memo::Operation::Create.(params: params, current_user: current_use )
Your steps use the existing API, and everything here is as it used to be before.
class Memo::Operation::Create < Trailblazer::Operation
step :create_model
def create_model(options, params:, **)
# ..
end
end
The new call
API is much more consistent and takes away another thing we kept explaining to new users - an indicator for a flawed API. Believe it or not, but not a single time has this topic come up again on our support forum.
The ctx
(or options
) object now behaves like a HashWithIndifferentAccess
, a pattern used for Railsâ params
object. In other words, you can now access ctx[:current_user]
instead of ctx["current_user"]
and conveniently use the keyword argument version of it, too.
Another handy addition to ctx
is aliasing. Before 2.1, you had to refer to, say, the main contract via ctx["contract.default"]
. While I designed that name âback in the daysâ with multiple contracts per operation in mind, it turned out to be quite clumsy since you couldnât access it as a keyword argument.
Youâre able to set aliases now for the ctx
object, allowing magical things to happen, amongst those an aliasing from "contract.default"
to :contract
.
class Memo::Operation::Create < Trailblazer::Operation
step :inspect_contract
def inspect_contract(ctx, contract:, **)
puts contract #=> same as ctx["contract.default"]
end
The ctx
object is now documented in its own section.
Developer Experience
Debugging deeply nested operations before 2.1 sucked. Even if there was no nesting applied, an exception thrown from a random step was hard to find, the native Ruby stacktrace didnât really help, the TRB and Rails framework noise made it a painful experience to spot the source of an exception.
This was one of the major arguments in many companies against Trailblazer, besides it âbeing too complexâ while it seemed much easier to program pure Ruby as we did in the 90s. And we liked it that way!
To tell you the truth, the new tracing feature was the original reason why I decided to write 2.1 and make you sit and wait in agony for years. Nevertheless, tracing is simply blowing my mind. I canât count how many hours and angering rushs of adrenaline Iâve saved since the introduction of the wtf?
method and its helpful higher-level stack trace.
Not only does tracing show you the path the execution took in a successful invokation, it also shows you where an exception happenend. And not âin Rubyâ, itâs âin Trailblazerâ, in the actual step where things broke.
With the new developer
gem, you can trace executions, find exceptions, observe certain variables in the ctx
throughout the flow, render operations and activities to visually understand whatâs going on, and much more.
As this new gem turns out to be a massive time saver for everyone, yes, even for us, weâre focusing on extending and adding functions such as visualizing nested activities.
It might sound a bit overly sentimental or even cheesy, but I personally think that this is lifting the dev experience to a new level that will be hard to come by in pure Ruby. Yes, Trailblazer is adding new abstractions and concepts and they are different to the 90s-Ruby, but now, at the latest, it becomes obvious how this improves the developing process. Weâre no longer talking in two-dimensional method stack traces or byebug hoops, the language and conception is changing to the actual higher level code flow, to activities sitting in activities structured into smaller step
units.
Trailblazer::Developer.wtf?(Memo::Operation::Create, [ctx])
Once youâve used the wtf?
method, you will feel it!
Unlimited Wiring
Over the past years, harnessing the flow control and error handling mechanics we added in 2.0, one thing became obvious pretty quickly: users want more wiring possibilities. The linear railway and âfast trackâ concepts have been a helpful tool for structuring code flow, but people need to be able to model arbitrary flows.
Needless to say that 2.1 provides you just that. We added the concepts of outputs to steps that can be added or âre-wiredâ using the Wiring API.
You may add additional outputs, rewire, go back, error-out in additional termini, go nuts, and all that using a super simple DSL.
class Execute < Trailblazer::Activity::Railway
step :model, Output(:failure) => End(:not_found)
step :validate
step :save, Output(:retry) => Id(:model)
end
This works for both Activity
subclasses and operations (which are just special-flavored Activity::FastTrack
s).
While youâre still wondering whether weâve completely lost it, we already use those new wiring mechanics to build everything from simple edge cases in code flows to long-running, complex business workflows.
A major improvement here is the ability to maintain more than two explicit termini. In 2.0, you had the success
and the failure
termini (or âendsâ as we used to call them). Now, additional ends such as not_found
can be leveraged to communicate a non-binary outcome of your activity or operation.
Using a terminus to indicate a certain outcome - in turn - allows for much stronger interfaces across nested activities and less guessing! For example, in the new endpoint
gem, the not_found
terminus is then wired to a special â404 trackâ that handles the case of âmodel not foundâ. The beautiful thing here is: there is no guessing by inspecting ctx[:model]
or the like - the not_found
end has only one meaning!
Check the endpoint
gem if you want to see that in action.
Core team
In the past 1 ½ years something weird happened: a real core team formed around the Trailblazer gems. I say ârealâ because in the past 15 years of OSS, Iâve had people come and go, being of great help but never staying and taking over long-term responsibilities - which I found to be the pivotal element of a core team.
Eventually, those kids convinced me to start the Trailblazer organization on Github and move over all âapotonick gemsâ. Over the course of time, I saw myself giving away that aforementioned responsibility with a smile on my face, adding owners and collaborators to gems, yes, even giving away entire gems, letting people work on documentation and just trusting someone and their skills.
I have no words to describe how good that feels!
For me, a dream has come true. I work with crazy geniuses who share many of my opinions (not all, and thatâs good). I learned to âlet goâ and simply trust others to maintain certain gem suites. Messages like âdonât worry, Iâll do itâ combined with a pull requests minutes later - things I literally dreamed of a few years ago, are now part of my daily routine.
Here are those kids, in the order of appearance: <3
- Celso Fernandes whoâs been part of TRB since the 12ths commit or something. Heâs currently focusing on the business side, consulting, and is the magical sysop of our infrastructure.
- Emanuele Magliozzi worked for many years as a TRB consultant and is now the master of
reform
and its 2.x line and interrupting our important daily work with Aussie bantering. - Abdelkader Boudih whoâs still unaware that I secretly call him âThe Machineâ since Iâm not sure heâs an AI bot trained to fulfill basically any request an OSS author might have.
- Pedro Visintin, besides keeping me busy talking about punk rock guitars, is the person behind the PRO editor and our sharply-dressed agile marketing director.
- Kamil Milewski has been contributing to TRB for years both intellectually and with many code additions and even more deletions.
- Krzysztof Piotrowski has been tirelessly helping with writing example applications and turned out to be the refactoring expert on board.
- Adam Piotrowski is resisting my deceiving calls to work on the code side and rather manages our processes - so we gladly let him do just this!
- Yogesh Khater is now working full-time for Trailblazer, working on all 48 repositories at the same time (plus our private ones!), releasing gems when I sleep, leaving me jobless and peacefully napping in the sun.
In addition to the organically formed core team, I started Trailblazer GmbH 4 years ago with my relocation from Australia back to Europe. One of our consulting clients is the central police department of a German state that has kept me busy for more than three years now. And yes, at TRB GmbH, we do pay people to work on OSS!
License and PRO
Around 2 years ago I decided to end the experiment of âTRB PROâ as I felt I didnât provide enough value to paying users. In the end, we had around 150 companies and individuals signed up, which was epic and a great funding source for more development.
Weâre now relaunching PRO, but instead of a paid chat and (never existing) paid documentation, your team gets access to paid gems, our visual editor for workflows, and a commercial license.
The editor (that mostly plays along with the workflow
gem) is already in use for a bunch of commercial projects that we consult, but will be launched in its stable version somewhere this year.
The core gems of Trailblazer have been re-licensed to LGPLv3 to raise awareness for this. For 99% of all our users, this has no legal implication at all - go use TRB for free if you donât feel like paying money for OSS.
With all this âmonetizationâ happening around Trailblazer, we will also make sure that all free and paid parts of the project grow adult and maintan an LTS - or long-term support - status. Those are good news to all you users out there having been scared to use gems of this project, not knowing whether or not theyâre being maintained, breaking code in the future or making your developers addicted to and then cutting off the supply chain. Trailblazer 2.1 onwards is LTS, and the last 1 ½ years of collaboration have proven that.
Upcoming gems and plans
We use Trello boards now to organize our work. Yeah, you can teach old dogs new tricks! Not only does it help to structure myself, it also manifests how many ideas and improvements we got lined up for the near future.
- COMPILE-TIME PERFORMANCE While weâre pretty good on the runtime performance, compiling all those activities takes more time than it could. We already started improving the compilation process and will ship updates in the next months. Optimization in this case is nothing crazy, just something I neglected while designing the framework.
- TRAILBLAZER-TEST The official stable release is only weeks away bringing you a bunch of new assertions that drastically reduce coding effort for tests! Of course, Minitest and RSpec will both be supported.
- TRAILBLAZER-STORY will follow as it turned out to be inevitable for setting up application state for tests. Instead of fumbling around with factories and traits in your tests, you âtell a storyâ about what to create in which order, easily customizable, and all written using activities. Currently, Iâm working on designing the interfaces and itâs real fun!
- TRAILBLAZER-WORKFLOW is another dream âo mine come true. It allows creating long-term processes (or state machines) based on BPMN diagrams that can be modeled using our editor. Iâm pretty confident that you, coming from AASM or your own state machine engine, might find this approach highly interesting. Weâre using this gem in several commercial apps already, so the first release should be out this summer (I didnât say which hemisphere).
- TRAILBLAZER-ENDPOINT Iâve been promising for many years and it turned out I couldnât have fully designed it without the tools we do have now. Endpoint is the missing link between your routing (Rails, Hanami, âŚ) and the âoperationâ to be called. It provides standard behavior for all cases 404, 401, 403, etc and lets you hook in your own logic like Devise or Tyrant authentication, again, using TRB activity mechanics.
- TYRANT is an authentication framework fully written in TRB activities, interacting with Rails, workflows and your database. Itâs highly customizable using the TRB mechanics and definitely less rough to work with than Devise. Looking forward to the release here!
[A leaked snippet of the endpoint architectural design draft document, highly confidential.]
Documentation
Writing documentation for the new website has been fun. Yes, fun! Mainly because we have a helpful little framework behind the page rendering that pulls code snippets from real tests out of the actual gems!
Many users have asked whether there will be an updated version of the Trailblazer book. While itâs really cute and humbling to still have around $10 per month income from the now free book, keep in mind it was published end of June 2016, thatâs pretty much four years ago!
What this means is: I better refrain from writing a new book and we rather focus on more and better docs. Writing examples and technical docs as a team has been working out pretty well recently and we will add more Trailblazer tutorials in the next half year. A comprehensive workflow tutorial demonstrating long-running processes and all things Trailblazer is also in the works! The legendary cfp-app
will become a Rails-to-TRB refactoring tutorial.
We decided against paid documentation, so all will be freely available on our shiny new website.
Staying up-to-date
All new publications, changes, ideas, conferences and parties will be posted regularly on our Facebook page. Yepp, thatâs true. We found it much easier to maintain and follow than most other social media platforms.
And if you have anything to talk about with us, use the new Zulip chat. Weâre looking forward meeting you! See you on the trail!
Nick Sutterer, @apotonick
Subscribe to our newsletter!
We send you news about Trailblazer's components, posts, events and rants straight to your inbox, every 2-3 months.