Did a close read of @jeremyphoward et al's Drivetrain Approach
Define goal → Consider levers → Consider data → Implement assembly line
Most important is having actions that fulfill the objective
Predictions are more data, not actions
Key takeaways 🧵
Define goal → Consider levers → Consider data → Implement assembly line
Most important is having actions that fulfill the objective
Predictions are more data, not actions
Key takeaways 🧵
Jun 23, 2026 · 8:32 AM UTC
2
3
25
28,474
Clearly define an objective; not a metric
The objective is a change you want to see in the world
The objective is a change you want to see in the world
1
0
1
641
For a search engine, "return relevant search results is a poor objective". "Complete the user's objective" is a better one
"Recommend books the user would like" is poor. "Recommend books that the user wouldn't have bought if not recommended" is better.
"Recommend books the user would like" is poor. "Recommend books that the user wouldn't have bought if not recommended" is better.
1
0
0
569
A metric is a way to measure the change you've made. An objective can have multiple metrics, but a metric cannot take the form of an objective. Don't mix up a metric for an objective.
Don't mix "liking" for a "purchase". Don't mix "purchase" for "increased sales"
Don't mix "liking" for a "purchase". Don't mix "purchase" for "increased sales"
1
0
0
404
An objective is not "implement the thing" or "train the model". An objective is not an action either.
An objective is the outcome/change you want.
An objective is the outcome/change you want.
1
0
1
329
Next up is to consider your levers.
What things can you change? What things can't you change?
Does your lever has sublevers?
If you flip a lever, will the objective change?
What things can you change? What things can't you change?
Does your lever has sublevers?
If you flip a lever, will the objective change?
1
0
0
285
Next is to consider your data.
Can loop between steps 2 and 3.
There's a reason why you wan to confirm your data before looking at the model.
Can loop between steps 2 and 3.
There's a reason why you wan to confirm your data before looking at the model.
1
0
0
235
Consider the reverse. You pick a fancy architecture, you hunt for data in for that model, and then tune the model until the metric looks great.
You've optimized for a metric rather than the objective.
Checking data before hand is, "Can my existing data answer my objective?"
You've optimized for a metric rather than the objective.
Checking data before hand is, "Can my existing data answer my objective?"
1
0
0
213
Then, you construct the model assembly line.
- Construct your models
- Assemble the models together and simulate actual actions
- From the action outcomes, determine the optimal position all levers should be at
- Construct your models
- Assemble the models together and simulate actual actions
- From the action outcomes, determine the optimal position all levers should be at
1
0
0
190
- The modeler will tell you how the world will work
- The simulator will tell you if you take a certain action, how the world will change with all models together
- The simulator will tell you if you take a certain action, how the world will change with all models together
1
0
0
178
- The optimizer does not replace your own judgement
- The predictions are not the end point; an actionable outcome is
- The same lever can be modeled differently under different conditions
- The predictions are not the end point; an actionable outcome is
- The same lever can be modeled differently under different conditions
1
0
0
179
Succinctly
- Ask how you want to change the world
- List out all possible actions
- List out all your data
- Model, simulate, and determine the optimal solution
Or,
starting from the goal, use a model assembly line to turn predictions into the most optimal solution
- Ask how you want to change the world
- List out all possible actions
- List out all your data
- Model, simulate, and determine the optimal solution
Or,
starting from the goal, use a model assembly line to turn predictions into the most optimal solution
1
0
0
181
Created a Polya x Drivetrain approach, for a more structured way to help me ask myself the right questions
share.solveit.pub/d/1801b478f778…
share.solveit.pub/d/1801b478f778…
0
0
1
167
@ForBo7_s@jeremyphowarding is underrated, most ML projects fail because they optimize the wrong objective. The Drivetrain Approach forces you to define what lever you're pulling before touching a single model. More on building ML systems that actually drive business outcomes: datacamp.com/blog/the-many-…
0
0
2
203