Did a close read of @jeremyphoward et al's Drivetrain Approach
Define goal → Consider levers → Consider data → Implement assembly line
Most important is having actions that fulfill the objective
Predictions are more data, not actions
Key takeaways 🧵
Define goal → Consider levers → Consider data → Implement assembly line
Most important is having actions that fulfill the objective
Predictions are more data, not actions
Key takeaways 🧵
2
3
25
28,482
Clearly define an objective; not a metric
The objective is a change you want to see in the world
The objective is a change you want to see in the world
1
0
1
641
A metric is a way to measure the change you've made. An objective can have multiple metrics, but a metric cannot take the form of an objective. Don't mix up a metric for an objective.
Don't mix "liking" for a "purchase". Don't mix "purchase" for "increased sales"
Don't mix "liking" for a "purchase". Don't mix "purchase" for "increased sales"
Jun 23, 2026 · 8:32 AM UTC
1
0
0
404
An objective is not "implement the thing" or "train the model". An objective is not an action either.
An objective is the outcome/change you want.
An objective is the outcome/change you want.
1
0
1
329
Next up is to consider your levers.
What things can you change? What things can't you change?
Does your lever has sublevers?
If you flip a lever, will the objective change?
What things can you change? What things can't you change?
Does your lever has sublevers?
If you flip a lever, will the objective change?
1
0
0
285
Next is to consider your data.
Can loop between steps 2 and 3.
There's a reason why you wan to confirm your data before looking at the model.
Can loop between steps 2 and 3.
There's a reason why you wan to confirm your data before looking at the model.
1
0
0
235
Consider the reverse. You pick a fancy architecture, you hunt for data in for that model, and then tune the model until the metric looks great.
You've optimized for a metric rather than the objective.
Checking data before hand is, "Can my existing data answer my objective?"
You've optimized for a metric rather than the objective.
Checking data before hand is, "Can my existing data answer my objective?"
1
0
0
213
Then, you construct the model assembly line.
- Construct your models
- Assemble the models together and simulate actual actions
- From the action outcomes, determine the optimal position all levers should be at
- Construct your models
- Assemble the models together and simulate actual actions
- From the action outcomes, determine the optimal position all levers should be at
1
0
0
190
- The modeler will tell you how the world will work
- The simulator will tell you if you take a certain action, how the world will change with all models together
- The simulator will tell you if you take a certain action, how the world will change with all models together
1
0
0
178
- The optimizer does not replace your own judgement
- The predictions are not the end point; an actionable outcome is
- The same lever can be modeled differently under different conditions
- The predictions are not the end point; an actionable outcome is
- The same lever can be modeled differently under different conditions
1
0
0
179
Succinctly
- Ask how you want to change the world
- List out all possible actions
- List out all your data
- Model, simulate, and determine the optimal solution
Or,
starting from the goal, use a model assembly line to turn predictions into the most optimal solution
- Ask how you want to change the world
- List out all possible actions
- List out all your data
- Model, simulate, and determine the optimal solution
Or,
starting from the goal, use a model assembly line to turn predictions into the most optimal solution
1
0
0
181
Created a Polya x Drivetrain approach, for a more structured way to help me ask myself the right questions
share.solveit.pub/d/1801b478f778…
share.solveit.pub/d/1801b478f778…
0
0
1
167