How to Interpret a Decile AnalysisReading time: 3 minutes
After building a predictive model, there are several ways to determine how well the model describes your data. One visual way to get an idea of how well a model fits your data is to examine the decile analysis. Here we’ll look at what the decile analysis represents, how it’s created, and how to spot a good model.
What a Decile Analysis Represents
After building a statistical model, a decile analysis is created to test the model’s ability to predict the intended outcome. Each column in the analysis chart represents a collection of records scored by the model. The height of each column represents the average of those records’ actual behavior.
How the Decile Analysis is Calculated
1. The hold-out or validation sample is scored according to the model being tested.
2. The records are sorted by their predicted scores in descending order and divided into ten equal-sized bins or deciles. The top decile contains 10% of the population most likely to respond and the bottom decile contains 10% of the population least likely to respond, based on the model scores.
3. The deciles and their actual response rates are graphed on the x and y axes, respectively.
After the decile analysis is built, you’ll want to take a look at the height of the bars in relation to one another. Deciding whether a model is worth moving forward with depends on the pattern you see when viewing the decile analysis.
Ideal Situation: The Staircase Effect
When you’re looking at a decile analysis, you want to see a staircase effect; that is, you’ll want the bars to descend in order from left to right, as shown below.
The staircase effect tells you that the model “binned” your constituents correctly from most likely to respond to least likely to respond. A model exhibiting a good staircase decile analysis is one you can consider moving forward with.
In contrast, if the bars seem out of order (as shown below), the analysis tells you that the model is not doing a good job of predicting actual responses.
If the bars seem to be the same height, or the decile analysis looks “flat,” it’s telling you that the model isn’t performing any better than randomly binning people into deciles would. In both cases, you should improve your model before moving forward with it.
Our fully transparent, single-click predictive modeling software allows you to generate predictive models and analyze their integrity with decile analysis, percent concordance, and other accuracy measures. If you’re interested in learning more, click the button below to schedule a personalized walkthrough of our data tools.