Relative Net Lift

 

Relative Net Lift is an innovative measure of the percentage of the possible lift achieved by a model. (Lift is a generic term used to measure how well a model ranks the records in an analysis in terms of the target variable.)

 

Relative Net Lift is calculated in the Compare Analyses tool. The value in the summary statistics section uses Test Data if available (and Training Data if it is not) and either the Modeled or Modified Target depending on whether or not the Include Curve Modification box is checked (if it is checked, Modified Target is used, if it is not, Modeled Target is used). When two analyses are clicked to compare, the Relative Net Lift is calculated using both training and test data as well as with and without the curve modification in the detailed comparison in the bottom left of the window.

 

The calculation is performed as follows:

 

The data used for the calculation is the Exposure, Target, and either Modeled or Modified Target for each record and can be found in either the Training Detail Table or the Test Detail Table (these tables can be found in SQL or exported as csv files from the File menu) depending on whether the calculation is being performed on Training Data or Test Data respectively. This data is then sorted three different ways to produce three different curves: the highest possible lift (High), the base amount of lift (Low), and the actual model lift (Mod). Relative Net Lift then measures the percentage of the possible lift that was captured by the model. These curves, as well as a line assuming the average amount of Target in each record (Mean), can be seen on a graph of cumulative Target below:

 

 

 

The sorting criteria and description of each curve are as follows:

 

Curve

Sorting Criteria

Description

Mean

Not Sorted

This curve is calculated simply by taking the total Target divided by the number of records and applying this average Target amount to each record, resulting in a straight line.

High

Primary:

       Target (descending)

Secondary:

       Exposure (descending)

This curve represents the best the model could do (with regard to lift), or 100% Relative Net Lift, since sorting by Target ranks the records perfectly in terms of Target by definition.

Low

Primary:

       Exposure (descending)

Secondary:

       Target (ascending)

This curve represents the baseline lift for the model, or 0% Relative Net Lift. Since Exposure is expected to have a strong relationship with the Target variable, simply sorting by Exposure will likely give more lift than just using the mean Target. A model is expected to improve on this basic expectation, and so this is considered the baseline. In cases where no Exposure field was selected, the Low curve is simply the Mean curve.

Mod

 

Primary:

       Modeled/Modified

           Target (descending)

Secondary:

       Exposure (ascending)

If curve modification is being used, Modified Target is used, if it is not, Modeled Target is used. This curve represents the amount of lift created by the model being used.

 

 

Next, the lift value for each curve, defined as L, is calculated for each of the High, Low, and Mod curves (LHigh, LLow, and LMod). This can be intuitively thought of as the net area under the curve and above the Mean line. More precisely, it is the sum of the distances between the the curve and the Mean line for each curve multiplied by the exposure of each individual record, as seen in the formula below. In cases where no exposure is provided (so 1 is used as the exposure for every record), L will exactly equal the area.

 

 

It is important to note that in cases where the Cumulative Target drops below Cumulative Average Target (the Mean line), the summand becomes negative and counts against the lift. In the case where no exposure is provided (so 1 is used as the exposure for every record), L is equal to the net area under the curve and above the Mean curve. On the graph below, L = - A + B - C.

 

 

 

Whenever LLow is negative, it is automatically set to zero. This can occur when the chosen Exposure field is in actuality not predictive of the Target field. In such cases, rather than suggest that a model is good based on comparison to a bad starting point, the model is compared to the result as if no Exposure field were used.

 

Once these A values have been calculated, they are plugged into the formula below to calculate Relative Net Lift:

 

 

In some cases the value of LMod may be less than LLow. This reflects a model that is achieving less lift than the baseline, which will result in a negative Relative Net Lift.