Monitoring Your Model Performance

  • Published on Nov 26, 2024
  • 4 minute(s) read

Overview

The Model Performance page contains information regarding the performance of your MadKudu customer fit model, and the impact of it on your revenue and conversions. This is also a hub to gather insights on your conversion rates, average deal size, and lead value by segment. This guide will walk you through the modules that are included in the page and shed some light on examples of insights you may gather.

Notes:

  • We will be including charts from a fake business along with some examples of insights that we can gather from these charts.

  • The conversion charts below are based on conversions that happened in the past 6 months only from leads created in the past 6 months.

  • 'SQO' is based on a custom definition that MadKudu's models are optimized on. Ask your Customer Success Manager for the definition configured for your business.

Guide

Model Performance

How to Interpret This:

This section aims to showcase how many conversions over the past 6 months came from leads that were scored as 'Very Good' or 'Good' by MadKudu. Note that having a 100% model performance is almost impossible!

How MadKudu Computes This: The model performance recall is calculated by taking the sum of the ‘Very Good’ and ‘Good’ conversions and dividing that by all the conversions in the past 6 months (uncohorted).

MadKudu Fit Quality Distribution

How to Interpret This:

This section aims to showcase the model’s ability to predict a majority of the MadKudu SQOs from a small subset of leads and how that translates to revenue. See mapping>conversion for definition of “MadKudu SQO”.

How MadKudu Computes This:

Leads: what percentage of net new emails captured in the last 6 months were scored very good or good.

MadKudu SQO:

what percentage of MadKudu SQOs created in the last 6 months came from leads scored as very good or good. (see mapping>conversion for definition)

Revenue:

what percentage of the closed-won revenue from opportunities created in the last 6 months came from leads scored as very good or good.

Leads to Revenue

How to Interpret This: Compare conversion rates and deal sizes by MadKudu predicted segment. There is a chance your very good leads might convert at a lower rate than some other segment but at a higher average value. This is meant to showcase how each metric plays a different role in the predictive model.

MadKudu Tip: $5,197,137 (89%) of their revenue comes from these ‘very good’ and ‘good’ leads. Show Sales this chart to educate them on how little value the low and medium leads are actually worth!

How MadKudu Computes This:

Leads: how many net new emails were captured in the last 6 months.

Revenue: how much revenue did those leads bring through closed-won opportunities (see mapping>conversion for amount and closed-won definitions). This does not include recently closed revenue from leads that were created more than 6 months ago.

Conversion Rate

How to Interpret This:

Here you can see the how well each of your segments convert. This is an essential chart to show sales how much better the ‘Very Good’ and ‘Good’ leads convert. One question we can answer with this chart alone is “What’s the conversion rate of my ‘X’ leads?” - but in order to gather more insights we need to look at thecharts alongside it that provide context. We will pair this with ‘Average Deal Size’ and apply some simple math to draw some conclusions.

How MadKudu Computes This:

Conversion Rate is calculated by taking the number of unique closed won conversions that occurred in the past 6 months from leads that were created in the past 6 months and dividing it by the number of leads that were created in the past 6 months. Note that this does not look at any conversions that happened from leads that were created before the 6 months mark.

Average Deal Size

How to Interpret This:

Now with additional context - you can see the average deal size of every segment and how well they convert.

But what if we want to answer “Which segment’s leads are the most valuable?” At first glance it looks like a toss up between ‘Very Good’, ‘Good’, and ‘Medium’.

We need the ‘Predicted Lead Value’.

How MadKudu Computes This:

Average Deal Size is calculated by taking the sum of revenue (closed won conversions in dollar value) and dividing it by the number of closed conversions that occurred in the past 6 months. Note that these conversions were taken from the leads that were created in the past 6 months and do not account for any conversions that occurred from leads that were created before the 6 months mark.

How to Interpret This:

Predicted Value requires both Conversion Rate and Average Deal Size the calculation being (Conversion Rate X Average Deal Size = Predicted Lead Value) this is why the page has the charts going from left to right.

After multiplying the two we can now see your ‘Very Good’ leads are the clear winners, and now have the ability to answer questions like “How much is a ‘Very Good’ lead worth?”

MadKudu Tip:

This is the first step to putting a dollar value on your top-of-funnel leads. What this means is that you can predict how much a lead is worth before they actually convert into a paying customer, and this can help you quantify your marketing pipeline in order to measure success.

How MadKudu Computes This:

Predicted value looks at the value of a lead before they have reached the end of the funnel (aka made a purchase), based on the historical value of similar leads who have made the purchase. Thus, the computation is: average conversion rate * average deal size for each customer fit segment. Average deal size will have to be based on Closed Won deals, since we are computing the value when they actually made a purchase, not if they simply just open an opportunity.