Lead Scoring & Machine Learning in Mautic

Download: Master Thesis on Scoring and Machine Learning

What is lead scoring?

Lead scoring is a way of prioritizing a company's contacts so that only time and resources are invested in leads that actually have a high buying potential. It can also ensure that leads are only contacted by the sales team or receive appropriate CTAs when they are ready to buy.

How does lead scoring work?

Traditionally, lead scoring involves experts first developing a points system that determines how many points leads receive if they perform certain actions or exhibit certain characteristics. This points system is created in collaboration between marketing and sales. An example of a points system that is mapped in a scorecard is shown in the following figure.

A score is then calculated for all our leads based on this scorecard. If this score exceeds a threshold value defined by us, a lead is considered qualified and is passed on to sales or used with special marketing measures for qualified leads or marketing qualified leads (MQLs).

Advanced use cases of lead scoring

By default, lead scoring evaluates whether a lead is interested in our company and whether their buyer profile matches our company. However, there are also advanced use cases:

• Account-based scoring: With account-based scoring, it is not individual leads that are scored, but the company that is superior to the leads. This can be useful, as several people are usually involved in the purchasing process in the business-to-business (B2B) sector. As a result, it can happen that none of the company's individual leads exceed the defined score, but the leads together have a very high score, which is a clear indication of the company's interest in buying. If the characteristics of the company still match our ideal buyer profile, it is considered qualified.
• Product-based scoring: Product-based scoring is an extended use case of lead scoring in which the interest of leads in our individual products, product groups or business areas is measured rather than their interest in our company as a whole. A separate scoring system is set up for each product group for this purpose. If a lead exceeds the threshold value of the score for a product group, it can be processed in a more targeted manner by marketing and sales.

Procedure for creating a lead scoring model

Two process models for developing a lead scoring system in Mautic were developed as part of the work. One of them is an optimized traditional lead scoring process model, the other is a predictive process model that uses a machine learning model.

Optimized traditional lead scoring model

The advantage of the optimized traditional lead scoring model is that its scoring model is based on a data analysis. This prevents distorted subjective assessments from negatively influencing the quality of the model. The following steps are carried out for this purpose.

1. Developing a service level agreement: In the first step, marketing and sales set out the objectives and framework conditions of the lead scoring project in what is known as a service level agreement.
2. Data generation: As part of data generation, lead data is generated, which is later used as the basis for data analysis.  In addition to data on the characteristics and actions of the leads, data on the quality of the leads is also collected. For example, an assessment by the sales team (1-10) or whether a lead has bought or not bought.
3. Data preparation: This data is then prepared so that it can be analyzed later.
4. Data analysis: The data analysis step examines how the actions and characteristics of high quality leads differ from those of lower quality leads.
5. Defining a points system: A points system is then defined by marketing and sales based on the findings of the data analysis.
6. Defining a threshold value: A threshold value is then defined. This involves checking for which threshold value the best results are achieved.
7. Iterative adjustment of the system: Finally, it is recommended to iteratively adjust the scoring system and the threshold value in order to further optimize the results of the system.

The system can then be used productively. It is also advisable to measure the success of the model using KPIs and to update it regularly.

Predictive lead scoring model

In addition to the traditional lead scoring model, a predictive lead scoring model was also developed. This involves training a machine learning model that can later automatically assess the quality of our leads. The following steps are carried out for this purpose:

1. Developing a service level agreement: As with traditional scoring, a service level agreement is also defined here.
2. Data generation: The lead data is then generated, which is later used to train a machine learning model. In addition to the action data and the data on the lead characteristics, an assessment of the lead quality is also required here.
3. Data preparation: In the third step, the lead data is prepared so that it is suitable for training a machine learning model.
4. Developing a machine learning model: A machine learning model is now trained using the processed data.
5. Evaluating and adapting the machine learning model: The evaluation phase involves looking for weaknesses in the model and its design process. If any are identified, the model is optimized.
6. Implementation of the machine learning model: In this phase, the model is integrated into the MAS. In addition, all automations are created so that the MQLs predicted by the machine learning model are automatically transferred to the sales team or provided with marketing materials for qualified leads.

 The model can then be used productively. With the predictive model, it is also advisable to measure the success of the model using KPIs and to update it regularly.

The use of predictive lead scoring systems is currently being increasingly recommended, as they enable more accurate predictions of lead quality. This is because predictive models are able to recognize more complex relationships that cannot be mapped in traditional lead scoring systems. For example, a predictive system can recognize actions that always indicate high lead quality if they are carried out together with other actions. With traditional scoring, on the other hand, points can only be awarded for each of these actions individually and therefore no interaction between several actions can be taken into account.

In a practical test with data from an online store, a traditional lead scoring system was therefore able to predict with 77% accuracy which leads would or would not become customers. Predictive lead scoring, on the other hand, was 95 percent accurate.

Insights into lead scoring in Mautic

As part of the master's thesis, all of the previously described approaches and use cases for lead scoring in Mautic were tested. Several suggestions for improving Mautic were developed.

Expansion of the "Points" menu item

In Mautic, the menu item "Points" is provided for lead scoring. This contains the three sub-items:

• Manage Groups: Additional scores can be created here, which is useful, for example, if several scoring systems are created for different product groups
• Manage Actions: The points system can be defined here by assigning points to individual actions.
• Manage Triggers: The threshold values for the individual scores are set here and actions are defined that are carried out when the scores are exceeded.

However, there are inefficiencies in each of these three points that increase the complexity of the lead scoring process:

• Manage Groups: Point groups created under Manage Groups do not have the same functions as new custom fields. This can complicate several processes. For example, in the lead scoring process, it can be useful to recycle leads qualified by the system by decision of the sales team and reintegrate them into the lead scoring process. To do this, the score must be reset to a previously defined value, as otherwise the lead would be directly reclassified as an MQL. As the reset for "Point Groups" cannot be carried out via the "Update Contact" campaign action, a relatively complex campaign had to be created as a workaround.
• Manage Actions: Only actions can be defined here to increase or decrease the score of a lead. However, it is not possible to define that the score of a lead is increased on the basis of its characteristics, such as its position or the size of its company. In addition, some actions, such as unsubscribing from the newsletter, are not included in the actions that can be selected under "Manage Actions". As a result, the scoring of some actions and all properties must be intercepted via campaigns, which increases the complexity of the system.
• Manage Triggers: Under "Manage Triggers", actions can only be executed based on a score when a threshold value is reached. However, if, for example, there is a score based on the activities of the lead and one based on the properties of the lead, it is not possible to specify that actions are only executed when both threshold values are exceeded. It would also be useful if filters could also be taken into account there. This would allow, for example, leads that are in a "Do Not Score" segment to be ignored if a threshold value is exceeded.

Account Based Scoring in Mautic currently not possible

At present, it is not possible to carry out effective account-based scoring in Mautic, as the functionalities for companies are significantly limited compared to leads. It would therefore make sense to bring the functionalities for companies up to a similar level to those for leads, which would also enable account-based scoring.

External tools required for data analysis and the development of a machine learning model

In the context of the two proposed models, it was necessary to evaluate the data externally, for example using Python codes, as part of the data preparation and data analysis or machine learning model development. Therefore, the integration of additional capacities for data evaluation in Mautic could bring considerable added value in terms of lead scoring. It would also make it possible to carry out predictive lead scoring automatically and in real time. Currently, the data has to be exported each time for evaluation, fed into the trained machine learning model and the predictions then imported back into Mautic.

Recommendations for developing your own lead scoring system

If you want to set up your own lead scoring systems, it may be that no data is available at the start that would allow you to create a data-based model. In this case, it is advisable to set up a regular, traditional system first. As soon as sufficient data is available to carry out statistical analyses, you can then switch to a data-driven system such as optimized traditional lead scoring or, ideally, predictive lead scoring. More detailed information on the implementation of the models in practice can be found in the master's thesis.