Are you curious about how long it takes for MadKudu to score your records? Look no further than the Processes page! If you're interested in learning more about the different steps of the batch data pipeline and scoring process, this article is for you.

Prerequisites

• Page only available to Admin and Architect users

The Processes page tool (Settings > Processes) helps you understand when MadKudu is processing and scoring your data, and helps you see if MadKudu encountered any issue between the pull of your data, all the way to the sync (push) of the scores to your destinations in batch.

When was the last time MadKudu scored my records in batch?

1. Log into app.madkudu.com 

2. Go to Settings then Processes

3. Read the line with step "sync": the last success of the sync process indicates when your records were last scored by the batch update.

When is the next batch update?

MadKudu processes your data in a specific order to send back scores to your integration. 

The data pipeline is executed continuously and usually runs end to end within 2h to 12h, or more depending on

• the volume of data (leads, contacts, accounts, events) from your integrations to process

• the number of models and the complexity of customizations to process. 

What are the steps within the process?

Steps

• merge: Madkudu records the data pulled from your integrations into its own Database’s tables.

• map: Your data is abstracted into a format Madkudu can ingest. See mapping section for more details.

• augment: Your mapped data is then automatically enriched (ex: adding datapoints like number of employees from Clearbit to the record).

• compute: In this step, all computations are calculated. Computations are the building blocks of Customer Fit scoring & Copilot explore columns. Learn more here.

• aggregations: In this step, aggregations are calculated to feed your behavioral model.

• analysis: All scores & scoring signals are actually computed in this step.

• sync: This step pushes the data to your integration.

How to read the Status column

There are different reasons why a process may be failing or delayed (pipeline congested, server unavailable, connection interrupted ...etc). The batch process will retry continuously to process the steps.  

Status

The different statuses can be 

• idle: not running at the moment, waiting for its turn to process

• processing: currently being executed 

• queued: process in stand-by 

• failed: an error occurred, the process was not fully executed

• processing (last error: ): last try failed, the process is being executed at the moment

Process error codes

• worker shutdown: the server running the process on MadKudu side was momentarily down

  • Resolution: The process will retry by itself. 

• 1023 or 1018: several queries processing your data are trying to run at the same time and competing for the same resources

  • Resolution: The process will retry by itself. 

• 1040: the database where your data is processed is full

  • Resolution: The process will retry by itself. 

• Query cancelled on user's request: query processing your data timed out. The process will retry by itself.

  • Resolution: The process will retry by itself. 

• deadlock detected: there was an interaction with other active transactions that prevented the transaction from being completed.

  • Resolution: The process will retry by itself. 

• Invalid exponent, Value 'e', Pos 5, Type: Decimal: a SQL query is failing when executing your data processes, likely a syntax error in the definition of a Computation

  • Resolution: Please review the type and definition of the computations deployed. For instance, if your new computation type is number, make sure the fields it's based on are also exclusively numbers.

• Could not open relation with OID: query processing your data timed out. The process will retry by itself.

  • Resolution: The process will retry by itself. 

For other errors, please refer to the error message you receive for more information.