In this post, we continue describing the CRISP-DM (Cross Industry Standard Process for Data Mining) methodology, after our previous post Phase IV: Modeling. In this case, we discuss the fifth phase of the data analysis project, known as Evaluation.

It is an extracted version of:

Chapman, Pete (NCR); Clinton, Julian (SPSS); Kerber, Randy (NCR); Khabaza, Thomas (SPSS); Reinartz, Thomas (DaimlerChrysler); Shearer, Colin (SPSS); Wirth, Rüdiger (DaimlerChrysler). Step-by-step data mining guide. 2000. 

DataPrix. Metodología CRISP-DM para minería de datos. 2007.

CRISP-DM Fase V. Evaluation.

Figure below outlines the different tasks that need to be carried out in this phase.

Evaluate results

This task assesses the degree to which the model meets the business objectives, and determines if there is any business decision that the model doesn’t cover. An option is to test the model on test applications. Evaluation also verifies other generated data mining results.

The objective is to summarize assessment results in terms of business success criteria, including a final statement related to whether the project already meets the initial business objectives.

Once the evaluation of the model is done with respect to business success criteria, generated models that cover selected criteria are approved.

Checklist below represents the tasks to be done in this task:

  • Understand the data mining results.
  • Interpret the results in terms of the application.
  • Check effect on data mining goals.
  • Evaluate data mining results against the given knowledge base to check if the discovered information is new and useful.
  • Assess and estimate results with respect to business success criteria, and determine if the project has achieved the original business objectives.
  • Compare evaluation results and interpretation.
  • Classify results according to business success criteria.
  • Check effect of results on initial application goal.
  • Determine if there are new business objectives to be addressed later in the project, or in new projects.
  • State recommendations for future data mining projects.

Review process

At this point, it is appropriate to make a more thorough review of the data mining engagement in order to determine if any important factor is missing. It would be useful to summarize the process review, highlighting the activities that have been missed or should be repeated.

Checklist below represents the tasks to be done in this task:

  • Provide an overview of the data mining process used.
  • Analyze data mining process. For each stage of the process, ask:
    • Was it necessary?
    • Was it executed optimally?
    • How could it be improved?
  • Identify failures.
  • Identify deviations.
  • Identify possible alternative actions and unexpected paths in the process.
  • Review data mining results with respect to business success criteria.

Determine next steps

The project team decides how to advance in the process, whether to finish the project or initiate further iterations. This task involves the analysis of remaining results and budget. At the end of this task, it would be useful to list possible further actions along with the reasons for and against each option, as well as the decisions made regarding on how to proceed and reasons.

Checklist below represents the tasks to be done in this task:

  • Analyze the potential for deployment of each result.
  • Estimate potential for improvement of current process.
  • Check remaining resources to determine if they allow additional process iterations or whether additional resources can be made available.
  • Recommend alternatives to continue.
  • Refine process plan.
  • Classify possible actions, select one of them, and document reasons for the choice.

This is the last tasks in Phase V, Evaluation, that focuses on results and evaluation. The next post will describe the last phase of CRISP-DM methodology, which is phase VI, known as Deployment.

Our professional team can effectively address Data Analytics projects in any complex scenario with the maximum guarantees of success applying CRISP-DM methodology. If you would like more information about this area, please do not hesitate to contact us. We will be glad to help you.

[translated by Marta Villegas González]