Analysis of the KDD Cup 2009: Fast
Scoring on a Large Orange Customer Database
Isabelle Guyon, Vincent Lemaire, Marc Boullé,
Gideon Dror and David Vogel ; JMLR W & CP 7:1-22,
2009.
Abstract
We organized the KDD cup 2009 around a marketing problem with
the goal of identifying data mining techniques capable of
rapidly building predictive models and scoring new entries on a
large database. Customer Relationship Management (CRM) is a key
element of modern marketing strategies. The KDD Cup 2009
offered the opportunity to work on large marketing databases
from the French Telecom company Orange to predict the
propensity of customers to switch provider (churn), buy new
products or services (appetency), or buy upgrades or add-ons
proposed to them to make the sale more profitable (up-selling).
The challenge started on March 10, 2009 and ended on May 11,
2009. This challenge attracted over 450 participants from 46
countries. We attribute the popularity of the challenge to
several factors: (1) A generic problem relevant to the Industry
(a classification problem), but presenting a number of
scientific and technical challenges of real practical interest
(many missing values, large number of features - 15000 - and
large number of training examples - 50000, unbalanced class
proportions - fewer than 10% of the examples of the positive
class), noisy data, many missing values, presence of
categorical variables with many different values. (2) Prizes -
Orange offers 10000 Euros in prizes. (3) A well designed
protocol and web site - we benefitted from past experience. (4)
An effective advertising campaign using mailings and a
teleconference to answer potential participants questions. The
results of the challenge were discussed at the KDD conference
(June 28, 2009). The principal conclusions are that ensemble
methods are very effective and that ensemble of decision trees
offer off-the-shelf solutions to problems with large numbers of
samples and attributes, mixed types of variables, and lots of
missing values. The data and the platform of the challenge
remain available for research and educational purposes at
http://www.kddcup-orange.com/.
