Success Stories

Meet MESA alumnus Scott Frasard

Global Manager of Training Measurement and Evaluation at eBay, Inc.

The Challenge:

Scott Frasard, global manager of training measurement and evaluation at eBay, Inc., wanted to help his company become more data-driven in its training practices. In particular, Scott wanted to become more data-driven in designing eBay training and in delivering it, both in-person and online.

The Methodology and Solution:

Scott used proper survey design principles from his UIC MESA courses to:

  • Create a measurement framework for “effective training”
  • Generate potential survey questions to measure constructs
  • Narrow the list of survey questions to a manageable number to prototype test
  • Conduct various statistical analyses to identify the best questions to ask

He created four constructs within his framework:

  1. Instructional design
  2. Trainer/learner interaction
  3. Perceived learning outcomes
  4. Context

And then created three metrics:

  • Design score
  • Interaction score
  • Outcomes score

To supplement his quantitative data and generate additional qualitative insights, Scott included in his survey open-ended questions that evaluated the same four constructs.

Scott designed all survey questions in a format that would help eBay take specific actions in low-scoring areas. He also weighted all questions based upon their contribution to an overall outcome, in order to prioritize the specific actions to be taken.

Results:

Scott’s new post-training survey yielded better, more useful, and more reliable results than previous eBay surveys. Scott and his team then replicated the survey development process in several global regions, yielding different yet similar surveys, with metrics comparable to the U.S. survey.

The U.S. team also calculated “normal” score ranges for each region, allowing them to:

  • Numerically define what they would expect each region’s scores to be
  • Identify excellent and substandard performances without being influenced by factors outside of a region

For the first time ever, eBay was able to compare training survey results across the globe, with confidence that the scores were indeed comparable and shared a common meaning.

A Word from Scott:

“I greatly enjoyed my UIC MESA studies. I liked the challenges and the collaboration, and I learned a great deal that was, literally, directly applicable to my work.

“I’ve advanced my career, helped my company, and positioned myself for even greater future success. I highly recommend the UIC MESA program for anyone interested in leading the charge in evidence-based decision making.”