For anyone who knows me and the work I do professionally, knows that I love data and analytics. In fact I love all things mathematics, data, and optimization/ strategy focused. One of the responsibilities that I have looked after since day one of working at Monumental Sports & Entertainment, has been the oversight of our consumer insights initiatives (surveys, focus groups, other empirical research). Although this is only one of the several major responsibilities that I look after, I have always wanted to implement a new model as to how to survey more efficiently, but even more important, accurately.
Everyone has taken a survey of some sort, I’m sure of it. Do you know what one of the major problems with surveys are? Although things like unrepresentative samples, measurement error, sampling error, and survey bias are all major pitfalls to avoid, it’s not what comes to my mind first. To me, it’s the fact that many surveys are snapshots of sentiments at a single point in time. For many institutions this might be okay. But in an industry like mine, or any industry in which seasonality plays a major factor, this is inherently problematic.
Every year my organization asks me to put together our annual brand study. Simply explained, our brand study is the big gigantic survey that goes out to a large portion of the contacts in our database asking a multitude of questions across a wide variety of topics. We try to be smart with it – we send it before the season begins so that any hot or cold start that the team has isn’t included, and whatever heartbreak the season before ended with is in the rearview mirror. However, if we’re being totally honest with ourselves, this isn’t an ideal situation to be in. Surveying on major topics that we then use the data from to implement new strategies. The results generally aren’t ready until we’re already in season a few months later and at that point, implementing anything new is nearly impossible. Then the season ends and the data we collected is outdated.
Another example. We surveys our season tickets holders this past year on their likelihood to renew their tickets at the end of the season – a pretty typical survey that all teams send out in some way shape or form. The only problem is we sent the survey the week before our team went on a 18-0 home win streak. The week prior to that 18-0 home win streak… team was one of the worst in the league. As you can imagine, the perception of the team, and thus the data we collected would have been totally different had we waited six weeks to send it out. That being said, it wouldn’t have been good timing to send it six weeks later since feeling and emotions about the team would have been overly inflated at the time – also problematic.
What if however, we created a survey method that was able to combat this? Recently I have been able to start the implementation of an idea I had several years ago – high frequency surveying across multiple dimensions and topics. The idea here is to create a view of consumer data over time, rather than having data snapshots of a single moment in time. The ability to report on data throughout a season would help standardize data points and offer valuable insights into how seasonality influences different data.
The beauty of all this is, of course, the ability to understand how certain times of the year (or in our case the season) affect feedback and scores. As it relates to sports, much of the data may still highly depend on how the team is performing at any given moment (individuals tend to give higher scores across dimensions when the team is doing well and lower when the team is performing poorly in the standings). Because of this, data living in the peaks and valleys of trends is often hard to act on. However, tracking these peaks and valleys and how attitudes move over time will allow us to better understand where the underlying data and tendencies actually lives, thus allowing us to more accurately identify the levers we can pull to enact effective and meaningful change.
So what do we need to do to do it? For one, a great deal of preparation and organization is required. One would have to determine and build out survey topics in advance. You’d need these questions and topics to be standardized in order to consistently collect and analyze data. You’ll also need to sort out your segmentation practices far in advance. However, these are all things that are possible with enough preparation and time spent.
What do you think? What am I missing?
-Alex