The challenge of integrating new data and collection technology with historical data

To further strengthen our commitment to providing industry-leading data technology coverage, VentureBeat is pleased to welcome Andrew Brust and Tony Baer as regular contributors. Look for their articles in the Data Pipeline.

Whenever new technologies or systems become too valuable or ubiquitous not to integrate for companies across the industry, there are tenacity that sticks to the old ways or prioritizes the familiar over the innovative. Those organizations usually don’t last long.

But even among adopters, there are companies that try to fuse the old with the new, while others make it happen. We see this fully reflected in sport, where organizations are challenged to integrate legacy data with new collection technologies and datasets. What distinguishes the success stories?

When a sports organization is faced with waves of new data due to advances in automation and data collection methods, a sports organization must first recognize that it is a good problem to have. For example, with technology such as lidar (a laser-based motion tracking system), which aims to improve the accuracy, depth of information and seamlessness of data collection, performance evaluators now have access to a vast, untapped wealth of data that can be used to better inform their decisions. The question then becomes: how does a club deal with this influx of new data?

Preach patience first. Remember that organizations and their data teams have been using the same methods and approaches for years, with the same assumptions and associations. Old habits are hard to unlearn. And because advanced analytics can be applied to everything from game strategy to the optimal types of soda served at stadium concession stands, an organization adopting these technologies for the first time will require broad buy-in. It takes time.

The biggest challenge, however, is integrating an organization’s historical data with modern information. Collection technologies and methods are not the only thing that has changed in this area. Today’s data looks very different from the past, and in some cases the types of measurements do not match previous data sets. How do an organization’s data teams solve this problem? Start here:

Perform translation exercises. Provide a transition period in which a detailed analysis of all data and methods – both modern and historical – is performed. Collect a statistically significant amount of data. Avoid statistical noise or false positives that could result in too small a sample size. You want to get this right the first time. Be aware of prejudice. Certain preferences may arise during the calibration of the system. Identifying and correcting them is important to avoid building bias into your baselines and future calculations. Be aware of differences in data collection methods. Different sports venues use different tracking technologies, some of which have inherent limitations that affect the data collected. Note that some translations may be probabilistic in nature. Measure to a constant: In other words, player X runs at a speed of Y, so the new measurement output must equal Y. Integrating old and new data can be cumbersome. Ensuring that old data sets are not lost while embracing the insights that new data unlocks can be costly and time consuming. But it is important to remember after the exercise that an organization will be better able to make personnel decisions.

The key for sports organizations integrating old and new technologies, methodologies and information is to take a deep, in-depth dive into the data. Raw historical data does not help most clubs. Data needs to be easily understood by new user profiles to make it viable, which takes precious time and can lose all of its usefulness in the process.

A schism may exist between data sets that track similar or identical movements using different technologies or approaches. When measuring the force of a kick on the pitch, for example, data collected from wearables attached to a player’s shoe cannot be easily integrated with data collected that measured that same kick using on-field kicks. laser based lidar.

And because wearable technologies limit where and how often those measurements can be tracked, there could be gaps in the technology’s feedback due to missing data points. Data matching cannot merge this information.

Upgrading to new technologies is of course often worthwhile. Take lidar, which is more accurate while also being more portable and unobtrusive from the player’s point of view than previous technology. The challenge of data integration is the only significant drawback of adopting lidar for a club’s player evaluation department. And with the right plan, even that challenge can be solved.

Raf Keustermans is the CEO of Sportlight Technology.

DataDecision makers

Welcome to the VentureBeat Community!

DataDecisionMakers is where experts, including the technical people who do data work, can share data-related insights and innovation.

If you want to read about the very latest ideas and up-to-date information, best practices and the future of data and data technology, join us at DataDecisionMakers.

You might even consider contributing an article yourself!