All case studies
Data Cleansing

Optimizing Data Processing for Fantasy Football Analytics

Significantly Reduced Data Refresh Times Through Parallel Processing

By Jeff BartonJanuary 1, 2022

The Situation

We partnered with a company specializing in fantasy football analytics. They had a sophisticated website where the data and front-end code were tightly intertwined. This coupling caused their weekly data refresh process to be time-consuming and a potential bottleneck for scaling their product.

The Solution

After a comprehensive code review, we suggested decoupling the data and front-end code. We developed a Python package dedicated to handling all their data needs. To drastically reduce their weekly refresh times, we implemented parallel processing.

The Result

Our solution reduced the weekly refresh times from several hours to about 5 minutes. This dramatic speed improvement made their platform significantly more scalable and easier to troubleshoot. Furthermore, it provided a clear path for adding automated testing in the future. Our client could now confidently push for new subscribers, knowing their platform could handle the increased load. This project reaffirmed our proficiency in data optimization and scalable system design.

Tech stack

PythonPandasGitHubSQL