kohera-logo-regular.svg

De Tijdloze 2020: follow-up

We and several other Koherians once again spent New Year’s Eve glued to our radios, wondering if we’d adapted the predictive model successfully enough to get De Tijdloze Top 100 from Studio Brussels right? Or at least improved compared to last year? Find out below!

Longer wait for a hit

We had to wait a bit longer to hear our first hit this year. But there it was at number 87! A perfect prediction for With or Without You by U2. And then it was the same story again for the rest of the list this year as it was for last year: the closer to the top 10, the better the prediction. We got eight out of the top ten songs right, with five of them even in the correct position.

ARIMA, Python, AIC and SARIMA

Predicting De Tijdloze can be seen as a forecast over a period of time – like an edition. We developed a Python script using the Autoregressive integrated moving average – or ARIMA model – for this. One improvement we made compared to our previous model was to the minimum Akaike Information Criteria (AIC). We now calculated this per song rather than having just one optimal AIC for the entire model, and then trained the model on the basis of these individual AIC values.

It also turned out that the seasonal variation that can be added (SARIMA) did not result in a better prediction, so the seasonal parameter was no longer included this year.

Insufficient data

The biggest weakness in our prediction undoubtedly came from our source data, which consisted exclusively of historical results from De Tijdloze. Not only did this mean we lost the ability to predict new songs; we also had to drop songs that did not appear at least five times. Indeed, the fewer historical data points there are, the weaker the ARIMA model becomes.

Another problem we noticed was that we were very poor at predicting songs which did not appear last year as an exception, but which had otherwise become established. This is an important insight to take into account for future attempts because it highlights one of the model’s weaknesses.

Looking ahead to next year

The best way to improve our prediction is of course to simply try again next year! With an extra year of data, and an extra year of experience in particular, we will again try to get a step closer to the perfect prediction!

Top 10 reality versus prediction

Place Reality Prediction
1 Pearl Jam – Black Pearl Jam – Black
2 Fleetwood Mac – The Chain Nirvana – Smells Like Teen Spirit
3 Queen – Bohemian Rhapsody Queen – Bohemian Rhapsody
4 Gorky – Mia Gorky – Mia
5 Pink Floyd – Wish You Were Here Pink Floyd – Wish You Were Here
6 Eddie Vedder – Society Lou Reed – Walk on the Wild Side
7 Dire Straits – Sultans of Swing Metallica – One
8 Nirvana – Smells Like Teen Spirit Led Zeppelin – Stairway to Heaven
9 Pink Floyd – Shine On You Crazy Diamond Pink Floyd – Shine On You Crazy Diamond
10 The Cure – A Forest Dire Straits – Sultans of Swing

 

Parameter sniffing solved with new Parameter Sensitive Plan Optimization feature

If you’re someone that works a lot with Microsoft SQL Server, there’s no doubt that you’ve had issues with an issue called “Parameter sniffing” before. Up until now, you had...

Creating maps with R and Power BI

The possibilities are infinite when it comes to creating custom visuals in Power BI. As long as you have creativity and knowledge about the right programming language, you can let...

Sending monitoring alerts through Telegram

What if you could get the ease of phone notifications for whatever monitoring alerts you need? Then we have a solution for you with the app Telegram. Some of you...

Send mails with Azure Elastic Database Jobs

The DatabaseMail feature in SQL Server and Managed Instance is widely used by many professionals. But what if you want a similar functionality in Azure SQL Database? There are options,...

Sorting matrices in Power BI

Recently I worked on a Power BI project for a client. They had a SharePoint site where they regularly published articles and wanted to pour view data into a report...

The world of data is evolving

The data landscape has changed dramatically over recent years. In the past, we mainly heard that we needed to do as much as possible “cloud-only”—but this trend has become more...