The world of data is evolving

The data landscape has changed dramatically over recent years, the world of data is evolving. In the past, we mainly heard that we needed to do as much as possible “cloud-only”—but this trend has become more nuanced now. There has also been plenty of discussion about Big Data or actually the ‘Right Data’ recently. In this article, we look at these two trends and add our own two cents.

Custom data method

Nowadays, we’re organizing our data and applications more in line with “best fit” practices. So the “cloud-only” credo has been toned down a bit. This means we need to look at where the best place for storing your data is. We’re increasingly evolving to a hybrid model—the best of both worlds. But it’s true that the cloud is still generally preferred overall—especially among new set-ups. Aiming for the highest possible level of cloud adoption is the goal, but there are an increasing number of options available to also connect your on-premise environment and gradually move over all your resources to the cloud as the final stage. You don’t need to perform acrobatics to transfer your local systems to the cloud anymore.

Cloud possibilities on-premise

Doesn’t this mean we lose the benefits of the cloud? Isn’t this exactly what Microsoft is giving us? Just because we still have our data on-premise, for example, doesn’t mean that we can’t take advantage of the Azure cloud services. With Azure Arc Data Services, for instance, we can manage and run database services through Azure’s control panel without them being in the cloud.

We need to connect the on-premise data center to Microsoft Azure for this, though, and manage it from there. Then it’s even possible to make your own cross-cloud vendor connections and manage your data centrally that way. This approach and associated services are still new and under development, and will increasingly become an option over the coming years. So it will make much less difference where your data is saved in the future. The most important thing is that you manage the various back-ends from a single, centralized location.

What do we need to do as a company?

Organizations currently wanting to start with a database modernization assessment can soon become lost with all the various options that Azure now offers. Do we want IaaS or PaaS? Are we using an Azure SQL database managed instance or opting for an Azure SQL Database? These are all new or different terms, and each option has its own benefits. So it’s about choosing the right outfit for the right situation. We wouldn’t go to a gala ball in our running clothes—just like we wouldn’t go running dressed for a ball!

The first step in this situation is therefore to map out the existing resources. We see that many companies don’t know exactly how many database servers are running, or who is using and accessing data from where. In the next steps, you can then develop a plan for how to manage your data. We do this together with our customers, usually based on the Cloud Adoption Framework principles.

Big Data often isn’t even “big” yet

Another hot topic in the world of data is the Big Data trend—a discussion that has already filled many hard drives. Because from what size can we start to speak of “big” data? This is a question we’re already guilty of answering, but a question that cannot be answered unequivocally. There’s a good reason for the IT cliché: “It depends.”

When people speak about Big Data, in some cases they’re still too often talking about “regular large volumes of structured data.” We can usually still manage these volumes using relational systems, without pulling out all the stops for big data processing. But the impact on the processing time is sometimes greater because the data isn’t “big” enough. Big Data systems only really come into their own with vast amounts of data.

Better to look at Right Data

I recently read a tweet from Thomas La Rock that said: “The era of Big Data is done. It’s over. Now is the era of getting the Right Data. That is far more critical and Difficult.” (26/08/2021 – Thomas LaRock – Twitter)

This is something I can relate to. It’s dead easy to store all kinds of data these days, which is why it’s being done more readily—leading to an overload of available data. Lots and different kinds of data is one thing; extracting the right and relevant information out of it at a good time is another. Thankfully, the cloud makes it possible to sift through data very quickly and find connections and contexts that have previously been hidden.

Each situation is unique

We wrote it down just a moment ago: We’re increasingly working with a “best fit” principle. But this doesn’t just apply to choosing between cloud and on-premise. Any data initiative always needs to weigh up, by asking the right questions, what the correct choices are for the specific project and its context. No environment or situation is the same.

At Kohera, we help our customers make the necessary choices here according to the time, context, and stage that the customer is at in the process and their growth. The advantage of data and the cloud is its flexibility, and being able to use it in the right circumstances. We’re not physically bound to hardware in the data center for the next four years. And there’s no need to build your own supercomputer anymore, only for it to sit gathering dust for over half the month.

An other extremely important benefit is that we get unmatched security features straight out-of-the box, and you are always up to date with the latest definitions and best practices in this area. There are a gazillion of options, properties and characteristics that you can tailor to fulfill your security requirements and needs in the Microsoft Azure cloud.

How do we tackle this at Kohera?

At Kohera, we always describe and use various set-ups. We explain the pros and cons, and possible strategies. The reality is nuanced, with an increasing number of possible solutions, after all. And one is probably a bit better than the others. With one that has more limitations that the others.

So, in a process like this, the most important thing is to be open to alternatives—to keep an open mind and be less likely to consider options by definition. That’s why, at Kohera, we have specific frameworks for certain contexts which we use to make decisions without immediately ruling out other alternatives. For example, we recently wrote a research paper and whiteboards about Azure Synapse Analytics in which we explore when it might now best be applied.

The data hub as uncharted territory

We’re finding that companies are increasingly recognizing the importance of collecting different types of data in some sort of data hub. Combining all these types of data again gives us multiple opportunities to reassemble something new. And that’s where the uncharted territory currently is: “The era of the right data.”

It’s time to start looking for new insights in the available data. The technology and tools are there. The data is there. Now it’s time to get stuck in and work it out.

Want to learn more? Take a look at our other blog articles!