Azure Cloud

How to send emails with an SMTP server in Azure Databricks

A client asked if we could provide a simple form of monitoring on a part of a provided solution. For this particular source, our client asked us to send a simple email with record counts to a mailing list. No problem! Let's get to work

Read more

Caching in Databricks?
Yes, you can!

Sometimes, Databricks can be a bit sluggish. Caching with, for instance, the Delta Cache Accelerated Worker can help you speed it up. We'll explain how it does that.

Read more

XML fun on Azure SQL

While XML has been suitable as a SQL Server native datatype since SQL 2005, for one reason or the other it’s not often used - with only a very few working examples. This blog post is meant for T-SQL teams looking for a working XML example on SQL Server, including schema binding and XML indexes.

Read more

Kohera’s Modern Data Platform: Choosing the model – Pt.1

Kohera’s Modern Data platform does not come with a canned data model by default. This makes the framework extremely flexible to adapt to a variety of project workflows, but it also means during the Analysis phase we will need to think about how to structure the data before continuing. A blogpost.

Read more

The hidden gems of the function Web.Contents()

Some of our customers want us to import their data from a standard on-premises or cloud-based (Azure) SQL-database, others want us to import their data from a REST API. If ever, you’ll have to import data from a REST API, you’ll surely encounter some obstacles.

Read more

Should I use serverless for all my Azure SQL Databases?

Azure SQL Database serverless is a new compute tier made for single databases. This new model automatically scales the compute based on the actual workload per second. This means that you only have to pay for the compute resources (memory & vCores) you use. But there's more...

Read more

Custom email notification in Azure Data Factory

Tired of logging on to Azure Data Factory every day and checking the status of your pipeline(s)? In this blog, Data Wizard Wout Cardoen shows you how to develop an automatic email that alerts you when a pipeline fails.

Read more

Creating a Secure Databricks Environment

Azure Databricks is a cloud native (Big) Data analytics service, offered as a managed PaaS environment. It’s designed to hide the underlying distributed systems and networking complexity as much as possible from the end user, so you can focus on developing rather than having to stress over infrastructure management.

Read more

Azure Synapse Analytics: Where Azure DWH, Spark & ADF meet

When I started to jump from DBA to Data Architect, I thought that these roles would diverge further and further until they became two separate roles and functions. The current evolution Microsoft launched proved me wrong though. Enter Azure Synapse Analytics.

Read more

The Chamber of SQL Part 4: What is a managed instance?

Stefanie was planning to update her knowledge on Microsoft Azure. After a good night rest, she was ready for this new adventure. She opened up her laptop, turned on the power and immediately stopped and stood up. First things first. Black gold. Now she was ready to learn about Managed Instance.

Read more

Data Lake Security and Governance best practices

Data Lakes are the foundations of the new data platform, enabling companies to represent their data in an uniform and consumable way. In this document, we’ll describe how to setup a Data Lake in such a way that it will become the efficient Data Lake that users are looking for.

Read more

All I want for Christmas is you

Oh, this is cheesy, can’t believe we could ever dare to mention Mariah Carey in our blogpost series.  Well, the holiday season is starting, so there is no escape at all 😊. By the way, what I meant with the title was: All I need for Christmas is you… Azure Data Flow. 

Read more

When an Azure runbook does not recognize your cmdlets…

Automating a Data Factory pipeline using a runbook seemed like an easy last step in a Azure DWH setup I developed lately. The part in my script responsible to start and stop the DWH ran without problems, but then ...

Read more

Connecting to an (Azure) database from within Databricks

I made a very useful ELT program in python and wanted it do run inside a Databricks cluster. Databricks on Azure fully supports python 3, so I thought I was up for a walk in the park. Trying to import the database connection classes, already gave a small hint of the troubles ahead...

Read more

OPINION – Is SSRS on Azure happening?

Is SSRS on Azure happening? Well as always in Azure it depends… If you mean that ‘SQL Service Reporting Services’ is going to be provided on Azure, the answer is NO as SSRS is a product name as well as a part of SQL Server. But...

Read more

Using CTAS to efficiently load large tables

An Azure data warehouse is a real powerhouse. Personally, I consider it to be one of the most performant components that can be used and build into the Lambda architecture. Ludicrous power comes with a drawback of course and that drawback is potential inefficiency.

Read more

Azure pt.3: Parameterize and trigger

In previous blogposts we have been creating the Linked Services in Azure and CTAS’ing files from Azure Data Lake into tables on Azure DWH.  I finished my previous post by advising you to use Azure Data Factory V2 (ADF) as an orchestration tool for your ELT load. This is exactly what we’ll be doing in this post. 

Read more

Azure pt.2: The story continues… Enter Polybase

Enter Polybase External Tables.  A technology introduced in SQL Server Parallel Datawarehouse (PDW) around 2013 and became more generally available since SQL 2016, allowing you to read flat files as easy as if they were database tables. 

Read more

Azure pt.1: A word on Linked Services in Azure

How to best describe Azure Data Factory (ADF) V2?  A friend of mine described it as the ETL of the skies.  And since I based almost my entire career on ETL tools, this sounded like music to my ears. After running a nice project in ADF I can now look back on this statement, and conclude there is some truth behind it.

Read more

Azure hybrid solutions with on-premise SSRS Server

For the moment, SSRS (SQL server reporting services) is currently not an Azure component. Will it be available in the near future…? I don’t know. SSRS- functionalities become available in other dashboards and reporting tools like Power BI. Today, you’ll need to work with an on-premise installed SSRS server or you'll have to setup one in an Azure VM.

Read more

Store documents in Azure Blob storage using SQL

One little step in a migration track from on-premise to Azure is related to binary files like images, pdf and others. These files are often stored on many different places. When these files are stored in SQL, you probably have already created a variety of SQL procedures. But you can also use Azure blob-storage.

Read more

Using Azure Blob storage in SSRS reports

Attractive reports are not only showing the hard material statistics and numbers. Visualization of your data leads more and more to graphical decision making. One particular part of your reports are images for logo’s, product images etc. When moving to the cloud, where do we need to store and find our data and images?

Read more

Hybrid cloud solutions

I truly believe in the azure environment. The possibilities are huge and growing every day. Do we all need to migrate to the cloud ‘sito-presto’…yes, I think so. Why I’m so sure about this, is easy to explain. The most important reasons for me are: cost-efficient architecture possibilities and lower maintenance due to very stable architecture solutions.

Read more

How-To create an ultra-fast, concurrent DataMart for Self-service reports on Azure DWH and Azure DB

With Analysis Services, you can mashup and combine data from multiple data sources, define metrics, and secure your data in a single, trusted semantic data model. The data model provides an easier and faster way for your users to browse massive amounts of data with client applications.

Read more

Azure SQL Datawarehouse “Gen 2”: a mayor game changer

Moving Azure SQL Datawarehouse to the cloud made it available for the majority of all companies, enabling them to use this workhorse as a powerful engine to drive their analytical needs. Now the "Gen 2" is five times faster than the already blazing fast Gen1.

Read more

Azure Data Architectures: Lambda

Are you familiar with the lambda architecture? It actually splits data flows into two components, receiving data centrally and doing as little as possible processing before copying and splitting the data stream into two streams, namely the real time and batch layers.

Read more

Why invest in data science?

As you might have noticed, data science is not just a buzz word anymore. It has become serious business! Why?

Read more

How to get your business ready for the cloud?

Cloud computing is no longer just a buzzword, it is here to stay. Companies should prepare by looking into cloud technology and getting their cloud strategy sorted out. If you look at cloud computing in its most basic form, you could simply call it store stuff on the internet. Although this is partially true, the cloud has much more to offer.

Read more