Automate Databricks Infrastructure as Code with Terraform
This article describes how to Implement Infrastructure as Code Databricks with Terraform.
Automate Databricks Infrastructure as Code with Terraform Read More »
Databricks
Databricks VACUUM Command
Databricks is a unified big data processing and analytics cloud platform for transforming and processing vast volumes of data. Apache Spark is the building block of Databricks, an in-memory analytics engine for big data and machine learning. In this article, we will see how to use the Databricks VACUUM command to remove unused files from
Databricks VACUUM Command Read More »
Kafka Streaming vs Spark Streaming
Kafka Streams and Spark Streams are potent tools for real-time processing, Here are the key differences Kafka Streaming vs Spark Streaming.
Kafka Streaming vs Spark Streaming Read More »
Databricks Certified Data Engineer Associate
In this post, We have documented how to conquer the Databricks Certified Data Engineer Associate certification in this article. Databricks has introduced the Data Engineering Associate exam which consists of 45 multiple-choice questions. The time slot allocated is 1.30 hr. The passing score is 32 which is 70%. High-level topics with their weightage in the exam are:
Databricks Certified Data Engineer Associate Read More »
Manage Secret Scopes in Databricks
Databricks platform is used to connect to multiple applications. Databricks requires credentials or secrets to connect to these applications. Databricks or Azure Key Vault can store these secrets securely. Secret scopes are used to manage the secrets which are stored in Azure Key Vault or Databricks.
Manage Secret Scopes in Databricks Read More »
Mount and Unmount Data Lake in Databricks
Mounting object storage to Databricks fie system allows easy access to object storage as if they were on the local file system. In this article, we will see how to mount and unmount data lake in Databricks.
Mount and Unmount Data Lake in Databricks Read More »
Call a notebook from another notebook in Databricks
In this article, we will see how to call a notebook from another notebook in Databricks and how to manage the execution context of a notebook.
Call a notebook from another notebook in Databricks Read More »