Databricks is a unified big data processing and analytics cloud platform for transforming and processing vast volumes of data. Apache Spark is the building block of Databricks, an in-memory analytics engine for big data and machine learning. In this article, we will see how to use the Databricks VACUUM command to remove unused files from […]
Databricks VACUUM Command Read More »
Kafka Streams and Spark Streams are potent tools for real-time processing, Here are the key differences Kafka Streaming vs Spark Streaming.
Kafka Streaming vs Spark Streaming Read More »
In this post, We have documented how to conquer the Databricks Certified Data Engineer Associate certification in this article. Databricks has introduced the Data Engineering Associate exam which consists of 45 multiple-choice questions. The time slot allocated is 1.30 hr. The passing score is 32 which is 70%. High-level topics with their weightage in the exam are:
Databricks Certified Data Engineer Associate Read More »
Databricks platform is used to connect to multiple applications. Databricks requires credentials or secrets to connect to these applications. Databricks or Azure Key Vault can store these secrets securely. Secret scopes are used to manage the secrets which are stored in Azure Key Vault or Databricks.
Manage Secret Scopes in Databricks Read More »
Mounting object storage to Databricks fie system allows easy access to object storage as if they were on the local file system. In this article, we will see how to mount and unmount data lake in Databricks.
Mount and Unmount Data Lake in Databricks Read More »
In this article, we will see how to call a notebook from another notebook in Databricks and how to manage the execution context of a notebook.
Call a notebook from another notebook in Databricks Read More »
