Azure Data Factory for Beginners - Build Data Ingestion - High-Level Plan

The video tutorial provides an overview of metadata-driven ingestion, focusing on the process of transporting data from various sources to a storage medium, specifically within Azure environments. It explains the roles of different systems, such as Bylot Online web store and Azure BLOB storage, and how they interact with the data engineering data lake. The tutorial also covers the structure and function of the metadata database, including system info and batch run tables, and details the batch initialization and execution process. Finally, it discusses the source to target information necessary for data ingestion pipelines.

What is the primary function of data ingestion?

• To analyze data from various sources

• To transport data from various sources to a storage medium

• To visualize data in dashboards

• To delete data from storage

Which system is known as the source system in the context of this video?

• Data Engineering Data Lake

• Bylot Online web store

• Azure Data Lake

• Azure Gen. 2 Data Lake

What is stored in the data engineering metadata database?

• Source to target information

• User credentials

• Data visualization reports

• Network configurations

What is the role of the batch orchestration pipeline?

• To initialize and manage batch ingestion

• To delete old data

• To create new datasets

• To visualize data

What does batch initialization involve?

• Preparing the next batch for execution

• Creating new data lakes

• Deleting old batches

• Visualizing batch data

Where is the source to target information stored?

• In the data visualization tool

• In the source to target metadata table

• In the user profile database

• In the network configuration file

What does the source tool target view do?

• Creates new datasets

• Visualizes data in dashboards

• Queries the source to target metadata table and joins it with other tables

• Deletes old data