Continuous Data: Ingesting Continuous Data in Snowflake

Snowflake 2022    |    Intermediate
  • 7 Videos | 48m 20s
  • Earns a Badge
Data is generally processed using a batch or stream methodology depending on how much time between data generation and processing is acceptable. The Snowflake feature Snowpipes processes data in micro-batches which fall in between these two scenarios. In this course, you will cover the implementation of Snowpipes when data is sourced from an internal Snowflake stage. You will kick things off by looking at data ingestion options in Snowflake from a theoretical standpoint, including the differences between bulk data loading and Snowpipes. Then, you get hands-on to set up the infrastructure for data ingestion: an internal stage for CSV data, a destination table for a data load, and a pipe to carry out the load in micro-batches. Next, you will ingest the data into the destination table and explore how this process can be monitored by tracking the pipe status. Finally, you will implement a Snowflake task to trigger a Snowpipe at regular time intervals.

WHAT YOU WILL LEARN

  • discover the key concepts covered in this course
    outline how batch, stream, and micro-batch processing works
    recognize how the Snowpipe service works in Snowflake
    set up the stage, file format, and data for loading continuous data
  • create a Snowpipe and view its metadata
    load data into a table automatically using a Snowflake pipe
    summarize the key concepts covered in this course

IN THIS COURSE

  • Playable
    1. 
    Course Overview
    2m 10s
    UP NEXT
  • Playable
    2. 
    Processing Data in Micro-batches
    7m 50s
  • Locked
    3. 
    Continuous Data Ingestion with Snowpipes
    9m 5s
  • Locked
    4. 
    Setting Up a Table and Stage for Data Ingestion
    11m 3s
  • Locked
    5. 
    Creating a Snowflake Pipe
    6m 39s
  • Locked
    6. 
    Ingesting Data with a Snowpipe
    9m 47s
  • Locked
    7. 
    Course Summary
    1m 47s

EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion of this course, which can be shared on any social network or business platform

Digital badges are yours to keep, forever.