SRE Data Pipelines & Integrity: Data Integrity

SRE    |    Intermediate
  • 16 videos | 1h 6m 50s
  • Includes Assessment
  • Earns a Badge
Likes 22 Likes 22
Data integrity is vital as it ensures end-user data accuracy and consistency in conjunction with an adequate level of service and availability. In this course, you'll learn how to choose a strategy for data integrity, including how to account for any potential upsides and tradeoffs. You'll explore various types of failures that lead to data loss and the existence of the many data failure modes. You'll also identify data integrity challenges. Next, you'll examine in detail the soft deletion, back up and recovery, and early detection layers of defense-in-depth, before investigating the data integrity challenges a cloud developer may encounter in high-velocity environments. Finally, you'll outline considerations for implementing out-of-band data validation and successful data recovery and identify how the primary SRE principles apply to data integrity.


  • discover the key concepts covered in this course
    define data integrity and recognize how it relates to site reliability engineering and user data, its associated risks, and how to mitigate them
    list vital data integrity requirements and considerations and differentiate between data integrity and security
    outline how to choose a strategy for data integrity
    define what is meant by SRE objectives and outline how they can help maintain data integrity and ensure data availability
    differentiate between different types of failures resulting in data loss
    recognize challenges for maintaining data integrity
    state what is meant by combinations of data integrity failure modes and the multiple layers of defense in depth
  • define the soft deletion layer of defense-in-depth and recognize the benefits of a soft deletion strategy
    name and describe some important considerations when implementing backup and recovery strategies
    define what is meant by the early detection layer of defense-in-depth and describe how it relates to data loss prevention
    list some common challenges faced by cloud developers
    describe what comprises out-of-band data validation and recognize some important considerations when implementing this strategy
    recognize the importance of setting up systems and knowing that data recovery will be successful
    name and define some of the core principles of SRE and outline some best practices for applying these
    summarize the key concepts covered in this course


  • 1m 38s
  • 5m 5s
    In this video, you'll learn more about data integrity in site reliability engineering. You'll learn that most systems are built around data. Everything relies on the data flowing through the system. In this video, you'll learn about data integrity. FREE ACCESS
  • Locked
    3.  Data Integrity Requirements
    In this video, you'll learn more about data integrity. You'll learn that it's a no-brainer to say a system should have data integrity. Data integrity is essential in a system that relies on data. But what are the requirements for data integrity in a system? What needs to be done? And what does it mean to say your data integrity is high? You'll first learn about the requirements for data integrity in a system. FREE ACCESS
  • Locked
    4.  Data Integrity Strategies
    5m 10s
    In this video, you'll learn more about strategies for data integrity, including data cleaning. This is the act of finding and correcting the errors and inconsistencies in your historical data. You'll discover that it can be costly to do because it may involve creating software that can detect and correct issues or buying software to do so.You'll also learn how to schedule regular data maintenance. This means there will be less data to look at all at once. FREE ACCESS
  • Locked
    5.  SRE Objectives in Data Integrity and Availability
    3m 47s
    In this video, you'll learn more about how to meet the goal of maintaining data availability and integrity as an SRE. You'll learn that users expect data to be accurate and available when they need it. However, as an SRE, you can't focus on just ensuring data is accurate and available when it first enters the system. You need to maintain this throughout the lifetime of the data. FREE ACCESS
  • Locked
    6.  Data Loss Failure Types
    4m 13s
    In this video, you will learn more about the common system failures that lead to data loss. These examples should help you as an SRE to target trouble areas and minimize data loss. You will discover that one common cause of data loss is site catastrophes. If your data center is hit by a flood that wipes out your server farms, you can potentially lose a huge amount of data. Other causes include human error and hardware issues. FREE ACCESS
  • Locked
    7.  Data Integrity Challenges
    4m 26s
    In this video, you'll learn more about the challenges of maintaining data integrity in large distributed systems. You'll learn how these systems are required to perform real-time operations such as analytics on data streams receiving large amounts of data. FREE ACCESS
  • Locked
    8.  Data Integrity Failure Modes
    3m 55s
    Data integrity failure is a term used to describe any event that causes the loss of data or its integrity. Data integrity failure can happen at any time, and can be caused by a variety of factors. The three layers of defense against data integrity failure are prevention (layers 1 and 2), retrieval (layer 3), and destruction (layer 4). Prevention measures include following best practices and using proper authorization mechanisms, while retrieval focuses on restoring lost data and destroying corrupted data. FREE ACCESS
  • Locked
    9.  The Soft Deletion Strategy
    5m 25s
    In this video, you'll learn about the first layer of defense against data integrity loss. You'll discover how to define the soft deletion layer of defense-in-depth and recognize the benefits of a soft deletion strategy. The soft delete is when data is flagged as deleted but not physically deleted. It may be represented by some rows in a database with a special flag that tells the rest of the system to treat those rows as if they've actually been deleted. FREE ACCESS
  • Locked
    10.  Backup and Recovery Considerations
    4m 2s
    In this video, you'll learn more about layer two of the three layers of data protection against corruption and loss, which is backup. You'll learn that backups are simply a detail and that without a proper recovery strategy, data backups can be useless. You'll discover that there are a number of practical considerations to keep in mind when implementing backup and recovery strategies. FREE ACCESS
  • Locked
    11.  Early Detection and Data Loss Prevention
    5m 8s
    In this video, you'll learn more about the third layer of protection against data loss of data integrity. You'll learn that prevention is critical because data loss costs you as a company. It will cut into your productivity timelines and hurt your client relations. This video will outline what causes data loss and what you can do to prevent it. FREE ACCESS
  • Locked
    12.  Cloud Developer Challenges
    4m 25s
    In this video, you'll learn how to list some of the unique challenges faced by cloud system developers in regard to data integrity. You'll discover that cloud services lend themselves to adaptability, which translates to frequent changes, including schema changes in your data. This is a challenge because data backups may reflect one schema. FREE ACCESS
  • Locked
    13.  The Out-of-band Data Validation Strategy
    4m 41s
    In this video, you'll learn more about cloud services and other large distributed systems. They must be constantly vigilant to detect data loss. The problem is that data loss may occur in so many ways that a distributed effort may not be sufficient. Another approach is to have a centralized out-of-band data validation pipeline for checking the system for data loss. This out-of-band process is dedicated to data integrity across the system and helps to quickly find and possibly even remediate dangerous data corruption. FREE ACCESS
  • Locked
    14.  Successful Data Recovery
    3m 41s
    In this video, you'll learn when it's the worst time to discover your data recovery plan doesn't work. You'll learn there's no such thing as a fail-safe backup and recovery solution. This is because failures are inevitable in any system, and you need to prepare for them as if they're going to happen tomorrow. Watch this video to learn more about testing your recovery process regularly and know ahead of time if you have a problem. FREE ACCESS
  • Locked
    15.  Fundamental SRE Principles
    4m 3s
    The goal of an SRE is to protect the company's data while allowing for as much functionality as possible. To do this, they must use a multitiered strategy that includes cybersecurity measures, backups and recovery, and low-level data deletion strategies. When one strategy fails, they must have another in place to minimize the chances of data loss. They should also expect bugs and operator errors and be prepared for them. FREE ACCESS
  • Locked
    16.  Course Summary
    1m 12s


Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.