Faust: Stream Processing Using Models, Agents, & Channels

Faust 1.10    |    Intermediate
  • 12 videos | 1h 31m 42s
  • Includes Assessment
  • Earns a Badge
Likes 7 Likes 7
Stream processing in Faust uses native Python code, meaning you don't have to learn a new domain-specific language to perform data transformations. All you need to know is how existing concepts, such as models, work within a Faust context. Faust models allow you to specify fields and their data types and use this well-defined data structure to access streaming data. In this course, you'll learn how to represent the individual elements of a stream using Faust models. You'll work with agents, which are at the heart of every Faust stream processing application. You'll perform operations using agents and invoke agents synchronously and asynchronously from within your application. You'll then work with channels in Faust. At the end of this course, you'll have the confidence to use models, agents, and channels in the right way to build a successful application.


  • discover the key concepts covered in this course
    use models to represent stream elements
    create models with multiple fields and different data types
    publish messages to a Kafka topic using the pykafka library
    invoke the ask() method to await processing results from an agent
    invoke the cast() method to await processing results from an agent
  • use multiple agents for chained processing of data
    use multiple agents to process elements from different topics
    use channels to send and receive messages
    create streams manually using the .stream() method
    define stream processors to transform stream elements
    summarize the key concepts covered in this course


  • 2m 21s
    In this video, you’ll learn more about the course and your instructor. In this course, you’ll learn how to represent the individual elements of a stream using Faust models. Models allow you to specify fields and their data types, and then access your streaming data using this data structure. Next, you will learn how to work with agents. FREE ACCESS
  • 12m 25s
    In this video, you’ll learn more about Faust Stream Processing. In this demo, you’ll take a look at your code. In the folder you’re working in, you have your current working directory. There’s a subfolder called Models, and under Models, there’s a Python script that contains your Faust application. We’ll take a look at this code. Notice the import Faust on line 1, and on line 4 you instantiate your Faust application. FREE ACCESS
  • Locked
    3.  Specifying Multiple Model Fields in Faust
    5m 48s
    In this video, you’ll watch a demo. In this demo, you’ll continue working with models to represent the data that you’ve read from a Kafka topic. The automatic deserialization of Kafka messages provided when you use these model objects is useful. You’ll learn that when you're working with stream processing using Faust, it's common to represent your input data using models. Next, you’ll take a look at the changes you've introduced in this code. FREE ACCESS
  • Locked
    4.  Programmatically Publishing Messages to Kafka
    8m 14s
    In this video, you’ll watch a demo. You’ll take records from the input stream and print the details to the Console Window. The main change you’ll make is you’ll publish messages to your Kafka topic using Python code. Instead of manually publishing messages using a Console producer, you'll write Python code to publish messages you read in from a CSV file. To publish messages to a Kafka topic, you'll use the pykafka client. FREE ACCESS
  • Locked
    5.  Invoking the ask() Method on Faust Agents
    12m 13s
    In this video, you’ll watch a demo. You'll learn about agents in Faust in more detail, and learn how these agents work. First, you’ll look at the code onscreen. You’ll use the working directory under the Skillsoft folder is Agents. Under the Agent subfolder, you’ll find two files you'll be working with. There is loans.py, which contains the code for your stream processing application, and send_loan_records.py. Next, you’ll explore the stream processing code you’ve written. FREE ACCESS
  • Locked
    6.  Invoking the cast() Method on Faust Agents
    4m 32s
    In this video, you’ll watch a demo. You'll explore another method available on our agents. You’ll once again work with loans.py. You’ll import Faust, set up the LoanRecord model, and you’ll see the fields present in every LoanRecord. This LoanRecord inherits from the faust.Record base class. Onscreen you’ll see the Faust application has been instantiated using faust.App. The topic our agent will listen to is the loan-details topic. The app you’ll use is app.topic. FREE ACCESS
  • Locked
    7.  Using Multiple Faust Agents for Chained Processing
    12m 59s
    In this video, you’ll watch a demo. You’ll define multiple agents. You’ll learn you can have multiple agent definitions within the same Python script. You’ll see you can have as many agents as needed to process your code across your Faust applications. You’ll continue using the class LoanRecord. You’ve instantiated the Faust application, and you've set up the loan details topic. This time, there are two app agent specifications and two decorators decorating two functions.  FREE ACCESS
  • Locked
    8.  Using Multiple Faust Agents for Multiple Topics
    4m 56s
    In this video, you’ll watch a demo. In this demo, you’ll see space characters are removed from this gender field. Following along onscreen, you’ll see the applicant for this loan has the gender female. Next, you’ll print the details of the loan. You’ll print the loan type. Then, you’ll look at the second agent where we process loan records by income and credit score. You’ll run an async for loop to iterate every record. FREE ACCESS
  • Locked
    9.  Using Faust Channels to Send and Receive Messages
    9m 45s
    In this video, you’ll watch a demo. In this demo, you’ll learn the channel construct in Faust Streaming. You’ll access the top-level folder Skillsoft and the Channels subfolder. Within Channels you’ll find the data_loan.csv file and you’ll continue working with the same dataset. This demo will focus on channels. You’ll learn that a channel in Faust is a special construct that Faust uses to send and receive messages. FREE ACCESS
  • Locked
    10.  Manually Creating Streams in Faust
    8m 1s
    In this video, you’ll watch a demo. You’ll explore streams. You’ll learn a stream can be thought of as an infinite async iterable consuming messages from a channel. The channel you’ve used so far has been a Kafka topic, because Faust streaming has built-in support for the Kafka topic. You’ll learn streams can read from any kind of channel. A stream is essentially an infinite source of streaming events processed with code. FREE ACCESS
  • Locked
    11.  Defining Stream Processors in Faust
    8m 6s
    In this video, you’ll watch a demo. You’ll learn that when you're working with events or messages from an input stream, you’ll want to pre-process those streams before you send them along to agents for actual processing. You’ll need to format the fields that already exist in the records that you process. In this demo, you'll see how to specify pre-processors for the records in our input stream when you manually create a stream. FREE ACCESS
  • Locked
    12.  Course Summary
    2m 24s
    In this video, you’ll summarize what you’ve learned in the course. You explored the features available in Faust to enable stream processing. You learned that models describe the fields of the data structures that can be used to represent streaming elements, in the form of key-value pairs. You learned Faust deserializes JSON strings in the input stream to represent elements using the models we’ve defined. You installed the PyKafka library on your local machine. FREE ACCESS


Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.


Likes 254 Likes 254