Data Infrastructures with Apache Kafka Competency (Intermediate Level)

  • 18m
  • 18 questions
The Data Infrastructures with Apache Kafka Competency (Intermediate Level) benchmark measures your ability to recognize the basics, use cases, and core concepts of Apache Kafka. You will be evaluated on your skills in building Python applications that produce messages for and consume messages from Kafka topics, creating and using consumer groups, and developing Apache Spark applications that can manipulate streaming data. A learner who scores high on this benchmark demonstrates that they have the ability to work with Apache Kafka with minimal supervision.

Topics covered

  • configure Kafka brokers using properties files
  • contrast batch, streaming, and micro-batch processing
  • create a Kafka producer and consumer in Python
  • create and use a consumer group
  • create and use topics with partitions and replicas
  • create multiple topics and perform cleanup operations for topics
  • describe how Apache Hadoop and Spark work
  • enable and disable autogeneration of topics
  • install and set up Apache Kafka
  • outline a use case for and the services offered by Apache Kafka
  • process messages at the consumer using Python
  • produce and consume basic messages with a Kafka topic
  • publish fake data to a Kafka topic
  • recall the architecture and features of Apache Spark
  • recall why brokers, partitions, and replicas are used in Kafka
  • recognize the different options available to improve the performance of a Kafka environment
  • recognize the use cases of Spark in general and specifically, its structured streaming engine
  • write out data to a partitioned Kafka topic and configure partition splits