Sign in

Software Engineer | Graduate of University of Kelaniya

An introduction to Hadoop, its services & architecture

Photo by Richard Jacobs on Unsplash



What is Kafka?

  1. Publish and subscribe to streams of records
  2. Store replicas of the streams of records
  3. Process streams of records in real-time

Kafka Usages

  1. As a messaging system
  2. As a storage system
  3. As a stream-processor

As a Messaging System

Messaging system model
  1. A Producer (Publisher)- The client applications that send some messages.
  2. A Broker- Receives messages from publishers and store them.
  3. A Consumer- Reads the messages from the broker.


What is Redis?

  1. Redis is an in-memory, key-value store.
  • In-memory store: Redis keeps the data in the cache and it does not write to the disk. This makes reading/writing data very fast. (However, Redis has an option to write data to the disk)
  • Key-value store: Redis can store data as key-value pairs.
eg: SET "name" "Varuni"
"name" is the Key and "Varuni" is…


  • Stream: An established bidirectional connection which may carry one or more messages
  • Message: A complete sequence of frames that represents a request or response message
  • Frame: The smallest unit of communication in HTTP/2. Each frame contains a frame header which could identify the stream to which the frame belongs.
  • h2: Short term for HTTP/2

Varuni Punchihewa

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store