Browse Definitions :
Definition

Apache Kafka

Apache Kafka is a distributed publish-subscribe messaging system that receives data from disparate source systems and makes the data available to target systems in real time. Kafka is written in Scala and Java and is often associated with real-time event stream processing for big data.

Like other message brokers systems, Kafka facilitates the asynchronous data exchange between processes, applications and servers. Unlike other messaging systems, however, Kafka has very low overhead because it does not track consumer behavior and delete messages that have been read. Instead, Kafka retains all messages for a set amount of time and makes the consumer responsible for tracking which messages have been read.

Kafka software runs on one or more servers and each node in a Kafka cluster is called a broker. Kafka uses Apache ZooKeeper to manage clusters; the broker's job is to help producer applications write data to topics and consumer applications read from topics. Topics are divided into partitions to make them more manageable and Kafka guarantees strong ordering for each partition. Because messages are written into a partition in a particular order and are read in the same order, each partition essentially becomes a commit log that can function as a single source of truth (SSoT) for a distributed system’s events.

Kafka’s code base, which was originally developed at LinkedIn to provide a mechanism for parallel load in Hadoop systems, became an open source project under the Apache Software Foundation in 2011. In 2014, the developers at LinkedIn who created Kafka started a company called Confluent to facilitate Kafka deployments and support enterprise-level Kafka-as-a-service products. Version 5.0 of the Confluent Platform, which was commercially released in 2018, improves the handling of application client failover for disaster recovery (DR) and reduces reliance on the Java programming language for data streaming analytics applications.

This was last updated in March 2019

Continue Reading About Apache Kafka

Networking
  • WAN optimization (WAN acceleration)

    WAN optimization -- also known as WAN acceleration -- is a collection of technologies and techniques used to improve the ...

  • Multiprotocol Label Switching (MPLS)

    Multiprotocol Label Switching (MPLS) is a switching mechanism used in wide area networks (WANs).

  • computer network

    A computer network is a group of interconnected nodes or computing devices that exchange data and resources with each other.

Security
  • three-factor authentication (3FA)

    Three-factor authentication (3FA) is the use of identity-confirming credentials from three separate categories of authentication ...

  • cyber espionage

    Cyber espionage (cyberespionage) is a type of cyber attack that malicious hackers carry out against a business or government ...

  • role-based access control (RBAC)

    Role-based access control (RBAC) is a method of restricting network access based on the roles of individual users within an ...

CIO
  • knowledge-based systems (KBSes)

    Knowledge-based systems (KBSes) are computer programs that use a centralized repository of data known as a knowledge base to ...

  • Sarbanes-Oxley Act

    The Sarbanes-Oxley Act of 2002 is a federal law that established sweeping auditing and financial regulations for public companies.

  • project charter

    A project charter is a formal short document that states a project exists and provides project managers with written authority to...

HRSoftware
  • employee engagement

    Employee engagement is the emotional and professional connection an employee feels toward their organization, colleagues and work.

  • talent pool

    A talent pool is a database of job candidates who have the potential to meet an organization's immediate and long-term needs.

  • diversity, equity and inclusion (DEI)

    Diversity, equity and inclusion is a term used to describe policies and programs that promote the representation and ...

Customer Experience
  • sales development representative (SDR)

    A sales development representative (SDR) is an individual who focuses on prospecting, moving and qualifying leads through the ...

  • service level indicator

    A service level indicator (SLI) is a metric that indicates what measure of performance a customer is receiving at a given time.

  • customer data platform (CDP)

    A customer data platform (CDP) is a type of software application that provides a unified platform of customer information that ...

Close