Chapter 1- Introduction to Bigdata

I'm a Hadoop component that stores data with ease, A distributed file system, that's my expertise. I'm designed to handle large amounts of data, And provide high availability, that's my mantra. I use replication to ensure data is safe, And can be accessed from anywhere, that's my aim. I'm the storage backbone of Hadoop's might, And help process data, day and night!

• Hadoop

• HDFS

• IPFS

• EXT4

I collect, aggregate, move unstructured or semi structured data into HDFS.

• Hive

• Pig

• Sqoop

• Flume

I’m responsible for managing resources in a Hadoop cluster.

• YARN

• HBase

• MapReduce

• Hive

I efficiently store large tables with sparse data in Hadoop.

• Flume

• Hive

• HBase

• Oozie

I schedule and manage Hadoop jobs.

• Oozie

• Cassandra

• NameNode

• HCatalog

My 10 lines of script is around 200 lines of MapReduce.

• Apache

• Kafka

• Pig Latin

• PySpark

I am a distributed streaming platform used for building real-time data pipelines and streaming applications.

• Apache

• Kafka

• Pig Latin

• PySpark

I allow users to process large volumes of data and create machine learning models that runs on Hadoop MapReduce

• Mahoot

• Sqoop

• Oozie

• Zookeeper

I'm a tool used in Hadoop's processing part, To sort and filter data, I play a smart start. I take inputs, process them with care, And output key-value pairs, that's my share. I'm used in conjunction with another tool, To generate outputs, that's the final rule. Together we work, in a harmonious way, To help Hadoop process data, every single day!

• YARN

• HDFS

• MapReduce

• Hadoop Utilities

I'm a framework that's divided into two, One for processing, one for storing too. My processing part is like a map, Helping to sort and filter, that's a fact. My storage part is like a file system vast, Distributed and scalable, that's a must. I'm used by companies of all sizes, you see, To manage their data, that's just me!

• Hadoop

• MapReduce

• YARN

• HDFS