Introduction to Big Data and Hadoop

What is big data?

• Big data is large and complex datasets that require advanced tools and techniques for processing and analysis.

• Big data is only relevant to social media platforms.

• Big data is a type of software used for data entry.

• Big data refers to small datasets that can be easily managed.

List the 5 V's of big data.

• Volume, Variety, Velocity, Veracity, Value

• Volume, Variety, Velocity, Veracity, Viability

• Volume, Variety, Velocity, Viscosity, Value

• Volume, Variety, Vortex, Veracity, Value

What is Hadoop?

• Hadoop is an open-source framework for distributed storage and processing of large data sets.

• Hadoop is a cloud storage service.

• Hadoop is a programming language for data analysis.

• Hadoop is a type of database software.

Explain the role of HDFS in Hadoop.

• HDFS is a programming model for data processing in Hadoop.

• HDFS is the primary storage system of Hadoop, providing distributed storage, fault tolerance, and high throughput access to large datasets.

• HDFS is a web interface for managing Hadoop clusters.

• HDFS is used for real-time data analytics in Hadoop.

What is MapReduce?

• A database management system for small data sets.

• MapReduce is a programming model for processing large data sets using distributed algorithms.

• A cloud storage service for big data.

• A web framework for building applications.

Name the components of the Hadoop ecosystem.

• Spark

• HDFS, MapReduce, YARN, Hive, Pig, HBase, Sqoop, Flume, Zookeeper, Oozie

• Kafka

• Redis

What is the purpose of YARN in Hadoop?

• To store data in a centralized database.

• To perform data analysis and visualization.

• The purpose of YARN in Hadoop is to manage and schedule resources in a distributed environment.

• To provide a user interface for Hadoop.

How do you install Hadoop on a local machine?

• Use a cloud service to install Hadoop without local setup.

• Install Hadoop by setting up Java, downloading and extracting Hadoop, configuring environment variables, editing configuration files, formatting the filesystem, and starting services.

• Run a single command to install Hadoop automatically.

• Install Hadoop by only downloading the software without Java.

What are the system requirements for Hadoop installation?

• Minimum 8 GB RAM, multi-core processor, 100 GB disk space, Linux OS, and Java (JDK 8+).

• 16 GB RAM, dual-core processor, 200 GB disk space, Mac OS, and Java (JDK 7).

• 8 GB RAM, multi-core processor, 100 GB disk space, Linux OS, and C++.

• 4 GB RAM, single-core processor, 50 GB disk space, Windows OS, and Python.

Explain the difference between Hadoop 1.x and 2.x.

• Hadoop 1.x uses a distributed file system, whereas Hadoop 2.x relies on a local file system.

• Hadoop 1.x has a monolithic architecture with a single NameNode and JobTracker, while Hadoop 2.x uses YARN for resource management, allowing for better scalability and performance.

• Hadoop 1.x supports multiple NameNodes for redundancy, while Hadoop 2.x does not.

• Hadoop 1.x is designed for real-time processing, while Hadoop 2.x is optimized for batch processing.