Prerequisites for Big data certification
Before you start proceeding with this tutorial, we will assume that you have some prior exposure to Core Java, database concepts, and Linux operating system flavors.
Syllabus for big data certification in Bangalore
Module 1: Hadoop Architecture
Learning Objective: In this module, you will understand what is Big Data, What are its limitations of the existing solutions for Big Data problem; How Hadoop solves the Big Data problem, What are the common Hadoop ecosystem components, Hadoop Architecture, HDFS and Map Reduce Framework, and Anatomy of File Write and Read.
Topics,
- Hadoop Cluster Architecture
- Hadoop Cluster Mods
- Multi-Node Hadoop Cluster
- A Typical Production Hadoop Cluster
- Map Reduce Job execution
- Common Hadoop Shell Commands
- Data Loading Technique: Hadoop Copy Commands
- Hadoop Project: Data Loading
- Hadoop Cluster Architecture
Module 2: Hadoop Cluster Configuration and Data Loading
Learning Objective: In this module, you will learn the Hadoop Cluster Architecture and Setup, Important Configuration in Hadoop Cluster and Data Loading Techniques.
Topics,
- Hadoop 2.x Cluster Architecture
Federation and High Availability Architecture
- Typical Production Hadoop Cluster
- Hadoop Cluster Modes
- Common Hadoop Shell Commands
- Hadoop 2.x Configuration Files
- Single Node Cluster & Multi-Node Cluster set up
- Basic Hadoop Administration
Module 3: Hadoop Multiple node cluster and Architecture
Learning Objective: This module will help you understand multiple Hadoop server roles such as Name node & Data node, and Map Reduce data processing. You will also understand the Hadoop 1.0 cluster setup and configuration, steps in setting up Hadoop Clients using Hadoop 1.0, and important Hadoop configuration files and parameters.
Topics,
- Hadoop Installation and Initial Configuration
- Deploying Hadoop in the fully-distributed mode
- Deploying a multi-node Hadoop cluster
- Installing Hadoop Clients
- Hadoop server roles and their usage
- Rack Awareness
- Anatomy of Write and Read
- Replication Pipeline
- Data Processing
Module 4: Backup, Monitoring, Recovery, and Maintenance
Learning Objective: In this module, you will understand all the regular Cluster Administration tasks such as adding and removing data nodes, name node recovery, configuring backup and recovery in Hadoop, Diagnosing the node failure in the cluster, Hadoop upgrade, etc.
Topics,
- Setting up Hadoop Backup
- White list and Blacklist data nodes in the cluster
- Setup quotas, upgrade Hadoop cluster
- Copy data across clusters using distcp
- Diagnostics and Recovery
- Cluster Maintenance
- Configure rack awareness
Module 5: Flume (Dataset and Analysis)
Learning Objective: Flume is a standard, simple, robust, flexible, and extensible tool for data ingestion from various data producers (webservers) into Hadoop.
Topics,
- What is Flume?
- Why Flume
- Importing Data using Flume
- Twitter Data Analysis using hive
Module 6: PIG (Analytics using Pig) & PIG LATIN
Learning Objective: In this module, we will learn about analytics with PIG. About Pig Latin scripting, complex data type, different cases to work with PIG. Execution environments, operation & transformation.
Topics,
- Execution Types
- Grunt Shell
- Pig Latin
- Data Processing
- Schema on reading Primitive data types and complex data types and complex data types
Tuples Schema
- BAG Schema and MAP Schema
- Loading and storing
- Validations in PIG, Typecasting in PIG
- Filtering, Grouping & Joining, Debugging commands (Illustrate and Explain)
Working with function
- Types of JOINS in pig and Replicated join in detail
- SPLITS and Multi query execution
- Error Handling
- FLATTEN and ORDER BY parameter
- Nested for each
- How to LOAD and WRITE JSON data from PIG
- Piggy Bank
- Hands-on exercise
Module 7: Sqoop (Real-world dataset and analysis)
Learning Objective: This module will cover Import & Export Data from RDBMS (MySql, Oracle) to HDFS & Vice Versa
Topics,
- What is Sqoop
- Why Sqoop
- Importing and exporting data using sqoop
- Provisioning Hive Metastore
- Populating HBase tables
- SqoopConnectors
- What are the features of the scoop
- Multiple cases with HBase using client
- What are the performance benchmarks in our cluster for the scoop
Module 8: HBase and Zookeeper
Learning Objectives: This module will cover advance HBase concepts. You will also learn what Zookeeper is all about, how I help in monitoring a cluster, why HBase uses zookerper and how to build an application with zookeeper.
Topics,
- The Zookeeper Service: Data Model
- Operations
- Implementations
- Consistency
- Sessions
- States
Module 9: Hadoop 2.0, YARN, MRv2
Learning Objective: in this module, you will understand the newly added features in Hadoop 2.0, namely MRv2, Name node High Availability, HDFS Federation, and support for Windows, etc.
Topics,
- Hadoop 2.0 New Feature: Name Node High Availability
- HDFS Federation
- MRv2
- YARN
- Running MRv1 in YARN
- Upgrade your existing MRv1 to MRv2
Module 10: Map-Reduce Basics and Implementation
This module, will work on Map-Reduce Framework. How Map Reduce implements on Data which is stored in HDFS. Know about input split, input format & output format. Overall Map Reduce process & different stages to process the data.
Topics
- Map Reduce Concepts
- Mapper Reducer
- Driver
- Record Reader
- Input Split (Input Format (Input Split and Records, Text Input, Binary Input, Multiple Input
- Overview of InputFileFormat
- Hadoop Project: Map-Reduce Programming
Module 11: Hive and HiveQL
In this module, we will discuss a data warehouse package that analysis structure data. About Hive installation and loading data. Storing Data in different tables.
Topics,
- Hive Services and Hive Shell
- Hive Server and Hive Web Interface (HWI)
- Meta Store
- Hive QL
- OLTP vs. OLAP
- Working with Tables
- Primitive data types and complex data types
- Working with Partitions
- User-Defined Functions
- Hive Bucketed Table and Sampling
- External partitioned tables, Map the data to the partition in the table
- Writing the output of one query to another table, multiple inserts
- Differences between ORDER BY, DISTRIBUTE BY and SORT BY
- Bucketing and Sorted Bucketing with Dynamic
- RC File, ORC, SerDe: Regex
- MAPSIDE JOINS
- INDEXES and VIEWS
- Compression on Hive table and Migrating Hive Table
- How to enable update in HIVE
- Log Analysis on Hive
- Access HBase tables using Hive
- Hands-on Exercise
Module 12: Oozie
Learning Objective: Apache Oozie is the tool in which all sorts of programs can be pipelined in the desired order to work in Hadoop’s distributed environment. Oozie also provides a mechanism to run the job at a given schedule.
Topics:
- What is Oozie?
- Architecture
- Kinds of Oozie Jobs
- Configuration Oozie Workflow
- Developing & Running an Oozie Workflow (Map Reduce, Hive, Pig, Sqoop)
- Kinds of Nodes
Module 13: Spark
Learning Objectives: This module includes Apache Spark Architecture, How to use Spark with Scala and How to deploy Spark projects to the cloud Machine Learning with Spark. Spark is a unique framework for big data analytics which gives one unique integrated API by developers for the purpose of data scientists and analysts to perform separate tasks.
Topics,
- Spark Introduction
- Architecture
- Functional Programming
- Collections
- Spark Streaming
- Spark SQL
- Spark MLLib
