{"id":1551,"date":"2019-04-28T09:37:52","date_gmt":"2019-04-28T09:37:52","guid":{"rendered":"https:\/\/prwatech.in\/blog\/?p=1551"},"modified":"2024-04-06T06:56:58","modified_gmt":"2024-04-06T06:56:58","slug":"hadoop-hbase-serialization-and-deserialization-tutorial","status":"publish","type":"post","link":"https:\/\/prwatech.in\/blog\/hadoop\/hbase\/hadoop-hbase-serialization-and-deserialization-tutorial\/","title":{"rendered":"Hadoop HBase Serialization and DeSerialization"},"content":{"rendered":"

Hadoop HBase Serialization and DeSerialization Tutorial<\/h2>\n

 <\/p>\n

Hadoop HBase Serialization and DeSerialization Tutorial<\/strong>, are you looking for the information of Serialization and DeSerialization in hadoop? Or the one who is casually glancing for the best platform which is providing bucketing in hive with examples for beginners? Then you\u2019ve landed on the Right Platform which is packed with tons of Tutorials of bucketing in Hadoop hive. Follow the below mentioned Bucketing in Hive tutorial for Beginners which were originally designed by the world-class Trainers of Hadoop Training institute<\/a> Professionals.<\/p>\n

 <\/p>\n

If you are the one who is a hunger to become the certified Pro Hadoop Developer<\/a>? Or the one who is looking for the best Hadoop Training institute in Bangalore<\/a>\u00a0 which offering advanced tutorials and Hadoop certification course<\/a> to all the tech enthusiasts who are eager to learn the technology from starting Level to Advanced Level.<\/p>\n

 <\/p>\n

HBase Configuration Serialization<\/h2>\n

 <\/p>\n

Add these libraries to your java project build path.<\/strong><\/h3>\n

 <\/p>\n

avro-1.8.1.jar<\/p>\n

avro-tools-1.8.1.jar<\/p>\n

log4j-api-2.0-beta9.jar<\/p>\n

log4j-core-2.0-beta9.jar<\/p>\n

 <\/p>\n

Step 1 – Change the directory to \/home\/cloudera\/Desktop\/Avro<\/strong><\/h3>\n

 <\/p>\n

$ cd \/home\/cloudera\/Desktop\/Avro<\/p>\n

 <\/p>\n

Step 2 – Make a new directory schema in \/home\/cloudera\/Desktop\/Avro<\/strong><\/h3>\n

 <\/p>\n

$ mkdir \/home\/hduser\/Desktop\/AVRO\/schema<\/p>\n

 <\/p>\n

Step 3 – Change the directory to \/home\/cloudera\/Desktop\/Avro\/Schema<\/strong><\/h3>\n

 <\/p>\n

$ cd \/home\/cloudera\/Desktop\/Avro\/Schema<\/p>\n

 <\/p>\n

Step 4 – Create a new avro schema Prwatech.avsc in \/home\/cloudera\/Desktop\/Avro\/schema. It creates a new Prwatech.avsc file if it doesn’t exists and opens for editing.<\/strong><\/h3>\n

 <\/p>\n

$ gedit Prwatech.avsc<\/p>\n

 <\/p>\n

\"Hadoop<\/p>\n

 <\/p>\n

 <\/p>\n

Step 5 – Add these following lines to Prwatech.avsc \u00a0file. Save and close it.<\/strong><\/h3>\n

 <\/p>\n

>> Open eclipse and write following program<\/strong>:<\/p>\n

package<\/strong> com.test.avro;<\/p>\n

import<\/strong> java.io.File;<\/p>\n

import<\/strong> java.io.IOException;<\/p>\n

import<\/strong> org.apache.avro.Schema;<\/p>\n

import<\/strong> org.apache.avro.file.DataFileWriter;<\/p>\n

import<\/strong> org.apache.avro.generic.GenericData;<\/p>\n

import<\/strong> org.apache.avro.generic.GenericDatumWriter;<\/p>\n

import<\/strong> org.apache.avro.generic.GenericRecord;<\/p>\n

import<\/strong> org.apache.avro.io.DatumWriter;<\/p>\n

 <\/p>\n

public<\/strong> class<\/strong> SerializeNew {<\/p>\n

public<\/strong> static<\/strong> void<\/strong> main(String args[]) throws<\/strong> IOException {<\/p>\n

\/\/ Instantiating the Schema.Parser class.<\/p>\n

Schema schema = new<\/strong> Schema.Parser().parse(new<\/strong> File(<\/p>\n

“\/home\/cloudera\/Desktop\/Avro\/Schema\/Prwatech.avsc”));<\/p>\n

\/\/ Instantiating the GenericRecord class.<\/p>\n

GenericRecord e1 = new<\/strong> GenericData.Record(schema);<\/p>\n

\/\/ Insert data according to schema<\/p>\n

e1.put(“name”, “Ramman”);<\/p>\n

e1.put(“id”, 001);<\/p>\n

e1.put(“course”, “Hadoop”);<\/p>\n

e1.put(“DOQ”, “Feb”);<\/p>\n

e1.put(“location”, “Bangalore”);<\/p>\n

GenericRecord e2 = new<\/strong> GenericData.Record(schema);<\/p>\n

e2.put(“name”, “Akash”);<\/p>\n

e2.put(“id”, 002);<\/p>\n

e2.put(“course”, “Python”);<\/p>\n

e2.put(“DOQ”, “March”);<\/p>\n

e2.put(“location”, “Pune”);<\/p>\n

DatumWriter<GenericRecord> datumWriter = new<\/strong> GenericDatumWriter<GenericRecord>( schema);<\/p>\n

DataFileWriter<GenericRecord> dataFileWriter = new<\/strong> DataFileWriter<GenericRecord>( datumWriter);<\/p>\n

dataFileWriter.create(schema, new<\/strong> File( “\/home\/cloudera\/Desktop\/Avro\/data.txt”));<\/p>\n

dataFileWriter.append(e1);<\/p>\n

dataFileWriter.append(e2);<\/p>\n

dataFileWriter.close();<\/p>\n

System.out<\/em><\/strong>.println(“data successfully serialized”);<\/p>\n

}<\/p>\n

}<\/p>\n

<\/h3>\n

Now compile the program<\/strong><\/h3>\n

 <\/p>\n

 <\/p>\n

\"Hadoop<\/p>\n

 <\/p>\n

Output will be<\/strong><\/h3>\n

 <\/p>\n

\"Hadoop<\/p>\n

 <\/p>\n

<\/h2>\n

HBase Configuration DeSerialization<\/h2>\n

 <\/p>\n

Input Data will be<\/strong><\/h3>\n

 <\/p>\n

\"Hadoop<\/p>\n

 <\/p>\n

Run following command in eclipse within same package (but create different class)<\/strong><\/h3>\n

 <\/p>\n

package<\/strong> com.test.avro;<\/p>\n

import<\/strong> java.io.File;<\/p>\n

import<\/strong> org.apache.avro.Schema;<\/p>\n

import<\/strong> org.apache.avro.file.DataFileReader;<\/p>\n

import<\/strong> org.apache.avro.generic.GenericDatumReader;<\/p>\n

import<\/strong> org.apache.avro.generic.GenericRecord;<\/p>\n

import<\/strong> org.apache.avro.io.DatumReader;<\/p>\n

public<\/strong> class<\/strong> DeserializeNew {<\/p>\n

public<\/strong> static<\/strong> void<\/strong> main(String args[]) throws<\/strong> Exception {<\/p>\n

\/\/ Instantiating the Schema.Parser class.<\/p>\n

Schema schema = new<\/strong> Schema.Parser().parse(new<\/strong> File(<\/p>\n

\/home\/cloudera\/Desktop\/Avro\/Schema\/Prwatech.avsc”));<\/p>\n

DatumReader<GenericRecord> datumReader = new<\/strong> GenericDatumReader<GenericRecord>(schema);<\/p>\n

DataFileReader<GenericRecord> dataFileReader<\/u> = new<\/strong> DataFileReader<GenericRecord>( new<\/strong> File(“\/home\/cloudera\/Desktop\/Avro\/data.txt”),datumReader);<\/p>\n

GenericRecord emp = null<\/strong>;<\/p>\n

while<\/strong> (dataFileReader.hasNext()) {<\/p>\n

emp = dataFileReader.next(emp);<\/p>\n

System.out<\/em><\/strong>.println(emp);<\/p>\n

}<\/p>\n

System.out<\/em><\/strong>.println(“data deserialized”);<\/p>\n

}<\/p>\n

}<\/p>\n

 <\/p>\n

After DeSerialization the output will be<\/h3>\n

 <\/p>\n

 <\/p>\n

\"Hadoop<\/p>\n

 <\/p>\n

 <\/p>\n

Get success in your career as a Hadoop developer<\/a> by being a part of the Prwatech<\/a>, India’s leading Hadoop training institute in Bangalore<\/a>.<\/p>\n

 <\/p>\n

 <\/p>\n