Forum

This topic contains 0 replies, has 1 voice, and was last updated by  sumansinharoy 8 months, 2 weeks ago.

Viewing 1 post (of 1 total)
  • Author
    Posts
  • #3099 Reply

    sumansinharoy
    Participant

    Start of Steps

    Step 1 : Input splits in key/value pairs (k1,v1)
    k1 = offset of the record (hexadecimal value)
    v1 = content of the record

    example,
    record 1 = [Big Data and Hadoop Design of HDFS]
    record 2 = [Anatomy of a File Read Data Flow]

    Step 2 : map(LongWritable,Text,IntWritable,Text)

    Mapper output is
    list[k2,v2] = [
    3 Big,
    4 Data,
    3 and,
    6 Hadoop,
    6 Design,
    2 of,
    4 HDFS
    ]

    Step 3 : Suffling and Sorting of mapper output

    list[k3,v3] = [
    2 of,
    3 Big,
    3 and,
    4 Data,
    4 HDFS
    6 Hadoop,
    6 Design,
    ]

    Step 4 : reduce(IntWritable, Text, Text, Text)

    Reducer output is
    k4,v4 = “2”, “1 [of]”
    “3”, “2 [Big, and]”
    “4”, “2 [Data, HDFS]”
    “6”, “2 [Hadoop, Design]”

    End of Steps

Viewing 1 post (of 1 total)
Reply To: Algorithm for Word Size for MR job (Assignment)
Your information:




cf22

Your Name (required)

Your Email (required)

Subject

Phone No

Your Message

Cart

  • No products in the cart.