Forum

This topic contains 0 replies, has 1 voice, and was last updated by  Tajunnisa 1 year, 6 months ago.

Viewing 1 post (of 1 total)
  • Author
    Posts
  • #1983 Reply

    Tajunnisa
    Participant

    The below reference is from Hadoop Apache.org site

    Some of the map reduce configuration parameters are given below with their description.

    1) mapreduce.reduce.shuffle.retry-delay.max.ms – The maximum number of milli seconds the reducer will delay before retrying to download the map data (value – 6000).
    2) mapreduce.reduce.shuffle.parallelcopies-The default number of parallel transfers run by reduce during the cope(shuffle) phase(value – 5).
    3) mapreduce.reduce.shuffle.connect.timeout-The maximum amount of time(milli seconds) reduce task spends to contact the tasktracker to get the output(value – 180000).
    4) mapreduce.reduce.shuffle.read.timeout – The maximum amount of time(milli seconds) reduce task waits for map output data to be available for reading after obtaining connection. (Value – 180000)
    5) mapreduce.shuffle.connection-keep-alive.enable – set to true to keep alive connections. (Value – False).
    6) mapreduce.shuffle.connection-keep-alive.timeout – The number of seconds a shuffle client attempts to retain http connection (value – 5).
    7)mapreduce.task.timeout – The number of ms taken before a task is terminated before it neither reads an input writes an output nor updates its status string. A value of 0 disables the timeout(value – 600,000).
    8) mapreduce.tasktracker.map.tasks.maximum – The max number of map tasks that run simultaneously by the task tracker(value – 2).
    9) mapreduce.tasktracker.reduce.tasks.maximum – The max number of reduce tasks that are run simultaneously by the task tracker (value – 2).
    10) mapreduce.map.memory.mb – amount of memory to request from the scheduler for each map task.(value – 1024)
    11) mapreduce.map.cpu.vcores – The amount of virtual cores to request from scheduler for each map task(value – 1)
    12) mapreduce.reduce.memory.mb – The amount of memory to request from scheduler for each reduce task(value – 1024)
    13) mapreduce.reduce.cpu.vcores – The number of virtual cores to request from scheduler for each reduce task(value – 1)
    14) mapreduce.jobtracker.retiredjobs.cache.size – the number of retired job status to be kept in cache(value – 1000)
    15) mapreduce.tasktracker.outofband.heartbeat – Set this true for the task tracker to send out-of-band heart beat after the task completion for better matency. (Value – False)
    16) mapreduce.jobtracker.jobhistory.lru.cache.size – The number of job history files loaded in memory. jobs are loaded when they are first accessed and the cache are cleared based on LRU. (Value – 5)
    17) mapreduce.jobtracker.instrumentation – The instrumentation class to associate each with jobtracker.
    18) mapred.child.java.opts – Java opts for the task processes.
    19) mapred.child.env – User added environment variables.
    20) mapreduce.admin.user.env – Additional execution environment for map and reduce task processes.
    21) mapreduce.map.log.level – The logging level for map task.
    22) mapreduce.reduce.log.level – The logging level for reduce task.

Viewing 1 post (of 1 total)
Reply To: Map reduce configuration parameters from 40-60
Your information:




cf22

Your Name (required)

Your Email (required)

Subject

Phone No

Your Message

Cart

  • No products in the cart.