Forum

This topic contains 0 replies, has 1 voice, and was last updated by  Nirmal Kumar 5 months ago.

Viewing 1 post (of 1 total)
  • Author
    Posts
  • #3063 Reply

    Nirmal Kumar

    Hi Everyone,

    Whenever we run any query either Piglatin or HQl the query is referred as a job in hadoop & the execution engine generates a job id for that
    particular query,on further diving in the log path you get a series of random name & value pairs irrespective of the cloudera or the ubuntu version you are using under the jobtracker.Here are some of the necessary NV pairs that need some dive-in.

    1)hive.exec.job.debug.capture.stacktraces TRUE

    Des:-Whether or not stack traces parsed from the task logs of a sampled failed task for each failed job should be stored in the
    SessionState

    Like a normal computer programme memory is allocated as per the code is run,so the memory continuosly allocated to the stack in comparison to heap,stack trace allows tracking of the nested sequence functions upto which the stack trace is generated,Stacktrace helps us to trace the nearest point at which the error in code is generated.
    Hadoop is a group of clusters so whenever a client is connected to one of the nodes of the clusters the information about it should be maintained either the job executes successfully or not,so that when the the stack trace is generated proper location of the error prompted by the machine could be traced out.

    & which is set here as the value as TRUE…

    2)hive.stats.reliable FALSE

    Desc:- This part refers form the query optimization part,statics or stats are very important in the case of query optimization, a dev can easily figure out how will its query will behave in real time by running the saved query stats rather than running long execution plans.
    But there are hard times when queries fails to collect the possible statics correctly so in hive.stats.reliable can be used to get the stats correctly but if the stats are not collected as requirement the query is made to fail by default it is set to false.

Viewing 1 post (of 1 total)
Reply To: CDH 4 Hive Propeties (1 to 10)
Your information:




cf22

Your Name (required)

Your Email (required)

Subject

Phone No

Your Message

Cart

  • No products in the cart.