In this article, we will explain how to resolve the error in the log file.
2021-07-14 13:14:05,157 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_Id
2021-07-14 13:14:05,287 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster:
/************************************************************
[system properties]
os.name: Linux
os.version: 3.10.0-1062.12.1.el7.x64
java.home: /usr/java/jdk1.8.0_151/jre
java.runtime.version: 1.8.0_151-b12
java.vendor: Oracle Corporation
java.version: 1.8.0_151
java.vm.name: Java HotSpot(TM) 64-Bit Server VM
java.class.path: /scratch/yarn6/usercache/ /appcache/ application_Id/container_id:/etc/hadoop/conf.cloudera.yarn:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-protobuf.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/commons-daemon-1.0.13.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-hdfs-httpfs-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/mssql-jdbc-6.2.1.jre7.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jetty-webapp-9.3.25.v20180904.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/spark-2.4.0-cdh6.3.3-yarn-shuffle.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jackson-xc-1.9.13.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/kerb-crypto-1.0.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-format.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/objenesis-1.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/okhttp-2.7.5.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jsp-api-2.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/commons-io-2.6.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/commons-beanutils-1.9.4.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-mapreduce-client-jobclient-3.0.0-cdh6.3.3-tests.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-yarn-applications-unmanaged-am-launcher-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/commons-net-3.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jersey-guice-1.19.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jackson-jaxrs-1.9.13.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/json-io-2.5.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/logredactor-2.0.7.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/commons-math3-3.1.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/commons-cli-1.2.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jackson-core-2.9.10.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/kerby-util-1.0.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/javax.activation-api-1.2.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/httpclient-4.5.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/snappy-java-1.1.4.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-openstack-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/netty-resolver-4.1.17.Final.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-mapreduce-client-nativetask-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-mapreduce-examples-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/curator-framework-2.12.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/kerb-client-1.0.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/accessors-smart-1.2.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jetty-util-ajax-9.3.25.v20180904.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jersey-servlet-1.19.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-gridmix-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/tt-instrumentation-6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jaxb-api-2.2.11.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-distcp-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/nimbus-jose-jwt-4.41.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/lz4-java-1.5.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/azure-keyvault-core-0.8.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-cascading3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/aliyun-sdk-oss-2.8.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-mapreduce-client-shuffle-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/log4j-api-2.8.2.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jetty-security-9.3.25.v20180904.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jetty-util-9.3.25.v20180904.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/avro-1.8.2-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/commons-codec-1.11.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/kafka-clients-2.2.1-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-sls-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-yarn-client-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/gson-2.2.4.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jaxb-impl-2.2.3-1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/kerb-simplekdc-1.0.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/httpcore-4.4.6.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jdom-1.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/protobuf-java-2.5.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/kerb-identity-1.0.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/netty-handler-4.1.17.Final.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/ehcache-3.3.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jetty-io-9.3.25.v20180904.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-mapreduce-client-hs-plugins-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/audience-annotations-0.5.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-streaming-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-jackson.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/kerb-util-1.0.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/guice-4.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/stax2-api-3.1.4.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-generator.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jsch-0.1.54.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-common-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/wildfly-openssl-1.0.4.Final.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-cascading.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/xz-1.6.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/commons-logging-1.1.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/javax.inject-1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jersey-server-1.19.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-common-3.0.0-cdh6.3.3-tests.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jetty-servlet-9.3.25.v20180904.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/log4j-1.2.17.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/kerby-xdr-1.0.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-format-javadoc.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-hdfs-native-client-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-archives-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/kerb-admin-1.0.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jetty-server-9.3.25.v20180904.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/kerby-pkix-1.0.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/bcpkix-jdk15on-1.60.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jackson-annotations-2.9.10.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/kerby-asn1-1.0.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jul-to-slf4j-1.7.25.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/re2j-1.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-hdfs-client-3.0.0-cdh6.3.3-tests.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jcip-annotations-1.0-1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-thrift.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/commons-configuration2-2.1.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/commons-compress-1.19.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-yarn-api-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/aws-java-sdk-bundle-1.11.271.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/curator-client-2.12.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-mapreduce-client-core-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-archive-logs-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/slf4j-log4j12.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/slf4j-api-1.7.25.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/zstd-jni-1.3.8-1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-aws-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/ojalgo-43.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/log4j-core-2.8.2.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/netty-codec-4.1.17.Final.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/java-util-1.9.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/json-smart-2.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-kafka-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/netty-common-4.1.17.Final.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jsr305-3.0.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/HikariCP-java7-2.4.12.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/json-simple-1.1.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-avro.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-auth-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/netty-3.10.6.Final.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/zookeeper-3.4.5-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-format-sources.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/commons-lang-2.6.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/curator-recipes-2.12.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jettison-1.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-hdfs-native-client-3.0.0-cdh6.3.3-tests.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-yarn-registry-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/event-publish-6.3.3-shaded.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/guava-11.0.2.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-azure-datalake-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jackson-jaxrs-json-provider-2.9.10.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-pig.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/asm-5.0.4.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-azure-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/kerb-common-1.0.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-extras-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-encoding.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/fst-2.50.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/paranamer-2.8.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-common.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/azure-storage-5.4.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/commons-lang3-3.7.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-hadoop.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jersey-core-1.19.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-hdfs-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-mapreduce-client-hs-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-pig-bundle.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jsr311-api-1.1.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-hdfs-3.0.0-cdh6.3.3-tests.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-yarn-applications-distributedshell-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jackson-module-jaxb-annotations-2.9.10.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/guice-servlet-4.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-hdfs-nfs-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jersey-client-1.19.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-datajoin-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/avro.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-resourceestimator-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-rumen-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jersey-json-1.19.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/bcprov-jdk15on-1.60.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-kms-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/leveldbjni-all-1.8.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-yarn-server-nodemanager-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-mapreduce-client-jobclient-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/kerb-server-1.0.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/netty-buffer-4.1.17.Final.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/netty-transport-4.1.17.Final.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/aopalliance-1.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jetty-http-9.3.25.v20180904.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-column.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/kerby-config-1.0.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-aliyun-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-mapreduce-client-uploader-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-scala_2.11.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/netty-codec-http-4.1.17.Final.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/okio-1.6.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jackson-mapper-asl-1.9.13-cloudera.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/javax.servlet-api-3.1.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-yarn-server-common-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jetty-xml-9.3.25.v20180904.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-mapreduce-client-common-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/commons-collections-3.2.2.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jackson-databind-2.9.10.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/zookeeper.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/metrics-core-3.0.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/kerb-core-1.0.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jackson-core-asl-1.9.13.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/woodstox-core-5.0.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/gcs-connector-hadoop3-1.9.10-cdh6.3.3-shaded.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-yarn-server-tests-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/azure-data-lake-store-sdk-2.2.9.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-mapreduce-client-app-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/jackson-jaxrs-base-2.9.10.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-yarn-common-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/parquet-hadoop-bundle.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-hdfs-client-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/htrace-core4-4.1.0-incubating.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-nfs-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-yarn-server-web-proxy-3.0.0-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/mr-framework/hadoop-annotations-3.0.0-cdh6.3.3.jar::job.jar/job.jar:job.jar/classes/:job.jar/lib/*:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/job.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/ant-contrib-1.0b3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/ant-eclipse-1.0-jvm1.2.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/audience-annotations-0.7.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/avro-ipc.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/avro-mapred.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/avro.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/bonecp-0.8.0.RELEASE.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/commons-codec-1.9.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/commons-collections-3.2.2.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/commons-compress-1.19.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/commons-io-2.4.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/commons-jexl-2.1.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/commons-lang3-3.4.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/commons-logging-1.2.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/commons-pool-1.6.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/datanucleus-api-jdo-4.2.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/datanucleus-core-4.1.6.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/datanucleus-rdbms-4.1.7.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/derby-10.14.1.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/fastutil-7.2.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/guava-11.0.2.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/hsqldb-1.8.0.10.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/jackson-annotations-2.9.10.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/jackson-core-2.9.10.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/jackson-core-asl-1.9.13.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/jackson-databind-2.9.10.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/jackson-mapper-asl-1.9.13-cloudera.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/javax.jdo-3.2.0-m3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/javax.servlet-api-3.1.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/jetty-http-9.3.25.v20180904.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/jetty-io-9.3.25.v20180904.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/jetty-security-9.3.25.v20180904.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/jetty-server-9.3.25.v20180904.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/jetty-servlet-9.3.25.v20180904.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/jetty-util-9.3.25.v20180904.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/jsr305-2.0.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/jtds-1.3.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/kite-data-core.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/kite-data-hive.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/kite-data-mapreduce.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/kite-hadoop-compatibility.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/netty-3.10.6.Final.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/opencsv-2.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/paranamer-2.8.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/parquet-avro.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/parquet-column.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/parquet-common.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/re2j-1.0.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/parquet-encoding.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/parquet-format.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/parquet-hadoop.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/parquet-jackson.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/slf4j-api-1.7.25.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/snappy-java-1.1.4.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/sqoop-1.4.7-cdh6.3.3.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/transaction-api-1.1.jar:/scratch/yarn6/usercache/ /appcache/ application_Id/container_id/xz-1.6.jar
java.io.tmpdir: /scratch/yarn6/usercache/ /appcache/ application_Id/container_id/tmp
user.dir: /scratch/yarn6/usercache/ /appcache/ application_Id/container_id
user.name:
************************************************************/
2021-07-14 13:14:05,344 INFO [main] org.apache.hadoop.security.SecurityUtil: Updating Configuration
2021-07-14 13:14:05,618 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens: [Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: cluster_timestamp: timestamp } attemptId: 1 } keyId: -key_id), Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:nameservice1, Ident: (token for : HDFS_DELEGATION_TOKEN owner= @NA.CORP.cluster.COM, renewer=yarn, realUser=, issueDate=1663161229100, maxDate=1663766029100, sequenceNumber=130462154, masterKeyId=2337)]
2021-07-14 13:14:05,656 INFO [main] org.apache.hadoop.conf.Configuration: resource-types.xml not found
2021-07-14 13:14:05,656 INFO [main] org.apache.hadoop.yarn.util.resource.ResourceUtils: Unable to find 'resource-types.xml'.
2021-07-14 13:14:05,672 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred newApiCommitter.
2021-07-14 13:14:05,673 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null
2021-07-14 13:14:05,711 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 2
2021-07-14 13:14:05,711 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2021-07-14 13:14:06,460 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2021-07-14 13:14:06,623 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
2021-07-14 13:14:06,624 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
2021-07-14 13:14:06,625 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
2021-07-14 13:14:06,626 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
2021-07-14 13:14:06,627 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
2021-07-14 13:14:06,628 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
2021-07-14 13:14:06,628 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
2021-07-14 13:14:06,629 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
2021-07-14 13:14:06,657 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://nameservice1:8020]
2021-07-14 13:14:06,670 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://nameservice1:8020]
2021-07-14 13:14:06,683 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://nameservice1:8020]
2021-07-14 13:14:06,702 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Emitting job history data to the timeline server is not enabled
2021-07-14 13:14:06,742 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
2021-07-14 13:14:06,978 WARN [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-mrappmaster.properties,hadoop-metrics2.properties
2021-07-14 13:14:07,021 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
2021-07-14 13:14:07,021 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system started
2021-07-14 13:14:07,027 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for job_timestamp_ to jobTokenSecretManager
2021-07-14 13:14:07,174 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing job_timestamp_ because: not enabled; too many maps; too much RAM;
2021-07-14 13:14:07,192 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job job_timestamp_ = 0. Number of splits = 12
2021-07-14 13:14:07,192 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for job job_timestamp_ = 0
2021-07-14 13:14:07,192 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_timestamp_ Job Transitioned from NEW to INITED
2021-07-14 13:14:07,194 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal, non-uberized, multi-container job job_timestamp_ .
2021-07-14 13:14:07,222 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue queueCapacity: 100 scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler
2021-07-14 13:14:07,227 INFO [Socket Reader #1 for port 38499] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 38499
2021-07-14 13:14:07,490 INFO [main] org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server
2021-07-14 13:14:07,507 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2021-07-14 13:14:07,507 INFO [IPC Server listener on 38499] org.apache.hadoop.ipc.Server: IPC Server listener on 38499: starting
2021-07-14 13:14:07,508 INFO [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated MRClientService at peanut-worker12.cluster.com/10.12.36.28:38499
2021-07-14 13:14:07,536 INFO [main] org.eclipse.jetty.util.log: Logging initialized @3353ms
2021-07-14 13:14:07,607 INFO [main] org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
2021-07-14 13:14:07,609 INFO [main] org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.mapreduce is not defined
2021-07-14 13:14:07,613 INFO [main] org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2021-07-14 13:14:07,646 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context mapreduce
2021-07-14 13:14:07,646 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context static
2021-07-14 13:14:07,648 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /mapreduce/*
2021-07-14 13:14:07,648 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
2021-07-14 13:14:07,916 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules
2021-07-14 13:14:07,917 INFO [main] org.apache.hadoop.http.HttpServer2: Jetty bound to port 45220
2021-07-14 13:14:07,918 INFO [main] org.eclipse.jetty.server.Server: jetty-9.3.25.v20180904, build timestamp: 2018-09-04T21:11:46Z, git hash: 3ce520221d0240229c862b122d2b06c12a625732
2021-07-14 13:14:07,950 INFO [main] org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.s.ServletContextHandler@e322ec9{/static,jar:file:/scratch/yarn6/filecache/397443/3.0.0-cdh6.3.3-mr-framework.tar.gz/hadoop-yarn-common-3.0.0-cdh6.3.3.jar!/webapps/static,AVAILABLE}
2021-07-14 13:14:08,632 INFO [main] org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.w.WebAppContext@1e1232cf{/,file:///scratch/yarn6/usercache/ /appcache/ application_Id/container_id/tmp/jetty-0.0.0.0-45220-mapreduce-_-any-7502685259599448688.dir/webapp/,AVAILABLE}{/mapreduce}
2021-07-14 13:14:08,636 INFO [main] org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@6588b715{HTTP/1.1,[http/1.1]}{0.0.0.0:45220}
2021-07-14 13:14:08,636 INFO [main] org.eclipse.jetty.server.Server: Started @4453ms
2021-07-14 13:14:08,636 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Web app mapreduce started at 45220
2021-07-14 13:14:08,640 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue queueCapacity: 3000 scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler
2021-07-14 13:14:08,641 INFO [Socket Reader #1 for port 36437] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 36437
2021-07-14 13:14:08,658 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2021-07-14 13:14:08,659 INFO [IPC Server listener on 36437] org.apache.hadoop.ipc.Server: IPC Server listener on 36437: starting
2021-07-14 13:14:08,697 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true
2021-07-14 13:14:08,697 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3
2021-07-14 13:14:08,697 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33
2021-07-14 13:14:08,699 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: 0% of the mappers will be scheduled using OPPORTUNISTIC containers
2021-07-14 13:14:08,773 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: maxContainerCapability: <memory:71680, vCores:15>
2021-07-14 13:14:08,773 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: queue: root.users.
2021-07-14 13:14:08,777 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper limit on the thread pool size is 500
2021-07-14 13:14:08,777 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: The thread pool initial size is 10
2021-07-14 13:14:08,785 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_timestamp_ Job Transitioned from INITED to SETUP
2021-07-14 13:14:08,787 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_SETUP
We didn’t find any resolution on this please post on the comment section.