MapReduce程序运行异常Container exited with a non-zero exit code 1. Error file: prelaun

网络投稿 02-07 6565

我在搭建集群后，所有服务均可正常启动，但是在测试mapreduce任务的时候，总是提示错误，我在修改了各项配置，可是还是提示错误

以下是报错信息

Number of Maps = 1 Samples per Map = 1 Wrote input for Map #0 Starting Job 2021-11-21 19:58:02,262 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/xiaoman/.staging/job_1637495728045_0002 2021-11-21 19:58:02,417 INFO input.FileInputFormat: Total input files to process : 1 2021-11-21 19:58:02,534 INFO mapreduce.JobSubmitter: number of splits:1 2021-11-21 19:58:02,784 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1637495728045_0002 2021-11-21 19:58:02,785 INFO mapreduce.JobSubmitter: Executing with tokens: [] 2021-11-21 19:58:02,950 INFO conf.Configuration: resource-types.xml not found 2021-11-21 19:58:02,950 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'. 2021-11-21 19:58:03,027 INFO impl.YarnClientImpl: Submitted application application_1637495728045_0002 2021-11-21 19:58:03,072 INFO mapreduce.Job: The url to track the job: http://hadoop-168-0-112:8088/proxy/application_1637495728045_0002/ 2021-11-21 19:58:03,073 INFO mapreduce.Job: Running job: job_1637495728045_0002 2021-11-21 19:58:17,299 INFO mapreduce.Job: Job job_1637495728045_0002 running in uber mode : false 2021-11-21 19:58:17,300 INFO mapreduce.Job: map 0% reduce 0% 2021-11-21 19:58:17,325 INFO mapreduce.Job: Job job_1637495728045_0002 failed with state FAILED due to: Application application_1637495728045_0002 failed 2 times due to AM Container for appattempt_1637495728045_0002_000002 exited with exitCode: 1 Failing this attempt.Diagnostics: [2021-11-21 19:58:16.237]Exception from container-launch. Container id: container_e05_1637495728045_0002_02_000001 Exit code: 1 ? [2021-11-21 19:58:16.298]Container exited with a non-zero exit code 1. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. ? ? [2021-11-21 19:58:16.299]Container exited with a non-zero exit code 1. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. ? ? For more detailed output, check the application tracking page: http://hadoop-168-0-112:8088/cluster/app/application_1637495728045_0002 Then click on links to logs of each attempt. . Failing the application. 2021-11-21 19:58:17,391 INFO mapreduce.Job: Counters: 0 Job job_1637495728045_0002 failed!

我在确定不启动集群，只是在一个节点运行mapreduce任务的时候，是没有问题的，但是启动集群后，反而不能运行成功，怀疑可能是集群配置有问题，在集群启动后ResourceManager这边出现了问题（没有启动集群的时候，是没有运行ResourceManager服务的，所以不需要用到ResourceManager调度，这时执行任务没有出现问题，而启动后ResourceManager会对整个集群的资源进行调度，这时跑任务反而出现了问题，所以ResourceManager嫌疑最大），查了几天资料都没找到相关信息，结果一个偶然找到了下面博主的一篇文章Mapreduce运行异常Container exited with a non-zero exit code 1

终其原因还是配置的问题，在hadoop3.0需要加上ResourceManager的webapp地址参数配置，yarn.resourcemanager.webapp.address.rm1， yarn.resourcemanager.scheduler.address.rm2，加上这两个配置后任务就可以成功运行了。

下面是博主的原文，有兴趣的可以去看看。

Mapreduce运行异常Container exited with a non-zero exit code 1_danielchan2518的专栏-CSDN博客用idea编写mapreduce读写hbase，并打包jar放到集群服务器上运行时出现下面错误：[2019-01-05 04:03:01.278]Container exited with a non-zero exit code 1. Error file: prelaunch.err.Last 4096 bytes of prelaunch.err :Last 4096 bytes ...https://blog.csdn.net/danielchan2518/article/details/85887597