在使用MultipleOutputs输出时报could only be replicated to 0 nodes instead of minReplication (=1). There are 3 datanode(s) running and no node(s) are excluded in this operation.
即只能复制到0个节点，而不是minReplication（=1）。有3个数据节点正在运行，此操作中不排除任何节点。
1. 我的防火墙也关了
2. 在不使用MultipleOutputs的代码时没有任何问题
3. 在本地跑没有问题，在集群上一跑到就出了问题，疯狂报这个错
could only be replicated to 0 nodes instead of minReplication (=1). There are 3 datanode(s) running and no node(s) are excluded in this operation.
4. 向HDFS上传文件等都没这个问题
查了好多资料有说空间不足的
但我的群集节点上的hdfs dfsadmin -report显示的空间应该是没问题的：
Configured Capacity: 143516123136 (133.66 GB)
Present Capacity: 109557473280 (102.03 GB)
DFS Remaining: 103351971840 (96.25 GB)
DFS Used: 6205501440 (5.78 GB)
DFS Used%: 5.66%
Under replicated blocks: 2
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0
-------------------------------------------------
Live datanodes (3):
Name: 192.168.100.100:50010 (node01)
Hostname: node01
Decommission Status : Normal
Configured Capacity: 47838707712 (44.55 GB)
DFS Used: 2064539648 (1.92 GB)
Non DFS Used: 9394999296 (8.75 GB)
DFS Remaining: 33928613888 (31.60 GB)
DFS Used%: 4.32%
DFS Remaining%: 70.92%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Wed Jan 29 02:18:12 CST 2020
Name: 192.168.100.102:50010 (node02)
Hostname: node02
Decommission Status : Normal
Configured Capacity: 47838707712 (44.55 GB)
DFS Used: 2069651456 (1.93 GB)
Non DFS Used: 8647630848 (8.05 GB)
DFS Remaining: 34670870528 (32.29 GB)
DFS Used%: 4.33%
DFS Remaining%: 72.47%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Wed Jan 29 02:18:13 CST 2020
Name: 192.168.100.103:50010 (node03)
Hostname: node03
Decommission Status : Normal
Configured Capacity: 47838707712 (44.55 GB)
DFS Used: 2071310336 (1.93 GB)
Non DFS Used: 8564355072 (7.98 GB)
DFS Remaining: 34752487424 (32.37 GB)
DFS Used%: 4.33%
DFS Remaining%: 72.65%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Wed Jan 29 02:18:13 CST 2020
这是我的Driver代码
package com.czxy;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.NullWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.mapreduce.lib.output.MultipleOutputs;
import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;
import org.apache.hadoop.util.Tool;
import org.apache.hadoop.util.ToolRunner;
/**
* @Author 嗨皮骚
* @贴吧用户_00JG4KJ615 v 1.0
* @date 2020/2/14
*/
public class Driver extends Configured implements Tool {
@Override
public int run(String[] args) throws Exception {
Job job = Job.getInstance(super.getConf(), "xm");
job.setNumReduceTasks(3);
job.setJarByClass(Driver.class);
// job.setInputFormatClass(TextInputFormat.class);
FileInputFormat.setInputPaths(job,new Path("hdfs://node01:8020/cs/sj.txt"));
job.setMapperClass(Map.class);
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(Text.class);
job.setReducerClass(Reduce.class);
job.setOutputKeyClass(NullWritable.class);
job.setOutputValueClass(Text.class);
MultipleOutputs.addNamedOutput(job, "output", TextOutputFormat.class, NullWritable.class, Text.class);
// job.setOutputFormatClass(TextOutputFormat.class);
FileOutputFormat.setOutputPath(job,new Path("hdfs://node01:8020/cs/wc"));
return job.waitForCompletion(true)?0:-1;
}
public static void main(String[] args) throws Exception {
Configuration configuration = new Configuration();
int run = ToolRunner.run(configuration, new Driver(), args);
System.out.println(run==0?"成功":"失败");
System.exit(run);
}
}
求大佬帮忙分析分析这是什么问题