MapReduce 多表连接

题目描述：

现在有两个文件，1为存放公司名字和城市ID，2为存放城市ID和城市名

表一：

factoryname,addressed
Beijing Red Star,1
Shenzhen Thunder,3
Guangzhou Honda,2
Beijing Rising,1
Guangzhou Development Bank,2
Tencent,3
Back of Beijing,1

 

表2：

1,Beijing
2,Guangzhou
3,Shenzhen
4,Xian

 

现在要求输出公司名和城市名。例如：

Beijing Red Star Beijing

 

这个类似数据库里的多表连接。整体思路和单表连接差不多。还是利用reduce阶段对城市ID进行归并，我们在map阶段统一输出key=城市ID value=falg+“+”+城市名or公司名。然后通过reduce对flag的解析，分析后者是城市名还是公司名，并放到两个数组中，最后利用笛卡尔积将其输出

具体代码

public class MyMapper extends Mapper<LongWritable, Text, Text, Text> {

 

                 public void map(LongWritable ikey, Text ivalue, Context context )

                                                 throws IOException, InterruptedException {

                                String line=ivalue.toString();

                                StringTokenizer st= new StringTokenizer(line,"," );

                                String value0=st.nextToken();

                                String value1=st.nextToken();

                                 if(value0.compareTo("factoryname" )!=0){

                                                 if(value0.length()==1){

                                                                 context.write(new Text(value0), new Text("1" +"+"+value1));

                                                } else{

                                                                 context.write(new Text(value1), new Text("2" +"+"+value0));

                                                }

                                }

                }

 

}

 

 

public class MyReducer extends Reducer<Text, Text, Text, Text> {

 

                 public void reduce(Text _key, Iterable<Text> values, Context context)

                                                 throws IOException, InterruptedException {

                                 // process values

                                ArrayList<String> address= new ArrayList<String>();

                                ArrayList<String> factory= new ArrayList<String>();

                                 for (Text val : values) {

                                                String line=val.toString();

                                                StringTokenizer st=new StringTokenizer(line,"+" );

                                                 int flag=Integer.parseInt(st.nextToken());

                                                 if(flag==1){

                                                                String addressname=st.nextToken();

                                                                 address.add(addressname);

                                                                

                                                                

                                                } else if (flag==2){

                                                                String factoryname=st.nextToken();

                                                                factory.add(factoryname);

                                                }

                                }

                                 if(address.size()!=0&&factory.size()!=0){

                                                 for(int i=0;i<address.size();i++){

                                                                 for(int j=0;j<factory.size();j++){

                                                                                context.write( new Text(address.get(i)),new Text(factory.get(j)));

                                                                }

                                                }

                                }

                }

 

}