【Storm初探】wordcount demo

目前工作下需要接触storm，遂写个demo练练手。
代码主要借鉴的是storm官方的github项目：https://github.com/apache/storm

略做修改。
1.pom.xml引入

<dependency>
     <groupId>org.apache.storm</groupId>
     <artifactId>storm-core</artifactId>
     <type>jar</type>
     <version>1.0.0</version>
</dependency>

2.Spout
Spout是strom里产生元数据（tuple）流的类，通常情况下Spout会读取外部数据，然后转换为Topology内部的元数据（tuple），主动角色。
nextTuple方法不断执行，数据在此方法内产生。
此处用在数组中随机获得句子作为元数据流做示例。

public class RandomSentenceSpout extends BaseRichSpout {
  private static final Logger LOG = LoggerFactory.getLogger(RandomSentenceSpout.class);

  SpoutOutputCollector _collector;
  Random _rand;

  @Override
  public void open(Map conf, TopologyContext context, SpoutOutputCollector collector) {
    _collector = collector;
    _rand = new Random();
  }

  @Override
  public void nextTuple() {
    Utils.sleep(100);
    String[] sentences = new String[]{sentence("the cow jumped over the moon"), sentence("an apple a day keeps the doctor away"),
            sentence("four score and seven years ago"), sentence("snow white and the seven dwarfs"), sentence("i am at two with nature")};
    final String sentence = sentences[_rand.nextInt(sentences.length)];

    LOG.info("Emitting tuple: {}", sentence);

    _collector.emit(new Values(sentence));
  }

  protected String sentence(String input) {
    return input;
  }

  @Override
  public void ack(Object id) {
  }

  @Override
  public void fail(Object id) {
  }

  @Override
  public void declareOutputFields(OutputFieldsDeclarer declarer) {
    declarer.declare(new Fields("word"));
  }
}

3.Bolt
在一个Topology中接收数据并处理的组件，被动角色。
execute函数处理接收到的数据，并生成新元组转发（emit）出去。
此处定义了两个bolt，第一个blot先把得到Sentence分隔成word，然后下一个bolt对word计数。

public class SplitSentenceBlot extends BaseBasicBolt {

    @Override
    public void execute(Tuple tuple, BasicOutputCollector collector) {
        String word = tuple.getString(0);
        String[] words = word.split(" ");
        for (String s : words) {
            System.out.println("==========" + s);
            collector.emit(new Values(s));
        }
    }

    @Override
    public void declareOutputFields(OutputFieldsDeclarer declarer) {
        declarer.declare(new Fields("word"));
    }
}

public class WordCountBlot extends BaseBasicBolt {
    Map<String, Integer> counts = new HashMap<String, Integer>();

    @Override
    public void execute(Tuple tuple, BasicOutputCollector collector) {
        String word = tuple.getString(0);
        Integer count = counts.get(word);
        if (count == null)
            count = 0;
        count++;
        counts.put(word, count);
        System.out.println("==============[" + word + "]:" + count);
        collector.emit(new Values(word, count));
    }

    @Override
    public void declareOutputFields(OutputFieldsDeclarer declarer) {
        declarer.declare(new Fields("word", "count"));
    }
}

4.Topology
拓扑，storm里的关键性概念，本质上就是在storm里运行的一个实时应用程序，也就是此处要定义的主类。
具体看注释即可…

public class SimpleTopology {
    public static void main(String[] args) throws Exception{
        // 实例化TopologyBuilder类。
        TopologyBuilder topologyBuilder = new TopologyBuilder();
        // 设置喷发节点并分配并发数，该并发数将会控制该对象在集群中的线程数。
        topologyBuilder.setSpout("spout", new RandomSentenceSpout(), 1);
        // 设置数据处理节点并分配并发数。指定该节点接收喷发节点的策略为随机方式。
        topologyBuilder.setBolt("split", new SplitSentenceBlot(), 3).shuffleGrouping("spout");
        topologyBuilder.setBolt("count", new WordCountBlot(), 3).fieldsGrouping("split", new Fields("word"));
        String topologyName = "word-count";

        Config config = new Config();
//        config.setDebug(true);
        if (args != null && args.length > 0) {
            config.setNumWorkers(1);
            StormSubmitter.submitTopology(args[0], config, topologyBuilder.createTopology());
        } else {
            // 这里是本地模式下运行的启动代码。
            config.setMaxTaskParallelism(1);
            LocalCluster cluster = new LocalCluster();
            cluster.submitTopology(topologyName, config, topologyBuilder.createTopology());
        }
    }
}