In this paper, we present a new method that summarizes multiple sentences into one sentence. Such a task is useful in domains such as short text summarization. Despite the usefulness of this issue, many studies have not been proposed on the matter. One of the reasons is the training data scarcity: it is impractical to prepare a large data set that contains a sufficiently diverse set of combinations to ensure generalization. We aim to alleviate this difficulty by reformulating the original problem into a problem that produces sentences sequentially. That is, we construct a recursive generation model that takes two sentences in order: the input sentence of the current time step and the generalized sentence so far. Since there is no need to consider the combination of the total input sentences, less complex training data is required than the original problem. Based on the recursive generation model, we propose to summarize the sequentially-input sentences in an online, stochastic manner, where the summary is updated with probability proportional to semantic similarity. Thanks to this stochastic generation scheme, our approach is more flexible for handling situations where inputs are not similar, thus produces summaries with better semantic compression.
