Temporal Difference-based Weighted Instance Segmentation Applied on Consecutive Images for Food Recognition

Abstract

This paper presents a TD (Temporal difference) based weighted instance segmentation algorithm for consecutive images. The motivation behind this study is to enhance the segmentation capabilities of service robots, specifically those involved in restaurant cooking assistance. The autonomy of robots heavily relies on visual information through their sensors, and deep neural networks have shown promise in object segmentation. The proposed method employs a weighted segmentation method based on combined probabilities and segmentation history across consecutive images. It accumulates segmentation results in each frame and uses them in subsequent segmentation to reduce segmentation errors. The temporal difference method is based upon a probability map derived from instance segmentation, specifically the mask region-based convolutional neural network (Mask R-CNN) method. The experimental results focus on the segmentation of raw chicken parts for cooking materials, comparing the proposed method with instance segmentation. The experiments demonstrated that 29% of the images exhibited improved segmentation of target objects compared with the existing methods.