Textual variations in social media text processing applications: challenges, solutions, and trends

Abstract

Being an informal communication source, social media text is susceptible to several intentional and unintentional textual variations. These variations lead to various out-of-vocabulary (OOV) words, making social media text processing more challenging. This work analyses and discusses such challenges by providing a detailed overview of different sources of intentional and unintentional OOV words and associated challenges. We provide a detailed survey of pre-processing techniques, including traditional and application-specific methods proposed in the literature to handle intentional and unintentional textual variations, while highlighting their pros and cons. The paper analyses the implications of text normalization (standardization) in different social media text-processing applications. Moreover, the paper provides an overview of the recent challenges and trends in handling social media textual variations, and it is expected to provide a baseline for future research.