Indonesian Chatbot Message Classification Using TF-IDF and Multinomial Naive Bayes

Abstract

Customer-service chatbots receive messages with different communicative purposes, including abusive content that may require further review. This study examines the classification of Indonesian chatbot messages using term frequency–inverse document frequency (TF-IDF) and multinomial naive Bayes. The reported preprocessing comprises case folding, text normalization, removal of punctuation and nonalphabetic characters, and stopword removal. Messages were assigned to four categories: harassment, neutral, racist, and violence. An 80:20 stratified training–test split was reported. The available confusion matrix contains 2,141 test messages, of which 1,674 were correctly classified, giving an accuracy of 78.19%. Harassment had the highest precision (0.893), while violence had the highest recall (0.844). Neutral messages had the lowest F1-score (0.738), and 98 neutral messages were classified as violence. These results establish the performance of the reported classifier on the available test set, but they do not establish customer satisfaction, because the labels describe message content rather than satisfaction ratings. Incomplete documentation of corpus construction, annotation, model settings, and timing procedures limits reproducibility and deployment claims. The model therefore provides a preliminary baseline for content categorization, with further validation needed before its outputs inform customer-service decisions.