INTENT CLASSIFICATION IN CUSTOMER SUPPORT TICKETS USING NLP

Abstract

Efficient and accurate intent classification is essential for customer support automation, yet practical deployment is hindered by dataset heterogeneity, uncertainty handling, and tooling fragmentation. This study presents a web-based intent recognition system built on a modern transformer architecture, leveraging DeBERTa‑v3‑base with the Hugging Face Transformers stack and the CLINC150 (plus) benchmark covering 151 intents with out‑of‑scope coverage. The objective is to deliver a robust, portable, and real‑time solution for classifying user queries in support workflows. Our pipeline standardizes preprocessing (tokenization with max_length=64, consistent intent-to-label mapping) and incorporates lightweight smoke/one‑step validation to guarantee loss wiring and training stability across library versions. Trained for three epochs, the system attains strong test performance of 90.31% accuracy, 91.61% weighted precision, 90.31% weighted recall, and 89.86% weighted F1 on CLINC150 (plus). To ensure practicality and accessibility, we integrate the trained model into a Streamlit application that supports single and batch inference with top‑k predictions, per‑query confidence visibility, and embedded evaluation metrics—enabling rapid qualitative assessment by support operations and product teams. The resulting framework combines competitive accuracy with a lightweight, reproducible toolchain suitable for real‑world iteration and extension (e.g., calibrated uncertainty and zero/few‑shot onboarding)