Seq2Seq Model-Based Chatbot with LSTM and Attention Mechanism for Enhanced User Interaction

2024-12-27

AI Summary

To address the prevalent issues of high deployment cost, low flexibility, and vendor lock-in in tourism-domain dialogue systems, this paper proposes a lightweight end-to-end dialogue system tailored to the Drâa-Tafilalet region of Morocco. We introduce a customized Seq2Seq architecture integrating an LSTM-based encoder-decoder with Bahdanau attention—operating entirely without external APIs, thereby eliminating third-party dependencies and significantly reducing operational overhead. The model is trained exclusively on domain-specific tourism corpora, ensuring strong task adaptation while maintaining low computational and memory requirements. Experimental results demonstrate training, validation, and test accuracies of 99.58%, 98.03%, and 94.12%, respectively. The system substantially improves response relevance, coherence, and user satisfaction. This work establishes a reusable technical paradigm for low-resource, minority-language tourism applications.

Technology Category

• Natural Language Processing: Conversational AI/Dialog Systems
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

A chatbot is an intelligent software application that automates conversations and engages users in natural language through messaging platforms. Leveraging artificial intelligence (AI), chatbots serve various functions, including customer service, information gathering, and casual conversation. Existing virtual assistant chatbots, such as ChatGPT and Gemini, demonstrate the potential of AI in Natural Language Processing (NLP). However, many current solutions rely on predefined APIs, which can result in vendor lock-in and high costs. To address these challenges, this work proposes a chatbot developed using a Sequence-to-Sequence (Seq2Seq) model with an encoder-decoder architecture that incorporates attention mechanisms and Long Short-Term Memory (LSTM) cells. By avoiding predefined APIs, this approach ensures flexibility and cost-effectiveness. The chatbot is trained, validated, and tested on a dataset specifically curated for the tourism sector in Draa-Tafilalet, Morocco. Key evaluation findings indicate that the proposed Seq2Seq model-based chatbot achieved high accuracies: approximately 99.58% in training, 98.03% in validation, and 94.12% in testing. These results demonstrate the chatbot's effectiveness in providing relevant and coherent responses within the tourism domain, highlighting the potential of specialized AI applications to enhance user experience and satisfaction in niche markets.

Problem

• Chatbots
• Domain-specific Applications
• User Experience

Innovation

• Seq2Seq Model
• Attention Mechanism
• LSTM