Leveraging Data Engineering and Machine Learning for Enhanced Product Recommendation in E-Commerce

Abstract

Abstract: In the rapidly evolving e-commerce landscape, effective product recommendation systems are vital for enhancing customer satisfaction and fostering loyalty. This study leverages advanced data engineering techniques and machine learning algorithms to develop a high-performing, personalized recommendation system. Methods include the implementation of supervised machine learning models such as Linear Support Vector Classification (SVC), Decision Tree, K-Nearest Neighbor (KNN), and Naive Bayes, applied to a dataset of over 10,000 customer reviews and ratings from Amazon’s fashion products. The proposed method integrates Long Short-Term Memory (LSTM) with an attention mechanism and a refined loss function to capture temporal patterns and prioritize high-impact interactions. Natural Language Processing (NLP) was utilized for preprocessing, sentiment analysis, and classification of customer reviews into categories—Good, Moderate, and Not Recommended. Exploratory data analysis using Pearson correlation and Ordinary Least Squares (OLS) regression identified influential product attributes. Among the baseline models, Linear SVC achieved an accuracy of 89% with fast prediction speeds, while the proposed LSTM-based method outperformed all with an accuracy of 94% and superior precision, recall, and F1-score. The results demonstrate that the proposed method effectively addresses limitations in traditional recommendation systems by adapting to user preferences in real time, improving engagement, and increasing conversion rates. These findings provide actionable insights for e-commerce practitioners aiming to optimize recommendation systems for a more personalized and efficient shopping experience.