data mining

Extracting patterns and structured knowledge from large datasets using algorithms such as classification, clustering, association rules and NLP techniques (tokenization, sentiment analysis), typically implemented with scikit-learn, Spark MLlib or specialized NLP libraries to discover trends and build predictive models.

datamining

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

Must-Read Papers

Most classic and influential ideas
View more

Text2Struct: A Machine Learning Pipeline for Mining Structured Data from Text

Dec 18, 2022
CZ
Chaochao Zhou
🏛️ Northwestern University Feinberg School of Medicine | University of Illinois Urbana-Champaign

The lack of standardized annotation schemas and general-purpose extraction methods for unstructured medical text hinders structured data mining. Method: This paper proposes an open-domain, end-to-end framework for numerical triplet (value–measure–unit) extraction. It introduces the first template-free, general numerical association annotation scheme and jointly models medical text preprocessing, sequence labeling, and relation extraction to simultaneously identify and align numerical values, clinical measures, and associated units. Results: Evaluated on a thrombectomy literature dataset, the framework achieves a Dice coefficient of 0.82; random sampling validation confirms high accuracy in value–entity relation matching. This work establishes a transferable methodology and practical paradigm for structuring template-free medical texts.

Extracting structured data from unstructured textsLack of annotation scheme and training datasetMining metrics and units associated with numerals

Learning Algorithms Made Simple

Oct 11, 2024
NA
Noorbakhsh Amiri Golilarz
🏛️ Mississippi State University | University of Calgary | Brown University

To address the limited robustness to noise, poor cross-domain generalization, and weak interpretability of existing learning algorithms, this paper proposes a unified adaptive dynamic network architecture with three-tiered collaboration among convolutional neural networks (CNNs), traditional machine learning (ML), and large language models (LLMs). Methodologically, we integrate CNNs with ML to construct a hybrid base model and—novelly—introduce an LLM as a high-level semantic guidance and decision verification module, augmented by a noise-robust training strategy. Key contributions include: (1) the first CNN-ML-LLM collaborative framework, significantly improving noise resilience and cross-domain generalization; (2) enhanced model interpretability and end-to-end multi-task adaptability; and (3) empirical validation on image recognition and text generation tasks, achieving high accuracy (average +3.2%) and strong generalization in real-world applications such as medical image classification and financial risk prediction.

Addressing noise vulnerability in classification tasksExploring AI, ML, DL, and hybrid model integrationSimplifying learning algorithms for pattern identification

Performance Analysis of Supervised Machine Learning Algorithms for Text Classification

Aug 31, 2025
SZ
Sadia Zaman Mishu
🏛️ Rajshahi University of Engineering and Technology

This study addresses the comparative performance evaluation of supervised machine learning models for multiclass text classification. We propose and implement an independent, reproducible text classification benchmarking platform that systematically evaluates artificial neural networks (ANNs), backpropagation networks (BPNs), and classical classifiers—including support vector machines (SVM), random forests, and naive Bayes—on a unified, manually annotated dataset. Experiments follow standardized preprocessing, k-fold cross-validation, and accuracy as the primary evaluation metric. Results reveal substantial performance variation across algorithms on real-world text data, with optimal model selection strongly dependent on dataset characteristics. The principal contribution is a scalable, open benchmarking framework for text classification; empirical findings demonstrate that ANNs consistently achieve superior classification accuracy across most test scenarios, thereby providing data-driven guidance for model selection in practical text classification tasks.

Analyzing performance of supervised ML algorithms for text classificationComparing classifier accuracy on labeled text documentsEvaluating Neural Network models for supervised text categorization

This study addresses critical challenges in multilingual NLP—model bias, insufficient robustness, and difficulty in ethical alignment—by proposing a fine-tuning and deployment framework for large language models (LLMs) targeting low bias and high robustness. Methodologically, it integrates the Hugging Face ecosystem with Transformer architectures, incorporating multilingual tokenization, domain-aware data cleaning and augmentation, and a progressive fine-tuning strategy that jointly optimizes fairness and task performance. Contributions include: (1) a lightweight, cross-lingual fine-tuning paradigm resilient to bias-induced interference; (2) empirical validation across high-stakes domains (e.g., healthcare and finance), demonstrating significant improvements in generalization and fairness for classification and named entity recognition; and (3) an interpretable, auditable, and production-ready LLM deployment pipeline that advances the practical implementation of ethically aligned AI.

Addressing data preprocessing and transformer model implementationExploring NLP and LLMs in machine learning intersectionSolving multilingual data handling and AI bias reduction

EmbeddedML: A New Optimized and Fast Machine Learning Library

Sep 16, 2025
HH
Halil Hüseyin Çalışkan
🏛️ Bursa Technical University

To address the low training efficiency and excessive computational cost of traditional machine learning algorithms on large-scale datasets, this paper proposes a high-efficiency training framework grounded in mathematical reformulation and numerical optimization. By rigorously rederiving and restructuring the core optimization procedures of classical algorithms—including logistic regression, support vector machines (SVM), and linear regression—the framework eliminates redundant computations and integrates sparse acceleration with low-rank approximations, all while preserving theoretical accuracy. Experiments demonstrate significant speedups: up to 4× faster than scikit-learn on multivariate linear regression, logistic regression, and SVM tasks; and as high as 800× acceleration for SVM on large-scale datasets, with zero loss in predictive accuracy. The framework unifies support for regression, classification, clustering, and dimensionality reduction, offering both generality and scalability. It establishes a new paradigm for large-scale machine learning that simultaneously ensures high precision and high computational efficiency.

Mathematically enhancing ML algorithms for speedOptimizing training time for large datasetsReducing computation without accuracy loss

Latest Papers

What's happening recently
View more

High-Dimensional Data Processing: Benchmarking Machine Learning and Deep Learning Architectures in Local and Distributed Environments

Dec 11, 2025
JJ
José Julián Rodríguez Gutiérrez
🏛️ División de Ingenierías Campus Irapuato-Salamanca

To address the lack of unified benchmarks for model performance evaluation on high-dimensional big data in both local and distributed environments, this work designs an end-to-end evaluation framework covering three representative tasks—Epsilon (numerical regression), RestMex (text classification), and IMDb (movie feature analysis). Leveraging Apache Spark (Scala), we establish a reproducible heterogeneous computing experimental infrastructure to systematically compare traditional machine learning and deep learning models across accuracy, training efficiency, and resource consumption. This study presents the first pedagogically implemented standardized benchmark supporting multiple models, multimodal data, and diverse deployment scenarios, empirically uncovering performance bottlenecks and architectural trade-offs inherent in distributed scaling. The outcomes include an open-source evaluation pipeline, a standardized reporting template, and a reusable teaching paradigm—providing empirical foundations for AI system selection and optimization in big data contexts.

Benchmark machine learning architectures for high-dimensional data processingCompare local and distributed computing environments for big dataImplement workflows for text analysis and classification tasks

LLM-Based Information Extraction to Support Scientific Literature Research and Publication Workflows

Oct 06, 2025
SA
Samy Ateia
🏛️ University of Regensburg | University of Bayreuth

With the exponential growth of scientific literature, automated extraction of key concepts remains challenging, particularly due to poor cross-disciplinary adaptability. Method: This paper proposes a lightweight LLM-based semantic extraction method supporting FAIR implementation in scholarly workflows. It introduces a context learning–driven zero-/few-shot domain adaptation mechanism that enables rapid, fine-tuning–free adaptation to new disciplines. We systematically benchmark multiple open-source and commercial LLMs on concept identification tasks and develop an interactive online prototype system. Contribution/Results: Empirical evaluation in computer science—complemented by user studies—demonstrates the method’s effectiveness in structured literature review, knowledge graph construction, and information retrieval. It significantly improves both accuracy and cross-domain generalization of concept extraction, offering a scalable technical pathway for intelligent, full-lifecycle scholarly knowledge services.

Enabling rapid domain adaptation for scientific information extractionExtracting key concepts from scientific documents using LLMsSupporting FAIR principles in scientific publishing workflows

Traditional systematic literature reviews suffer from low efficiency and poor reproducibility, particularly when synthesizing fragmented research in emerging fields such as financial narrative. To address this limitation, this study proposes an algorithmic review framework that integrates natural language processing, clustering, and explainable AI techniques to automatically retrieve and structurally analyze scholarly publications from the Scopus database. Applying this framework to the domain of financial narrative—a first for the field—the analysis reveals a predominant focus on sentiment analysis and topic modeling, yet a notable absence of theoretical integration. The results demonstrate that the proposed approach significantly enhances the efficiency, quality, and reproducibility of literature reviews, thereby advancing financial narrative research toward more unified theoretical modeling.

conceptual fragmentationfinancial narrativesnarrative modeling

Leveraging Association Rules for Better Predictions and Better Explanations

Oct 21, 2025
GA
Gilles Audemard
🏛️ Univ. Artois | CNRS | CRIL | Institut Universitaire de France

This study addresses the longstanding trade-off between predictive accuracy and interpretability in tree-based models (e.g., decision trees, random forests). Methodologically, it introduces a novel framework that integrates negation-aware association rules: (1) high-confidence, generalizable rules are mined from training data via an enhanced association rule mining algorithm; (2) these rules are encoded as auxiliary features or logical constraints and incorporated into the tree learning process; and (3) a first-order logic–based abductive reasoning mechanism is developed to generate concise, generalizable explanations covering multiple instances. The key contribution lies in the first systematic use of association rules—particularly those containing negative items—to simultaneously enhance both classification accuracy and explanation universality of tree models. Experiments on multiple benchmark datasets demonstrate statistically significant improvements in classification accuracy, a >40% reduction in average explanation length, and markedly increased cross-instance applicability of generated explanations.

Combining data mining and knowledge for better classification outcomesEnhancing predictive accuracy of tree-based classification models using association rulesImproving explanation quality through more general abductive reasoning

Ontolearn-A Framework for Large-scale OWL Class Expression Learning in Python

Oct 13, 2025
CD
Caglar Demir
🏛️ Paderborn University

This study addresses the automated learning of OWL class expressions and their application to instance classification in large-scale knowledge graphs (KGs). We propose a scalable framework that synergistically integrates symbolic learning with neuro-symbolic learning: it incorporates efficient algorithms—including EvoLearner and DRILL—and introduces an LLM-driven natural language generation module to produce human-readable semantic interpretations of learned class expressions. Additionally, we design a SPARQL query translation mechanism enabling semantically consistent querying and reasoning over remote triple stores. Compared to state-of-the-art approaches, our framework significantly improves accuracy, interpretability, and computational efficiency for learning complex OWL class expressions—particularly those involving conjunction, disjunction, complement, and existential restrictions. Empirical evaluation on real-world large-scale KGs demonstrates superior classification performance and practical deployability.

Implementing symbolic and neuro-symbolic class expression learnersLearning OWL class expressions over large knowledge graphsTranslating complex OWL expressions into natural language

Hot Scholars

CZ

Chengzhi Zhang

Nanjing University of Science and Technology
Text MiningNatural Language ProcessingScience of Science
IG

Iryna Gurevych

Full Professor, TU Darmstadt; Adjunct Professor, MBZUAI, UAE; Affiliated Professor, INSAIT, Bulgaria
Natural Language ProcessingLarge Language ModelsArtificial Intelligence
PN

Preslav Nakov

Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)
Computational LinguisticsLarge Language ModelsFact-checkingFake News
PS

Philip S. Yu

Professor of Computer Science, University of Illinons at Chicago
Data miningDatabasePrivacy
BP

Barbara Plank

Professor, LMU Munich, Visiting Prof ITU Copenhagen
Natural Language ProcessingComputational LinguisticsMachine LearningTransfer Learning