hugging face

The proliferation of models on platforms like Hugging Face has led users to rely heavily on popularity metrics—such as download counts and likes—for model selection, often overlooking actual performance. Method: This work systematically evaluates 500 sentiment analysis models, integrating ~80,000 human-annotated test instances, multi-dimensional evaluation (accuracy, robustness, bias), and audit of model card documentation. Contribution/Results: We empirically demonstrate that neither download count nor like count correlates significantly with true performance. Furthermore, 80% of model cards omit critical training or evaluation details, and 88% overstate performance. Based on these findings, we propose a user-oriented, actionable model selection checklist; quantify the prevalence of documentation deficiencies; and rigorously establish that popularity metrics are invalid proxies for model capability—thereby providing both methodological grounding and practical guidance for trustworthy model reuse.

This study identifies critical issues—namely, inconsistent naming conventions, absent semantic versioning, and opaque documentation—in the release practices of 52,227 open-source pre-trained language models (PTLMs) on Hugging Face, undermining version reliability and experimental reproducibility. Employing a mixed empirical methodology—including large-scale web crawling, naming-pattern clustering, version-change provenance tracing, foundational model lineage reconstruction, and documentation completeness assessment—the authors reveal that PTLM version identifiers (e.g., v1.2) are largely arbitrary; 40.87% of weight updates lack explicit representation in version names or documentation. They further identify 148 distinct naming practices and reconstruct 299 foundational model derivation lineages. Based on these findings, the work proposes the first semantic versioning framework specifically designed for PTLMs, aiming to standardize model releases and enhance reproducibility across the NLP research community.

This study addresses critical challenges in multilingual NLP—model bias, insufficient robustness, and difficulty in ethical alignment—by proposing a fine-tuning and deployment framework for large language models (LLMs) targeting low bias and high robustness. Methodologically, it integrates the Hugging Face ecosystem with Transformer architectures, incorporating multilingual tokenization, domain-aware data cleaning and augmentation, and a progressive fine-tuning strategy that jointly optimizes fairness and task performance. Contributions include: (1) a lightweight, cross-lingual fine-tuning paradigm resilient to bias-induced interference; (2) empirical validation across high-stakes domains (e.g., healthcare and finance), demonstrating significant improvements in generalization and fairness for classification and named entity recognition; and (3) an interpretable, auditable, and production-ready LLM deployment pipeline that advances the practical implementation of ethically aligned AI.

Scaling Transformer models is prohibitively expensive due to fixed-parameter linear projection layers; architectural modifications necessitate full retraining. Method: We propose TokenFormer, the first architecture introducing *parameter tokenization*, which models model parameters as learnable tokens and replaces all linear layers with token-parameter self-attention—unifying parameter and input token representations in a shared latent space. Contribution/Results: Our method enables zero-shot, progressive parameter expansion without retraining, overcoming classical scaling bottlenecks. Without altering network topology, we scale model parameters from 124M to 1.4B while matching the performance of fully trained baselines, achieving substantial training cost reduction. The code and models are publicly released.

To address insufficient integration of lexical distributions and structured knowledge in biomedical named entity recognition (NER), this paper proposes a data-centric knowledge injection paradigm. Specifically, it pioneers the direct conversion of the Unified Medical Language System (UMLS) semantic network and concept hierarchy into structured textual sequences, seamlessly incorporated into BERT’s pretraining pipeline without architectural modifications. The method jointly optimizes masked language modeling and graph-based knowledge reconstruction objectives, enabling co-learning of terminological hierarchical relationships and contextual semantics. A multi-stage pretraining strategy—comprising continued pretraining followed by de novo pretraining—is employed. Empirical evaluation demonstrates significant performance gains across multiple mainstream biomedical NER benchmarks. All models, knowledge serialization code, preprocessing pipelines, and evaluation scripts are publicly released.

This study investigates the evolutionary dynamics and structural characteristics of open-source machine learning models during downstream fine-tuning. Leveraging metadata and model cards from 1.86 million models on Hugging Face, it pioneers the application of evolutionary biology frameworks to AI model ecosystems—constructing fine-grained model phylogenies and integrating network analysis, genetic similarity metrics, and textual feature extraction to systematically characterize inheritance, mutation, and diffusion patterns. Key findings include: (1) a trend toward more permissive licensing; (2) significant degradation in multilingual support, with increasing English dominance; and (3) growing template-driven homogenization of model documentation. The work further identifies “family resemblance” as a structural invariant and reveals directed, rapid mutational bursts—demonstrating that AI model evolution follows quantifiable, predictable systemic drift. These insights establish a novel paradigm for model governance, reproducibility assessment, and ecosystem health monitoring.

This work investigates the intrinsic mechanisms by which pretraining enhances downstream classification performance in NLP. Using a BERT-6 architecture pretrained on Wikipedia and fine-tuned on FewRel and DBpedia, we analyze token-level confusion matrices and representation dynamics across layers. We find that pretraining progressively strengthens high-level semantic modeling by breaking token symmetry and inducing robust semantic clustering layerwise. We propose the Alignment-Perception Threshold (APT) as an order parameter to quantitatively characterize pretraining efficacy; its evolutionary trajectory aligns with findings in vision, suggesting a cross-modal universal mechanism. Experiments demonstrate that pretraining substantially improves fine-tuning accuracy, with output-layer representations vastly outperforming embedding-layer ones, and label prediction confidence remaining stable—confirming effective capture of abstract linguistic structure.

This work addresses the rapid yet often unstructured evolution of Transformer-based language models by proposing a practical, four-dimensional evaluation framework to distinguish substantive advances from incremental improvements and to guide cross-domain deployment. The framework holistically assesses model architecture, alignment methodologies, energy-efficiency trade-offs, and domain adaptability, encompassing key techniques such as encoder/decoder variants, long-context modeling, mixture-of-experts (MoE), retrieval augmentation, instruction tuning, and preference optimization. Through empirical analysis across vertical domains—including healthcare, finance, and law—the study quantifies the trade-offs between model scale and computational cost, redefines what constitutes an “advanced” model in real-world settings, identifies critical research gaps, and offers actionable guidelines for model selection and deployment.

Selecting the optimal pre-trained text-to-image (T2I) diffusion model for fine-tuning remains an open challenge. Method: This paper proposes M&C, the first systematic model selection framework, which constructs a model–dataset matching graph to explicitly model performance correlations between models and datasets, and employs a graph neural network to jointly encode model architectures, dataset characteristics, and graph topology for predicting relative fine-tuning performance in target domains. Contribution/Results: M&C eliminates costly exhaustive fine-tuning. Experiments across 10 T2I models and 32 datasets show that M&C identifies the top-performing model in 61.3% of cases and consistently recommends high-performing alternatives otherwise—substantially improving both efficiency and accuracy of model selection. To our knowledge, this is the first work to introduce graph-structured modeling into T2I model selection, establishing a novel paradigm for efficient large-model adaptation.

Tokenizer effects on language model performance and behavior remain confounded with other architectural and training factors, hindering principled analysis. Method: We isolate tokenizer impact by training 14 models with identical architectures, datasets, and optimization configurations—differing only in their tokenizers (BPE, WordPiece, SentencePiece, etc.)—and introduce the first benchmark for tokenizer sensitivity to realistic text perturbations (whitespace, punctuation, spelling variants). Contribution/Results: Quantitative evaluation reveals up to 5–12% downstream task performance variance across tokenizers, accompanied by nontrivial behavioral shifts—not merely accuracy fluctuations—but systematic differences in robustness, generalization, and computational efficiency. We identify fundamental trade-offs among these dimensions and localize the critical transformer layers and attention patterns governing tokenizer robustness, providing the first rigorous, decoupled characterization of tokenizer-induced effects in modern language models.