Selected work

Representative Papers

Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought

Apr 08, 2025

This work addresses the challenge of efficiently scaling large language models (LLMs) to multimodal vision-language understanding while preserving strong textual reasoning capabilities. We propose a lightweight multimodal reasoning framework that avoids retraining language or vision backbones, instead leveraging a learnable visual projector for text-image joint reasoning. To enhance cross-modal alignment, we innovatively integrate supervised fine-tuning (SFT) with group-relative policy optimization (GRPO). Furthermore, we introduce adaptive-length chain-of-thought distillation, dynamically optimizing reasoning chain length to balance inference efficiency and accuracy. Evaluated on benchmark suites, our 38B-parameter model achieves 69.0 on MMMU and 67.5 on MathVista, while maintaining state-of-the-art textual reasoning performance (72.0 on AIME and 94.0 on MATH500). All model weights are publicly released.

1 citationsRead paper

MoE-TTS: Enhancing Out-of-Domain Text Understanding for Description-based TTS via Mixture-of-Experts

Aug 15, 2025

To address the poor generalization of descriptive text-to-speech (TTS) systems on out-of-domain user descriptions, this paper proposes a modality-oriented Mixture of Experts (MoE) architecture. Without fine-tuning the frozen pre-trained large language model (LLM), the method introduces trainable speech modality-specific expert networks and employs a gating mechanism to dynamically fuse their outputs. This enables efficient adaptation of pre-trained textual knowledge to speech generation while preserving linguistic capability. Experiments on a custom out-of-domain test set demonstrate significant improvements in semantic alignment between generated speech and input descriptions, as well as enhanced naturalness—particularly for complex or unseen descriptions. The proposed approach substantially outperforms existing commercial TTS systems in accuracy under such challenging conditions. These results validate the effectiveness of modality-decoupled MoE for cross-modal generalization in descriptive TTS.

0 citationsRead paper

Skywork-R1V3 Technical Report

Jul 08, 2025

This work addresses the challenge of efficiently transferring the strong reasoning capabilities of pure-text large language models (LLMs) to vision-language tasks while achieving high-fidelity cross-modal alignment and multimodal reasoning. To this end, we introduce Skywork-R1V3—the first open-source vision-language model trained via reinforcement learning–based post-training (RLPT), enabling effective activation and transfer of textual reasoning abilities without additional pretraining. Key contributions include: (i) identifying the critical role of connector modules in cross-modal alignment; (ii) proposing an interpretable evaluation metric based on entropy of key reasoning tokens; and (iii) integrating curriculum learning with reinforcement fine-tuning. On the MMMU benchmark, the 38B-parameter model achieves an accuracy improvement from 64.3% to 76.0%, reaching human-entry-level performance, and demonstrates exceptional generalization on discipline-specific reasoning tasks—particularly mathematics—matching state-of-the-art closed-source models.

0 citationsRead paper

Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMs

Jun 23, 2025

Software engineering (SWE) datasets suffer from manual curation, high cost, and limited scale. Method: This paper proposes an incremental, automated data cleaning and generation pipeline that leverages real-world GitHub tasks to construct a high-quality SWE dataset comprising 10,169 Python tasks—supporting long-context and multi-turn interactive code generation. It innovatively integrates natural-language task descriptions with domain-specific execution environment images to enable fully automated unit test validation. Contribution/Results: Empirical analysis reveals a sustained scaling law for SWE capability with increasing data volume—no saturation observed—constituting the first such characterization in the field. The Skywork-SWE-32B model trained on this dataset achieves 38.0% pass@1 on SWE-bench Verified, improving to 47.0% under test-time scaling—outperforming all open-source models ≤32B and establishing a new state-of-the-art within the Qwen2.5-Coder-32B series.

0 citationsRead paper

CSVQA: A Chinese Multimodal Benchmark for Evaluating STEM Reasoning Capabilities of VLMs

May 30, 2025

Existing multimodal vision-language model (VLM) benchmarks lack rigorous evaluation of higher-order scientific reasoning—specifically, reasoning that integrates domain-specific knowledge with visual evidence in authentic STEM contexts. To address this gap, we introduce CSVQA, the first Chinese multimodal scientific reasoning benchmark grounded in STEM disciplines, comprising 1,378 expert-constructed question-answer pairs spanning physics, chemistry, and biology. CSVQA emphasizes domain-grounded reasoning through a novel, verifiable stepwise evaluation protocol: it incorporates human-authored questions, expert-generated chain-of-thought explanations, structured vision–text alignment annotations, and fine-grained stepwise plausibility assessment. Extensive evaluation across 15 state-of-the-art VLMs reveals severe limitations in scientific reasoning capability—the top-performing commercial model achieves only 49.6% accuracy—highlighting a critical bottleneck in current VLM development for scientific applications.

0 citationsRead paper

Recent publications

Latest Papers

MoE-TTS: Enhancing Out-of-Domain Text Understanding for Description-based TTS via Mixture-of-Experts

Aug 15, 2025

To address the poor generalization of descriptive text-to-speech (TTS) systems on out-of-domain user descriptions, this paper proposes a modality-oriented Mixture of Experts (MoE) architecture. Without fine-tuning the frozen pre-trained large language model (LLM), the method introduces trainable speech modality-specific expert networks and employs a gating mechanism to dynamically fuse their outputs. This enables efficient adaptation of pre-trained textual knowledge to speech generation while preserving linguistic capability. Experiments on a custom out-of-domain test set demonstrate significant improvements in semantic alignment between generated speech and input descriptions, as well as enhanced naturalness—particularly for complex or unseen descriptions. The proposed approach substantially outperforms existing commercial TTS systems in accuracy under such challenging conditions. These results validate the effectiveness of modality-decoupled MoE for cross-modal generalization in descriptive TTS.

0 citationsRead paper

Skywork-R1V3 Technical Report

Jul 08, 2025

This work addresses the challenge of efficiently transferring the strong reasoning capabilities of pure-text large language models (LLMs) to vision-language tasks while achieving high-fidelity cross-modal alignment and multimodal reasoning. To this end, we introduce Skywork-R1V3—the first open-source vision-language model trained via reinforcement learning–based post-training (RLPT), enabling effective activation and transfer of textual reasoning abilities without additional pretraining. Key contributions include: (i) identifying the critical role of connector modules in cross-modal alignment; (ii) proposing an interpretable evaluation metric based on entropy of key reasoning tokens; and (iii) integrating curriculum learning with reinforcement fine-tuning. On the MMMU benchmark, the 38B-parameter model achieves an accuracy improvement from 64.3% to 76.0%, reaching human-entry-level performance, and demonstrates exceptional generalization on discipline-specific reasoning tasks—particularly mathematics—matching state-of-the-art closed-source models.

0 citationsRead paper

Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMs

Jun 23, 2025

Software engineering (SWE) datasets suffer from manual curation, high cost, and limited scale. Method: This paper proposes an incremental, automated data cleaning and generation pipeline that leverages real-world GitHub tasks to construct a high-quality SWE dataset comprising 10,169 Python tasks—supporting long-context and multi-turn interactive code generation. It innovatively integrates natural-language task descriptions with domain-specific execution environment images to enable fully automated unit test validation. Contribution/Results: Empirical analysis reveals a sustained scaling law for SWE capability with increasing data volume—no saturation observed—constituting the first such characterization in the field. The Skywork-SWE-32B model trained on this dataset achieves 38.0% pass@1 on SWE-bench Verified, improving to 47.0% under test-time scaling—outperforming all open-source models ≤32B and establishing a new state-of-the-art within the Qwen2.5-Coder-32B series.

0 citationsRead paper

CSVQA: A Chinese Multimodal Benchmark for Evaluating STEM Reasoning Capabilities of VLMs

May 30, 2025

Existing multimodal vision-language model (VLM) benchmarks lack rigorous evaluation of higher-order scientific reasoning—specifically, reasoning that integrates domain-specific knowledge with visual evidence in authentic STEM contexts. To address this gap, we introduce CSVQA, the first Chinese multimodal scientific reasoning benchmark grounded in STEM disciplines, comprising 1,378 expert-constructed question-answer pairs spanning physics, chemistry, and biology. CSVQA emphasizes domain-grounded reasoning through a novel, verifiable stepwise evaluation protocol: it incorporates human-authored questions, expert-generated chain-of-thought explanations, structured vision–text alignment annotations, and fine-grained stepwise plausibility assessment. Extensive evaluation across 15 state-of-the-art VLMs reveals severe limitations in scientific reasoning capability—the top-performing commercial model achieves only 49.6% accuracy—highlighting a critical bottleneck in current VLM development for scientific applications.

0 citationsRead paper

Skywork Open Reasoner 1 Technical Report

May 28, 2025

To address the insufficient reasoning capability of large language models (LLMs) on long-chain-of-thought (CoT) tasks, this paper proposes a scalable reinforcement learning (RL) framework specifically optimized for extended CoT reasoning. Methodologically, it integrates Proximal Policy Optimization (PPO), multi-stage reward modeling, entropy regularization to mitigate entropy collapse, and CoT distillation augmented with large-scale synthetic reasoning data. Key contributions include: (i) the first demonstration of efficient, large-scale RL training for long-CoT models; (ii) open-sourcing of full-stack resources—including model weights, training code, and datasets; and (iii) the Skywork-OR1 series, which achieves state-of-the-art performance on AIME24/25 and LiveCodeBench, significantly outperforming DeepSeek-R1 and Qwen3-32B—yielding a +15.0% average accuracy gain for the 32B variant and establishing new SOTA for the 7B variant among models of comparable scale.

0 citationsRead paper