Institution profile

Korea University

Academic institutionasia · kr
Official website
Research library913linked papers
Opportunities0open roles
Selected work

Representative Papers

Too Many Frames, not all Useful: Efficient Strategies for Long-Form Video QA

Jun 13, 2024arXiv.org

Long-form video question answering (LVQA) suffers from high visual redundancy and sparse salient information, while existing methods inefficiently process uniformly sampled frames via independent vision-language model (VLM) descriptions, leading to poor semantic utilization. To address this, we propose the Hierarchical Keyframe Selector (HKFS), the first framework to jointly perform question-guided dynamic temporal segment localization and semantic keyframe selection. HKFS integrates multi-granularity temporal modeling, question-driven visual attention, and a lightweight VLM adaptation architecture—LVNet—to substantially reduce visual-language modeling overhead. Our approach achieves state-of-the-art performance on three major LVQA benchmarks—EgoSchema, NExT-QA, and IntentQA—and demonstrates strong generalization on VideoMME. Notably, it supports LVQA over videos up to one hour in length, enabling scalable, efficient, and semantically grounded long-video understanding.

12 citations2 influentialRead paper

SC-Rec: Enhancing Generative Retrieval with Self-Consistent Reranking for Sequential Recommendation

Aug 16, 2024arXiv.org

In generative recommendation, inconsistent outputs for identical user histories arise from discrepancies between prompt templates and item indexing schemes, limiting sequential recommendation performance. To address this, we propose a generative retrieval framework that jointly incorporates heterogeneous item indexing and multi-template prompting to leverage large language models (LLMs) for candidate generation. We further introduce the first self-consistency–based re-ranking mechanism for generative recommendation, which jointly models dual-path preferences—textual semantics and collaborative signals—via voting and confidence-weighted aggregation over multi-source LLM generations. Evaluated on three real-world datasets, our method significantly outperforms state-of-the-art approaches, achieving up to a 12.7% improvement in Recall@10. This work marks the first successful integration and trustworthy ranking of multi-source heterogeneous knowledge—spanning semantic and collaborative modalities—within a generative recommendation paradigm.

6 citations2 influentialRead paper

Manipulating Feature Visualizations with Gradient Slingshots

Jan 11, 2024arXiv.org

This work exposes a critical credibility vulnerability in feature visualization (FV) for deep neural network interpretability: FV outputs are susceptible to stealthy manipulation, leading to erroneous attribution of neuron semantics. To address this, we propose the first model-architecture-agnostic targeted FV manipulation method. Our approach integrates gradient redirection (via Slingshot optimization), adversarial latent-space perturbations, and neuron-activation-constrained regularization to achieve “semantic masking”—i.e., seamless substitution of a target neuron’s original FV explanation with an arbitrary user-specified semantic concept. Experiments across CNNs and Vision Transformers demonstrate successful concealment of functionally critical neurons: model accuracy degrades by less than 0.3%, yet FV-based auditing yields a 92% false-negative rate in detecting manipulated neurons. These results underscore the fragility of prevailing FV techniques and establish a new paradigm for robust model auditing and interpretability governance.

6 citationsRead paper

2DMamba: Efficient State Space Model for Image Representation with Applications on Giga-Pixel Whole Slide Image Classification

Dec 01, 2024arXiv.org

Modeling long-range 2D dependencies in gigapixel whole-slide image (WSI) classification remains challenging: existing Transformers suffer from quadratic complexity and spatial distortion due to 1D tokenization, while conventional 2D state space models (SSMs) incur prohibitive computational overhead. This paper introduces the first hardware-aware 2D selective SSM, which natively processes images in 2D via raster-scan order, enables efficient state propagation across rows and columns, and leverages custom GPU kernels for optimal throughput. The method preserves linear complexity and high parallelism while faithfully capturing intrinsic 2D continuity. Extending the Mamba architecture, it synergizes with VMamba for hierarchical visual representation learning. Evaluated on ten public WSI datasets, our approach achieves +2.48% AUC and +3.11% F1; it also improves mIoU by 0.5–0.7 on ADE20k and top-1 accuracy by 0.2% on ImageNet-1K.

2 citationsRead paper

Causal K-Means Clustering

May 05, 2024arXiv.org

This paper addresses the challenge of identifying heterogeneous causal effects when subgroup structures are unknown and difficult to estimate accurately. We propose “Causal k-Means Clustering”—the first extension of k-means to the counterfactual function space—to automatically discover subgroups exhibiting similar causal responses. Our method innovatively integrates plug-in estimation with double machine learning–based bias correction, ensuring √n-consistency and asymptotic normality of the estimator, while accommodating multiple treatment levels and partially observed outcomes. Theoretically, we derive the nonparametric efficiency bound and establish the semiparametric efficiency of our estimator. Empirically, we demonstrate its superiority over existing approaches in both simulation studies and an application to adolescent substance abuse intervention data. The proposed method thus offers strong theoretical guarantees—including optimality and robustness—alongside practical implementability for real-world causal heterogeneity analysis.

2 citationsRead paper
Recent publications

Latest Papers