Can LLMs Use Relational Transformer Embeddings?
研究通过将关系变换器嵌入注入大型语言模型来处理多表结构和语言推理问题，但实验结果表明该方法不稳定且表现不佳。
0 citationsRead paper
Selected work
Representative Papers
Context Window Failures in Relational Foundation Models
研究针对关系型基础模型处理高基数数据时的窗口限制问题，通过引入合成金融数据集Animus并采用时间预聚合方法，显著提高了预测性能。
0 citationsRead paper
Latent Fact-Checking: Detecting Misinformation through Activation Engineering
Online misinformation detection faces challenges in scalability and reliance on external knowledge. This work proposes a novel method that requires no fine-tuning, retrieval, or task-specific supervision, treating truthfulness as a geometric property within the representation space of pretrained language models. By contrasting activation patterns between true and false statements, the approach identifies a “falsehood direction” in the residual stream and classifies inputs via projection of their final-layer activations. Combining contrastive activation addition (CAA) with an MLP classifier, the method demonstrates strong performance across mainstream architectures—including Gemma, Llama, and Qwen—matching or surpassing zero- and few-shot prompting on benchmarks LIAR and FACTors, with particularly notable gains for smaller models. Its performance is limited on AVeriTeC, which relies on annotated evidence, underscoring the method’s paradigm of evidence-free detection.
0 citationsRead paper
Recent publications
Latest Papers
Can LLMs Use Relational Transformer Embeddings?
研究通过将关系变换器嵌入注入大型语言模型来处理多表结构和语言推理问题，但实验结果表明该方法不稳定且表现不佳。
0 citationsRead paper
Context Window Failures in Relational Foundation Models
研究针对关系型基础模型处理高基数数据时的窗口限制问题，通过引入合成金融数据集Animus并采用时间预聚合方法，显著提高了预测性能。
0 citationsRead paper
Latent Fact-Checking: Detecting Misinformation through Activation Engineering
Online misinformation detection faces challenges in scalability and reliance on external knowledge. This work proposes a novel method that requires no fine-tuning, retrieval, or task-specific supervision, treating truthfulness as a geometric property within the representation space of pretrained language models. By contrasting activation patterns between true and false statements, the approach identifies a “falsehood direction” in the residual stream and classifies inputs via projection of their final-layer activations. Combining contrastive activation addition (CAA) with an MLP classifier, the method demonstrates strong performance across mainstream architectures—including Gemma, Llama, and Qwen—matching or surpassing zero- and few-shot prompting on benchmarks LIAR and FACTors, with particularly notable gains for smaller models. Its performance is limited on AVeriTeC, which relies on annotated evidence, underscoring the method’s paradigm of evidence-free detection.
0 citationsRead paper