A Tool-Augmented, GPT-4 Chatbot for Real-Time Repository Data Analysis
为解决非技术用户难以访问软件仓库数据的问题,本文提出一种基于GPT-4的聊天机器人架构,通过解析查询、选择工具和生成详细回应来提高数据可访问性和分析准确性。
为解决非技术用户难以访问软件仓库数据的问题,本文提出一种基于GPT-4的聊天机器人架构,通过解析查询、选择工具和生成详细回应来提高数据可访问性和分析准确性。
针对利用Windows合法工具进行的APT攻击难以检测的问题,提出了SENTINEL架构,结合BERT语义编码、字符级CNN等方法有效提升了检测准确率。
本文通过构建包含3000个孟加拉语梗图的多模态数据集BanglaMemeX,旨在提升AI模型对文化隐喻图像的理解能力。
LLM-based IDE agents are increasingly used to generate repository-grounded unit tests, yet common evaluations often rely on execution success or coverage. These metrics can miss deeper quality issues such as weak assertions, missing edge cases, poor isolation, and limited maintainability. This paper presents VibeCheck, an empirical study of unit test generation across 15 student-developed Python and JavaScript/TypeScript repositories. We evaluate Kiro, Antigravity, and Cursor with Claude Sonnet 4.5 as the underlying agent, under repository-only, zero-shot conditions using a five-dimensional rubric covering runnability, assertion strength, logic and edge-case coverage, isolation/determinism, and maintainability. We also apply leave-one-out cross-agent peer evaluation to compare tools and identify failure patterns. Results show a clear execution-adequacy gap: generated tests are often runnable but frequently lack strong assertions and meaningful behavioral coverage. Weak assertions and missing edge cases occur more often than blocking failures, showing that runnable tests can still be shallow. VibeCheck provides a reliability-oriented framework for evaluating LLM-generated tests beyond pass/fail outcomes.
研究发现,在孟加拉湾海洋模拟器中,将飓风路径作为预设输入会因信号稀有导致模型表现下降,提出移除该输入以改善预测。
针对利用Windows合法工具进行的APT攻击难以检测的问题,提出了SENTINEL架构,结合BERT语义编码、字符级CNN等方法有效提升了检测准确率。
为解决非技术用户难以访问软件仓库数据的问题,本文提出一种基于GPT-4的聊天机器人架构,通过解析查询、选择工具和生成详细回应来提高数据可访问性和分析准确性。
本文通过构建包含3000个孟加拉语梗图的多模态数据集BanglaMemeX,旨在提升AI模型对文化隐喻图像的理解能力。
LLM-based IDE agents are increasingly used to generate repository-grounded unit tests, yet common evaluations often rely on execution success or coverage. These metrics can miss deeper quality issues such as weak assertions, missing edge cases, poor isolation, and limited maintainability. This paper presents VibeCheck, an empirical study of unit test generation across 15 student-developed Python and JavaScript/TypeScript repositories. We evaluate Kiro, Antigravity, and Cursor with Claude Sonnet 4.5 as the underlying agent, under repository-only, zero-shot conditions using a five-dimensional rubric covering runnability, assertion strength, logic and edge-case coverage, isolation/determinism, and maintainability. We also apply leave-one-out cross-agent peer evaluation to compare tools and identify failure patterns. Results show a clear execution-adequacy gap: generated tests are often runnable but frequently lack strong assertions and meaningful behavioral coverage. Weak assertions and missing edge cases occur more often than blocking failures, showing that runnable tests can still be shallow. VibeCheck provides a reliability-oriented framework for evaluating LLM-generated tests beyond pass/fail outcomes.
研究发现,在孟加拉湾海洋模拟器中,将飓风路径作为预设输入会因信号稀有导致模型表现下降,提出移除该输入以改善预测。