llama

To address the challenge of deteriorating retrieval relevance and generation quality in RAG systems caused by continuous evolution of external data sources, this paper proposes RAGOps—the first operations paradigm spanning the full RAG lifecycle. Methodologically, it introduces a four-dimensional architectural view, a dual-track (data and model) lifecycle model, and a cross-stage quality trade-off framework; integrates the 4+1 architectural style, LLMOps practices, automated data evaluation, and retrieval-generation co-monitoring with feedback mechanisms. Contributions include: (i) systematic identification of six core operational challenges; (ii) establishment of reusable RAGOps design principles and an evaluation methodology; and (iii) empirical validation on two industrial-scale RAG applications, demonstrating a 3.2× improvement in responsiveness to data changes and a 27.6% increase in end-to-end output reliability.

This study addresses the critical gap in systematic evaluation of large language models (LLMs) against the OWASP Top 10 security risks specific to LLM applications. Leveraging the OWASP Top 10 for LLM Applications framework, the authors conduct the first comprehensive assessment of five standard Llama models and five Llama Guard variants. They construct a benchmark comprising 100 adversarial prompts spanning all ten vulnerability categories and evaluate threat detection accuracy, response safety, and computational overhead on the FABRIC platform using NVIDIA A30 GPUs. Results reveal that the lightweight specialized model Llama-Guard-3-1B achieves the best performance with a 76% detection rate and 0.165-second latency, whereas base models such as Llama-3.1-8B exhibit 0% detection, indicating an inverse relationship between model scale and security efficacy. The benchmark dataset, including adversarial prompts and attack metadata, is publicly released.

This work presents the first comprehensive synthesis of the technical foundations of the Meta Llama 4 model family, addressing the current lack of systematic documentation. It details core architectural innovations—including the routed/shared mixture-of-experts design, early-fusion multimodal integration, iRoPE-based long-context extension, and lightweight alignment strategies such as light supervised fine-tuning (SFT), online reinforcement learning (RL), and light direct preference optimization (DPO). The study integrates the complete training pipeline, evaluation results, and deployment constraints, offering an authoritative technical reference. Furthermore, it compiles performance benchmarks for both base and instruction-tuned variants across standard datasets and clarifies practical considerations for inference, including context-length limitations and quantization-aware deployment practices.

LLaMA 2 inference faces significant efficiency bottlenecks on resource-constrained devices—particularly Apple Silicon—due to suboptimal framework and language support. Method: We systematically benchmark ten programming languages and frameworks—including TensorFlow, PyTorch, Python, C++, Java, Rust, Zig, Go, Julia, and the emerging Mojo SDK—across inference latency, memory footprint, and development complexity. This constitutes the first comprehensive performance evaluation of Mojo against mainstream systems programming languages for LLM inference on the M1 Max platform. Results: Mojo SDK achieves C++-class inference latency while reducing memory overhead by 12–18% relative to C++ and other high-performance alternatives. Crucially, it retains Python-level usability and ecosystem compatibility without requiring manual memory management or low-level hardware abstraction. Our work empirically validates Mojo as a viable paradigm for efficient, lightweight, and deployable LLM inference on edge devices, providing both empirical evidence and practical engineering guidance for on-device large language model deployment.

Open-source large language models (LLMs) significantly underperform closed-source counterparts like GPT-4-Turbo in ultra-long-context understanding and retrieval-augmented generation (RAG). Method: We introduce a Llama 3.0-based LLM with native 128K-context support, proposing a novel three-stage joint training paradigm: (i) context window expansion (from 8K to 128K tokens), (ii) RAG-aware supervised fine-tuning, and (iii) multi-granularity long-text evaluation. Contribution/Results: Empirical results demonstrate that robust long-context capability enables RAG performance to consistently surpass pure long-context inference as the number of retrieved chunks increases. Our model achieves state-of-the-art results on benchmarks involving >100K-token documents and 4K-RAG tasks, outperforming GPT-4-Turbo, Qwen2-72B, and Llama-3.1-70B. To foster reproducibility and community advancement, we fully open-source the model weights, training code, and evaluation framework.

Deploying large language models in web browsers is constrained by limited memory and hardware heterogeneity, making it challenging to simultaneously achieve efficiency, privacy, and portability. This work addresses these limitations by introducing a novel WebGPU-based backend for llama.cpp, featuring static memory planning, templated GPU kernels, and multi-precision quantization support. Coupled with a cross-vendor device tuning mechanism, the proposed approach enables efficient and portable browser-side inference. Experimental evaluation across 16 heterogeneous devices demonstrates a 29–33% reduction in memory consumption and a 45–69% increase in decoding throughput compared to existing browser frameworks. In several scenarios, the performance even matches or surpasses that of native backends.

This study investigates the trade-offs among generation quality, inference latency, and computational resources in retrieval-augmented generation (RAG) systems. Leveraging a human-curated benchmark of 5,144 Kubernetes question-answer pairs and a hybrid retrieval pipeline—combining BGE-M3 dense and sparse vectors, reciprocal rank fusion (RRF), and cross-encoder reranking—the authors systematically evaluate 20 LoRA configurations on Llama-3.1-8B and Llama-3.2-3B. For the first time, they quantitatively characterize LoRA’s multi-dimensional trade-offs in RAG settings, demonstrating that LoRA variants adapting only the query and value projections consistently dominate the Pareto frontier, with this advantage stemming from architectural design rather than parameter count. Evaluation metrics include token-level F1, LLM-judged groundedness and correctness (pass@4), latency, memory usage, and training cost. The dataset, code, and optimized adapters are publicly released.

Large language model (LLM) training datasets are prohibitively large, largely inaccessible, and lack auditability, hindering transparency, safety evaluation, and reproducible research. Method: This work introduces the first publicly available offline indexing system for ultra-large-scale LLM training data. Built upon the Apertus dataset, it deploys and optimizes Elasticsearch on the ARM64-based green supercomputer Alps—marking the first such deployment on ARM64—and constructs a high-performance full-text index over 8.6 trillion tokens (56.6% of the 15.2-trillion-token corpus). Contribution/Results: The system serves dual purposes—as an open web search engine and an LLM safety auditing tool—enabling fine-grained, verifiable data provenance tracing and content inspection without requiring model jailbreaking. Key innovations include an ARM64-optimized distributed indexing architecture, an energy-efficient retrieval framework for trillion-token unstructured text, and the first publicly accessible, reproducible, and auditable infrastructure for large-scale LLM training data.

This work addresses two key challenges in large language model (LLM) research: the difficulty of training data provenance attribution and low efficiency in lossless text compression. We propose Llamazip—a unified, LLaMA3-based framework for joint lossless text compression and training set membership inference. Leveraging LLaMA3’s token-level predictive capability, Llamazip encodes only residual tokens that the model fails to predict accurately, achieving high compression efficiency. Crucially, the model’s per-token prediction confidence distribution serves as a discriminative signal for membership inference—marking the first direct use of LLM memorization for training set membership detection. Through quantized compression, context window optimization, and systematic evaluation, Llamazip achieves superior compression ratios over conventional algorithms on standard text corpora and attains high membership detection accuracy (AUC > 0.92). This work establishes a novel paradigm for enhancing model transparency, safeguarding data copyright, and enabling perception-aware LLM compression.

To address challenges in cloud-native communication/networking services—including ambiguous SLI/SLO definitions, high expertise barriers for specialized monitoring, and low cross-organizational trust in metrics—this paper proposes the first SRE platform integrating generative AI, federated learning, and blockchain. Methodologically, it introduces federated learning for collaborative, privacy-preserving SLI metric discovery across distributed environments; employs QLoRA-finetuned Llama-3-8B to enable intelligent, context-aware SLI/SLO generation; and leverages smart contracts and NFTs on-chain to immutably attest and audit metrics. The platform is compatible with Prometheus/Mimir, supports lightweight deployment, and was validated on Open5GS 5G core network, demonstrating effective automated SLO management. It simultaneously ensures data privacy, system transparency, and engineering practicality.