Exploring Influence Factors on LLM Suitability for No-Code Development of End User IoT Applications

2025-05-07

AI Summary

Non-technical users face significant challenges in no-code IoT development due to inadequate support from large language models (LLMs) for natural language understanding, event-action logic generation, and dynamic code synthesis. Method: We systematically investigate determinants of LLM suitability through multi-model comparative experiments (Llama, GPT, Claude), prompt engineering analysis, and a novel automated evaluation framework for functional correctness and usability of IoT applications. Contribution/Results: We identify and empirically validate three decisive factors—LLM architecture, IoT-domain coverage of pretraining data, and instruction-tuning quality—and propose user-centric evaluation dimensions tailored to end-user IoT development. Results demonstrate that enhancing IoT-domain data coverage and instruction-tuning quality significantly improves accuracy in natural language interpretation, event-action mapping, and runtime code generation, thereby increasing application generation accuracy and user task success rate.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Application Domains: Internet of Things, Sensor Networks & Smart Cities

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

With the increasing popularity of IoT applications, end users demand more personalized and intuitive functionality. A major obstacle for this, however, is that custom IoT functionality today still requires at least some coding skills. To address this, no-code development platforms have been proposed as a solution for empowering non-technical users to create applications. However, such platforms still require a certain level of technical expertise for structuring process steps or defining event-action relations. The advent of LLMs can further enhance no-code platforms by enabling natural language-based interaction, automating of complex tasks, and dynamic code generation. By allowing users to describe their requirements in natural language, LLMs can significantly streamline no-code development. As LLMs vary in performance, architecture, training data used, and the use cases they target, it is still unclear which models are best suited and what are the influence factors determining this fit. In particular, no-code development of IoT applications by non-technical users will have completely different demands on LLMs than, e.g., code generation for more open-ended applications or for supporting professional developers. In this paper, we explore the factors influencing the suitability of LLMs to no-code development of IoT applications. We also examine the role of input prompt language on accuracy and quality of generated applications as well as the influence of LLM training data. By conducting comprehensive experiments with a range of LLMs, we provide valuable insights for optimizing LLM-powered no-code platforms, guiding the selection of the suitable LLMs and their effective application. Our findings contribute to improving the accessibility, efficiency, and user experience of no-code IoT development, ultimately enabling broader adoption of IoT technologies among non-expert users.

Problem

• Identify factors affecting LLM suitability for no-code IoT development
• Assess impact of prompt language on application accuracy and quality
• Evaluate influence of LLM training data on no-code platform performance

Innovation

• LLMs enable natural language-based IoT development
• Dynamic code generation automates complex tasks
• Comprehensive experiments guide optimal LLM selection