Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the real-world usage of ML models in 173 open-source GitHub projects across 16 application domains, focusing on decision-making scope, risk mitigation, and compliance with terms of use. The study finds evidence of potentially non-compliant ML automation, including applications in safety-critical contexts and violations of usage terms. This highlights the need for guidelines and automated tools to detect regulatory violations in ML-powered software.
Open-source projects are quietly deploying ML models in ways that may violate terms of use and even governmental regulations, raising serious questions about responsible AI adoption.
The increasing availability of Machine Learning (ML) models, particularly foundation models, enables their use across a range of downstream applications, from scenarios with missing data to safety-critical contexts. This, in principle, may contravene not only the models'terms of use, but also governmental principles and regulations. This paper presents a preliminary investigation into the use of ML models by 173 open-source projects on GitHub, spanning 16 application domains. We evaluate whether models are used to make decisions, the scope of these decisions, and whether any post-processing measures are taken to reduce the risks inherent in fully autonomous systems. Lastly, we investigate the models'compliance with established terms of use. This study lays the groundwork for defining guidelines for developers and creating analysis tools that automatically identify potential regulatory violations in the use of ML models in software systems.