Managing Patient Use of Generative Health AI

AbstractPatients' use of artificial intelligence, or patient AI (PAI), is now widespread, promising both benefits and risks. There is an urgent need to assist patients with using these new technologies as safely and effectively as possible. Focusing on patients' use of large language models (LLMs) for health care purposes, this article explores critical issues in the management of generative PAI in the United States, including constitutional limits on government's regulatory authority, and identifies opportunities for public and private actors to help patients take advantage of generative PAI safely. With respect to the public sector, there is a critical need for federal action to fund research on the benefits and risks of PAI and on the development of valid metrics and performance standards for PAI. The federal government also needs to better protect the privacy and security of patient data shared with PAI, and to require transparency with respect to how LLMs used for health care purposes are funded, how they are developed, and how they perform. (Supported by the Commonwealth Fund.).