Public values in public R&D through natural language processing.

Area of Science:

• Artificial Intelligence R&D Policy
• Societal Impact Assessment
• Public Value Evaluation

Background:

• South Korea faces R&D budget cuts, necessitating efficient policy management.
• Existing R&D evaluation methods lack systematic assessment of qualitative and societal impacts.
• A data-driven approach is needed to evaluate public R&D beyond output metrics.

Purpose of the Study:

• To introduce a novel framework for evaluating the social value of public R&D outputs using Public Value (PV).
• To integrate R&D outputs with external data using Natural Language Processing (NLP) and Deep Learning.
• To systematically capture and interpret the societal and economic impacts of public R&D in emerging technologies.

Main Methods:

• Utilized NLP for patent document parsing into subsections.
• Employed a Large Language Model (LLM) classifier to categorize segments into predefined Public Value types.
• Integrated 1642 patents from publicly funded AI R&D projects with 422 news columns for societal perspective.

Main Results:

• Extracted Public Value elements across six categories: industrial advancements, safe society, sustainable environment, job creation, human health, and convenience of life.
• Analyzed top five AI R&D programs based on patent counts.
• Demonstrated a novel, data-driven R&D policy evaluation approach using integrated data sources and advanced NLP/LLM.

Conclusions:

• The study presents a systematic, data-driven framework for evaluating societal impacts of public R&D.
• Findings provide evidence-based insights for R&D policy prioritization and resource allocation.
• This approach advances R&D policy management by enabling impact-oriented evaluation systems.