State-wise linear modulation (SLim): A novel approach for steering large language models.

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Human-Computer Interaction

Background:

• Large Language Models (LLMs) show advanced language capabilities but lack affective adaptability for human-AI interaction.
• Current prompt engineering methods for steering LLMs have limitations in reliability and scope.
• Effective human-AI interaction requires AI systems to be adaptable on affective and emotional levels.

Purpose of the Study:

• To investigate methods for guiding LLM output characteristics by modifying internal model states.
• To enhance LLM adaptability for more nuanced human-AI interactions beyond lexical generation.

Main Methods:

• Experiments were conducted using datasets labeled with sentiment, emotion, and stylistic attributes.
• Specific state or style-based vectors were used to scale and shift LLM activations.
• LLM outputs were analyzed for alignment with target attributes based on internal state modifications.

Main Results:

• Modifying internal LLM states effectively guides output characteristics to align with specified attributes like sentiment and emotion.
• State-engineered outputs showed more pronounced effects when prompted with subjective inputs compared to factual ones.
• This approach offers a new method for controlling LLM output beyond traditional prompt engineering.

Conclusions:

• Internal state manipulation provides a promising direction for enhancing LLM control and adaptability.
• This research bridges the gap between current LLM capabilities and the requirements for sophisticated human-AI interaction.
• Future work can explore further applications of state-based control for affective AI systems.