Complexity-entropy analysis at different levels of organisation in written language

Area of Science:

• Computational Linguistics
• Information Theory
• Bioinformatics

Background:

• Written language presents a complex system constrained by rules, context, and redundancy.
• Unpredictability is a crucial characteristic of natural language, enabling effective communication.

Purpose of the Study:

• To introduce and demonstrate the application of entropic measures for analyzing written text.
• To quantify the balance between predictability and surprise within textual data.
• To explore the generalizability of this analytical approach to other complex systems.

Main Methods:

• Utilizing entropic measures to assess the interplay of predictability and surprise.
• Analyzing textual data at different hierarchical levels of organization.
• Applying the methodology to written language and biological sequences (DNA).

Main Results:

• Demonstrated that entropic measures can effectively quantify innovation and context preservation in documents.
• Showcased the ability to analyze this balance across multiple organizational levels within a text.
• Established the potential applicability of the method to other complex message systems like DNA.

Conclusions:

• Entropic measures provide a robust framework for understanding the dynamics of complex communication systems.
• The balance between predictability and surprise is a key feature measurable in language and potentially other information-rich structures.
• This approach offers a novel perspective for analyzing information content and structure in diverse fields.