Data science for pattern recognition in agricultural large time series data: A case study on sugarcane sucrose yield

Area of Science:

• Agricultural Science
• Data Science
• Computational Statistics

Background:

• Data science (DS) offers versatile applications across disciplines, enabling process optimization and large-scale data analysis.
• DS integrates programming with mathematical and statistical tools in environments like Python, R, and Julia.
• Analyzing historical sugarcane production data is crucial for optimizing sucrose yield and understanding production dynamics.

Purpose of the Study:

• To propose and validate a data science protocol for organizing, visualizing, and analyzing historical sugarcane production data.
• To identify key patterns and variables associated with sucrose content in tropical sugarcane systems.
• To provide an alternative to traditional analytical methods in agriculture using a structured DS approach.

Main Methods:

• A four-phase protocol was implemented: data collection/organization, data management/cleaning, visualization, and multi-approach analysis (frequentist, regularized regression, machine learning).
• Python software and libraries (Pandas, Numpy, Scikit-learn, etc.) were utilized for automating data processing and analysis.
• Variable importance was assessed using Pearson correlation, exploratory analysis, and model fitting to eliminate non-informative parameters.

Main Results:

• The protocol successfully organized, visualized, and analyzed historical sugarcane data to reveal patterns related to sucrose.
• Climatic variables were identified as the most informative factors influencing sucrose content, while soil-related variables showed minor contributions.
• The analysis facilitated the elimination of obscuring variables, highlighting key drivers of sucrose behavior.

Conclusions:

• The proposed data science protocol offers a systematic and effective method for analyzing complex agricultural datasets.
• This approach provides valuable insights into sugarcane production systems, particularly regarding sucrose variability.
• The study demonstrates the potential of responsible data science implementation in the agricultural sector for enhanced decision-making.