Efficient use of binned data for imputing univariate time series data
Jay Darji1, Nupur Biswas1,2, Vijay Padul1
1Rhenix Lifesciences, Hyderabad, Telangana, India.
Frontiers in Big Data
|September 5, 2024
Summary
Imputing missing time series data using binned data significantly reduces errors, especially for longer gaps. This method improves accuracy for both biological and non-biological datasets.
Area of Science:
- Data Science
- Signal Processing
- Biomedical Informatics
Background:
- Time series data are prevalent across many sectors, but often suffer from missing values, impacting analysis.
- Existing imputation algorithms show varied performance, influenced by data characteristics and missing data patterns.
Purpose of the Study:
- To investigate the effectiveness of using binned time series data for imputation.
- To evaluate imputation performance across different missing data spans and algorithms, using heart rate and power consumption data.
Main Methods:
- Generated missing data in time series datasets (heart rate, power consumption) for various durations.
- Imputed missing data using different algorithms on binned data of varying sizes.
- Evaluated imputation performance using the root mean square error (RMSE) metric.
Main Results:
- Using binned data reduced RMSE compared to using the entire dataset, notably with the expectation-maximization (EM) algorithm.
- RMSE reduction was observed for 1-, 5-, and 15-minute missing data intervals, with greater improvements for longer gaps.
- Data fluctuation and sampling frequency influenced the effectiveness of binned data imputation.
Conclusions:
- The utility of binned data for imputation is contingent on the missing data span, data sampling frequency, and inherent data fluctuation.
- Binned data imputation is effective for diverse datasets, including IoT-derived biological data (smartwatch heart rate) and non-biological data (power consumption).
Related Concept Videos
00:57Modified Boxplots
9.2K
A standard box and whisker plot informs us about the spread of the data in a given sample. One can identify the minimum value, maximum value, first quartile value, second quartile or median value, and third quartile.
However, the box plot does not tell the reader about outliers - values that lie far from the center of the data. We can modify the standard box and whisker plot to identify the outliers and visualize the actual spread of the data in a sample.
Initially, we calculate the adjusted...
However, the box plot does not tell the reader about outliers - values that lie far from the center of the data. We can modify the standard box and whisker plot to identify the outliers and visualize the actual spread of the data in a sample.
Initially, we calculate the adjusted...
9.2K
01:19Survival Tree
73
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
73
00:54Time-Series Graph
4.3K
A time-series graph is a line graph with repeated measurements taken at successive intervals of time. It is also called a time series chart. To construct a time-series graph, one must look at both pieces of a paired data set. The horizontal axis is used to plot the time increments, and the vertical axis is used to plot the values of the variable that one is measuring. By using the axes in this way, each point on the graph will correspond to time and a measured quantity. The points on the graph...
4.3K
01:26Distributions to Estimate Population Parameter
4.0K
The accurate values of population parameters such as population proportion, population mean, and population standard deviation (or variance) are usually unknown. These are fixed values that can only be estimated from the data collected from the samples. The estimates of each of these parameters are sample proportion, the sample mean, and sample standard deviation (or variance). To obtain the values of these sample statistics, data are required that have particular distribution and central...
4.0K
01:05Noncompartmental Analysis: Mean Residence Time
119
According to statistical moment theory, mean residence time (MRT) is an important measure in pharmacokinetics. MRT can be defined as the expected mean of a probability density function distribution. It provides valuable insights into drug disposition in the body.
After the administration of a drug through intravenous bolus injection, the drug molecules are distributed throughout the body and remain there for varying periods. The MRT represents the average time these drug molecules stay in the...
After the administration of a drug through intravenous bolus injection, the drug molecules are distributed throughout the body and remain there for varying periods. The MRT represents the average time these drug molecules stay in the...
119
01:045-Number Summary
4.2K
In a dataset, the 5-number summary includes the minimum data value, the data value of the first quartile, the median data value or data value of the second quartile, the data value of the third quartile, and the maximum data value. These 5 data values can be visualized as a box and whisker plot.
In a box plot, the minimum and maximum data values represent the lower and upper whiskers in the graph, and the median is designated as the center of the box in the chart. The first quartile and third...
In a box plot, the minimum and maximum data values represent the lower and upper whiskers in the graph, and the median is designated as the center of the box in the chart. The first quartile and third...
4.2K