Many short traces are nearly as informative as one uninterrupted trace in recovering hidden-state kinetics

Hidden Markov models are widely used to infer state-to-state kinetics from noisy traces (i.e., time series), but many experiments do not produce one uninterrupted trace. In single-molecule fluorescence experiments, for example, fluorophores photobleach or molecules leave the observation volume, yielding multiple short traces. In addition, detectors introduce dead time, which leads to gaps and missed events. This naturally raises the question: to what degree can we learn about transition probabilities, and ultimately reaction kinetics, by jointly considering multiple independent traces as compared to a long uninterrupted trace. We consider multiple scenarios under the same observation data budget: uninterrupted traces, segmentations of those traces analyzed jointly, and independent traces generated from the same kinetics. In the cases considered here, we find only very modest changes in the estimated transition probabilities, justifying the use of many short, jointly analyzed traces with the same total observation budget as a long uninterrupted trace. We also consider best practices for dealing with missing data and find that when gaps in data are small, inaccuracies in the estimation of the transition probability can be mitigated by integrating the unobserved dynamics over the gap as compared to treating traces before and after the gap as independent. We tie our findings back to the notion of Shannon information and the number of bits gained on the distribution learned over the transition probability under consideration of the three scenarios described above.