Off-chip prefetching based on Hidden Markov Model for non-volatile memory architectures.

Area of Science:

• Computer Architecture
• Memory Systems
• Performance Optimization

Background:

• Non-volatile memory (NVM) in commodity hardware offers backup memory for DRAM caches without software changes.
• NVM's higher read/write latencies can worsen the memory wall problem.
• Complex off-chip memory access patterns in multicore processors pose performance challenges.

Purpose of the Study:

• To address the latency issues of NVM in external DRAM caches.
• To develop a novel prefetch technique for complex off-chip memory access patterns.
• To improve the efficiency of multicore memory systems.

Main Methods:

• Analysis of off-chip memory access patterns in multicore processors.
• Proposal of a Hidden Markov Model (HMM)-based prefetching module in the last-level cache (LLC).
• Implementation of a prefetching module with linear computational complexity relative to the number of threads.

Main Results:

• The HMM-based technique effectively tracks and clusters simultaneous memory access groups from multiple threads.
• Accurate identification of complex address groups and high-accuracy prefetching.
• Up to 76% improvement in off-chip DRAM cache hit ratio for multicore architectures.
• Reduced prefetch request overhead: 48% in single-core and 83% in multicore simulations.

Conclusions:

• The proposed HMM-based prefetching technique significantly enhances off-chip DRAM cache performance in multicore systems.
• This approach effectively mitigates latency problems associated with NVM and complex memory access patterns.
• The technique offers a practical solution for optimizing memory hierarchy performance without software modification.