A New Design of RDMA-based Small Message Channels for InfiniBand ClustersXin Yuan, Professor, Florida State University, Department of Computer Science, Nov 28, 2012
This talk has two independent parts. The first part is on a new design of RDMA-based small message channels for InfiniBand clusters that improves over the MVAPICH RDMA-based small message channels. I will first describe a technique that eliminates persistent buffer association, a scheme used in MVAPICH that not only results in significant memory requirement, but also imposes restrictions in memory management. Building upon this technique, we develop a novel shared RDMA-based small message channel design that allows MPI processes on the same SMP node to share small message channels, which greatly reduces the number of small message channels needed for an MPI program on clusters with SMP nodes. Our techniques considerably improve the scalability and reduce memory requirement in comparison to MVAPICH, allowing RDMA-based small message channels to be used by a much larger number of MPI processes. The experimental results demonstrate that our techniques achieve the improvements without adding noticeable overheads or sacrificing the performance benefits of RDMA.