Multi-GPU, Multi-core, Multi-phase Lattice-Boltzmann Simulations of Fluid Flow for the Geosciences

Lattice-Boltzmann methods are a popular technique for modeling laminar to turbulent flow of complex fluids, most notably multiphase and multicomponent fluid flow that can include reactions, as well as fluid flow through complex domain geometries. The flexibility and relative ease-of-implementation of these methods makes them an attractive option for a range of computational fluid-dynamics problems found in the geosciences. Lattice-Boltzmann methods are, however, computationally demanding, particularly for geoscience problems spanning wide spatial and/or temporal scales. Fortunately, Lattice-Boltzmann methods are also excellent candidates for parallelization, being especially suited to implementation in single-instruction multiple-data (SIMD) environments. Examples of these programming environments can be found in accelerators such as standard graphics processing units (GPU) of modern graphics cards which can consist of hundreds of simple, but for Lattice-Boltzmann simulations sufficient, processors per GPU. During the past couple of years the barriers to adapting this technology to non-graphics applications have dropped dramatically, with the release of general purpose graphics programming environments such as BrookGPU, Stream, and CUDA. Using these new programming tools, several researchers have demonstrated that GPU-based lattice Boltzmann simulations can achieve dramatic performance gains (x20-x40) at low costs compared to their single-CPU counterparts. However, single graphics card implementations limit the size of lattices that can be simulated, and there remains a question as to what extent the increased performance can be sustained across multiple machines. In this presentation, we report on the performance of lattice-Boltzmann simulations in multi-GPU clusters employing CUDA as well as both OpenMP and MPI. In addition to standard single-phase lattice-Boltzmann models, we also discuss how these systems perform with more complex multiphase and multicomponent lattice-Boltzmann methods.