307
Chapter 23
High-Performance
Computing Methods at
France’s HydrOcean
Erwan Jacquin
23.1 INTRODUCTION
HydrOcean is a consultancy company and CFD solver (a mathematical
soware engine embedded in an application) license provider specializ-
ing in numerical simulation in uid dynamics. Composed of 25 high-level
engineers, HydrOcean provides design support to industry with the use of
innovative numerical simulation tools capable of accurately simulating the
most simple to the most complex hydrodynamic phenomena. HydrOcean’s
services enable clients to save time at the design stage, decrease research
costs, improve product performance, and reduce design risks.
HydrOcean uses and distributes a wide range of numerical tools for
the whole industry, with a dedicated choice of solvers for marine appli-
cations. Part of these tools or solvers are developed by HydrOcean in
partnership with the Laboratoire d’Hydrodynamique, d’Énergétique et
d’Environnement Atmosphérique (LHEEA) of École Centrale de Nantes
(UMR ECN/CNRS 6598). HydrOcean and the LHEEA have set up a
highly ecient research partnership, which aims at promoting the most
CONTENTS
23.1 Introduction 307
23.2 SPH-Flow Description 308
23.3 HPC Overview 308
23.4 Future Plans and Conclusions 310
308 ◾ Industrial Applications of High-Performance Computing
innovative research tools developed in their laboratory and also making
them accessible to industrial partners in the form of service provision or
through user licenses.
e quality of HydrOcean’s expertise is based on its solvers. e use of
high-performance research solvers specically provides innovative solu-
tions to customers, requiring huge computing power. On the other hand,
and simplifying to the extreme, the accuracy of the solvers is directly
related to the calculation cost. ese calculation costs can reach up to
thousands of processors for several days. erefore, high-performance
computing (HPC) activities at HydrOcean have three objectives:
1. To be able to simulate very large problems (several hundred million
points)
2. To reduce the calculation cost to popularize numerical simulations
3. To decrease the recovery time to make it compatible with industrial
processes.
23.2 SPH-FLOW DESCRIPTION
Among the solvers developed by HydrOcean, SPH-Flow solver is one the most
advanced in terms of HPC requirements. SPH-Flow implements the smoothed
particle hydrodynamics (SPH) method. It is developed 11years ago at LHEEA
and adopted by HydrOcean for the last 7 years. SPH uses a particle method
based on a Lagrangian approach without constant connectivity between the
elements. is approach is well suited for catching the free surface without any
specic extra work. at is why the method is well suited for dynamic ow,
with complex and/or deformable bodies, or with highly deformable inter-
faces that might involve free-surface fragmentation and reconnections. is
method aims at being a complement to well-established mesh-based methods
and actual commercial solvers (Fluent, CFX, StarCCM+, etc.). e scientic
and industrial world is also very interested in the SPH method gathered in
the SPHERIC (an international organization representing the community
of researchers and industrial users of SPH) group of ERCOFTAC (European
Research Community on Flow, Turbulence and Combustion).
23.3 HPC OVERVIEW
SPH methods suer from high-computational costs that increase dra-
matically in 3D engineering applications. To increase the industrial use
of SPH methods, large simulations must be solved in reasonable return
High-Performance Computing Methods at France’s HydrOcean ◾ 309
times. Simulations involving several million particles require important
computational resources and ecient parallelization.
With the help of the European FP7 NextMuSE project, addressing large
parallel SPH simulations and interactive-enhanced visualization, the
parallel eciency of SPH-Flow has been improved to reach high perfor-
mances on simulations involving up to 3 billion particles and running
on 32,768 cores. e parallelization strategy adopted takes advantage
of a dynamic domain decomposition in which each subdomain with its
underlying particles is assigned to a processor. Interactions between pro-
cessors are performed using non-blocking MPI (node-to-node Message
Passing Interface) communications. During the NextMuSE project, the
domain decomposition has been enhanced adopting an ORB (Orthogonal
Recursive Bisection) technique, and the MPI communication idle times
have been fully masked by optimizing the computation overlapping. e
ORB algorithm now enables the decomposition of billions of particles on
thousands of cores in only a few minutes, and recent performance tests
remove previous parallelization bottlenecks. e nal scalability study
performed on Switzerland’s ETH Zurich machine “Monte Rosa” led to
a parallel eciency larger than 90% and quasi-linear speedups on up to
32,768 cores (Figure 23.1).
35,000
10
9
Particles
10
8
Particles
30,000
25,000
20,000
15,000
10,000
5,000
0
0 5,000 10,000 15,000 20,000
Number of cores
Speedup
25,000 30,000
35,000
FIGURE 23.1 Speedups for 10 time-steps and one domain decomposition on a
3D dam break problem.
Get Industrial Applications of High-Performance Computing now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
