Tue 24th of October 2006

Presentation
The MUMPS team organizes from time to time the MUMPS User Days.
This first MUMPS User Day in 2006 has been an opportunity for users to discuss their use of the MUMPS package and their requirements for the future. It was also of interest for people concerned with largescale simulations. Part of the day has been dedicated to presentations by users from various application fields, while another part focused on research on parallel sparse direct methods and on recent and planned functionalities of the MUMPS solver (e.g., outofcore execution, parallel analysis, processing singular matrices). It was also a good occasion for informal exchanges and discussions between users and developers of MUMPS.
Funding
Organisation
Program
08.30  08.55  Registration and Coffee  
08.55  09.00  Welcome and presentation of the day (Aurélia Fèvre)  
09.00  09.20 
(MUMPS team) Short presentation of MUMPS (History, Users, Functionalities) 

09.20  09.50 
Stéphane Pralet and JeanPierre Delsemme,
SAMTECH, BELGIUM Integration of MUMPS in SAMCEF Mecano 

09.50  10.20 
Stéphane Operto,
GEOAZUR, FRANCE
Seismic wave propagation modeling using a frequencydomain finite difference method: Application to seismic imaging 

10.20  10.50  Coffee Break  
10.50  11.10 
(MUMPS team) Controlling MUMPS accuracy and efficiency 

11.10  11.30 
(MUMPS team) Future functionalities and ongoing projetcs 

11.30  11.55 
Emmanuel Agullo, PhD student, ENS Lyon OutOfCore parallel factorization 

11.55  12.20 
Tzvetomila Slavova, PhD student, CERFACS OutOfCore parallel solution 

LUNCH 


13.50  14.20 
Guillaume Sylvand, EADS, FRANCE Simulation in electromagnetism at EADSCRC using MUMPS for coupled BEM/FEM 

14.20  14.50 
Ken Stanley, Interactive Supercomputing, Sandia National Labs, USA Power to the People: Bringing MUMPS to the masses 

14.50  15.20 
Hong Zhang, Dept. of Computer Science, Illinois Institute of Technology and
Mathematics and Computer Science Division, Argonne National Laboratory Design, Implementation and Applications of PETScMUMPS Interface 

15.20  15.50  Coffee Break  
15.50  16.20 
(MUMPS team) Parallelism in MUMPS 

16.20  16.50 
Luc Giraud, ENSEEIHTIRIT, FRANCE From direct to iterative substructuring: some parallel experiences in 2 and 3D 

16.50  17.50  General Discussion  
19.30  Dinner in Lyon 
Abstracts of users' and applications presentations
Integration of MUMPS in SAMCEF Mecano (Stéphane Pralet)
SAMCEF Mecano is general finite element software that solves nonlinear structural and mechanical problems. It provides specific answers regarding your analysis: MECANO Structure, for the nonlinear analysis of structures, MECANO Motion, for the analysis of flexible mechanisms and MECANO Cable, for the analysis of cables subjected to electrodynamic efforts. A NewtonRaphson strategy is used in order to solve nonlinear equations in an implicit scheme. At a given Newton iteration, a linear system of equations has thus to be solved. The 2 main linear solvers available in Mecano are BCSLIB (sequential multifrontal approach) and MUMPS (parallel multifrontal approach).
In the first part of our talk we will present the evolution of the interface between Mumps and Mecano. In a first implementation the linear solver and the problem generation were running on separate processes and data were exchanged through files. This interface has been substituted by a tighter approach where Mumps and Mecano use the same MPI processes and where data are exchanged through the main memory.
In the second part of our talk we will illustrate MUMPS advantages and we will discuss possible improvements. To our mind the strength of MUMPS is based on the alliance of a competitive parallel distributed factorization and of the availability of advanced functionalities. To show MUMPS efficiency, we will compare it with BCSLIB in the context of large test cases arising from mechanical engineering. To give an example of use of advanced functionalities, we will describe how the Schur complement feature is exploited to solve contact problems.
Seismic wave propagation modeling using a frequencydomain finite difference
method: Application to seismic imaging (Stéphane Operto)
[Slides available (.ppt.gz)]
Seismic methods are among the most powerful approaches to image the structure of the earth at different scales (from subsurface to the deep crust). Artificial sources such as explosions are used to propagate elastic waves into the basement. A receiver array records the seismic arrivals which carry indirect information on some physical properties of the basement (wave velocities, density, attenuation, anisotropy). The objective of seismic imaging is to image these physical properties (the model) from the seismic recordings (the data) using an optimization procedure (non linear inverse problem).
The non linear relationship between the seismic data and the physical properties of the basement is described by the wave equation whose heterogeneous coefficients are the medium properties. A key issue is to use an efficient tool for the numerical resolution of the wave equation. For this, we use a finitedifference frequencydomain (FDFD) method. The associated numerical problem is the resolution of a sparse system of linear equations with multiple righthand side (RHS) terms. Each RHS term correspond to a source of the experiment, the matrix coefficients depend on the medium properties and the solutions are the modeled wavefields. The rank of the matrix is the product of the dimensions of the basement model. A specificity of our applications is the large number of RHS terms which directs us towards direct methods since the matrix factorization is independent of the RHS terms. We use MUMPS on a PC cluster to solve this system of equations with multiple RHS terms.
Several applications were finalized in 2D. The factorization and the multiple resolution phases are computed in parallel with MUMPS. We exploit the distribution of the system solutions on several processors to compute in parallel the subsequent inverse problem (which is implemented as a weighted summation over the system solutions).Abstract Presently, we are developing a 3D FDFD code for seismic wave propagation modeling based on MUMPS to assess whether representative problems can be adressed with a direct solver keeping in mind that the memory requirement will be the key issue in 3D.
Simulation in electromagnetism at EADSCRC using MUMPS for coupled BEM/FEM (Guillaume Sylvand)
We are interested in the numerical simulation of wave propagations in electromagnetism governed by the Maxwell equations. We have to solve this kind of problem in fields such as stealth conception, antenna design and placement, etc. More precisely, we choose to solve these equations in the frequency domain and in integral form, which results in having to solve a full complex linear system. The default solver for these equations is a direct Cholesky algorithm, but iterative solvers and FMM method are available to speedup the resolution for large systems.
However, the BEM can not simulate heterogeneous propagation media which are sometime used for realistic industrial cases. We have chosen to treat these parts of the mesh with a volumic finite element method based on a tetrahedral meshing. The system resulting of this coupling scheme is a 2x2 linear system, with one full block (on the boundary) and three sparse block. Different solvers have been tested on this system (direct, iterative, Schur complement) using MUMPS whenever the inversion of a large sparse matrix is required. Problems with several millions of unknowns (surfacic + volumic) have been solved on parallel machines.
Power to the People: Bringing MUMPS to the masses (Ken Stanley)
MUMPS is a terrific solver and can harness the power of distributed computers to solve very large sparse matrices. Programming for distributed computing, however, continues to be a timeconsuming and hence expensive task. Amesos and StarP provide interfaces which greatly reduce the cost of programming for distributed computing, making MUMPS attractive to a larger customer base.
Amesos is an objectoriented interface to sparse direct solvers within the Trilinos package. Amesos allows users easy access to several solvers and to switch solvers by changing only a couple characters. Trilinos allows users to develop robust numerical software using modern obejctoriented techniques while leveraging the value of established numerical libraries.
StarP allows matlab users to harness the power of distributed computing with virtually no development effort and allows users to go straight from prototype to prodcution codes.
I will describe the Amesos and StarP and discuss how they make MUMPS easier to use.
Design, Implementation and Applications of PETScMUMPS Interface (Hong Zhang)
[Slides available (.ppt.gz)]
PETSc (Portable, Extensible Toolkit for Scientific Computation) is a suite of data structures and routines for the scalable solution of scientific applications modeled by partial differential equations. MUMPS is a highperformance direct sparse solver. The PETScMUMPS interface enables our users to easily invoke the MUMPS solver at runtime for either algorithmic study or solving computationalintensive problems under the PETSc environment.
In this talk, first, I will present the design and implementation of the PETScMUMPS interface. I will then demonstrate how MUMPS is used on solving largescale eigenvalue problems arising from the nanoscale materials simulation. Finally, I will discuss what improvements can be envisaged. If time permits, I will give a short demo on using PETSc MUMPS interface.
From direct to iterative substructuring: some parallel experiences in 2 and 3D
(Luc Giraud)
[Slides available (.pdf)]
One route to the solution of large sparse linear systems in parallel scientific computing is the use of hybrid methods that combine direct and iterative methods. These techniques inherit the advantages of each approach, namely the limited amount of memory and easy parallelization for the iterative component and the numerical robustness of the direct part.
In this talk, we will consider some parallel solution techniques for the solution of linear system arising from the discretization of PDEs. These approaches are often called the substructuring methods, referring to structural mechanics, historically its first area of application. It consists in splitting the mesh into nonoverlapping submeshes/subdomains (or equivalently splitting the graph of the matrix), first eliminating the unknowns associated with the interior nodes from the equations associated with the nodes on the interfaces, then solving the interface problem and finally solving back the internal problems on each subdomain. The solution of the interface problem, also referred to as condensed system, can be performed using either a factorization approach or a preconditioned Krylov solver.
We will present, how some of the uniques features of MUMPS enable us to implements these schemes on parallel platforms. Numerical experiments and parallel performances will be reported for the solution on 2D problems arising in semiconductor device modelling simulation. Preliminary results on 3D heterogeneous diffusion problems will also be described.
MUMPS team presentations
Short presentation of MUMPS (History, Users, Functionalities)
General introduction and presentation of the MUMPS package.
Controlling MUMPS accuracy and efficiency
The objective of this presentation is to present the main control parameters of MUMPS and explain how default parameters can be modified to solve a numerically difficult problem more accurately, solve a huge problem more efficiently, or adapt the use of MUMPS to a particular situation resulting from the application context. The various functionalities and the effects of the different control parameters will be illustrated on practical examples.
Future functionalities and ongoing projects
We will present the ongoing projects and collaborations, as well as the functionalities that we plan to develop in the next few years. These developments and research work result from common needs and/or limits observed by our users when using the package in various situations.
OutofCore parallel factorization and solution
Memory is usually the main bottleneck to solve arbitrarily large problems with a direct method. Because demands from numerical simulation lead to huge matrices for which the physical memory of modern computers is not sufficient, we have developed a beta version of an outofcore extension to MUMPS that manages the factors outofcore. Two PhD students, Emmanuel Agullo (LIPENS Lyon) and Tzvetomila Slavova (CERFACS) are currently working on research issues on the outofcore factorization phase and the outofcore solution phase, respectively. The objectives are to further reduce the incore working space of the factorization phase and improve the performance of the outofcore solution phase.
Parallelism in MUMPS
In the parallel context, MUMPS uses a combination of static and dynamic scheduling techniques to map the computationnal tasks over the working processors. In this context, it is crucial to take into account various constraints not only linked to performance but also to memory requirements or volume of I/O (in the outofcore context). We will describe different ways of managing parallelism and ongoing work in this topic (dynamic scheduling strategies, sophisticated static techniques, I/O constrained schedulers, ...)