M × N Communication and Parallel Interpolation in Community Climate System Model Version 3 Using the Model Coupling Toolkit

Abstract

The Model Coupling Toolkit (MCT) is a software library for constructing parallel coupled models from individual parallel models. MCT was created to address the challenges of creating a parallel coupler for the Community Climate System Model (CCSM). Each of the submodels that make up CCSM is a separate parallel application with its own domain decomposition, running on its own set of processors. This application contains multiple instances of the M × N problem, the problem of transferring data between two parallel programs running on disjoint sets of processors. CCSM also requires efficient data transfer to facilitate its interpolation algorithms. MCT was created as a generalized solution to handle these and other common functions in parallel coupled models. Here we describe MCT’s implementation of the data transfer infrastructure needed for a parallel coupled model. The performance of MCT scales satisfactorily as processors are added to the system. However, the types of decompositions used in the submodels can affect performance. MCT’s infrastructure provides a flexible and high-performing set of tools for enabling interoperability between parallel applications.

Keywords

Parallel coupling parallel communication climate modeling coupled models

References

Beckman, P. H. , Fasel, P. K. , and Humphrey, W. F. 1998. Efficient coupling of parallel applications Using PAWS . Proceedings of the 7th IEEE International Symposium on High Performance Distributed Computation, Chicago, IL, July.

Bettencourt, M. T. 2002. Distributed model coupling frame-work . Proceedings of the 11th IEEE Symposium on High Performance Distributed Computing, Edinburgh, UK, July, pp. 284-290 .

Bettge, T. , Craig, A. , James, R. , Wayland, V. , and Strand, G. 2001. The DOE Parallel Climate Model (PCM): the computational highway and backroads . Proceedings of the International Conference on Computational Science (ICCS) 2001, San Francisco, CA, May, Lecture Notes in Computer Science, Vol. 2073, Springer-Verlag, Berlin, pp. 148-156 .

Collins, W. D. et al. 2005. The Community Climate System Model: CCSM3 . Journal of Climate in press.

Craig, A. P. , Kaufmann, B. , Jacob, R. , Bettge, T. , Larson, J. , Ong, E. , Ding, C. , and He, H. 2005. Cpl6: the new extensible high performance parallel coupler for the Community Climate System Model . International Journal of High Performance Computing Applications 19(3).

Drummond, L. A. , Demmel, J. , Mechose, C. R. , Robinson, H. , Sklower, K. , and Spahr, J. A. 2001. A data broker for distributed computing environments . Proceedings of the 2001 International Conference on Computational Science (ICCS) 2001, San Francisco, CA, May, Lecture Notes in Computer Science, Vol. 2073, Springer-Verlag, Berlin, pp. 31-40 .

Edjlali, G. , Sussman, A. , and Saltz, J. 1997. Interoperability of data-parallel run-time Libraries . International Parallel Processing Symposium, Geneva, Switzerland.

Foster, I. and Kesselman, C. 1998. The GRID: Blueprint for a New Computing Infrastructure, Morgan Kaufmann, San Mateo, CA .

Geist, G. A. , Beguelin, A. , Dongarra, J. , Jiang, W. , Manchek, R. , and Sunderam, V. 1994. PVM: Parallel Virtual Machine. A User’s Guide and Tutorial for Networked Parallel Computing, MIT Press, Cambridge, MA .

10.

Geist, G. A. , Kohl, J. A. , and Papadopoulos, P. M. 1997. CUMULVS: providing fault tolerance, visualization and steering of parallel applications . International Journal of High Performance Computing Applications 11(3): 224-236 .

11.

Gropp, W. , Lusk, E. , and Skjellum, A. 1999. Using MPI: Portable Parallel Programming with the Message Passing Interface, 2nd edition, MIT Press, Cambridge, MA .

12.

Hill, C. , DeLuca, C. , Balaji, V. , Suarez, M. , da Silva, A. , and the ESMF Joint Specification Team . 2004. The architecture of the Earth System Modeling Framework . Computing in Science and Engineering 6: 18-28 .

13.

Jacob, R. , Schafer, C. , Foster, I. , Tobis, M. , and Anderson, J. 2001. Computational design and performance of the Fast Ocean Atmosphere Model . Proceedings of the International Conference on Computational Science (ICCS) 2001, San Francisco, CA, May, Lecture Notes in Computer Science, Vol. 2073, Springer-Verlag, Berlin, pp. 175-184 .

14.

Jiao, X. , Campbell, M. T. , and Heath, M. T. 2003. Roccom: an object-oriented, data centric software integration frame-work for multiphysics simulations . Proceedings of the 17th Annual ACM International Conference on Supercomputing, San Francisco, CA, June.

15.

Jones, P. W. 1998. A User’s Guide for SCRIP: A Spherical Coordinate Remapping and Interpolation Package, Los Alamos National Laboratory, Los Alamos, NM .

16.

Karonis, N. , Toonen, B. , and Foster, I. 2003. MPICH-G2: a grid-enabled implementation of the Message Passing Interface . Journal of Parallel and Distributed Computing 63(5): 551-563 .

17.

Keahey, K. and Gannon, D. 1997. PARDIS: a parallel approach to CORBA . Proceedings of the High Performance Distributed Computing Conference, Portland, OR, pp. 31-39 .

18.

Keahey, K. , Fasel, P. , and Mniszewski, S. 2001. PAWS: collective interactions and data transfers . Proceedings of the High Performance Distributed Computing Conference, San Francisco, CA.

19.

Larson, J. , Jacob, R. , and Ong, E. 2005. The Model Coupling Toolkit: a new Fortran90 toolkit for building multiphysics parallel coupled models . International Journal of High Performance Computing Applications 19(3).

20.

Lee, J. and Sussman, A. 2004. Efficient communication between parallel programs with InterComm. CS-TR-4557 and UMIACS-TR-2004-04, University of Maryland, Department of Computer Science and UMI-ACS .

21.

MPI Forum . 1994. MPI: a message passing interface standard . International Journal of Supercomputer Applications and High Performance Computing 8(3/4): 159-416 .

22.

Ong, E. , Larson, J. , and Jacob, R. 2002. A real application of the Model Coupling Toolkit . Proceedings of the International Conference on Computational Science, Amsterdam, the Netherlands, April, Lecture Notes in Computer Science, Vol. 2330, Springer-Verlag, Berlin, pp. 748-757 .

23.

Perez, C. , Priol, T. , and Ribes, A. 2003. A parallel COBRA component model for numerical code coupling . International Journal of High Performance Computing Applications 17(4): 417-429 .

24.

Ranganathan, M. , Acharya, A. , Edjlali, G. , Sussman, A. , and Saltz, J. 1996. Run-time coupling of data-parallel programs . Proceedings of the International Conference on Supercomputing, Philadelphia, PA.