Necessity is the mother of invention: a simple grid computing system using commodity tools

TitleNecessity is the mother of invention: a simple grid computing system using commodity tools
Publication TypeJournal Articles
Year of Publication2003
AuthorsMyers D.S, Cummings MP
JournalJ Parallel Distr ComJ Parallel Distr Com
Type of ArticleDOI 10.1016/S0743-7315(03)00004-2
KeywordsApache, computing, distributed, Grid, HTTP, java, Linux, Perl, SQL, Unix, XML-RPC

Access to sufficient resources is a barrier to scientific progress for many researchers facing large computational problems. Gaining access to large-scale resources (i.e., university-wide or federally supported computer centers) can be difficult, given their limited availability, particular architectures, and request/review/approval cycles. Simultaneously, researchers often find themselves with access to workstations and older clusters overlooked by their owners in favor of newer hardware. Software to tie these resources into a coherent Grid, however, has been problematic. Here, we describe our experiences building a Grid computing system to conduct a large-scale simulation study using "borrowed" computing resources distributed over a wide area. Using standard software components, we have produced a Grid computing system capable of coupling several hundred processors spanning multiple continents and administrative domains. We believe that this system fills an important niche between a closely coupled local system and a heavyweight, highly customized wide area system. (C) 2003 Elsevier Science (USA). All rights reserved.