In 2009 I got the opportunity to take part in Google’s great Google Summer of Code:
Google Summer of Code is a global program that offers student developers stipends to write code for various open source software projects.
Blog posts concerning this project can be found in the GSoC category.
Now the project’s description follows (below you can find related links to get more information):
Distribution of computing jobs among different clouds (Nimbus, AWS)
— My Google Summer of Code 2009 project mentored by Kate Keahey (leading the Nimbus project at The Globus Alliance) —
The aim of this project is a scheduler for scientific computing jobs that uses resources from local, scientific and commercial computing clouds dynamically. Users should feel as few changes as possible, so the scheduler should be an extension of currently well-established job submission systems.
Within the “Amazon Web Services for ATLAS Computing” (AWSAC) project, until now a proof-of-principle job system was developed for ATLAS Computing (ATLAS is one of six experiments at LHC, CERN (Geneva)) using Amazon’s cloud. This system is formed by the AWSAC tools. The starting point for this GSoC project is the infrastructure the AWSAC tools deliver. The first step is to make the system compatible to Nimbus by completely redesigning it. After this, it will be possible to distribute jobs among different clouds.
Additionally, the implementation of the scheduler as a backend of the GANGA tool would be convenient. GANGA delivers a standardized job description language and is in active use by many ATLAS and LHCb (another experiment at LHC) users to submit their computing jobs.
Finally this new infrastructure has to be evaluated and fine-tuned concerning things like cost-effectiveness or maximum performance.
related: