Google Summer of Code end: code upload and acknowledgement

The Google Summer of Code 2009 final evaluation deadline is today; 19 UTC. I don’t have time to summarize my summer here now, but there are two things I want to say to the world. First, I want to thank many people for enriching my summer. Second, I would like to announce the Clobi project on Google Code.
  »» Continue reading »»

new system successfully tested:

Hello you out there!

I just started running the first serious test of the system I’ve developed during this year’s Google Summer of Code. If I wanted to put it in sensational words, the test could be called “Distribution of Particle Physics High Performance Computing Jobs among Multiple Computing Clouds”; just to get some readers :-) . During the test, there will be some time I just sit around and watch my monitor, so I decided to share my experience about the new system with you and keep record of the test progress within this blog post.
  »» Continue reading »»

“What’s faster–a supercomputer or EC2?” A reconsideration…

In his blog, Ian Foster recently presented What’s faster — a supercomputer or EC2?.
  »» Continue reading »»

Counting stuff in Python and assembling a histogram: analyze thread communication via os.pipe()

These days I built up an inter-thread communication via os.pipe(). While one thread is only writing to the “write end” of the pipe, the other thread is only reading from the “read end”.
  »» Continue reading »»

CernVM: local ATLAS Software -- the clean solution

In my blog post CernVM: how to set up a local ATLAS Software Release, I presented a brutal approach how to override CVMFS (CernVM‘s filesystem with HTTP backend) to install a local ATLAS Software release. Now I worked out a very clean and smooth solution. This approach allows:

  • to use the local ATLAS Software without “hacking” anything
  • to use local software and software provided by CVMFS at the same time.


  »» Continue reading »»

ATLAS Software: How to run The Full Chain

During development, a system running ATLAS Software has to be tested and validated. There are some standard tests that almost don’t need any input data, stress the system and — if they run properly — are a (very) good indicator that everything is set up correctly. I talk about the so-called JobTransforms. By combining these JobTransforms, the so-called Full Chain can be run — a convenient test. In this blog post I summarize what’s behind the Full Chain and provide a shell script to easily run it.
  »» Continue reading »»

EC2: Install ATLAS Software to an EBS volume

In CernVM: local ATLAS Software — the clean solution I proposed to install an ATLAS Software release to an EBS volume. I did it to make it locally available in CernVM running on EC2. The approach allows to move an EBS volume from one EC2 instance to another without losing important components and functionality. Here are some hints (both, CernVM specific and general) to follow before installing the software via pacman.
  »» Continue reading »»

Amazon Web Services: about REST API, Query API, SOAP API and possible inconsistencies

In this blog post I dedicate myself to the definitions and differences of SOAP, Query and REST API you can use to manage Amazon Web Services. By reading a good book, I could solve some confusion and find answers on upcoming questions. It turns out that Amazon does not use consistent API terms. I would like to share this information.
  »» Continue reading »»

CernVM: how to set up a local ATLAS Software Release (dirty version)

One of the main features of CernVM is its special filesystem CVMFS with http backend (based on FUSE). Using CernVM in the standard way, the different experiment softwares work out-of-the-box and are made accessible over the web via CVMFS. Although this is a great feature, I like to set up an ATLAS Software release locally — as real offline version — to be independent of the software-providing webservers.
  »» Continue reading »»

Problems at Boston University's ATLAS Software mirror

While installing an ATLAS Software release from Boston University’s mirror, I discovered a broken archive file. It was the fault of a bad network card. The mirror had to be rebuilt from scratch.
  »» Continue reading »»