****** processjobs.py start Mon Dec 15 16:49:20 2008 ****** # parsing the sessioninfofile /mnt/awsac/sessioninfo/awsac.sessinfo ... Session ID: 081215_1646--eventgen--1816 ; CoresPerInstance: 2 # getting information about this instance from EC2... the ami-launch-index of this instance (i-32f64f5b) is 1 the Availability Zone of this instance is us-east-1b # parsing the jobsconfigfile /mnt/awsac/jobs.cfg ... # jobsdatadicts_list read from /mnt/awsac/jobs.cfg # my job numbers are: [3, 4] [{'jobnr': '3', 'snap_id': 'snap-9fd433f6', 'shscript': '/mnt/awsac/gen_1000.sh'}, {'jobnr': '4', 'snap_id': 'snap-9fd433f6', 'shscript': '/mnt/awsac/gen_1000.sh'}] # prepare the creation of EBS volume(s) from the needed ATLAS Release snapshot(s)... detected following different snap_ids: ['snap-9fd433f6'] planing to assign snap_id(s) to following device(s): snap-9fd433f6->/dev/sdh1 # create EBS volume(s)... instructed EC2 to create EBS from snapshot snap-9fd433f6 status: creating status: available # attach EBS volume(s)... instructed EC2 to attach volume vol-95e400fc to /dev/sdh1 status: attaching status: attached # mount EBS volume(s)... invoke mounting: subprocess.Popen() with args ['mount', '/dev/sdh1', '/mnt/atlas'] mount subprocess ended. returncode: 0 # this is the list of successfully mounted EBS - snapshots: ['snap-9fd433f6'] # the following jobs now have their desired EBS running in system: [{'jobnr': '3', 'snap_id': 'snap-9fd433f6', 'shscript': '/mnt/awsac/gen_1000.sh'}, {'jobnr': '4', 'snap_id': 'snap-9fd433f6', 'shscript': '/mnt/awsac/gen_1000.sh'}] # initialize running jobs... creating SimpleDB jobitems... # preparing job 3... cwd for job: /mnt/atlasworkarea/3 (created) calling subprocess.Popen() with args: ['/bin/sh', '/mnt/awsac/gen_1000.sh'] runstart: updating sDB item job3 # preparing job 4... cwd for job: /mnt/atlasworkarea/4 (created) calling subprocess.Popen() with args: ['/bin/sh', '/mnt/awsac/gen_1000.sh'] runstart: updating sDB item job4 # wait for subprocesses to finish... # subprocess for job 4 ended. returncode: 0 runend: updating sDB item job4 savestart: updating sDB item job4 found /mnt/atlasworkarea/4/results.tar.bz2 (251734 Byte) 08-12-15 16:51:11: start upload. bucket:atlassessions; key:081215_1646--eventgen--1816/results_job_4.tar.bz2 08-12-15 16:51:12: finished. outputfile of job 4 found: /mnt/atlasworkarea/4/stdouterr_job_4.log bundled /mnt/atlasworkarea/4/stdouterr_job_4.log to /mnt/atlasworkarea/4/stdouterr_job_4.tar.bz2 planing to upload /mnt/atlasworkarea/4/stdouterr_job_4.tar.bz2 (307 Byte)... 08-12-15 16:51:12: start upload. bucket:atlassessions; key:081215_1646--eventgen--1816/stdouterr_job_4.tar.bz2 08-12-15 16:51:12: finished. saveend: updating sDB item job4 # subprocess for job 3 ended. returncode: 0 runend: updating sDB item job3 savestart: updating sDB item job3 found /mnt/atlasworkarea/3/results.tar.bz2 (249481 Byte) 08-12-15 16:51:29: start upload. bucket:atlassessions; key:081215_1646--eventgen--1816/results_job_3.tar.bz2 08-12-15 16:51:29: finished. outputfile of job 3 found: /mnt/atlasworkarea/3/stdouterr_job_3.log bundled /mnt/atlasworkarea/3/stdouterr_job_3.log to /mnt/atlasworkarea/3/stdouterr_job_3.tar.bz2 planing to upload /mnt/atlasworkarea/3/stdouterr_job_3.tar.bz2 (317 Byte)... 08-12-15 16:51:29: start upload. bucket:atlassessions; key:081215_1646--eventgen--1816/stdouterr_job_3.tar.bz2 08-12-15 16:51:29: finished. saveend: updating sDB item job3 # all subprocesses ended # summary: Jobnumber: 3 starttime: 08-12-15 16:49:33 endtime: 08-12-15 16:51 executiontime: 00:01 returncode: 0 ---------------------- Jobnumber: 4 starttime: 08-12-15 16:49:39 endtime: 08-12-15 16:51 executiontime: 00:01 returncode: 0 ---------------------- # unmount EBS volume(s)... invoke unmounting: subprocess.Popen() with args ['umount', '/dev/sdh1'] umount subprocess ended. returncode: 0 # detach EBS volume(s)... instructed EC2 to detach volume vol-95e400fc status: detaching status: available # delete EBS volume(s)... instructed EC2 to delete volume vol-95e400fc status: True # close processjobs-logfile... it will then be bundled and uploaded. then EC2 will be instructed to terminate this instance