****** processjobs.py start Mon Dec 15 16:48:42 2008 ****** # parsing the sessioninfofile /mnt/awsac/sessioninfo/awsac.sessinfo ... Session ID: 081215_1646--eventgen--1816 ; CoresPerInstance: 2 # getting information about this instance from EC2... the ami-launch-index of this instance (i-33f64f5a) is 0 the Availability Zone of this instance is us-east-1b # parsing the jobsconfigfile /mnt/awsac/jobs.cfg ... # jobsdatadicts_list read from /mnt/awsac/jobs.cfg # my job numbers are: [1, 2] [{'jobnr': '1', 'snap_id': 'snap-9fd433f6', 'shscript': '/mnt/awsac/gen_30.sh'}, {'jobnr': '2', 'snap_id': 'snap-9fd433f6', 'shscript': '/mnt/awsac/gen_30.sh'}] # prepare the creation of EBS volume(s) from the needed ATLAS Release snapshot(s)... detected following different snap_ids: ['snap-9fd433f6'] planing to assign snap_id(s) to following device(s): snap-9fd433f6->/dev/sdh1 # create EBS volume(s)... instructed EC2 to create EBS from snapshot snap-9fd433f6 status: creating status: available # attach EBS volume(s)... instructed EC2 to attach volume vol-92e400fb to /dev/sdh1 status: attaching status: attached # mount EBS volume(s)... invoke mounting: subprocess.Popen() with args ['mount', '/dev/sdh1', '/mnt/atlas'] mount subprocess ended. returncode: 0 # this is the list of successfully mounted EBS - snapshots: ['snap-9fd433f6'] # the following jobs now have their desired EBS running in system: [{'jobnr': '1', 'snap_id': 'snap-9fd433f6', 'shscript': '/mnt/awsac/gen_30.sh'}, {'jobnr': '2', 'snap_id': 'snap-9fd433f6', 'shscript': '/mnt/awsac/gen_30.sh'}] # initialize running jobs... creating SimpleDB jobitems... # preparing job 1... cwd for job: /mnt/atlasworkarea/1 (created) calling subprocess.Popen() with args: ['/bin/sh', '/mnt/awsac/gen_30.sh'] runstart: updating sDB item job1 # preparing job 2... cwd for job: /mnt/atlasworkarea/2 (created) calling subprocess.Popen() with args: ['/bin/sh', '/mnt/awsac/gen_30.sh'] runstart: updating sDB item job2 # wait for subprocesses to finish... # subprocess for job 1 ended. returncode: 0 runend: updating sDB item job1 savestart: updating sDB item job1 found /mnt/atlasworkarea/1/results.tar.bz2 (47507 Byte) 08-12-15 16:50:34: start upload. bucket:atlassessions; key:081215_1646--eventgen--1816/results_job_1.tar.bz2 08-12-15 16:50:34: finished. outputfile of job 1 found: /mnt/atlasworkarea/1/stdouterr_job_1.log bundled /mnt/atlasworkarea/1/stdouterr_job_1.log to /mnt/atlasworkarea/1/stdouterr_job_1.tar.bz2 planing to upload /mnt/atlasworkarea/1/stdouterr_job_1.tar.bz2 (314 Byte)... 08-12-15 16:50:34: start upload. bucket:atlassessions; key:081215_1646--eventgen--1816/stdouterr_job_1.tar.bz2 08-12-15 16:50:34: finished. saveend: updating sDB item job1 # subprocess for job 2 ended. returncode: 0 runend: updating sDB item job2 savestart: updating sDB item job2 found /mnt/atlasworkarea/2/results.tar.bz2 (47574 Byte) 08-12-15 16:50:51: start upload. bucket:atlassessions; key:081215_1646--eventgen--1816/results_job_2.tar.bz2 08-12-15 16:50:51: finished. outputfile of job 2 found: /mnt/atlasworkarea/2/stdouterr_job_2.log bundled /mnt/atlasworkarea/2/stdouterr_job_2.log to /mnt/atlasworkarea/2/stdouterr_job_2.tar.bz2 planing to upload /mnt/atlasworkarea/2/stdouterr_job_2.tar.bz2 (314 Byte)... 08-12-15 16:50:51: start upload. bucket:atlassessions; key:081215_1646--eventgen--1816/stdouterr_job_2.tar.bz2 08-12-15 16:50:52: finished. saveend: updating sDB item job2 # all subprocesses ended # summary: Jobnumber: 1 starttime: 08-12-15 16:48:56 endtime: 08-12-15 16:50 executiontime: 00:01 returncode: 0 ---------------------- Jobnumber: 2 starttime: 08-12-15 16:49:03 endtime: 08-12-15 16:50 executiontime: 00:01 returncode: 0 ---------------------- # unmount EBS volume(s)... invoke unmounting: subprocess.Popen() with args ['umount', '/dev/sdh1'] umount subprocess ended. returncode: 0 # detach EBS volume(s)... instructed EC2 to detach volume vol-92e400fb status: detaching status: available # delete EBS volume(s)... instructed EC2 to delete volume vol-92e400fb status: True # close processjobs-logfile... it will then be bundled and uploaded. then EC2 will be instructed to terminate this instance