********************************************************* * WMS Practicals * Guided Exercises on gLite Workload Management System * in comparison with LCG. * * gLite Tutorial, Catania 13-15 Giugno 2005. * * Authors: Valeria Ardizzone * Marco Pappalardo ********************************************************* Loading .pem files in .globus directory ======================================= (only for the first time) In the event that you intall a UI from scratch you need to put the .pem files obtained from your certificate in .globus subdirectory in your home dir. Create .globus directory in your HOME dir. Once you exported your certificate from your browser, in a .p12 files, use ssl tools to produce your certificate and private key. $ openssl pkcs12 -nocerts -in mycert.p12 -out userkey.pem $ openssl pkcs12 -clcerts -nokeys -in mycert.p12 -out usercert.pem $ chmod 0400 userkey.pem $ chmod 0600 usercert.pem You are now ready to submit to the Grid. Exercise 1 =========== Find out what are the best fitting resources for your computational request. Suppose to have a "job" that can be expressed by the following JDL (vi $HOME/Catania/JobSubmission/hostname.jdl) [ Type="Job"; JobType="Normal"; Executable = "/bin/sh"; Arguments = "start_hostname.sh"; StdError = "stderr.log"; StdOutput = "stdout.log"; InputSandbox = "start_hostname.sh"; OutputSandbox = {"stderr.log", "stdout.log"}; RetryCount = 7; ] The script start_hostname.sh is sent to the CE and it is passes as an argument to the executable "/bin/sh". The script two instructions: #!/bin/sh sleep 5 hostname -f Run the glite-job-list-match command to find out the appropriate resources list. [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-list-match hostname.jdl Selected Virtual Organisation name (from UI conf file): gilda Connecting to host glite-rb.ct.infn.it, port 7772 *************************************************************************** COMPUTING ELEMENT IDs LIST The following CE(s) matching your job requirements have been found: *CEId* cn01.be.itu.edu.tr:2119/jobmanager-lcglsf-infinite cn01.be.itu.edu.tr:2119/jobmanager-lcglsf-long cn01.be.itu.edu.tr:2119/jobmanager-lcglsf-short dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-short gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-long gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-short glite-ce.ct.infn.it:2119/blah-pbs-long grid-ce.bio.dist.unige.it:2119/jobmanager-lcgpbs-long grid-ce.bio.dist.unige.it:2119/jobmanager-lcgpbs-short grid002.mporzio.astro.it:2119/jobmanager-lcgpbs-infinite grid002.mporzio.astro.it:2119/jobmanager-lcgpbs-long grid002.mporzio.astro.it:2119/jobmanager-lcgpbs-short grid006.cecalc.ula.ve:2119/jobmanager-lcgpbs-infinite grid010.ct.infn.it:2119/jobmanager-lcgpbs-infinite grid010.ct.infn.it:2119/jobmanager-lcgpbs-long grid010.ct.infn.it:2119/jobmanager-lcgpbs-short grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-long grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-short glite-ce.ct.infn.it:2119/blah-pbs-short gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-infinite grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-infinite grid-ce.bio.dist.unige.it:2119/jobmanager-lcgpbs-infinite gildace.ct.astro.it:2119/jobmanager-lcgpbs-infinite gildace.ct.astro.it:2119/jobmanager-lcgpbs-long gildace.ct.astro.it:2119/jobmanager-lcgpbs-short glite-ce.ct.infn.it:2119/blah-pbs-infinite grid006.cecalc.ula.ve:2119/jobmanager-lcgpbs-short grid006.cecalc.ula.ve:2119/jobmanager-lcgpbs-long *************************************************************************** [pappalar@glite-tutor:~/Catania/JobSubmission]$ Enjoy the listmatch by repeating this exercise on LCG UI available using the command - edg-job-list-match finding the appropriate command line to use. Exercise 2 =========== Let's now run our "hostname" job on a CE (letting the WM choose the best suitable CE for execution). Consider again the hostname.jdl and start_hostname.sh. [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-submit -o myjobid hostname.jdl Selected Virtual Organisation name (from UI conf file): gilda Connecting to host glite-rb.ct.infn.it, port 7772 Logging to host glite-rb.ct.infn.it, port 9002 ================================ glite-job-submit Success ===================================== The job has been successfully submitted to the Network Server. Use glite-job-status command to check job current status. Your job identifier is: - https://glite-rb.ct.infn.it:9000/zGFA9ntyOY8gIISLPG94cg The job identifier has been saved in the following file: /home/pappalar/Catania/JobSubmission/myjobid ============================================================================================= Where myjobid contains: [pappalar@glite-tutor:~/Catania/JobSubmission]$ cat myjobid ###Submitted Job Ids### https://glite-rb.ct.infn.it:9000/zGFA9ntyOY8gIISLPG94cg [pappalar@glite-tutor:~/Catania/JobSubmission]$ Follow now the job life through job-status command in order to discover which the selected CE is. [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-status -i myjobid ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://glite-rb.ct.infn.it:9000/zGFA9ntyOY8gIISLPG94cg Current Status: Ready Status Reason: unavailable Destination: grid002.mporzio.astro.it:2119/jobmanager-lcgpbs-short Submitted: Mon Jun 13 10:29:11 2005 CEST ************************************************************* [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-status -i myjobid -v 3 ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://glite-rb.ct.infn.it:9000/zGFA9ntyOY8gIISLPG94cg Current Status: Ready Status Reason: unavailable Destination: grid002.mporzio.astro.it:2119/jobmanager-lcgpbs-short Submitted: Mon Jun 13 10:29:11 2005 CEST [.. Omissis ..] - stateEnterTimes = Submitted : Mon Jun 13 10:29:11 2005 CEST Waiting : Mon Jun 13 10:29:12 2005 CEST Ready : Mon Jun 13 10:29:16 2005 CEST Scheduled : --- Running : --- Done : --- Cleared : --- Aborted : --- Cancelled : --- Unknown : --- ************************************************************* As you can see from the previous report the choosen CE is: Destination: grid002.mporzio.astro.it:2119/jobmanager-lcgpbs-short It's obviosly an item in the set listed through glite-list-job-match command. By repeating again job-status command as before, in a quite short while, you'll have: - stateEnterTimes = Submitted : Mon Jun 13 10:29:11 2005 CEST Waiting : Mon Jun 13 10:29:12 2005 CEST Ready : Mon Jun 13 10:29:16 2005 CEST Scheduled : Mon Jun 13 10:34:45 2005 CEST Running : Mon Jun 13 10:34:45 2005 CEST Done : Mon Jun 13 10:34:51 2005 CEST Cleared : --- Aborted : --- Cancelled : --- Unknown : --- Another solution to follow the execution of your job with deeper detail uses the glite-job-logging-info command. As an example, try to inspect the output given by [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-logging-info -v 2 -i myjobid Repeat all using LCG commands as listed below: edg-job-submit -o myjobid hostname.jdl edg-job-status -i myjobid -v 3 edg-job-get-logging-info -v 2 -i myjobid Exercise 3 =========== Retrieve job output. As written in JDL OutputSandbox attribute we expect to get back two files containing the std output and the std error produced. First check if the 'JobOutput' subdirectory does exist either locally or in your $HOME directory. If not, just create it (preferrably local). We have choosen to create JobOutput inside JobSubmission directory, so the normal command line for glite-job-output produces an error: [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-output -i myjobid **** Error: UI_DIR_NOT_FOUND **** Unable to find the directory "/home/pappalar/JobOutput" Check the "OutputStorage" value specified in: /opt/glite/etc/glite_wmsui_cmd_var.conf [pappalar@glite-tutor:~/Catania/JobSubmission]$ Having used a non standard location for JobOutput we have to add another option to the c.l. indicating the destination path. [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-output -i myjobid --dir ./JobOutput Retrieving files from host: glite-rb.ct.infn.it ( for https://glite-rb.ct.infn.it:9000/zGFA9ntyOY8gIISLPG94cg ) ********************************************************************************* JOB GET OUTPUT OUTCOME Output sandbox files for the job: - https://glite-rb.ct.infn.it:9000/zGFA9ntyOY8gIISLPG94cg have been successfully retrieved and stored in the directory: /home/pappalar/Catania/JobSubmission/JobOuput/pappalar_zGFA9ntyOY8gIISLPG94cg ********************************************************************************* move to JobOutput/pappalar_zGFA9ntyOY8gIISLPG94cg and ls the containt: stderr.log stdout.log The stdout.log file contains, in fact: grid003.mporzio.astro.it With an LCG UI, plese refer to edg-job-get-output command. Exercise 4 =========== Let's now force the WM to choose a given CE as destination for the job. It can be done in two different ways. First of all, try the glite-job-submit option --resource ce_id -r ce_id If the command is launched with this option, the job-ad sent to the NS contains a line of the type SubmitTo = ce_id and the job is submitted by the WMS to the resource identified by ce_id without going through the match-making process. The standard format for a CEId is: :/jobmanager-- So: [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-submit -r grid010.ct.infn.it:2119/jobmanager-lcgpbs-long -o myjobid hostname.jdl Selected Virtual Organisation name (from UI conf file): gilda Connecting to host glite-rb.ct.infn.it, port 7772 Logging to host glite-rb.ct.infn.it, port 9002 ================================ glite-job-submit Success ===================================== The job has been successfully submitted to the Network Server. Use glite-job-status command to check job current status. Your job identifier is: - https://glite-rb.ct.infn.it:9000/HRtARhjGQArbw9EeW9hBtQ The job identifier has been saved in the following file: /home/pappalar/Catania/JobSubmission/myjobid ============================================================================================= [pappalar@glite-tutor:~/Catania/JobSubmission]$ As done before, let's inspect the containt of myjobid file: [pappalar@glite-tutor:~/Catania/JobSubmission]$ cat myjobid ###Submitted Job Ids### https://glite-rb.ct.infn.it:9000/zGFA9ntyOY8gIISLPG94cg https://glite-rb.ct.infn.it:9000/HRtARhjGQArbw9EeW9hBtQ [pappalar@glite-tutor:~/Catania/JobSubmission]$ It now contains two files! Did you expect that? :) After a short while, in order to investigate the status use glite-job-status and choose 2 when asked. [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-status -i myjobid -v 3 ------------------------------------------------------------------ 1 : https://glite-rb.ct.infn.it:9000/zGFA9ntyOY8gIISLPG94cg 2 : https://glite-rb.ct.infn.it:9000/HRtARhjGQArbw9EeW9hBtQ a : all q : quit ------------------------------------------------------------------ Choose one or more jobId(s) in the list - [1-2]all:2 ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://glite-rb.ct.infn.it:9000/HRtARhjGQArbw9EeW9hBtQ Current Status: Done (Success) Exit code: 0 Status Reason: Job terminated successfully Destination: grid010.ct.infn.it:2119/jobmanager-lcgpbs-long Submitted: Mon Jun 13 11:12:25 2005 CEST --- [.. Omissis ..] - stateEnterTimes = Submitted : Mon Jun 13 11:12:25 2005 CEST Waiting : Mon Jun 13 11:12:27 2005 CEST Ready : Mon Jun 13 11:12:30 2005 CEST Scheduled : Mon Jun 13 11:17:44 2005 CEST Running : Mon Jun 13 11:17:44 2005 CEST Done : Mon Jun 13 11:17:50 2005 CEST Cleared : --- Aborted : --- Cancelled : --- Unknown : --- ************************************************************* [pappalar@glite-tutor:~/Catania/JobSubmission]$ The job has gone exactly where we wanted. Procees as explained before until job status is done and then retrieve the Output. The LCG command to be used are the same than in previous cases. Don't forget the -r option, if you don't want to get crazy. Exercise 5 =========== Try now to get to the same result through setting "requirements" attribute inside the JDL file. Change our hostname.jdl according to what is shown below and save it as hostnamereq.jdl. (vi $HOME/Catania/JobSubmission/hostnamereq.jdl) [ Type="Job"; JobType="Normal"; Executable = "/bin/sh"; Arguments = "start_hostname.sh"; StdError = "stderr.log"; StdOutput = "stdout.log"; InputSandbox = "start_hostname.sh"; OutputSandbox = {"stderr.log", "stdout.log"}; Requirements = other.GlueCEUniqueId == "grid010.ct.infn.it:2119/jobmanager-lcgpbs-long"; RetryCount = 7; ] Try out with the glite-job-list-match. That will help us to understand whether we correctly set out criteria. [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-list-match hostnamereq.jdl Selected Virtual Organisation name (from UI conf file): gilda Connecting to host glite-rb.ct.infn.it, port 7772 *************************************************************************** COMPUTING ELEMENT IDs LIST The following CE(s) matching your job requirements have been found: *CEId* grid010.ct.infn.it:2119/jobmanager-lcgpbs-long *************************************************************************** [pappalar@glite-tutor:~/Catania/JobSubmission]$ Let's immediatly go with the glite-job-submit!!! [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-submit -o myjobid hostnamereq.jdl Selected Virtual Organisation name (from UI conf file): gilda Connecting to host glite-rb.ct.infn.it, port 7772 Logging to host glite-rb.ct.infn.it, port 9002 ================================ glite-job-submit Success ===================================== The job has been successfully submitted to the Network Server. Use glite-job-status command to check job current status. Your job identifier is: - https://glite-rb.ct.infn.it:9000/CWCwBjRvNvg8EYLCvYpHPg The job identifier has been saved in the following file: /home/pappalar/Catania/JobSubmission/myjobid ============================================================================================= [pappalar@glite-tutor:~/Catania/JobSubmission]$ cat myjobid ###Submitted Job Ids### https://glite-rb.ct.infn.it:9000/zGFA9ntyOY8gIISLPG94cg https://glite-rb.ct.infn.it:9000/HRtARhjGQArbw9EeW9hBtQ https://glite-rb.ct.infn.it:9000/CWCwBjRvNvg8EYLCvYpHPg [pappalar@glite-tutor:~/Catania/JobSubmission]$ Go ahead as usual until job status is done and then retrieve the Output. Exercise 6 =========== If you're interested into selecting the best CE in a set whose items respect a given criteria, instead of into a specific CE, you need to set up a filter through Requirements attribute and also a Rank criteria to get the entries ordered. In the following example, the JDL provides a filter based on LRMS type, explicitly asking for "PBS". In the selected set, the best Ranked CE will be choosen, that is the one having the highest number of Free CPUs. Save this JDL as hostnamerank.jdl. (vi $HOME/Catania/JobSubmission/hostnamerank.jdl) [ Type="Job"; JobType="Normal"; Executable = "/bin/sh"; Arguments = "start_hostname.sh"; StdError = "stderr.log"; StdOutput = "stdout.log"; InputSandbox = "start_hostname.sh"; OutputSandbox = {"stderr.log", "stdout.log"}; Requirements = other. GlueCEInfoLRMSType == "PBS"; Rank = other.GlueCEStateFreeCPUs; RetryCount = 7; ] [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-list-match hostnamerank.jdl Selected Virtual Organisation name (from UI conf file): gilda Connecting to host glite-rb.ct.infn.it, port 7772 *************************************************************************** COMPUTING ELEMENT IDs LIST The following CE(s) matching your job requirements have been found: *CEId* glite-ce.ct.infn.it:2119/blah-pbs-infinite glite-ce.ct.infn.it:2119/blah-pbs-long glite-ce.ct.infn.it:2119/blah-pbs-short grid002.mporzio.astro.it:2119/jobmanager-lcgpbs-infinite grid002.mporzio.astro.it:2119/jobmanager-lcgpbs-long grid002.mporzio.astro.it:2119/jobmanager-lcgpbs-short *************************************************************************** [pappalar@glite-tutor:~/Catania/JobSubmission]$ If no resources match your criteria you should have had: [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-list-match hostnamerank.jdl Selected Virtual Organisation name (from UI conf file): gilda Connecting to host glite-rb.ct.infn.it, port 7772 ===================== glite-job-list-match failure ====================== No Computing Element matching your job requirements has been found! ====================================================================== [pappalar@glite-tutor:~/Catania/JobSubmission]$ It is quite normal, at this point, the user expects that one among glite-ce.ct.infn.it:2119 and grid002.mporzio.astro.it:2119 is choosen for execution. In fact: [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-submit -o myjobid hostnamerank.jdl Selected Virtual Organisation name (from UI conf file): gilda Connecting to host glite-rb.ct.INFN.it, port 7772 Logging to host glite-rb.ct.INFN.it, port 9002 ================================ glite-job-submit Success ===================================== The job has been successfully submitted to the Network Server. Use glite-job-status command to check job current status. Your job identifier is: - https://glite-rb.ct.infn.it:9000/1LO8qDanqocshjhLp-HdJA The job identifier has been saved in the following file: /home/pappalar/Catania/JobSubmission/myjobid ============================================================================================= [pappalar@glite-tutor:~/Catania/JobSubmission]$ cat myjobid ###Submitted Job Ids### https://glite-rb.ct.infn.it:9000/zGFA9ntyOY8gIISLPG94cg https://glite-rb.ct.infn.it:9000/HRtARhjGQArbw9EeW9hBtQ https://glite-rb.ct.infn.it:9000/CWCwBjRvNvg8EYLCvYpHPg https://glite-rb.ct.infn.it:9000/1LO8qDanqocshjhLp-HdJA [pappalar@glite-tutor:~/Catania/JobSubmission]$glite-job-status -i myjobid -v 3 ------------------------------------------------------------------ 1 : https://glite-rb.ct.infn.it:9000/zGFA9ntyOY8gIISLPG94cg 2 : https://glite-rb.ct.infn.it:9000/HRtARhjGQArbw9EeW9hBtQ 3 : https://glite-rb.ct.infn.it:9000/CWCwBjRvNvg8EYLCvYpHPg 4 : https://glite-rb.ct.infn.it:9000/1LO8qDanqocshjhLp-HdJA a : all q : quit ------------------------------------------------------------------ Choose one or more jobId(s) in the list - [1-4]all:4 ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://glite-rb.ct.infn.it:9000/1LO8qDanqocshjhLp-HdJA Current Status: Ready Status Reason: unavailable Destination: glite-ce.ct.infn.it:2119/blah-pbs-long Submitted: Mon Jun 13 12:32:21 2005 CEST --- Feel free to continue the exercise as explained for previous cases and repeat it within LCG UI. Exercise 7 =========== Another option could be the case you need to filter available resources wrt the existence of a determined application. Mozart_K545 esample uses a musical application, called "C_SOUND" to reproduce sounds and music. That implies that mentioned application must be installed on the WN. In this way it is possible to express a preference not only on physical characteristics of the resource but also on applications available. [pappalar@glite-tutor:~]$ glite-job-list-match Mozart_K545.jdl Selected Virtual Organisation name (from UI conf file): gilda Connecting to host glite-rb.ct.infn.it, port 7772 *************************************************************************** COMPUTING ELEMENT IDs LIST The following CE(s) matching your job requirements have been found: *CEId* cn01.be.itu.edu.tr:2119/jobmanager-lcglsf-infinite cn01.be.itu.edu.tr:2119/jobmanager-lcglsf-long cn01.be.itu.edu.tr:2119/jobmanager-lcglsf-short dgt01.ui.savba.sk:2119/jobmanager-lcgpbs-short gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-long gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-short grid-ce.bio.dist.unige.it:2119/jobmanager-lcgpbs-long grid-ce.bio.dist.unige.it:2119/jobmanager-lcgpbs-short grid002.mporzio.astro.it:2119/jobmanager-lcgpbs-long grid006.cecalc.ula.ve:2119/jobmanager-lcgpbs-infinite grid010.ct.infn.it:2119/jobmanager-lcgpbs-infinite grid010.ct.infn.it:2119/jobmanager-lcgpbs-long grid010.ct.infn.it:2119/jobmanager-lcgpbs-short grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-long grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-short grid002.mporzio.astro.it:2119/jobmanager-lcgpbs-infinite grid002.mporzio.astro.it:2119/jobmanager-lcgpbs-short gilda-ce-01.pd.infn.it:2119/jobmanager-lcgpbs-infinite grid-ce.bio.dist.unige.it:2119/jobmanager-lcgpbs-infinite grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-infinite gildace.ct.astro.it:2119/jobmanager-lcgpbs-infinite gildace.ct.astro.it:2119/jobmanager-lcgpbs-long gildace.ct.astro.it:2119/jobmanager-lcgpbs-short grid006.cecalc.ula.ve:2119/jobmanager-lcgpbs-short grid006.cecalc.ula.ve:2119/jobmanager-lcgpbs-long *************************************************************************** [pappalar@glite-tutor:~]$ Submit it and just follow the execution: [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-status -v 3 https://glite-rb.ct.infn.it:9000/N1RGwYoph6I8Adpj80Lwyg ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://glite-rb.ct.infn.it:9000/N1RGwYoph6I8Adpj80Lwyg Current Status: Done (Success) Exit code: 0 Status Reason: Job terminated successfully Destination: grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-short Submitted: Mon Jun 13 12:43:56 2005 CEST --- [.. Omissis ..] - stateEnterTimes = Submitted : Mon Jun 13 12:43:56 2005 CEST Waiting : Mon Jun 13 12:43:58 2005 CEST Ready : Mon Jun 13 12:44:03 2005 CEST Scheduled : Mon Jun 13 12:49:25 2005 CEST Running : Mon Jun 13 12:49:25 2005 CEST Done : Mon Jun 13 12:49:37 2005 CEST Cleared : --- Aborted : --- Cancelled : --- Unknown : --- ************************************************************* Recover the output: [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-output -dir ./JobOutput/ https://glite-rb.ct.infn.it:9000/N1RGwYoph6I8Adpj80Lwyg Retrieving files from host: glite-rb.ct.infn.it ( for https://glite-rb.ct.infn.it:9000/N1RGwYoph6I8Adpj80Lwyg ) ********************************************************************************* JOB GET OUTPUT OUTCOME Output sandbox files for the job: - https://glite-rb.ct.infn.it:9000/N1RGwYoph6I8Adpj80Lwyg have been successfully retrieved and stored in the directory: /home/pappalar/Catania/JobSubmission/JobOutput/pappalar_N1RGwYoph6I8Adpj80Lwyg ********************************************************************************* [pappalar@glite-tutor:~/Catania/JobSubmission]$ Listing the containt of the directory: [pappalar@glite-tutor:~/Catania/JobSubmission/JobOutput/pappalar_N1RGwYoph6I8Adpj80Lwyg]$ ls Mozart_K545_csound.aiff Mozart_K545_csound.err Mozart_K545_csound.out [pappalar@glite-tutor:~/Catania/JobSubmission/JobOutput/pappalar_N1RGwYoph6I8Adpj80Lwyg]$ PLEASE, DON'T TRY TO REPRODUCE OUTPUT by yourselves! You don't have headphones!!! TRUST IN US: IT DOES REALLY SOUND. Exercise 8 =========== Run an ls command on the selected resource. (vi $HOME/Catania/JobSubmission/scriptls.jdl) [ Executable = "ls.sh"; Arguments = "-alt"; StdError = "stderr.log"; StdOutput = "stdout.log"; InputSandbox = "ls.sh"; OutputSandbox = {"stderr.log", "stdout.log"}; ] The executable is ls.sh that produces the execution of /bin/ls -alt on the resource, as shown: #!/bin/sh /bin/ls Try it out by yourselves. Also on LCG! Exercise 9 =========== Suppose you have your own c program source code and that you'll be willing to execute it on the selected resource. How to do it? Very simple: just compile and transfer it as executable properly setting the JDL file containt. source.c is a C program that prints a Welcome message on the resource. See the listing: (vi $HOME/Catania/JobSubmission/source.c) #include int main(int argc, char **argv) { char *name = argv[1]; /*system("clear");*/ printf("\n\n\n"); printf("Hello %s!\n",name); printf("Welcome to gLite tutorial, Catania 13-15 June,2005 \n\n\n"); exit(0); } Let's compile it, typing simply 'make'. Through makefile you can see in your directory, you'll obtain c_exe executable file. Just have a look at the JDL indicating that the executable must be transferred and executed. (vi $HOME/Catania/JobSubmission/c_exe.jdl) Type = "Job"; JobType = "Normal"; Executable = "/bin/sh"; StdOutput = "std.out"; StdError = "std.err"; OutputSandbox = {"std.err","std.out"}; InputSandbox = {"startExe.sh","c_exe"}; Arguments = "startExe.sh"; RetryCount = 7; So: [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-submit -o myjobid c_exe.jdl Selected Virtual Organisation name (from UI conf file): gilda Connecting to host glite-rb.ct.infn.it, port 7772 Logging to host glite-rb.ct.infn.it, port 9002 ================================ glite-job-submit Success ===================================== The job has been successfully submitted to the Network Server. Use glite-job-status command to check job current status. Your job identifier is: - https://glite-rb.ct.infn.it:9000/iSSPY5OAvL-pbrK4qE5IHw The job identifier has been saved in the following file: /home/pappalar/Catania/JobSubmission/myjobid ============================================================================================= [pappalar@glite-tutor:~/Catania/JobSubmission]$ cat myjobid ###Submitted Job Ids### https://glite-rb.ct.infn.it:9000/zGFA9ntyOY8gIISLPG94cg https://glite-rb.ct.infn.it:9000/HRtARhjGQArbw9EeW9hBtQ https://glite-rb.ct.infn.it:9000/CWCwBjRvNvg8EYLCvYpHPg https://glite-rb.ct.infn.it:9000/1LO8qDanqocshjhLp-HdJA https://glite-rb.ct.infn.it:9000/iSSPY5OAvL-pbrK4qE5IHw [pappalar@glite-tutor:~/Catania/JobSubmission]$ Let's get logging info on this job [pappalar@glite-tutor:~/Catania/JobSubmission]$ glite-job-logging-info -i myjobid ------------------------------------------------------------------ 1 : https://glite-rb.ct.infn.it:9000/zGFA9ntyOY8gIISLPG94cg 2 : https://glite-rb.ct.infn.it:9000/HRtARhjGQArbw9EeW9hBtQ 3 : https://glite-rb.ct.infn.it:9000/CWCwBjRvNvg8EYLCvYpHPg 4 : https://glite-rb.ct.infn.it:9000/1LO8qDanqocshjhLp-HdJA 5 : https://glite-rb.ct.infn.it:9000/iSSPY5OAvL-pbrK4qE5IHw a : all q : quit ------------------------------------------------------------------ Choose one or more jobId(s) in the list - [1-5]all:5 ********************************************************************** LOGGING INFORMATION: Printing info for the Job : https://glite-rb.ct.infn.it:9000/iSSPY5OAvL-pbrK4qE5IHw --- Event: RegJob - source = UserInterface - timestamp = Mon Jun 13 11:23:07 2005 CEST --- [.. Omissis ..] --- Event: Transfer - destination = LRMS - result = OK - source = LogMonitor - timestamp = Mon Jun 13 11:28:47 2005 CEST --- Event: Running - source = LogMonitor - timestamp = Mon Jun 13 11:28:47 2005 CEST --- Event: Done - source = LogMonitor - timestamp = Mon Jun 13 11:28:48 2005 CEST ********************************************************************** [pappalar@glite-tutor:~/Catania/JobSubmission]$ Since the job is Done, let's retrieve the output and look at the stdout. Exercise 10 =========== In this exercise you learn how the user job can access certain information, at execution time, concerning the job (for example the name of the CE, the SE close to the CE and so on...). At this scope is used a file named BrokerInfo that is created in the job working directory, that is the current directory on the Worker Node for the executable. It is possible to get the information contained on it by of a CLI edg-brokerinfo. The step are the following: 1)this is the JDL that you should submit: [vardizzo@grid019 vardizzo]$ cat scriptBrokerInfo.jdl [ Executable = "startScriptBrokerInfo.sh"; StdOutput = "std.out"; StdError = "std.err"; VirtualOrganisation = "gilda"; InputSandbox = {"startScriptBrokerInfo.sh"}; OutputSandbox = {"std.out","std.err",".BrokerInfo"}; RetryCount = 7; ] 2)Edit the startScriptBrokerInfo.sh inserting only your name and note the last command: [vardizzo@grid019 vardizzo]$ vi startScriptBrokerInfo.sh #!/bin/sh MY_NAME="ReplaceWithYourName" WORKER_NODE_NAME=`hostname` echo "Hello $MY_NAME, from $WORKER_NODE_NAME" echo "This job is running on this CE: " /opt/edg/bin/edg-brokerinfo getCE 3)Submit the job with the LCG command : [vardizzo@grid019 vardizzo]$ edg-job-submit -o myjobid scriptBrokerInfo.jdl Selected Virtual Organisation name (from JDL): gilda Connecting to host grid004.ct.infn.it, port 7772 Logging to host grid004.ct.infn.it, port 9002 ================================ edg-job-submit Success ===================================== The job has been successfully submitted to the Network Server. Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is: - https://grid004.ct.infn.it:9000/USpHqUQd1zYUKxLozX4n9g The edg_jobId has been saved in the following file: /home/vardizzo/myjobid ============================================================================================= 4)Querying for the status of the job: [vardizzo@grid019 vardizzo]$edg-job-status -i myjobid ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://grid004.ct.infn.it:9000/Rr8mfSk5jcY8dGVqxQHStg Current Status: Done (Success) Exit code: 0 Status Reason: Job terminated successfully Destination: grid006.cecalc.ula.ve:2119/jobmanager-lcgpbs-infinite reached on: Mon Jun 13 15:00:53 2005 ************************************************************* 5)Retrieving the output file with the command: [vardizzo@grid019 vardizzo]$ edg-job-get-output -i myjobid Retrieving files from host: grid004.ct.infn.it ( for https://grid004.ct.infn.it:9000/Rr8mfSk5jcY8dGVqxQHStg ) ********************************************************************************* JOB GET OUTPUT OUTCOME Output sandbox files for the job: - https://grid004.ct.infn.it:9000/Rr8mfSk5jcY8dGVqxQHStg have been successfully retrieved and stored in the directory: /tmp/jobOutput/vardizzo_Rr8mfSk5jcY8dGVqxQHStg ********************************************************************************* 6)Do a ls command of the directory where the output files are storage: [vardizzo@grid019 vardizzo]$ ls -a /tmp/jobOutput/vardizzo_Rr8mfSk5jcY8dGVqxQHStg/ .BrokerInfo std.err std.out 7)Watch the file std.out: [vardizzo@grid019 vardizzo]$ cat /tmp/jobOutput/vardizzo_Rr8mfSk5jcY8dGVqxQHStg/std.out Hello Valeria, from grid004.cecalc.ula.ve . .. .BrokerInfo startScriptBrokerInfo.sh std.err std.out This job is running on this CE: grid006.cecalc.ula.ve:2119/jobmanager-lcgpbs-infinite 8)Cat the .BrokerInfo file: [vardizzo@grid019 vardizzo]$ cat /tmp/jobOutput/vardizzo_Rr8mfSk5jcY8dGVqxQHStg/.BrokerInfo [ ComputingElement = [ CloseStorageElements = { [ GlueSAStateAvailableSpace = 14029724; GlueCESEBindCEAccesspoint = "/flatfiles/SE00"; mount = GlueCESEBindCEAccessPoint; name = "grid003.cecalc.ula.ve"; freespace = GlueSAStateAvailableSpace ] }; name = "grid006.cecalc.ula.ve:2119/jobmanager-lcgpbs-infinite" ]; InputFNs = { }; StorageElements = { }; VirtualOrganisation = "gilda" ] 9)Query the .BrokerInfo file on local machine with the command edg-brokerinfo getCE: [vardizzo@grid019 vardizzo]$ edg-brokerinfo -f /tmp/jobOutput/vardizzo_Rr8mfSk5jcY8dGVqxQHStg/.BrokerInfo getCE grid006.cecalc.ula.ve:2119/jobmanager-lcgpbs-infinite Exercise 11 =========== This is the mpi.jdl: Type = "Job"; JobType = "MPICH"; Executable = "MPItest.sh"; Arguments = "cpi 2"; NodeNumber = 2; StdOutput = "test.out"; StdError = "test.err"; InputSandbox = {"MPItest.sh","cpi"}; OutputSandbox = {"test.err","test.out","executable.out"}; Requirements = other.GlueCEInfoLRMSType == "PBS" || other.GlueCEInfoLRMSType == "LSF"; @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ This is the MPITest.sh #!/bin/sh # # this parameter is the binary to be executed EXE=$1 # this parameter is the number of CPU's to be reserved for parallel execution CPU_NEEDED=$2 # prints the name of the master node echo "Running on: $HOSTNAME" echo "*************************************" if [ -f "$PWD/.BrokerInfo" ] ; then TEST_LSF=`edg-brokerinfo getCE | cut -d/ -f2 | grep lsf` else TEST_LSF=`ps -ef | grep sbatchd | grep -v grep` fi if [ "x$TEST_LSF" = "x" ] ; then # prints the name of the file containing the nodes allocated for parallel execution echo "PBS Nodefile: $PBS_NODEFILE" # print the names of the nodes allocated for parallel execution cat $PBS_NODEFILE echo "*************************************" HOST_NODEFILE=$PBS_NODEFILE else # print the names of the nodes allocated for parallel execution echo "LSF Hosts: $LSB_HOSTS" # loops over the nodes allocated for parallel execution HOST_NODEFILE=`pwd`/lsf_nodefile.$$ for host in ${LSB_HOSTS} do echo $host >> ${HOST_NODEFILE} done fi echo "*************************************" # prints the working directory on the master node echo "Current dir: $PWD" echo "*************************************" for i in `cat $HOST_NODEFILE` ; do echo "Mirroring via SSH to $i" # creates the working directories on all the nodes allocated for parallel execution ssh $i mkdir -p `pwd` # copies the needed files on all the nodes allocated for parallel execution /usr/bin/scp -rp ./* $i:`pwd` # checks that all files are present on all the nodes allocated for parallel execution echo `pwd` ssh $i ls `pwd` # sets the permissions of the files ssh $i chmod 755 `pwd`/$EXE ssh $i ls -alR `pwd` echo "@@@@@@@@@@@@@@@" done # execute the parallel job with mpirun echo "*********************************" echo "Executing $EXE" chmod 755 $EXE ls -l mpirun -np $CPU_NEEDED -machinefile $HOST_NODEFILE `pwd`/$EXE > executable.out echo "*********************************" @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1) [glite-tutor] /home/vardizzo > glite-job-submit -o mioid mpi.jdl Selected Virtual Organisation name (from UI conf file): gilda Connecting to host glite-rb.ct.infn.it, port 7772 Logging to host glite-rb.ct.infn.it, port 9002 ================================ glite-job-submit Success ===================================== The job has been successfully submitted to the Network Server. Use glite-job-status command to check job current status. Your job identifier is: - https://glite-rb.ct.infn.it:9000/eE98NOnwepVNUzFRVs6vcw The job identifier has been saved in the following file: /home/vardizzo/mioid ============================================================================================= 2) [glite-tutor] /home/vardizzo > glite-job-status -i mioid ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://glite-rb.ct.infn.it:9000/eE98NOnwepVNUzFRVs6vcw Current Status: Done (Success) Exit code: 0 Status Reason: Job terminated successfully Destination: grid010.ct.infn.it:2119/jobmanager-lcgpbs-short Submitted: Mon Jun 13 18:39:39 2005 CEST ************************************************************* 3) [glite-tutor] /home/vardizzo > glite-job-output -i mioid Retrieving files from host: glite-rb.ct.infn.it ( for https://glite-rb.ct.infn.it:9000/eE98NOnwepVNUzFRVs6vcw ) ********************************************************************************* JOB GET OUTPUT OUTCOME Output sandbox files for the job: - https://glite-rb.ct.infn.it:9000/eE98NOnwepVNUzFRVs6vcw have been successfully retrieved and stored in the directory: /home/vardizzo/JobOutput/vardizzo_eE98NOnwepVNUzFRVs6vcw ********************************************************************************* 4) [glite-tutor] /home/vardizzo > ls /home/vardizzo/JobOutput/vardizzo_eE98NOnwepVNUzFRVs6vcw executable.out test.err test.out [glite-tutor] /home/vardizzo > 5) [glite-tutor] /home/vardizzo > cat /home/vardizzo/JobOutput/vardizzo_eE98NOnwepVNUzFRVs6vcw/executable.out Process 0 of 2 on grid037.ct.infn.it pi is approximately 3.1415926544231318, Error is 0.0000000008333387 wall clock time = 10.003797 Process 1 of 2 on grid037.ct.infn.it [glite-tutor] /home/vardizzo > Exercise 12 =========== DAG. see the document DAG_readme.txt. To submit the DAG with the new commands gLite is the same of the previous exercises also for DAG. Exercise 13 =========== A simple script to interact with the job is presented in this section. The function of the script is get the information sent by the interactive job, present it to the user, and send the user's response back to the job. In the follow the sequence of the commands: 1)Submit the interactive jdl file that you have in your $home. [vardizzo@grid019 vardizzo]$ edg-job-submit -o mioid --nogui interactive.jdl Selected Virtual Organisation name (from UI conf file): gilda Connecting to host grid004.ct.infn.it, port 7772 Logging to host grid004.ct.infn.it, port 9002 ================================ edg-job-submit Success ===================================== The job has been successfully submitted to the Network Server. Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is: - https://grid004.ct.infn.it:9000/u_Ugnvdlbrpm9a8w8UjTWw --- The Interactive Session Listener has been successfully launched with the following parameters: Host: 193.206.208.149 Port: 23017 The edg_jobId has been saved in the following file: /home/vardizzo/mioid ============================================================================================= *************************************** Interactive Job console started for https://grid004.ct.infn.it:9000/u_Ugnvdlbrpm9a8w8UjTWw Please press ^C to exit from the session *************************************** Welcome! What is your name? Valeria Bye Bye Valeria *********************************** * INTERACTIVE JOB FINISHED * ***********************************