HOWTO: Secure Data Access
with Condor, Parrot, Chirp, and Globus
Using Parrot and
Globus credentials,
it is easy to access remote data securely. However, what
happens when you wish to access remote data from within a batch job?
Your job will need access to your credentials in order to
authenticate to the file servers. This document explains how
to make this work.
Suppose that you wish to run a simulation called mysim.exe
through Condor
using Parrot to access
data on Chirp servers
using a CCL Globus certificate
to secure the connection.
(Note: Please do not try this example unless you are already
familiar with the above systems!)
If you have not done so already, obtain a CCL Globus certificate. Then, from your workstation, initialize your grid proxy:
% grid-proxy-init
Next, use the chirp tool to set the access control
on the necessary directories on the Chirp servers:
% chirp server.nd.edu
connected to server.nd.edu as globus:/O=Cooperative_Computing_Lab/CN=John_Q_Public
chirp:server.nd.edu:/> cd mydata
chirp:server.nd.edu:/mydata> setacl . globus:/O=Cooperative_Computing_Lab/CN=John_Q_Public rwla
chirp:server.nd.edu:/mydata> getacl
Create a directory on a local disk in order to construct your job:
% mkdir -p /tmp/jobs/job1
% cd /tmp/jobs/job1
Now, make symbolic links of several things in this directory:
% ln -s `which parrot` parrot
% ln -s `which mysim.exe` mysim.exe
% ln -s ${GLOBUS_LOCATION}/share/certficates/0224f2bf.0 .
% ln -s ${GLOBUS_LOCATION}/share/certficates/0224f2bf.signing_policy .
% ln -s /tmp/x509up_XXX userproxy.pem # (replace XXX with your Unix UID)
Then, create a Condor submit file (job.submit) that runs parrot with
your simulation and points to the CA certificates and your grid proxy certificate:
universe = vanilla
executable = parrot
# note the -H option is needed for Parrot
arguments = -H ./mysim.exe /chirp/server.nd.edu/mydata
output = output.txt
error = error.txt
log = log.txt
environment = X509_CERT_DIR=.;X509_USER_PROXY=userproxy.pem
transfer_input_files = 0224f2bf.0,0224f2bf.signing_policy,userproxy.pem,mysim.exe
transfer_files = on_exit
queue
Finally, submit the job:
condor_submit job.submit
Enjoy!
|