End-user python API
This section explains python API in panda-client for end-users. All functions are available through the API object.
[ ]:
from pandaclient import panda_api
c = panda_api.get_api()
Task submission API
Submit a task (low-level)
You need to prepare a dictionary of task parameters following an example. This function gives you a taskID, which is the unique identifier in the system, once it successfully submits the task. Note that it is highly recommended using another high-level API, such as execute_prun, execute_pathena, and execute_phpo, and only developers should this method since the task parameter dictionary is quite cryptic. Note that although all task submission functions have similar interface to subporcess functions they are executed in the same python interpreter, i.e., they don’t spawn child processes.
[ ]:
from pandaclient.example_task import taskParamMap
communication_status, o = c.submit_task(taskParamMap)
if communication_status:
server_return_code, dialog_message, task_id = o
if o == 0:
print ("taskID={}".format(task_id))
submit_task(self, task_params, verbose=False)
Description: Submit a task to PanDA
args:
task_params: a dictionary of task parameters
verbose: True to see debug messages
returns:
status code
0: communication succeeded to the panda server
255: communication failure
tuple of return code, message from the server, and task ID if successful
0: request is processed
1: duplication in DEFT
2: duplication in JEDI
3: accepted for incremental execution
4: server error
Submit a prun task
[ ]:
import uuid
com = "--exec ls --outDS user.hoge.{} --vo sphenix".format(str(uuid.uuid4()))
status, task_dict = c.execute_prun(com.split(), console_log=False)
if status:
print ("taskID={}".format(task_dict['jediTaskID']))
execute_prun(args, console_log=True)
Description: Execute prun command
args:
args: The arguments used to execute prun. This is a list of strings.
console_log: False to disable console logging
returns:
status: True if succeeded. Otherwise, False
a dictionary: Task submission attributes including jediTaskID
Submit a pathena task
execute_pathena(args, console_log=True)
Description: execute pathena command
args:
args: The arguments used to execute prun. This is a list of strings.
console_log: False to disable console logging
returns:
status: True if succeeded. Otherwise, False
a dictionary: Task submission attributes including jediTaskID
Submit a phpo task
execute_phpo(args, console_log=True)
Description: execute phpo command
args:
args: The arguments used to execute prun. This is a list of strings.
console_log: False to disable console logging
returns:
status: True if succeeded. Otherwise, False
a dictionary: Task submission attributes including jediTaskID
Task management API
Kill a task
[ ]:
communication_status, o = c.kill_task(task_id)
if communication_status:
server_return_code, dialog_message = o
if o == 0:
print('OK')
else:
print ("Not good with {} : {}".format(server_return_code, dialog_message))
kill_task(taskID, verbose=False)
Description: kill a task
args:
jediTaskID: jediTaskID of the task to be killed
returns:
status code
0: communication succeeded to the panda server
255: communication failure
tuple of return code and diagnostic message
0: request is registered
1: server error
2: task not found
3: permission denied
4: irrelevant task status
100: non SSL connection
101: irrelevant taskID
Finish a task
[ ]:
communication_status, o = c.finish_task(task_id)
if communication_status:
server_return_code, dialog_message = o
if o == 0:
print('OK')
else:
print ("Not good with {} : {}".format(server_return_code, dialog_message))
finish_task(task_id, wait_running=False, verbose=False)
Description: finish a task
args:
task_id: jediTaskID of the task to finish
wait_running: True to wait until running jobs are done
verbose: True to see debug messages
returns:
status code
0: communication succeeded to the panda server
255: communication failure
tuple of return code and diagnostic message
0: request is registered
1: server error
2: task not found
3: permission denied
4: irrelevant task status
100: non SSL connection
101: irrelevant taskID
Retry a task
[ ]:
communication_status, o = c.retry_task(task_id)
if communication_status:
server_return_code, dialog_message = o
if o == 0:
print('OK')
else:
print ("Not good with {} : {}".format(server_return_code, dialog_message))
retry_task(task_id, new_parameters=Non, verbose=False)
Description: finish a task
args:
task_id: jediTaskID of the task to finish
new_parameters: a dictionary of task parameters to overwrite
verbose: True to see debug messages
returns:
status code
0: communication succeeded to the panda server
255: communication failure
tuple of return code and diagnostic message
0: request is registered
1: server error
2: task not found
3: permission denied
4: irrelevant task status
100: non SSL connection
101: irrelevant taskID
Bookkeeping API
Get tasks
[ ]:
tasks = c.get_tasks()
for task in tasks:
print ('taskID={} status={}'.format(task['jeditaskid'], task['status']))
get_tasks(self, task_ids, limit=1000, days=14, status=None, username=None)
Description: get a list of task dictionaries
args:
task_ids: a list of task IDs, or None to get recent tasks
limit: the max number of tasks to fetch from the server
days: tasks for last N days to fetch
status: filtering with task status
username: user name of the tasks, or None to get own tasks
returns:
a list of task dictionaries
Show tasks
show_tasks(self, task_ids, limit=1000, days=14, status=None, username=None)
Description: show tasks on the console
args:
task_ids: a list of task IDs, or None to get recent tasks
limit: the max number of tasks to fetch from the server
days: tasks for last N days to fetch
status: filtering with task status
username: user name of the tasks, or None to get own tasks
returns:
None
Get metadata of all jobs in a task
get_job_metadata(self, task_id, output_json_filename)
Description: get metadata of all jobs in a task
args:
task_id: task ID
output_json_filename: output json filename