timing
- pilot.util.timing.add_to_pilot_timing(job_id, timing_constant, time_measurement, args, store=False)[source]
Add the given timing contant and measurement got job_id to the pilot timing dictionary.
- Parameters:
job_id – PanDA job id (string).
timing_constant – timing constant (string).
time_measurement – time measurement (float).
args – pilot arguments.
store – if True, write timing dictionary to file. False by default.
- Returns:
- pilot.util.timing.get_elapsed_real_time(t0=None)[source]
Return a time stamp corresponding to the elapsed real time (since t0 if requested). The function uses os.times() to get the current time stamp. If t0 is provided, the returned time stamp is relative to t0. t0 is assumed to be an os.times() tuple.
- Parameters:
t0 – os.times() tuple for the t0 time stamp.
- Returns:
time stamp (int).
- pilot.util.timing.get_final_update_time(job_id, args)[source]
High level function that returns the time for execution the final update for the given job_id.
- Parameters:
job_id – PanDA job id (string).
args – pilot arguments.
- Returns:
time in seconds (int).
- pilot.util.timing.get_getjob_time(job_id, args)[source]
High level function that returns the time for the getjob operation for the given job_id.
- Parameters:
job_id – PanDA job id (string).
args – pilot arguments.
- Returns:
time in seconds (int).
- pilot.util.timing.get_initial_setup_time(job_id, args)[source]
High level function that returns the time for the initial setup. The initial setup time is measured from PILOT_START_TIME to PILOT_PRE_GETJOB.
- Parameters:
job_id – PanDA job id (string).
args – pilot arguments.
- Returns:
time in seconds (int).
- pilot.util.timing.get_payload_execution_time(job_id, args)[source]
High level function that returns the time for the payload execution for the given job_id.
- Parameters:
job_id – PanDA job id (string).
args – pilot arguments.
- Returns:
time in seconds (int).
- pilot.util.timing.get_postgetjob_time(job_id, args)[source]
Return the post getjob time.
- Parameters:
job_id – job object.
args – pilot arguments.
- Returns:
post getjob time measurement (int). In case of failure, return None.
- pilot.util.timing.get_setup_time(job_id, args)[source]
High level function that returns the time for the setup operation for the given job_id.
- Parameters:
job_id – PanDA job id (string).
args – pilot arguments.
- Returns:
time in seconds (int).
- pilot.util.timing.get_stagein_time(job_id, args)[source]
High level function that returns the time for the stage-in operation for the given job_id.
- Parameters:
job_id – PanDA job id (string).
args – pilot arguments.
- Returns:
time in seconds (int).
- pilot.util.timing.get_stageout_time(job_id, args)[source]
High level function that returns the time for the stage-out operation for the given job_id.
- Parameters:
job_id – PanDA job id (string).
args – pilot arguments.
- Returns:
time in seconds (int).
- pilot.util.timing.get_time_difference(job_id, timing_constant_1, timing_constant_2, args)[source]
Return the positive time difference between the given constants. The order is not important and a positive difference is always returned. The function collects the time measurements corresponding to the given timing constants from the pilot timing file. The job_id is used internally as a dictionary key. The given timing constants and their timing measurements, belong to the given job_id. Structure of pilot timing dictionary:
{ job_id: { <timing_constant_1>: <time measurement in seconds since epoch>, .. }
job_id = 0 means timing information from wrapper. Timing constants are defined in pilot.util.constants. Time measurement are time.time() values. The float value will be converted to an int as a last step.
- Parameters:
job_id – PanDA job id (string).
timing_constant_1
timing_constant_2
args – pilot arguments.
- Returns:
time difference in seconds (int).
- pilot.util.timing.get_time_measurement(timing_constant, time_measurement_dictionary, timing_dictionary, job_id)[source]
Return a requested time measurement from the time measurement dictionary, read from the pilot timing file.
- Parameters:
timing_constant – timing constant (e.g. PILOT_MULTIJOB_START_TIME)
time_measurement_dictionary – time measurement dictionary, extracted from pilot timing dictionary.
timing_dictionary – full timing dictionary from pilot timing file.
job_id – PanDA job id (string).
- Returns:
time measurement (float).
- pilot.util.timing.get_time_since(job_id, timing_constant, args)[source]
Return the amount of time that has passed since the time measurement of timing_constant.
- Parameters:
job_id – PanDA job id (string).
timing_constant
args – pilot arguments.
- Returns:
time in seconds (int).
- pilot.util.timing.get_time_since_multijob_start(args)[source]
Return the amount of time that has passed since the last multi job was launched.
- Parameters:
args – pilot arguments.
- Returns:
time in seconds (int).
- pilot.util.timing.get_time_since_start(args)[source]
Return the amount of time that has passed since the pilot was launched.
- Parameters:
args – pilot arguments.
- Returns:
time in seconds (int).
- pilot.util.timing.get_total_pilot_time(job_id, args)[source]
High level function that returns the end time for the given job_id. This means the wall time that has passed from the start of the pilot until after the last job update.
- Parameters:
job_id – PanDA job id (string).
args – pilot arguments.
- Returns:
time in seconds (int).
- pilot.util.timing.read_pilot_timing()[source]
Read the pilot timing dictionary from file.
- Returns:
pilot timing dictionary (json dictionary).
- pilot.util.timing.time_stamp()[source]
Return ISO-8601 compliant date/time format
- Returns:
time information